2024-05-08T23:49:26.547 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T23:49:26.548 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857 --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/fs/test_o_trunc}} --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857/orig.config.yaml 2024-05-08T23:49:26.603 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T23:49:26.683 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857 branch: wip-pdonnell-testing-20240508.183908-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/fs/test_o_trunc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7698857' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240508.183908-debug ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.183908-debug sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1722 sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.183908-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ30cHpozKG5vr7SMaqBp+YxVUzW9/tuk/5qxJkTDIZj9QrxdNGrTstcjbK+jrgu4BzJfWyA+JrRDjjCYXeQ7do= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKOFtQ1vjbB8kCo2ft/5a9pPu/W3387j0Ne4oIT0lO7WPxDqRW8zFaNZCNJ7E2lkgnLQt5NC+TCA2v9PVwu/zUs= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATYXNdfokQFkFbK3BXRecvlBAe904IA5kdXE/qvbs8i8EblbYLVVgEzWVboP95u2H6GMOvj/ytQJ+NizWCqLsE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_22:06:20 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T23:49:26.683 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa; will attempt to use it 2024-05-08T23:49:26.684 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks 2024-05-08T23:49:26.684 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T23:49:26.685 INFO:teuthology.task.internal:Checking packages... 2024-05-08T23:49:26.709 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c9da4049f638c0f84bdd9b4e33c42e86046ac554' 2024-05-08T23:49:26.709 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T23:49:26.709 INFO:teuthology.packaging:ref: None 2024-05-08T23:49:26.709 INFO:teuthology.packaging:tag: None 2024-05-08T23:49:26.709 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.183908-debug 2024-05-08T23:49:26.709 INFO:teuthology.packaging:sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-08T23:49:26.710 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240508.183908-debug 2024-05-08T23:49:26.932 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3594.gc9da4049 2024-05-08T23:49:26.934 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T23:49:26.940 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T23:49:26.940 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T23:49:26.985 INFO:teuthology.task.internal:Saving configuration 2024-05-08T23:49:27.004 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T23:49:27.008 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T23:49:27.033 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 23:44:12.087074', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ30cHpozKG5vr7SMaqBp+YxVUzW9/tuk/5qxJkTDIZj9QrxdNGrTstcjbK+jrgu4BzJfWyA+JrRDjjCYXeQ7do='} 2024-05-08T23:49:27.054 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 23:44:12.085800', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKOFtQ1vjbB8kCo2ft/5a9pPu/W3387j0Ne4oIT0lO7WPxDqRW8zFaNZCNJ7E2lkgnLQt5NC+TCA2v9PVwu/zUs='} 2024-05-08T23:49:27.075 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 23:44:12.084170', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATYXNdfokQFkFbK3BXRecvlBAe904IA5kdXE/qvbs8i8EblbYLVVgEzWVboP95u2H6GMOvj/ytQJ+NizWCqLsE='} 2024-05-08T23:49:27.075 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T23:49:27.082 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-08T23:49:27.082 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-08T23:49:27.082 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-08T23:49:27.082 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T23:49:27.188 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5d06db8f70>, signals=[15]) 2024-05-08T23:49:27.188 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T23:49:27.194 INFO:teuthology.task.internal:Opening connections... 2024-05-08T23:49:27.195 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T23:49:27.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:49:27.282 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-05-08T23:49:27.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:49:27.361 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-05-08T23:49:27.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:49:27.435 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T23:49:27.442 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-05-08T23:49:27.461 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-05-08T23:49:27.461 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-05-08T23:49:27.519 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T23:49:27.520 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T23:49:27.521 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-05-08T23:49:27.544 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-05-08T23:49:27.563 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-05-08T23:49:27.563 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-05-08T23:49:27.619 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T23:49:27.620 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T23:49:27.621 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-05-08T23:49:27.645 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-05-08T23:49:27.663 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-05-08T23:49:27.663 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-05-08T23:49:27.721 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-05-08T23:49:27.721 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-05-08T23:49:27.721 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-05-08T23:49:27.721 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-05-08T23:49:27.721 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-05-08T23:49:27.721 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-05-08T23:49:27.721 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T23:49:27.721 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-05-08T23:49:27.722 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-05-08T23:49:27.722 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T23:49:27.722 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-05-08T23:49:27.722 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T23:49:27.722 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T23:49:27.722 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T23:49:27.722 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-05-08T23:49:27.744 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T23:49:27.755 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T23:49:27.777 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T23:49:27.777 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-05-08T23:49:27.780 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-05-08T23:49:27.782 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-05-08T23:49:27.799 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T23:49:27.804 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T23:49:27.804 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-05-08T23:49:27.838 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-05-08T23:49:27.840 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-05-08T23:49:27.946 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T23:49:28.024 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T23:49:28.039 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-08T23:49:28.039 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-08T23:49:28.039 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-08T23:49:28.040 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T23:49:28.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:49:28.057 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-05-08T23:49:28.115 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-437.el9.x86_64 2024-05-08T23:49:28.115 INFO:teuthology.task.kernel:Running kernel on smithi046: 5.14.0-437.el9.x86_64 2024-05-08T23:49:28.115 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum install -y kernel 2024-05-08T23:49:30.722 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.1 MB 00:00 2024-05-08T23:49:34.281 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-05-08T23:49:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-05-08T23:49:39.757 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 105 kB/s | 16 kB 00:00 2024-05-08T23:49:41.503 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-08T23:49:46.750 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-08T23:49:48.840 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T23:49:48.840 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T23:49:48.933 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-05-08T23:49:48.941 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T23:49:48.942 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T23:49:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T23:49:48.942 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T23:49:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T23:49:48.942 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi046.stdout:Install 4 Packages 2024-05-08T23:49:48.946 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:49:48.947 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 78 M 2024-05-08T23:49:48.947 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 123 M 2024-05-08T23:49:48.947 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T23:49:49.193 INFO:teuthology.orchestra.run.smithi046.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.2 MB/s | 484 kB 00:00 2024-05-08T23:49:50.985 INFO:teuthology.orchestra.run.smithi046.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 7.7 MB/s | 15 MB 00:02 2024-05-08T23:49:52.835 INFO:teuthology.orchestra.run.smithi046.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 7.7 MB/s | 28 MB 00:03 2024-05-08T23:49:53.335 INFO:teuthology.orchestra.run.smithi046.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 7.8 MB/s | 34 MB 00:04 2024-05-08T23:49:53.335 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:49:53.336 INFO:teuthology.orchestra.run.smithi046.stdout:Total 18 MB/s | 78 MB 00:04 2024-05-08T23:49:54.047 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T23:49:54.494 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T23:49:54.494 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T23:49:55.460 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T23:49:55.461 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T23:49:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T23:49:58.676 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T23:49:58.761 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T23:50:01.577 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T23:50:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T23:50:06.795 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T23:50:06.838 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T23:50:11.931 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T23:50:42.876 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T23:50:42.916 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T23:50:43.278 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T23:50:43.278 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T23:50:43.278 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T23:50:43.278 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T23:50:44.113 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T23:50:44.114 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:50:44.114 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T23:50:44.114 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T23:50:44.114 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T23:50:44.114 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T23:50:44.114 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T23:50:44.114 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:50:44.114 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T23:50:44.462 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T23:50:44.539 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T23:50:44.540 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T23:50:44.540 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T23:50:44.540 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T23:50:44.540 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T23:50:44.540 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-08T23:50:44.540 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T23:50:44.540 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T23:50:44.591 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T23:50:44.592 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T23:50:44.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -qi grub2-tools 2024-05-08T23:50:44.626 INFO:teuthology.orchestra.run.smithi046.stdout:Name : grub2-tools 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:Epoch : 1 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:Version : 2.06 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:Release : 68.el9 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: x86_64 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:Group : Unspecified 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:Size : 8264254 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:License : GPLv3+ 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T23:50:44.627 INFO:teuthology.orchestra.run.smithi046.stdout:Packager : builder@centos.org 2024-05-08T23:50:44.628 INFO:teuthology.orchestra.run.smithi046.stdout:Vendor : CentOS 2024-05-08T23:50:44.628 INFO:teuthology.orchestra.run.smithi046.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T23:50:44.628 INFO:teuthology.orchestra.run.smithi046.stdout:Summary : Support tools for GRUB. 2024-05-08T23:50:44.628 INFO:teuthology.orchestra.run.smithi046.stdout:Description : 2024-05-08T23:50:44.628 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:50:44.628 INFO:teuthology.orchestra.run.smithi046.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T23:50:44.628 INFO:teuthology.orchestra.run.smithi046.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T23:50:44.628 INFO:teuthology.orchestra.run.smithi046.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T23:50:44.628 INFO:teuthology.orchestra.run.smithi046.stdout:hardware devices. 2024-05-08T23:50:44.628 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:50:44.628 INFO:teuthology.orchestra.run.smithi046.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T23:50:44.629 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T23:50:44.629 INFO:teuthology.task.kernel:Updating grub on smithi046 to boot 5.14.0-444.el9.x86_64 2024-05-08T23:50:44.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T23:50:45.232 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-05-08T23:50:46.652 INFO:teuthology.orchestra.run.smithi046.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T23:50:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-05-08T23:50:46.722 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-05-08T23:50:46.742 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.jB6yrK7N5g 2024-05-08T23:50:46.742 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jB6yrK7N5g 2024-05-08T23:50:46.815 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.jB6yrK7N5g 2024-05-08T23:50:46.956 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.jB6yrK7N5g is 6KB 2024-05-08T23:50:47.005 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.jB6yrK7N5g 2024-05-08T23:50:47.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T23:50:47.087 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T23:50:47.087 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T23:50:47.088 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T23:50:47.088 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T23:50:47.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T23:50:47.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-05-08T23:51:17.244 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-08T23:51:17.245 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T23:51:17.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:51:51.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-05-08T23:52:00.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T23:52:00.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:52:19.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-05-08T23:52:31.231 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T23:52:31.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:52:32.550 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-05-08T23:52:33.004 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T23:52:33.004 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-08T23:52:33.005 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-05-08T23:52:33.023 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-444.el9.x86_64 2024-05-08T23:52:33.023 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T23:52:33.024 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T23:52:33.024 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T23:52:34.025 DEBUG:teuthology.parallel:result is None 2024-05-08T23:52:34.025 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T23:52:34.033 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T23:52:34.033 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T23:52:34.037 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T23:52:34.039 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T23:52:34.057 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T23:52:34.063 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T23:52:34.097 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T23:52:34.098 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T23:52:34.100 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T23:52:34.102 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T23:52:34.194 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T23:52:34.201 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T23:52:34.201 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T23:52:34.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:52:34.218 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T23:52:34.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:52:34.235 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T23:52:34.237 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T23:52:34.261 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T23:52:34.301 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:52:34.309 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:52:34.317 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:52:34.324 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:52:34.340 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:52:34.350 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:52:34.351 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T23:52:34.358 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T23:52:34.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T23:52:34.361 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T23:52:34.363 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T23:52:34.390 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T23:52:34.399 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T23:52:34.399 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T23:52:34.430 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T23:52:34.432 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T23:52:34.448 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:52:34.527 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:52:34.570 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:52:34.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:52:34.692 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:52:34.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T23:52:34.761 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:52:34.805 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:52:34.852 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:52:34.936 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:52:34.978 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-08T23:52:34.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T23:52:35.045 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:52:35.089 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:52:35.147 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:52:35.232 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:52:35.272 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-08T23:52:35.272 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T23:52:35.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-05-08T23:52:35.340 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-05-08T23:52:35.343 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-05-08T23:52:35.386 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:52:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:52:35.424 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:52:35.886 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T23:52:35.894 INFO:teuthology.task.internal:Starting timer... 2024-05-08T23:52:35.894 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T23:52:35.952 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T23:52:35.983 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-08T23:52:35.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-05-08T23:52:36.039 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-05-08T23:52:36.041 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-05-08T23:52:36.110 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-05-08T23:52:36.112 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-05-08T23:52:36.172 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-05-08T23:52:36.174 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T23:52:36.174 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-05-08T23:52:36.195 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-05-08T23:52:36.195 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-05-08T23:52:36.224 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-05-08T23:52:36.225 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-05-08T23:52:36.254 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-05-08T23:52:36.254 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2024-05-08T23:52:36.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T23:52:36.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:52:36.281 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T23:52:36.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:52:36.309 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T23:52:36.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:52:36.336 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T23:52:36.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-05-08T23:52:36.383 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-05-08T23:52:36.419 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-05-08T23:52:36.450 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T23:52:36.459 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T23:52:36.476 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T23:52:36.477 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi142.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T23:57:00.697 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-05-08T23:57:00.699 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T23:57:00.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:57:00.787 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-05-08T23:57:00.872 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T23:57:00.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-05-08T23:57:00.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:57:00.953 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-05-08T23:57:01.031 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-05-08T23:57:01.031 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-05-08T23:57:01.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:57:01.106 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-05-08T23:57:01.188 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-05-08T23:57:01.188 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T23:57:01.200 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T23:57:01.201 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:57:01.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:57:01.204 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:57:01.204 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:57:01.207 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:57:01.207 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:57:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:57:01.238 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:57:01.253 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:57:01.254 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:57:01.261 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:57:01.275 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:57:01.278 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-05-08T23:57:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-05-08T23:57:01.289 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-05-08T23:57:01.294 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-05-08T23:57:01.299 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-05-08T23:57:01.302 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:57:01.311 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-05-08T23:57:01.311 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:57:01.316 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:57:01.324 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:57:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:57:01.338 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:57:01.374 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-05-08T23:57:01.377 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:57:01.377 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-05-08T23:57:01.377 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:57:01.378 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:57:01.378 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:57:01.378 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:57:01.379 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-05-08T23:57:01.383 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:57:01.383 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-05-08T23:57:01.383 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:57:01.383 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:57:01.383 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:57:01.383 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:57:01.394 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-05-08T23:57:01.397 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:57:01.397 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-05-08T23:57:01.397 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:57:01.397 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:57:01.397 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:57:01.397 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:57:01.398 INFO:teuthology.run_tasks:Running task install... 2024-05-08T23:57:01.407 DEBUG:teuthology.task.install:project ceph 2024-05-08T23:57:01.407 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'}} 2024-05-08T23:57:01.407 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'} 2024-05-08T23:57:01.407 INFO:teuthology.task.install:Using flavor: default 2024-05-08T23:57:01.415 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T23:57:01.415 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T23:57:01.416 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-08T23:57:01.416 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-08T23:57:01.419 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-08T23:57:01.419 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-08T23:57:01.420 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-08T23:57:01.420 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-08T23:57:01.562 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-08T23:57:01.562 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-08T23:57:01.567 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-08T23:57:01.567 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-08T23:57:01.641 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-08T23:57:01.641 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-08T23:57:01.771 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:57:01.771 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-08T23:57:01.771 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:57:01.777 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:57:01.777 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-08T23:57:01.777 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:57:01.778 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:57:01.778 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:57:01.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:57:01.797 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:57:01.797 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:57:01.803 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:57:01.803 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:57:01.806 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:57:01.807 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:57:01.863 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:57:01.867 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:57:01.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:57:01.937 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:57:01.940 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:57:01.952 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:57:02.001 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-05-08T23:57:02.002 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-05-08T23:57:02.003 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-05-08T23:57:02.004 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-05-08T23:57:02.018 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-05-08T23:57:02.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-05-08T23:57:02.230 INFO:teuthology.orchestra.run.smithi142.stdout:54 files removed 2024-05-08T23:57:02.232 INFO:teuthology.orchestra.run.smithi169.stdout:54 files removed 2024-05-08T23:57:02.257 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:57:02.259 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:57:02.261 INFO:teuthology.orchestra.run.smithi046.stdout:54 files removed 2024-05-08T23:57:02.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:57:02.929 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 203 kB/s | 84 kB 00:00 2024-05-08T23:57:03.027 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 184 kB/s | 84 kB 00:00 2024-05-08T23:57:03.097 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 146 kB/s | 84 kB 00:00 2024-05-08T23:57:03.521 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 21 kB/s | 12 kB 00:00 2024-05-08T23:57:03.570 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-05-08T23:57:03.623 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-05-08T23:57:04.297 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-05-08T23:57:04.298 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-05-08T23:57:04.303 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-05-08T23:57:04.722 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.1 MB 00:00 2024-05-08T23:57:04.790 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.1 MB 00:00 2024-05-08T23:57:05.726 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.1 MB 00:01 2024-05-08T23:57:06.566 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-05-08T23:57:06.683 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-08T23:57:07.497 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 32 MB/s | 19 MB 00:00 2024-05-08T23:57:11.643 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.2 MB 00:01 2024-05-08T23:57:12.607 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.2 MB 00:02 2024-05-08T23:57:13.043 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 16 kB 00:00 2024-05-08T23:57:13.339 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-05-08T23:57:14.875 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-05-08T23:57:15.240 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-05-08T23:57:16.095 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-08T23:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-08T23:57:21.385 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 7.2 kB/s | 1.7 kB 00:00 2024-05-08T23:57:22.758 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-05-08T23:57:23.686 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T23:57:23.687 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T23:57:23.698 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T23:57:23.699 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T23:57:23.699 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T23:57:23.700 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T23:57:23.702 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T23:57:23.704 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T23:57:23.705 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T23:57:23.766 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-05-08T23:57:23.781 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-05-08T23:57:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-05-08T23:57:23.781 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-05-08T23:57:23.781 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-05-08T23:57:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T23:57:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-08T23:57:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-08T23:57:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-08T23:57:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-08T23:57:23.781 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-08T23:57:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-08T23:57:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-08T23:57:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-08T23:57:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-08T23:57:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-08T23:57:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-08T23:57:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-08T23:57:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 764 k 2024-05-08T23:57:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-08T23:57:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-08T23:57:23.782 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 161 k 2024-05-08T23:57:23.783 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 322 k 2024-05-08T23:57:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 300 k 2024-05-08T23:57:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-08T23:57:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-08T23:57:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-08T23:57:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-08T23:57:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T23:57:23.784 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-05-08T23:57:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T23:57:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T23:57:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T23:57:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T23:57:23.784 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-08T23:57:23.785 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-08T23:57:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-08T23:57:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-08T23:57:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T23:57:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T23:57:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T23:57:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T23:57:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T23:57:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T23:57:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T23:57:23.786 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T23:57:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-08T23:57:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T23:57:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T23:57:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T23:57:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T23:57:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T23:57:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-08T23:57:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T23:57:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-08T23:57:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T23:57:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T23:57:23.787 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T23:57:23.788 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T23:57:23.789 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T23:57:23.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T23:57:23.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T23:57:23.792 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T23:57:23.793 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T23:57:23.794 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-05-08T23:57:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T23:57:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:57:23.794 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-05-08T23:57:23.794 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-05-08T23:57:23.794 INFO:teuthology.orchestra.run.smithi142.stdout:Install 148 Packages 2024-05-08T23:57:23.794 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 8 Packages 2024-05-08T23:57:23.794 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:57:23.795 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 219 M 2024-05-08T23:57:23.795 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-05-08T23:57:24.628 INFO:teuthology.orchestra.run.smithi142.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-05-08T23:57:24.836 INFO:teuthology.orchestra.run.smithi142.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 4.1 MB/s | 863 kB 00:00 2024-05-08T23:57:24.886 INFO:teuthology.orchestra.run.smithi142.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 2.9 MB/s | 147 kB 00:00 2024-05-08T23:57:25.170 INFO:teuthology.orchestra.run.smithi142.stdout:(4/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 8.0 MB/s | 2.3 MB 00:00 2024-05-08T23:57:25.186 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T23:57:25.186 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T23:57:25.198 INFO:teuthology.orchestra.run.smithi046.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T23:57:25.199 INFO:teuthology.orchestra.run.smithi046.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T23:57:25.199 INFO:teuthology.orchestra.run.smithi046.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T23:57:25.200 INFO:teuthology.orchestra.run.smithi046.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T23:57:25.202 INFO:teuthology.orchestra.run.smithi046.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T23:57:25.204 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T23:57:25.205 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T23:57:25.272 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T23:57:25.287 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-08T23:57:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 764 k 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T23:57:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 161 k 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 322 k 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 300 k 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-05-08T23:57:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-08T23:57:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-08T23:57:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T23:57:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T23:57:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T23:57:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T23:57:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T23:57:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T23:57:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T23:57:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T23:57:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T23:57:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T23:57:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T23:57:25.299 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-05-08T23:57:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T23:57:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:57:25.299 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T23:57:25.299 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-05-08T23:57:25.299 INFO:teuthology.orchestra.run.smithi046.stdout:Install 148 Packages 2024-05-08T23:57:25.299 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 8 Packages 2024-05-08T23:57:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:57:25.300 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 219 M 2024-05-08T23:57:25.300 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T23:57:25.387 INFO:teuthology.orchestra.run.smithi142.stdout:(5/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-05-08T23:57:25.554 INFO:teuthology.orchestra.run.smithi142.stdout:(6/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 4.9 MB/s | 5.4 MB 00:01 2024-05-08T23:57:25.962 INFO:teuthology.orchestra.run.smithi142.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 8.5 MB/s | 4.9 MB 00:00 2024-05-08T23:57:26.026 INFO:teuthology.orchestra.run.smithi046.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-08T23:57:26.337 INFO:teuthology.orchestra.run.smithi142.stdout:(8/156): ceph-common-19.0.0-3594.gc9da4049.el9. 10 MB/s | 20 MB 00:01 2024-05-08T23:57:26.427 INFO:teuthology.orchestra.run.smithi142.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 281 kB/s | 25 kB 00:00 2024-05-08T23:57:26.452 INFO:teuthology.orchestra.run.smithi046.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 2.0 MB/s | 863 kB 00:00 2024-05-08T23:57:26.552 INFO:teuthology.orchestra.run.smithi046.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 1.4 MB/s | 147 kB 00:00 2024-05-08T23:57:27.221 INFO:teuthology.orchestra.run.smithi142.stdout:(10/156): ceph-radosgw-19.0.0-3594.gc9da4049.el 8.0 MB/s | 10 MB 00:01 2024-05-08T23:57:27.297 INFO:teuthology.orchestra.run.smithi142.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 2.9 MB/s | 224 kB 00:00 2024-05-08T23:57:27.380 INFO:teuthology.orchestra.run.smithi142.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 325 kB/s | 27 kB 00:00 2024-05-08T23:57:27.419 INFO:teuthology.orchestra.run.smithi046.stdout:(4/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 2.6 MB/s | 2.3 MB 00:00 2024-05-08T23:57:27.455 INFO:teuthology.orchestra.run.smithi142.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 423 kB/s | 32 kB 00:00 2024-05-08T23:57:27.756 INFO:teuthology.orchestra.run.smithi142.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 2.5 MB/s | 756 kB 00:00 2024-05-08T23:57:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:(5/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 3.8 MB/s | 1.5 MB 00:00 2024-05-08T23:57:27.831 INFO:teuthology.orchestra.run.smithi142.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 2.1 MB/s | 158 kB 00:00 2024-05-08T23:57:27.914 INFO:teuthology.orchestra.run.smithi142.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.5 MB/s | 127 kB 00:00 2024-05-08T23:57:28.073 INFO:teuthology.orchestra.run.smithi142.stdout:(17/156): libradosstriper1-19.0.0-3594.gc9da404 3.1 MB/s | 497 kB 00:00 2024-05-08T23:57:28.298 INFO:teuthology.orchestra.run.smithi142.stdout:(18/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 6.5 MB/s | 18 MB 00:02 2024-05-08T23:57:28.415 INFO:teuthology.orchestra.run.smithi142.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 387 kB/s | 45 kB 00:00 2024-05-08T23:57:28.532 INFO:teuthology.orchestra.run.smithi142.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 1.1 MB/s | 135 kB 00:00 2024-05-08T23:57:28.641 INFO:teuthology.orchestra.run.smithi142.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 1.5 MB/s | 161 kB 00:00 2024-05-08T23:57:28.669 INFO:teuthology.orchestra.run.smithi046.stdout:(6/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 5.7 MB/s | 4.9 MB 00:00 2024-05-08T23:57:28.741 INFO:teuthology.orchestra.run.smithi142.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 3.1 MB/s | 322 kB 00:00 2024-05-08T23:57:28.803 INFO:teuthology.orchestra.run.smithi046.stdout:(7/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-05-08T23:57:28.850 INFO:teuthology.orchestra.run.smithi142.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 2.7 MB/s | 300 kB 00:00 2024-05-08T23:57:28.967 INFO:teuthology.orchestra.run.smithi142.stdout:(24/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 5.8 MB/s | 5.2 MB 00:00 2024-05-08T23:57:28.992 INFO:teuthology.orchestra.run.smithi142.stdout:(25/156): python3-rgw-19.0.0-3594.gc9da4049.el9 701 kB/s | 99 kB 00:00 2024-05-08T23:57:29.034 INFO:teuthology.orchestra.run.smithi142.stdout:(26/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.2 MB/s | 86 kB 00:00 2024-05-08T23:57:29.118 INFO:teuthology.orchestra.run.smithi142.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 2.0 MB/s | 169 kB 00:00 2024-05-08T23:57:29.218 INFO:teuthology.orchestra.run.smithi142.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 342 kB/s | 34 kB 00:00 2024-05-08T23:57:29.310 INFO:teuthology.orchestra.run.smithi142.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 1.5 MB/s | 142 kB 00:00 2024-05-08T23:57:29.711 INFO:teuthology.orchestra.run.smithi142.stdout:(30/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 4.2 MB/s | 3.0 MB 00:00 2024-05-08T23:57:30.122 INFO:teuthology.orchestra.run.smithi142.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 4.4 MB/s | 3.5 MB 00:00 2024-05-08T23:57:30.219 INFO:teuthology.orchestra.run.smithi142.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 2.7 MB/s | 274 kB 00:00 2024-05-08T23:57:30.303 INFO:teuthology.orchestra.run.smithi142.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 592 kB/s | 49 kB 00:00 2024-05-08T23:57:30.394 INFO:teuthology.orchestra.run.smithi142.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 180 kB/s | 16 kB 00:00 2024-05-08T23:57:30.470 INFO:teuthology.orchestra.run.smithi142.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 3.6 MB/s | 273 kB 00:00 2024-05-08T23:57:30.519 INFO:teuthology.orchestra.run.smithi046.stdout:(8/156): ceph-radosgw-19.0.0-3594.gc9da4049.el9 5.9 MB/s | 10 MB 00:01 2024-05-08T23:57:30.612 INFO:teuthology.orchestra.run.smithi142.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 5.3 MB/s | 764 kB 00:00 2024-05-08T23:57:30.620 INFO:teuthology.orchestra.run.smithi046.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 252 kB/s | 25 kB 00:00 2024-05-08T23:57:30.662 INFO:teuthology.orchestra.run.smithi142.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 366 kB/s | 18 kB 00:00 2024-05-08T23:57:30.663 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 1.2 MB/s | 21 MB 00:17 2024-05-08T23:57:31.371 INFO:teuthology.orchestra.run.smithi142.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 113 kB/s | 80 kB 00:00 2024-05-08T23:57:31.488 INFO:teuthology.orchestra.run.smithi142.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-05-08T23:57:31.728 INFO:teuthology.orchestra.run.smithi046.stdout:(10/156): ceph-common-19.0.0-3594.gc9da4049.el9 3.2 MB/s | 20 MB 00:06 2024-05-08T23:57:31.795 INFO:teuthology.orchestra.run.smithi046.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 3.3 MB/s | 224 kB 00:00 2024-05-08T23:57:31.896 INFO:teuthology.orchestra.run.smithi046.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 270 kB/s | 27 kB 00:00 2024-05-08T23:57:32.005 INFO:teuthology.orchestra.run.smithi046.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 293 kB/s | 32 kB 00:00 2024-05-08T23:57:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 249 kB/s | 195 kB 00:00 2024-05-08T23:57:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 2.5 MB/s | 756 kB 00:00 2024-05-08T23:57:32.405 INFO:teuthology.orchestra.run.smithi046.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.5 MB/s | 158 kB 00:00 2024-05-08T23:57:32.489 INFO:teuthology.orchestra.run.smithi046.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.5 MB/s | 127 kB 00:00 2024-05-08T23:57:32.646 INFO:teuthology.orchestra.run.smithi142.stdout:(41/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 6.9 MB/s | 43 MB 00:06 2024-05-08T23:57:32.647 INFO:teuthology.orchestra.run.smithi046.stdout:(17/156): libradosstriper1-19.0.0-3594.gc9da404 3.1 MB/s | 497 kB 00:00 2024-05-08T23:57:32.680 INFO:teuthology.orchestra.run.smithi142.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 614 kB/s | 803 kB 00:01 2024-05-08T23:57:33.006 INFO:teuthology.orchestra.run.smithi046.stdout:(18/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 4.1 MB/s | 18 MB 00:04 2024-05-08T23:57:33.064 INFO:teuthology.orchestra.run.smithi046.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 777 kB/s | 45 kB 00:00 2024-05-08T23:57:33.380 INFO:teuthology.orchestra.run.smithi142.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 228 kB/s | 253 kB 00:01 2024-05-08T23:57:33.405 INFO:teuthology.orchestra.run.smithi142.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 147 kB/s | 106 kB 00:00 2024-05-08T23:57:33.465 INFO:teuthology.orchestra.run.smithi046.stdout:(20/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 15 MB/s | 43 MB 00:02 2024-05-08T23:57:33.490 INFO:teuthology.orchestra.run.smithi046.stdout:(21/156): python3-ceph-common-19.0.0-3594.gc9da 319 kB/s | 135 kB 00:00 2024-05-08T23:57:33.532 INFO:teuthology.orchestra.run.smithi046.stdout:(22/156): python3-cephfs-19.0.0-3594.gc9da4049. 2.4 MB/s | 161 kB 00:00 2024-05-08T23:57:33.590 INFO:teuthology.orchestra.run.smithi046.stdout:(23/156): python3-rados-19.0.0-3594.gc9da4049.e 3.1 MB/s | 322 kB 00:00 2024-05-08T23:57:33.616 INFO:teuthology.orchestra.run.smithi046.stdout:(24/156): python3-rbd-19.0.0-3594.gc9da4049.el9 3.5 MB/s | 300 kB 00:00 2024-05-08T23:57:33.630 INFO:teuthology.orchestra.run.smithi142.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:00 2024-05-08T23:57:33.699 INFO:teuthology.orchestra.run.smithi046.stdout:(25/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 4.9 MB/s | 5.2 MB 00:01 2024-05-08T23:57:33.725 INFO:teuthology.orchestra.run.smithi046.stdout:(26/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 788 kB/s | 86 kB 00:00 2024-05-08T23:57:33.750 INFO:teuthology.orchestra.run.smithi046.stdout:(27/156): python3-rgw-19.0.0-3594.gc9da4049.el9 626 kB/s | 99 kB 00:00 2024-05-08T23:57:33.783 INFO:teuthology.orchestra.run.smithi046.stdout:(28/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 2.8 MB/s | 169 kB 00:00 2024-05-08T23:57:33.809 INFO:teuthology.orchestra.run.smithi046.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3594.g 583 kB/s | 34 kB 00:00 2024-05-08T23:57:33.842 INFO:teuthology.orchestra.run.smithi046.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 2.4 MB/s | 142 kB 00:00 2024-05-08T23:57:34.059 INFO:teuthology.orchestra.run.smithi046.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 14 MB/s | 3.5 MB 00:00 2024-05-08T23:57:34.126 INFO:teuthology.orchestra.run.smithi046.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 4.0 MB/s | 274 kB 00:00 2024-05-08T23:57:34.155 INFO:teuthology.orchestra.run.smithi142.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 174 kB/s | 135 kB 00:00 2024-05-08T23:57:34.189 INFO:teuthology.orchestra.run.smithi142.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 159 kB/s | 125 kB 00:00 2024-05-08T23:57:34.210 INFO:teuthology.orchestra.run.smithi046.stdout:(33/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 6.0 MB/s | 3.0 MB 00:00 2024-05-08T23:57:34.235 INFO:teuthology.orchestra.run.smithi046.stdout:(34/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 455 kB/s | 49 kB 00:00 2024-05-08T23:57:34.269 INFO:teuthology.orchestra.run.smithi046.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3594.gc 281 kB/s | 16 kB 00:00 2024-05-08T23:57:34.302 INFO:teuthology.orchestra.run.smithi046.stdout:(36/156): ceph-volume-19.0.0-3594.gc9da4049.el9 4.0 MB/s | 273 kB 00:00 2024-05-08T23:57:34.361 INFO:teuthology.orchestra.run.smithi046.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 312 kB/s | 18 kB 00:00 2024-05-08T23:57:34.428 INFO:teuthology.orchestra.run.smithi046.stdout:(38/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 4.7 MB/s | 764 kB 00:00 2024-05-08T23:57:34.481 INFO:teuthology.orchestra.run.smithi142.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 270 kB/s | 215 kB 00:00 2024-05-08T23:57:34.553 INFO:teuthology.orchestra.run.smithi046.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 419 kB/s | 80 kB 00:00 2024-05-08T23:57:34.645 INFO:teuthology.orchestra.run.smithi046.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-08T23:57:34.737 INFO:teuthology.orchestra.run.smithi046.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-08T23:57:34.762 INFO:teuthology.orchestra.run.smithi046.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-05-08T23:57:34.813 INFO:teuthology.orchestra.run.smithi046.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-08T23:57:34.823 INFO:teuthology.orchestra.run.smithi142.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-05-08T23:57:34.856 INFO:teuthology.orchestra.run.smithi142.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 1.4 MB/s | 943 kB 00:00 2024-05-08T23:57:34.871 INFO:teuthology.orchestra.run.smithi046.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-08T23:57:34.898 INFO:teuthology.orchestra.run.smithi142.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 256 kB/s | 107 kB 00:00 2024-05-08T23:57:34.924 INFO:teuthology.orchestra.run.smithi142.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-05-08T23:57:34.930 INFO:teuthology.orchestra.run.smithi046.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-08T23:57:34.966 INFO:teuthology.orchestra.run.smithi142.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-08T23:57:35.013 INFO:teuthology.orchestra.run.smithi046.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-08T23:57:35.016 INFO:teuthology.orchestra.run.smithi142.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-05-08T23:57:35.047 INFO:teuthology.orchestra.run.smithi046.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 4.0 MB/s | 1.2 MB 00:00 2024-05-08T23:57:35.066 INFO:teuthology.orchestra.run.smithi142.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-05-08T23:57:35.117 INFO:teuthology.orchestra.run.smithi142.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-05-08T23:57:35.175 INFO:teuthology.orchestra.run.smithi142.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-08T23:57:35.217 INFO:teuthology.orchestra.run.smithi142.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-05-08T23:57:35.284 INFO:teuthology.orchestra.run.smithi142.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-05-08T23:57:35.364 INFO:teuthology.orchestra.run.smithi046.stdout:(48/156): ceph-mgr-diskprediction-local-19.0.0- 4.9 MB/s | 7.4 MB 00:01 2024-05-08T23:57:35.390 INFO:teuthology.orchestra.run.smithi046.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-05-08T23:57:35.423 INFO:teuthology.orchestra.run.smithi046.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 943 kB 00:00 2024-05-08T23:57:35.426 INFO:teuthology.orchestra.run.smithi142.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-05-08T23:57:35.441 INFO:teuthology.orchestra.run.smithi046.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-05-08T23:57:35.451 INFO:teuthology.orchestra.run.smithi142.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 152 kB/s | 25 kB 00:00 2024-05-08T23:57:35.490 INFO:teuthology.orchestra.run.smithi046.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-05-08T23:57:35.516 INFO:teuthology.orchestra.run.smithi046.stdout:(53/156): boost-program-options-1.75.0-8.el9.x8 709 kB/s | 107 kB 00:00 2024-05-08T23:57:35.526 INFO:teuthology.orchestra.run.smithi142.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-05-08T23:57:35.560 INFO:teuthology.orchestra.run.smithi142.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-08T23:57:35.565 INFO:teuthology.orchestra.run.smithi046.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-08T23:57:35.591 INFO:teuthology.orchestra.run.smithi046.stdout:(55/156): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-05-08T23:57:35.608 INFO:teuthology.orchestra.run.smithi046.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-05-08T23:57:35.610 INFO:teuthology.orchestra.run.smithi142.stdout:(64/156): libtool-2.4.6-45.el9.x86_64.rpm 1.4 MB/s | 582 kB 00:00 2024-05-08T23:57:35.636 INFO:teuthology.orchestra.run.smithi142.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-08T23:57:35.650 INFO:teuthology.orchestra.run.smithi046.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-08T23:57:35.652 INFO:teuthology.orchestra.run.smithi142.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 382 kB/s | 35 kB 00:00 2024-05-08T23:57:35.675 INFO:teuthology.orchestra.run.smithi046.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-05-08T23:57:35.842 INFO:teuthology.orchestra.run.smithi046.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-05-08T23:57:35.867 INFO:teuthology.orchestra.run.smithi046.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 3.0 MB/s | 582 kB 00:00 2024-05-08T23:57:35.884 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-05-08T23:57:35.892 INFO:teuthology.orchestra.run.smithi046.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 506 kB/s | 25 kB 00:00 2024-05-08T23:57:35.935 INFO:teuthology.orchestra.run.smithi046.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 706 kB/s | 47 kB 00:00 2024-05-08T23:57:35.945 INFO:teuthology.orchestra.run.smithi142.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-05-08T23:57:35.993 INFO:teuthology.orchestra.run.smithi046.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-05-08T23:57:36.053 INFO:teuthology.orchestra.run.smithi046.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-08T23:57:36.061 INFO:teuthology.orchestra.run.smithi142.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-05-08T23:57:36.069 INFO:teuthology.orchestra.run.smithi046.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 35 kB 00:00 2024-05-08T23:57:36.120 INFO:teuthology.orchestra.run.smithi142.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-05-08T23:57:36.170 INFO:teuthology.orchestra.run.smithi142.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-08T23:57:36.336 INFO:teuthology.orchestra.run.smithi046.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.5 MB/s | 3.0 MB 00:00 2024-05-08T23:57:36.454 INFO:teuthology.orchestra.run.smithi142.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-05-08T23:57:36.546 INFO:teuthology.orchestra.run.smithi142.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-05-08T23:57:36.604 INFO:teuthology.orchestra.run.smithi142.stdout:(73/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.8 MB/s | 4.7 MB 00:00 2024-05-08T23:57:36.629 INFO:teuthology.orchestra.run.smithi142.stdout:(74/156): python3-attrs-20.3.0-7.el9.noarch.rpm 498 kB/s | 87 kB 00:00 2024-05-08T23:57:36.711 INFO:teuthology.orchestra.run.smithi046.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-05-08T23:57:36.738 INFO:teuthology.orchestra.run.smithi142.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-08T23:57:36.847 INFO:teuthology.orchestra.run.smithi142.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-05-08T23:57:36.945 INFO:teuthology.orchestra.run.smithi046.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 3.8 MB/s | 911 kB 00:00 2024-05-08T23:57:36.989 INFO:teuthology.orchestra.run.smithi142.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-05-08T23:57:37.014 INFO:teuthology.orchestra.run.smithi142.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 115 kB/s | 19 kB 00:00 2024-05-08T23:57:37.020 INFO:teuthology.orchestra.run.smithi046.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-05-08T23:57:37.039 INFO:teuthology.orchestra.run.smithi142.stdout:(79/156): python3-devel-3.9.18-3.el9.x86_64.rpm 562 kB/s | 244 kB 00:00 2024-05-08T23:57:37.045 INFO:teuthology.orchestra.run.smithi046.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-05-08T23:57:37.098 INFO:teuthology.orchestra.run.smithi142.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-05-08T23:57:37.123 INFO:teuthology.orchestra.run.smithi142.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 319 kB/s | 35 kB 00:00 2024-05-08T23:57:37.204 INFO:teuthology.orchestra.run.smithi046.stdout:(71/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.1 MB/s | 4.7 MB 00:01 2024-05-08T23:57:37.206 INFO:teuthology.orchestra.run.smithi142.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-08T23:57:37.232 INFO:teuthology.orchestra.run.smithi142.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-05-08T23:57:37.254 INFO:teuthology.orchestra.run.smithi046.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-08T23:57:37.296 INFO:teuthology.orchestra.run.smithi046.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-05-08T23:57:37.315 INFO:teuthology.orchestra.run.smithi142.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 712 kB/s | 77 kB 00:00 2024-05-08T23:57:37.341 INFO:teuthology.orchestra.run.smithi142.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-05-08T23:57:37.388 INFO:teuthology.orchestra.run.smithi046.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-05-08T23:57:37.424 INFO:teuthology.orchestra.run.smithi142.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-05-08T23:57:37.458 INFO:teuthology.orchestra.run.smithi142.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-05-08T23:57:37.471 INFO:teuthology.orchestra.run.smithi046.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-08T23:57:37.522 INFO:teuthology.orchestra.run.smithi046.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-05-08T23:57:37.533 INFO:teuthology.orchestra.run.smithi142.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-05-08T23:57:37.559 INFO:teuthology.orchestra.run.smithi142.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-05-08T23:57:37.580 INFO:teuthology.orchestra.run.smithi046.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-05-08T23:57:37.642 INFO:teuthology.orchestra.run.smithi142.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 494 kB/s | 54 kB 00:00 2024-05-08T23:57:37.732 INFO:teuthology.orchestra.run.smithi046.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-05-08T23:57:37.798 INFO:teuthology.orchestra.run.smithi046.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 530 kB/s | 35 kB 00:00 2024-05-08T23:57:37.843 INFO:teuthology.orchestra.run.smithi142.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 208 kB/s | 42 kB 00:00 2024-05-08T23:57:37.943 INFO:teuthology.orchestra.run.smithi142.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-05-08T23:57:38.043 INFO:teuthology.orchestra.run.smithi142.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-05-08T23:57:38.073 INFO:teuthology.orchestra.run.smithi046.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 5.5 MB/s | 5.8 MB 00:01 2024-05-08T23:57:38.132 INFO:teuthology.orchestra.run.smithi046.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-05-08T23:57:38.143 INFO:teuthology.orchestra.run.smithi142.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 360 kB/s | 36 kB 00:00 2024-05-08T23:57:38.244 INFO:teuthology.orchestra.run.smithi142.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-05-08T23:57:38.244 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T23:57:38.245 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T23:57:38.255 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T23:57:38.256 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T23:57:38.257 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T23:57:38.258 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T23:57:38.259 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T23:57:38.262 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T23:57:38.262 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T23:57:38.274 INFO:teuthology.orchestra.run.smithi046.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-05-08T23:57:38.323 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-05-08T23:57:38.324 INFO:teuthology.orchestra.run.smithi046.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-08T23:57:38.344 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-05-08T23:57:38.344 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-05-08T23:57:38.344 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-05-08T23:57:38.344 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-05-08T23:57:38.344 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T23:57:38.344 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-08T23:57:38.344 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-08T23:57:38.344 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-08T23:57:38.344 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-08T23:57:38.344 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-08T23:57:38.344 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 764 k 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T23:57:38.345 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 161 k 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 322 k 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 300 k 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-08T23:57:38.346 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T23:57:38.347 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T23:57:38.348 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T23:57:38.349 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T23:57:38.350 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T23:57:38.351 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T23:57:38.352 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T23:57:38.353 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T23:57:38.354 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T23:57:38.355 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T23:57:38.356 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-05-08T23:57:38.356 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T23:57:38.356 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:57:38.356 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-05-08T23:57:38.356 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-05-08T23:57:38.356 INFO:teuthology.orchestra.run.smithi169.stdout:Install 148 Packages 2024-05-08T23:57:38.356 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 8 Packages 2024-05-08T23:57:38.356 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:57:38.357 INFO:teuthology.orchestra.run.smithi142.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-05-08T23:57:38.357 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 219 M 2024-05-08T23:57:38.357 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-05-08T23:57:38.408 INFO:teuthology.orchestra.run.smithi046.stdout:(84/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-05-08T23:57:38.433 INFO:teuthology.orchestra.run.smithi046.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-05-08T23:57:38.453 INFO:teuthology.orchestra.run.smithi142.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 578 kB/s | 58 kB 00:00 2024-05-08T23:57:38.466 INFO:teuthology.orchestra.run.smithi046.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-08T23:57:38.492 INFO:teuthology.orchestra.run.smithi046.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-08T23:57:38.526 INFO:teuthology.orchestra.run.smithi046.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-05-08T23:57:38.551 INFO:teuthology.orchestra.run.smithi046.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 865 kB/s | 51 kB 00:00 2024-05-08T23:57:38.553 INFO:teuthology.orchestra.run.smithi142.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 940 kB/s | 94 kB 00:00 2024-05-08T23:57:38.585 INFO:teuthology.orchestra.run.smithi046.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 914 kB/s | 54 kB 00:00 2024-05-08T23:57:38.653 INFO:teuthology.orchestra.run.smithi142.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 637 kB/s | 64 kB 00:00 2024-05-08T23:57:38.677 INFO:teuthology.orchestra.run.smithi046.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.8 MB/s | 5.1 MB 00:00 2024-05-08T23:57:38.702 INFO:teuthology.orchestra.run.smithi046.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 356 kB/s | 42 kB 00:00 2024-05-08T23:57:38.727 INFO:teuthology.orchestra.run.smithi046.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-05-08T23:57:38.761 INFO:teuthology.orchestra.run.smithi046.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-08T23:57:38.762 INFO:teuthology.orchestra.run.smithi142.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 413 kB/s | 45 kB 00:00 2024-05-08T23:57:38.778 INFO:teuthology.orchestra.run.smithi046.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-05-08T23:57:38.820 INFO:teuthology.orchestra.run.smithi046.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-05-08T23:57:38.870 INFO:teuthology.orchestra.run.smithi046.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-05-08T23:57:38.887 INFO:teuthology.orchestra.run.smithi142.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 179 kB/s | 22 kB 00:00 2024-05-08T23:57:38.895 INFO:teuthology.orchestra.run.smithi046.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-05-08T23:57:38.929 INFO:teuthology.orchestra.run.smithi046.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-05-08T23:57:38.954 INFO:teuthology.orchestra.run.smithi046.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-08T23:57:38.979 INFO:teuthology.orchestra.run.smithi046.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-05-08T23:57:39.021 INFO:teuthology.orchestra.run.smithi142.stdout:(102/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-05-08T23:57:39.029 INFO:teuthology.orchestra.run.smithi046.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-05-08T23:57:39.163 INFO:teuthology.orchestra.run.smithi046.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-05-08T23:57:39.219 INFO:teuthology.orchestra.run.smithi169.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-08T23:57:39.255 INFO:teuthology.orchestra.run.smithi046.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-05-08T23:57:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 468 kB/s | 19 kB 00:00 2024-05-08T23:57:39.406 INFO:teuthology.orchestra.run.smithi046.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-08T23:57:39.498 INFO:teuthology.orchestra.run.smithi046.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 284 kB/s | 26 kB 00:00 2024-05-08T23:57:39.536 INFO:teuthology.orchestra.run.smithi169.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 2.7 MB/s | 863 kB 00:00 2024-05-08T23:57:39.540 INFO:teuthology.orchestra.run.smithi046.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-05-08T23:57:39.598 INFO:teuthology.orchestra.run.smithi046.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-08T23:57:39.604 INFO:teuthology.orchestra.run.smithi142.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 939 kB/s | 548 kB 00:00 2024-05-08T23:57:39.611 INFO:teuthology.orchestra.run.smithi169.stdout:(3/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 8.5 MB/s | 5.4 MB 00:00 2024-05-08T23:57:39.636 INFO:teuthology.orchestra.run.smithi169.stdout:(4/156): ceph-immutable-object-cache-19.0.0-359 1.4 MB/s | 147 kB 00:00 2024-05-08T23:57:39.707 INFO:teuthology.orchestra.run.smithi046.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-05-08T23:57:39.729 INFO:teuthology.orchestra.run.smithi142.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-05-08T23:57:39.804 INFO:teuthology.orchestra.run.smithi169.stdout:(5/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 12 MB/s | 2.3 MB 00:00 2024-05-08T23:57:39.830 INFO:teuthology.orchestra.run.smithi142.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-05-08T23:57:39.949 INFO:teuthology.orchestra.run.smithi046.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-05-08T23:57:39.954 INFO:teuthology.orchestra.run.smithi169.stdout:(6/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 4.7 MB/s | 1.5 MB 00:00 2024-05-08T23:57:40.188 INFO:teuthology.orchestra.run.smithi142.stdout:(106/156): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-05-08T23:57:40.229 INFO:teuthology.orchestra.run.smithi169.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 11 MB/s | 4.9 MB 00:00 2024-05-08T23:57:40.274 INFO:teuthology.orchestra.run.smithi046.stdout:(112/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.4 MB/s | 16 MB 00:01 2024-05-08T23:57:40.280 INFO:teuthology.orchestra.run.smithi142.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 284 kB/s | 26 kB 00:00 2024-05-08T23:57:40.308 INFO:teuthology.orchestra.run.smithi046.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 1.5 MB/s | 548 kB 00:00 2024-05-08T23:57:40.366 INFO:teuthology.orchestra.run.smithi046.stdout:(114/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 4.4 MB 00:01 2024-05-08T23:57:40.372 INFO:teuthology.orchestra.run.smithi142.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-05-08T23:57:40.391 INFO:teuthology.orchestra.run.smithi046.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2024-05-08T23:57:40.417 INFO:teuthology.orchestra.run.smithi046.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-05-08T23:57:40.442 INFO:teuthology.orchestra.run.smithi046.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-05-08T23:57:40.464 INFO:teuthology.orchestra.run.smithi142.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 736 kB/s | 67 kB 00:00 2024-05-08T23:57:40.484 INFO:teuthology.orchestra.run.smithi046.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-08T23:57:40.526 INFO:teuthology.orchestra.run.smithi046.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 851 kB/s | 35 kB 00:00 2024-05-08T23:57:40.556 INFO:teuthology.orchestra.run.smithi142.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-08T23:57:40.560 INFO:teuthology.orchestra.run.smithi046.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-05-08T23:57:40.585 INFO:teuthology.orchestra.run.smithi046.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-05-08T23:57:40.610 INFO:teuthology.orchestra.run.smithi046.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-08T23:57:40.652 INFO:teuthology.orchestra.run.smithi046.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-05-08T23:57:40.673 INFO:teuthology.orchestra.run.smithi142.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-05-08T23:57:40.677 INFO:teuthology.orchestra.run.smithi046.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-08T23:57:40.703 INFO:teuthology.orchestra.run.smithi046.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-05-08T23:57:40.745 INFO:teuthology.orchestra.run.smithi046.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 465 kB/s | 19 kB 00:00 2024-05-08T23:57:40.771 INFO:teuthology.orchestra.run.smithi046.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-05-08T23:57:40.773 INFO:teuthology.orchestra.run.smithi142.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-05-08T23:57:40.796 INFO:teuthology.orchestra.run.smithi046.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-05-08T23:57:40.837 INFO:teuthology.orchestra.run.smithi046.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-05-08T23:57:40.879 INFO:teuthology.orchestra.run.smithi046.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-05-08T23:57:40.899 INFO:teuthology.orchestra.run.smithi142.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 347 kB/s | 43 kB 00:00 2024-05-08T23:57:40.921 INFO:teuthology.orchestra.run.smithi046.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-05-08T23:57:40.971 INFO:teuthology.orchestra.run.smithi046.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-05-08T23:57:40.991 INFO:teuthology.orchestra.run.smithi142.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 351 kB/s | 32 kB 00:00 2024-05-08T23:57:41.013 INFO:teuthology.orchestra.run.smithi046.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 395 kB/s | 16 kB 00:00 2024-05-08T23:57:41.038 INFO:teuthology.orchestra.run.smithi046.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-05-08T23:57:41.064 INFO:teuthology.orchestra.run.smithi046.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-08T23:57:41.091 INFO:teuthology.orchestra.run.smithi142.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-05-08T23:57:41.114 INFO:teuthology.orchestra.run.smithi046.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-05-08T23:57:41.139 INFO:teuthology.orchestra.run.smithi046.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-05-08T23:57:41.181 INFO:teuthology.orchestra.run.smithi046.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-05-08T23:57:41.191 INFO:teuthology.orchestra.run.smithi142.stdout:(116/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-05-08T23:57:41.205 INFO:teuthology.orchestra.run.smithi169.stdout:(8/156): ceph-radosgw-19.0.0-3594.gc9da4049.el9 10 MB/s | 10 MB 00:00 2024-05-08T23:57:41.207 INFO:teuthology.orchestra.run.smithi046.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 883 kB/s | 59 kB 00:00 2024-05-08T23:57:41.233 INFO:teuthology.orchestra.run.smithi046.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-05-08T23:57:41.263 INFO:teuthology.orchestra.run.smithi169.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 432 kB/s | 25 kB 00:00 2024-05-08T23:57:41.282 INFO:teuthology.orchestra.run.smithi046.stdout:(141/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.8 MB/s | 2.0 MB 00:00 2024-05-08T23:57:41.292 INFO:teuthology.orchestra.run.smithi142.stdout:(117/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-08T23:57:41.307 INFO:teuthology.orchestra.run.smithi046.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-05-08T23:57:41.333 INFO:teuthology.orchestra.run.smithi046.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-08T23:57:41.358 INFO:teuthology.orchestra.run.smithi046.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-08T23:57:41.383 INFO:teuthology.orchestra.run.smithi046.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-05-08T23:57:41.392 INFO:teuthology.orchestra.run.smithi142.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-05-08T23:57:41.417 INFO:teuthology.orchestra.run.smithi046.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-08T23:57:41.467 INFO:teuthology.orchestra.run.smithi046.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-08T23:57:41.484 INFO:teuthology.orchestra.run.smithi142.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-05-08T23:57:41.497 INFO:teuthology.orchestra.run.smithi169.stdout:(10/156): ceph-common-19.0.0-3594.gc9da4049.el9 7.8 MB/s | 20 MB 00:02 2024-05-08T23:57:41.585 INFO:teuthology.orchestra.run.smithi142.stdout:(120/156): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-08T23:57:41.606 INFO:teuthology.orchestra.run.smithi169.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 2.0 MB/s | 224 kB 00:00 2024-05-08T23:57:41.701 INFO:teuthology.orchestra.run.smithi046.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-05-08T23:57:41.714 INFO:teuthology.orchestra.run.smithi169.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 250 kB/s | 27 kB 00:00 2024-05-08T23:57:41.718 INFO:teuthology.orchestra.run.smithi142.stdout:(121/156): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-05-08T23:57:41.818 INFO:teuthology.orchestra.run.smithi142.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-08T23:57:41.831 INFO:teuthology.orchestra.run.smithi169.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 272 kB/s | 32 kB 00:00 2024-05-08T23:57:41.910 INFO:teuthology.orchestra.run.smithi142.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-08T23:57:42.002 INFO:teuthology.orchestra.run.smithi142.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 193 kB/s | 18 kB 00:00 2024-05-08T23:57:42.031 INFO:teuthology.orchestra.run.smithi169.stdout:(14/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x8 8.5 MB/s | 18 MB 00:02 2024-05-08T23:57:42.065 INFO:teuthology.orchestra.run.smithi169.stdout:(15/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 3.2 MB/s | 756 kB 00:00 2024-05-08T23:57:42.094 INFO:teuthology.orchestra.run.smithi142.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-05-08T23:57:42.132 INFO:teuthology.orchestra.run.smithi169.stdout:(16/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.5 MB/s | 158 kB 00:00 2024-05-08T23:57:42.157 INFO:teuthology.orchestra.run.smithi169.stdout:(17/156): librados-devel-19.0.0-3594.gc9da4049. 1.3 MB/s | 127 kB 00:00 2024-05-08T23:57:42.186 INFO:teuthology.orchestra.run.smithi142.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-05-08T23:57:42.217 INFO:teuthology.orchestra.run.smithi046.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 4.5 MB/s | 2.3 MB 00:00 2024-05-08T23:57:42.241 INFO:teuthology.orchestra.run.smithi169.stdout:(18/156): libradosstriper1-19.0.0-3594.gc9da404 4.5 MB/s | 497 kB 00:00 2024-05-08T23:57:42.295 INFO:teuthology.orchestra.run.smithi142.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-05-08T23:57:42.349 INFO:teuthology.orchestra.run.smithi169.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 417 kB/s | 45 kB 00:00 2024-05-08T23:57:42.435 INFO:teuthology.orchestra.run.smithi046.stdout:(150/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 3.2 MB/s | 3.1 MB 00:00 2024-05-08T23:57:42.453 INFO:teuthology.orchestra.run.smithi142.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2024-05-08T23:57:42.458 INFO:teuthology.orchestra.run.smithi169.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 1.2 MB/s | 135 kB 00:00 2024-05-08T23:57:42.542 INFO:teuthology.orchestra.run.smithi169.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 1.9 MB/s | 161 kB 00:00 2024-05-08T23:57:42.545 INFO:teuthology.orchestra.run.smithi142.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 505 kB/s | 46 kB 00:00 2024-05-08T23:57:42.568 INFO:teuthology.orchestra.run.smithi046.stdout:(151/156): NetworkManager-libnm-1.47.5-1.el9.x8 5.2 MB/s | 1.8 MB 00:00 2024-05-08T23:57:42.594 INFO:teuthology.orchestra.run.smithi046.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 263 kB/s | 42 kB 00:00 2024-05-08T23:57:42.634 INFO:teuthology.orchestra.run.smithi169.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 3.4 MB/s | 322 kB 00:00 2024-05-08T23:57:42.637 INFO:teuthology.orchestra.run.smithi142.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 861 kB/s | 79 kB 00:00 2024-05-08T23:57:42.717 INFO:teuthology.orchestra.run.smithi169.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 3.5 MB/s | 300 kB 00:00 2024-05-08T23:57:42.729 INFO:teuthology.orchestra.run.smithi142.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 628 kB/s | 58 kB 00:00 2024-05-08T23:57:42.744 INFO:teuthology.orchestra.run.smithi046.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.4 MB/s | 248 kB 00:00 2024-05-08T23:57:42.801 INFO:teuthology.orchestra.run.smithi169.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.2 MB/s | 99 kB 00:00 2024-05-08T23:57:42.830 INFO:teuthology.orchestra.run.smithi142.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-05-08T23:57:42.878 INFO:teuthology.orchestra.run.smithi046.stdout:(154/156): librados2-19.0.0-3594.gc9da4049.el9. 2.3 MB/s | 3.3 MB 00:01 2024-05-08T23:57:42.893 INFO:teuthology.orchestra.run.smithi169.stdout:(25/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 7.0 MB/s | 5.2 MB 00:00 2024-05-08T23:57:42.918 INFO:teuthology.orchestra.run.smithi169.stdout:(26/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 731 kB/s | 86 kB 00:00 2024-05-08T23:57:42.921 INFO:teuthology.orchestra.run.smithi142.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-05-08T23:57:43.002 INFO:teuthology.orchestra.run.smithi169.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 2.0 MB/s | 169 kB 00:00 2024-05-08T23:57:43.011 INFO:teuthology.orchestra.run.smithi046.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 2.4 MB 00:00 2024-05-08T23:57:43.013 INFO:teuthology.orchestra.run.smithi142.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 983 kB/s | 90 kB 00:00 2024-05-08T23:57:43.077 INFO:teuthology.orchestra.run.smithi169.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 457 kB/s | 34 kB 00:00 2024-05-08T23:57:43.078 INFO:teuthology.orchestra.run.smithi046.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 2.8 MB/s | 1.3 MB 00:00 2024-05-08T23:57:43.081 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:57:43.081 INFO:teuthology.orchestra.run.smithi046.stdout:Total 12 MB/s | 219 MB 00:17 2024-05-08T23:57:43.105 INFO:teuthology.orchestra.run.smithi142.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 308 kB/s | 28 kB 00:00 2024-05-08T23:57:43.197 INFO:teuthology.orchestra.run.smithi142.stdout:(136/156): python3-repoze-lru-0.7-16.el9.noarch 336 kB/s | 31 kB 00:00 2024-05-08T23:57:43.289 INFO:teuthology.orchestra.run.smithi142.stdout:(137/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-05-08T23:57:43.302 INFO:teuthology.orchestra.run.smithi169.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 632 kB/s | 142 kB 00:00 2024-05-08T23:57:43.361 INFO:teuthology.orchestra.run.smithi169.stdout:(30/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 6.5 MB/s | 3.0 MB 00:00 2024-05-08T23:57:43.381 INFO:teuthology.orchestra.run.smithi142.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 644 kB/s | 59 kB 00:00 2024-05-08T23:57:43.473 INFO:teuthology.orchestra.run.smithi142.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 391 kB/s | 36 kB 00:00 2024-05-08T23:57:43.565 INFO:teuthology.orchestra.run.smithi142.stdout:(140/156): python3-typing-extensions-4.4.0-2.el 557 kB/s | 51 kB 00:00 2024-05-08T23:57:43.621 INFO:teuthology.orchestra.run.smithi169.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 11 MB/s | 3.5 MB 00:00 2024-05-08T23:57:43.657 INFO:teuthology.orchestra.run.smithi142.stdout:(141/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-05-08T23:57:43.687 INFO:teuthology.orchestra.run.smithi169.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 4.0 MB/s | 274 kB 00:00 2024-05-08T23:57:43.748 INFO:teuthology.orchestra.run.smithi142.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 981 kB/s | 90 kB 00:00 2024-05-08T23:57:43.771 INFO:teuthology.orchestra.run.smithi169.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 591 kB/s | 49 kB 00:00 2024-05-08T23:57:43.846 INFO:teuthology.orchestra.run.smithi169.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 220 kB/s | 16 kB 00:00 2024-05-08T23:57:43.849 INFO:teuthology.orchestra.run.smithi142.stdout:(143/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-08T23:57:43.930 INFO:teuthology.orchestra.run.smithi169.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 3.2 MB/s | 273 kB 00:00 2024-05-08T23:57:43.941 INFO:teuthology.orchestra.run.smithi142.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-05-08T23:57:44.041 INFO:teuthology.orchestra.run.smithi142.stdout:(145/156): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-05-08T23:57:44.047 INFO:teuthology.orchestra.run.smithi169.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 6.4 MB/s | 764 kB 00:00 2024-05-08T23:57:44.139 INFO:teuthology.orchestra.run.smithi169.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 199 kB/s | 18 kB 00:00 2024-05-08T23:57:44.167 INFO:teuthology.orchestra.run.smithi142.stdout:(146/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-08T23:57:44.239 INFO:teuthology.orchestra.run.smithi169.stdout:(38/156): ceph-mgr-diskprediction-local-19.0.0- 8.4 MB/s | 7.4 MB 00:00 2024-05-08T23:57:44.335 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T23:57:44.505 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T23:57:44.505 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T23:57:44.508 INFO:teuthology.orchestra.run.smithi142.stdout:(147/156): librados2-19.0.0-3594.gc9da4049.el9. 9.7 MB/s | 3.3 MB 00:00 2024-05-08T23:57:44.664 INFO:teuthology.orchestra.run.smithi169.stdout:(39/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 13 MB/s | 43 MB 00:03 2024-05-08T23:57:44.667 INFO:teuthology.orchestra.run.smithi142.stdout:(148/156): python3-numpy-1.20.1-5.el9.x86_64.rp 684 kB/s | 5.1 MB 00:07 2024-05-08T23:57:44.689 INFO:teuthology.orchestra.run.smithi169.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 146 kB/s | 80 kB 00:00 2024-05-08T23:57:44.801 INFO:teuthology.orchestra.run.smithi142.stdout:(149/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 11 MB/s | 3.1 MB 00:00 2024-05-08T23:57:44.831 INFO:teuthology.orchestra.run.smithi169.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-05-08T23:57:44.857 INFO:teuthology.orchestra.run.smithi169.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-05-08T23:57:44.882 INFO:teuthology.orchestra.run.smithi169.stdout:(43/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-05-08T23:57:44.907 INFO:teuthology.orchestra.run.smithi169.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-08T23:57:44.932 INFO:teuthology.orchestra.run.smithi169.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-08T23:57:44.958 INFO:teuthology.orchestra.run.smithi169.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-08T23:57:45.017 INFO:teuthology.orchestra.run.smithi169.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-08T23:57:45.225 INFO:teuthology.orchestra.run.smithi169.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-05-08T23:57:45.259 INFO:teuthology.orchestra.run.smithi169.stdout:(49/156): python3-cryptography-36.0.1-4.el9.x86 2.9 MB/s | 1.2 MB 00:00 2024-05-08T23:57:45.284 INFO:teuthology.orchestra.run.smithi169.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-05-08T23:57:45.309 INFO:teuthology.orchestra.run.smithi169.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-05-08T23:57:45.334 INFO:teuthology.orchestra.run.smithi169.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 285 kB/s | 21 kB 00:00 2024-05-08T23:57:45.360 INFO:teuthology.orchestra.run.smithi169.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-05-08T23:57:45.385 INFO:teuthology.orchestra.run.smithi169.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-05-08T23:57:45.410 INFO:teuthology.orchestra.run.smithi169.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-05-08T23:57:45.436 INFO:teuthology.orchestra.run.smithi169.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-05-08T23:57:45.461 INFO:teuthology.orchestra.run.smithi169.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-08T23:57:45.486 INFO:teuthology.orchestra.run.smithi169.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-05-08T23:57:45.611 INFO:teuthology.orchestra.run.smithi169.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 4.6 MB/s | 582 kB 00:00 2024-05-08T23:57:45.637 INFO:teuthology.orchestra.run.smithi169.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-05-08T23:57:45.679 INFO:teuthology.orchestra.run.smithi169.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 378 kB/s | 25 kB 00:00 2024-05-08T23:57:45.704 INFO:teuthology.orchestra.run.smithi169.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-05-08T23:57:45.763 INFO:teuthology.orchestra.run.smithi169.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-05-08T23:57:45.784 INFO:teuthology.orchestra.run.smithi142.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.1 MB/s | 2.3 MB 00:01 2024-05-08T23:57:45.788 INFO:teuthology.orchestra.run.smithi169.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-08T23:57:45.814 INFO:teuthology.orchestra.run.smithi169.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-05-08T23:57:45.901 INFO:teuthology.orchestra.run.smithi142.stdout:(151/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-05-08T23:57:46.189 INFO:teuthology.orchestra.run.smithi169.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.5 MB/s | 3.0 MB 00:00 2024-05-08T23:57:46.460 INFO:teuthology.orchestra.run.smithi142.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 62 kB/s | 42 kB 00:00 2024-05-08T23:57:46.564 INFO:teuthology.orchestra.run.smithi169.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-05-08T23:57:46.639 INFO:teuthology.orchestra.run.smithi169.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-05-08T23:57:46.698 INFO:teuthology.orchestra.run.smithi169.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-08T23:57:46.735 INFO:teuthology.orchestra.run.smithi142.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 298 kB/s | 248 kB 00:00 2024-05-08T23:57:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T23:57:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T23:57:46.756 INFO:teuthology.orchestra.run.smithi169.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 4.6 MB/s | 911 kB 00:00 2024-05-08T23:57:46.965 INFO:teuthology.orchestra.run.smithi169.stdout:(71/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.0 MB/s | 4.7 MB 00:01 2024-05-08T23:57:46.999 INFO:teuthology.orchestra.run.smithi169.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 2024-05-08T23:57:47.024 INFO:teuthology.orchestra.run.smithi169.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-08T23:57:47.108 INFO:teuthology.orchestra.run.smithi169.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-08T23:57:47.193 INFO:teuthology.orchestra.run.smithi169.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-08T23:57:47.242 INFO:teuthology.orchestra.run.smithi169.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-05-08T23:57:47.301 INFO:teuthology.orchestra.run.smithi169.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-05-08T23:57:47.384 INFO:teuthology.orchestra.run.smithi169.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-05-08T23:57:47.453 INFO:teuthology.orchestra.run.smithi142.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 1.4 MB/s | 1.3 MB 00:00 2024-05-08T23:57:47.459 INFO:teuthology.orchestra.run.smithi169.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 464 kB/s | 35 kB 00:00 2024-05-08T23:57:47.818 INFO:teuthology.orchestra.run.smithi142.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-05-08T23:57:47.826 INFO:teuthology.orchestra.run.smithi169.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 7.2 MB/s | 6.0 MB 00:00 2024-05-08T23:57:47.885 INFO:teuthology.orchestra.run.smithi169.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-05-08T23:57:47.944 INFO:teuthology.orchestra.run.smithi169.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-08T23:57:48.044 INFO:teuthology.orchestra.run.smithi169.stdout:(83/156): postgresql-server-13.14-1.el9.x86_64. 4.3 MB/s | 5.8 MB 00:01 2024-05-08T23:57:48.069 INFO:teuthology.orchestra.run.smithi169.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 616 kB/s | 77 kB 00:00 2024-05-08T23:57:48.102 INFO:teuthology.orchestra.run.smithi169.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 722 kB/s | 42 kB 00:00 2024-05-08T23:57:48.128 INFO:teuthology.orchestra.run.smithi169.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-08T23:57:48.162 INFO:teuthology.orchestra.run.smithi169.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-05-08T23:57:48.195 INFO:teuthology.orchestra.run.smithi169.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-08T23:57:48.221 INFO:teuthology.orchestra.run.smithi169.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-05-08T23:57:48.246 INFO:teuthology.orchestra.run.smithi169.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-08T23:57:48.296 INFO:teuthology.orchestra.run.smithi169.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 836 kB/s | 42 kB 00:00 2024-05-08T23:57:48.355 INFO:teuthology.orchestra.run.smithi169.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-05-08T23:57:48.414 INFO:teuthology.orchestra.run.smithi169.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-08T23:57:48.472 INFO:teuthology.orchestra.run.smithi169.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-05-08T23:57:48.523 INFO:teuthology.orchestra.run.smithi169.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-05-08T23:57:48.581 INFO:teuthology.orchestra.run.smithi169.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-05-08T23:57:48.640 INFO:teuthology.orchestra.run.smithi169.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 992 kB/s | 58 kB 00:00 2024-05-08T23:57:48.707 INFO:teuthology.orchestra.run.smithi169.stdout:(98/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-05-08T23:57:48.732 INFO:teuthology.orchestra.run.smithi169.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-05-08T23:57:48.757 INFO:teuthology.orchestra.run.smithi169.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-08T23:57:48.783 INFO:teuthology.orchestra.run.smithi169.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-05-08T23:57:49.341 INFO:teuthology.orchestra.run.smithi169.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 38 kB/s | 22 kB 00:00 2024-05-08T23:57:49.725 INFO:teuthology.orchestra.run.smithi169.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 917 kB/s | 863 kB 00:00 2024-05-08T23:57:49.758 INFO:teuthology.orchestra.run.smithi169.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-05-08T23:57:49.833 INFO:teuthology.orchestra.run.smithi169.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-05-08T23:57:49.859 INFO:teuthology.orchestra.run.smithi169.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-05-08T23:57:49.926 INFO:teuthology.orchestra.run.smithi169.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-05-08T23:57:50.076 INFO:teuthology.orchestra.run.smithi169.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-08T23:57:50.093 INFO:teuthology.orchestra.run.smithi169.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 294 kB/s | 49 kB 00:00 2024-05-08T23:57:50.143 INFO:teuthology.orchestra.run.smithi169.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-08T23:57:50.169 INFO:teuthology.orchestra.run.smithi169.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-08T23:57:50.336 INFO:teuthology.orchestra.run.smithi169.stdout:(112/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.7 MB/s | 16 MB 00:02 2024-05-08T23:57:50.377 INFO:teuthology.orchestra.run.smithi169.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.5 MB/s | 839 kB 00:00 2024-05-08T23:57:50.403 INFO:teuthology.orchestra.run.smithi169.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-05-08T23:57:50.428 INFO:teuthology.orchestra.run.smithi169.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 487 kB/s | 43 kB 00:00 2024-05-08T23:57:50.470 INFO:teuthology.orchestra.run.smithi169.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-05-08T23:57:50.495 INFO:teuthology.orchestra.run.smithi169.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 162 kB/s | 15 kB 00:00 2024-05-08T23:57:50.521 INFO:teuthology.orchestra.run.smithi169.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-08T23:57:50.546 INFO:teuthology.orchestra.run.smithi169.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-05-08T23:57:50.588 INFO:teuthology.orchestra.run.smithi169.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-05-08T23:57:50.613 INFO:teuthology.orchestra.run.smithi169.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 386 kB/s | 35 kB 00:00 2024-05-08T23:57:50.638 INFO:teuthology.orchestra.run.smithi169.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-08T23:57:50.722 INFO:teuthology.orchestra.run.smithi169.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-05-08T23:57:50.747 INFO:teuthology.orchestra.run.smithi169.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-05-08T23:57:50.789 INFO:teuthology.orchestra.run.smithi169.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 72 kB/s | 11 kB 00:00 2024-05-08T23:57:50.814 INFO:teuthology.orchestra.run.smithi169.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-05-08T23:57:50.848 INFO:teuthology.orchestra.run.smithi169.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-05-08T23:57:50.882 INFO:teuthology.orchestra.run.smithi169.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-05-08T23:57:50.907 INFO:teuthology.orchestra.run.smithi169.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-05-08T23:57:50.966 INFO:teuthology.orchestra.run.smithi169.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-05-08T23:57:50.991 INFO:teuthology.orchestra.run.smithi169.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-05-08T23:57:51.016 INFO:teuthology.orchestra.run.smithi169.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 730 kB/s | 79 kB 00:00 2024-05-08T23:57:51.041 INFO:teuthology.orchestra.run.smithi169.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 770 kB/s | 58 kB 00:00 2024-05-08T23:57:51.100 INFO:teuthology.orchestra.run.smithi169.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-08T23:57:51.125 INFO:teuthology.orchestra.run.smithi169.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-05-08T23:57:51.150 INFO:teuthology.orchestra.run.smithi169.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-05-08T23:57:51.178 INFO:teuthology.orchestra.run.smithi169.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-05-08T23:57:51.201 INFO:teuthology.orchestra.run.smithi169.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-05-08T23:57:51.226 INFO:teuthology.orchestra.run.smithi169.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-08T23:57:51.251 INFO:teuthology.orchestra.run.smithi169.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-05-08T23:57:51.277 INFO:teuthology.orchestra.run.smithi169.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-05-08T23:57:51.302 INFO:teuthology.orchestra.run.smithi169.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-05-08T23:57:51.328 INFO:teuthology.orchestra.run.smithi169.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-08T23:57:51.353 INFO:teuthology.orchestra.run.smithi169.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-08T23:57:51.378 INFO:teuthology.orchestra.run.smithi169.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-05-08T23:57:51.403 INFO:teuthology.orchestra.run.smithi169.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-05-08T23:57:51.429 INFO:teuthology.orchestra.run.smithi169.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-08T23:57:51.487 INFO:teuthology.orchestra.run.smithi169.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-08T23:57:51.729 INFO:teuthology.orchestra.run.smithi169.stdout:(149/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 10 MB/s | 3.1 MB 00:00 2024-05-08T23:57:51.846 INFO:teuthology.orchestra.run.smithi169.stdout:(150/156): librados2-19.0.0-3594.gc9da4049.el9. 7.5 MB/s | 3.3 MB 00:00 2024-05-08T23:57:51.897 INFO:teuthology.orchestra.run.smithi169.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 5.7 MB/s | 2.3 MB 00:00 2024-05-08T23:57:51.922 INFO:teuthology.orchestra.run.smithi169.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 555 kB/s | 42 kB 00:00 2024-05-08T23:57:51.955 INFO:teuthology.orchestra.run.smithi169.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-05-08T23:57:52.089 INFO:teuthology.orchestra.run.smithi169.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 5.1 MB/s | 1.8 MB 00:00 2024-05-08T23:57:52.214 INFO:teuthology.orchestra.run.smithi169.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 4.6 MB/s | 1.3 MB 00:00 2024-05-08T23:57:52.323 INFO:teuthology.orchestra.run.smithi169.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 2.4 MB 00:00 2024-05-08T23:57:52.327 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:57:52.327 INFO:teuthology.orchestra.run.smithi169.stdout:Total 16 MB/s | 219 MB 00:13 2024-05-08T23:57:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T23:57:53.367 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T23:57:53.416 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T23:57:53.472 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-05-08T23:57:53.486 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T23:57:53.527 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T23:57:53.559 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T23:57:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T23:57:53.623 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-05-08T23:57:53.623 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-05-08T23:57:53.965 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T23:57:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-08T23:57:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-08T23:57:54.277 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-08T23:57:54.354 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-08T23:57:54.509 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-08T23:57:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T23:57:54.754 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T23:57:54.781 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-08T23:57:54.893 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-08T23:57:54.923 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-08T23:57:54.987 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-08T23:57:55.016 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T23:57:55.038 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T23:57:55.233 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T23:57:55.390 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-05-08T23:57:55.391 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-05-08T23:57:55.404 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T23:57:55.548 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T23:57:55.616 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T23:57:55.749 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T23:57:55.817 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T23:57:56.030 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T23:57:56.145 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T23:57:56.180 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T23:57:56.214 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T23:57:56.290 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T23:57:56.331 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T23:57:56.374 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T23:57:56.407 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T23:57:56.434 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T23:57:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T23:57:56.558 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T23:57:56.587 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T23:57:56.631 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T23:57:56.709 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T23:57:56.866 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-08T23:57:56.919 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-08T23:57:56.947 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-08T23:57:57.019 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T23:57:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T23:57:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T23:57:57.284 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T23:57:57.340 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T23:57:57.593 INFO:teuthology.orchestra.run.smithi142.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 831 kB/s | 16 MB 00:20 2024-05-08T23:57:57.596 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:57:57.596 INFO:teuthology.orchestra.run.smithi142.stdout:Total 6.5 MB/s | 219 MB 00:33 2024-05-08T23:57:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T23:57:57.978 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T23:57:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T23:57:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T23:57:58.094 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-08T23:57:58.137 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T23:57:58.304 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T23:57:58.363 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T23:57:58.459 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T23:57:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T23:57:58.594 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T23:57:58.631 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T23:57:58.676 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T23:57:58.724 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T23:57:58.759 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T23:57:58.773 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-05-08T23:57:58.802 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T23:57:58.845 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T23:57:58.922 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-05-08T23:57:58.923 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-05-08T23:57:58.989 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T23:57:59.033 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T23:57:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T23:57:59.105 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T23:57:59.204 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T23:57:59.246 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T23:57:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T23:57:59.320 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T23:57:59.344 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T23:57:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T23:57:59.584 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T23:57:59.618 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T23:57:59.647 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T23:57:59.680 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T23:57:59.720 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T23:57:59.759 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T23:57:59.789 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T23:57:59.823 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T23:57:59.857 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T23:57:59.899 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T23:57:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T23:58:00.023 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T23:58:00.073 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T23:58:00.111 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T23:58:00.160 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T23:58:00.279 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T23:58:00.331 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T23:58:00.746 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-05-08T23:58:00.746 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-05-08T23:58:01.093 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T23:58:01.151 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T23:58:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T23:58:01.289 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T23:58:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T23:58:01.442 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T23:58:01.478 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T23:58:01.480 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-05-08T23:58:01.561 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T23:58:01.662 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T23:58:01.680 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T23:58:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T23:58:01.728 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T23:58:01.761 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T23:58:01.781 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T23:58:01.790 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T23:58:01.830 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T23:58:02.164 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T23:58:02.288 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-08T23:58:02.413 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-08T23:58:02.447 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-08T23:58:02.484 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T23:58:02.538 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-08T23:58:02.568 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-08T23:58:02.603 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T23:58:02.719 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T23:58:02.773 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T23:58:02.801 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-08T23:58:02.905 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-08T23:58:02.927 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-08T23:58:02.981 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-08T23:58:03.004 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T23:58:03.023 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T23:58:03.212 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T23:58:03.350 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T23:58:03.518 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T23:58:03.578 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T23:58:03.685 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T23:58:03.754 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T23:58:03.934 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T23:58:03.943 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T23:58:04.012 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T23:58:04.040 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T23:58:04.051 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T23:58:04.073 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T23:58:04.112 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T23:58:04.152 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T23:58:04.217 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T23:58:04.259 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T23:58:04.303 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T23:58:04.336 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T23:58:04.364 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T23:58:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T23:58:04.433 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T23:58:04.452 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-08T23:58:04.486 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T23:58:04.517 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T23:58:04.544 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-08T23:58:04.552 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T23:58:04.574 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-08T23:58:04.611 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T23:58:04.628 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T23:58:04.663 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-08T23:58:04.706 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-08T23:58:04.735 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-08T23:58:04.806 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T23:58:04.859 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T23:58:04.933 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T23:58:05.068 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T23:58:05.120 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T23:58:05.126 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T23:58:05.163 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-08T23:58:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-08T23:58:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-08T23:58:05.658 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T23:58:05.757 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T23:58:05.787 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T23:58:05.830 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T23:58:05.868 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-08T23:58:05.909 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T23:58:06.054 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T23:58:06.112 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T23:58:06.204 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T23:58:06.268 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T23:58:06.341 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T23:58:06.372 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T23:58:06.400 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T23:58:06.447 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T23:58:06.482 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T23:58:06.526 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T23:58:06.568 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T23:58:06.705 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T23:58:06.816 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-05-08T23:58:06.913 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T23:58:06.927 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T23:58:06.951 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T23:58:06.954 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T23:58:06.995 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T23:58:07.005 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T23:58:07.039 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T23:58:07.065 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T23:58:07.091 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T23:58:07.107 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T23:58:07.128 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T23:58:07.168 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T23:58:07.194 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T23:58:07.219 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T23:58:07.293 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T23:58:07.421 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T23:58:07.435 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T23:58:07.452 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T23:58:07.504 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T23:58:07.547 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T23:58:07.548 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-08T23:58:07.587 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T23:58:07.618 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T23:58:07.645 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-08T23:58:07.648 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T23:58:07.673 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-08T23:58:07.677 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-08T23:58:07.682 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T23:58:07.700 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-08T23:58:07.715 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T23:58:07.757 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T23:58:07.791 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-08T23:58:07.792 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T23:58:07.820 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-08T23:58:07.878 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T23:58:07.922 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T23:58:07.953 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T23:58:07.968 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T23:58:08.000 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T23:58:08.048 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T23:58:08.070 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-08T23:58:08.099 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T23:58:08.155 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T23:58:08.175 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-08T23:58:08.205 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-08T23:58:08.267 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-08T23:58:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-08T23:58:08.289 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T23:58:08.308 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T23:58:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-08T23:58:08.413 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-08T23:58:08.413 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T23:58:08.413 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T23:58:08.413 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:08.440 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-08T23:58:08.497 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T23:58:08.626 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T23:58:08.786 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T23:58:08.846 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T23:58:08.883 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T23:58:08.935 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T23:58:08.946 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T23:58:08.969 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T23:58:09.039 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T23:58:09.069 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T23:58:09.127 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T23:58:09.215 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T23:58:09.238 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T23:58:09.254 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T23:58:09.352 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T23:58:09.385 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T23:58:09.420 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T23:58:09.434 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T23:58:09.468 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T23:58:09.484 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T23:58:09.518 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T23:58:09.546 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T23:58:09.562 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T23:58:09.594 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T23:58:09.656 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T23:58:09.725 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T23:58:09.778 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T23:58:09.809 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T23:58:09.844 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T23:58:09.920 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T23:58:09.954 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-08T23:58:09.989 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-08T23:58:10.011 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-08T23:58:10.090 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T23:58:10.133 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T23:58:10.194 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T23:58:10.208 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T23:58:10.300 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T23:58:10.342 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T23:58:10.393 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T23:58:10.920 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T23:58:10.998 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T23:58:11.026 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T23:58:11.069 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T23:58:11.107 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-08T23:58:11.149 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T23:58:11.307 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T23:58:11.368 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T23:58:11.459 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T23:58:11.539 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T23:58:11.548 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T23:58:11.621 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T23:58:11.642 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T23:58:11.652 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T23:58:11.669 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T23:58:11.697 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T23:58:11.711 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T23:58:11.744 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T23:58:11.780 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T23:58:11.823 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T23:58:11.865 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T23:58:12.000 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T23:58:12.032 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T23:58:12.045 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T23:58:12.075 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T23:58:12.091 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-08T23:58:12.117 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T23:58:12.182 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-08T23:58:12.212 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T23:58:12.212 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-08T23:58:12.249 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T23:58:12.250 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T23:58:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T23:58:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T23:58:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T23:58:12.415 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T23:58:12.542 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T23:58:12.573 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T23:58:12.601 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T23:58:12.635 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T23:58:12.674 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T23:58:12.705 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T23:58:12.735 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T23:58:12.737 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T23:58:12.769 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-08T23:58:12.770 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T23:58:12.803 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T23:58:12.823 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-08T23:58:12.845 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T23:58:12.846 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-08T23:58:12.879 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T23:58:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T23:58:13.009 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T23:58:13.049 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T23:58:13.104 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T23:58:13.201 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T23:58:13.251 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T23:58:13.968 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T23:58:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T23:58:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T23:58:14.146 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T23:58:14.213 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T23:58:14.301 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T23:58:14.373 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T23:58:14.576 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T23:58:14.605 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T23:58:14.683 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T23:58:15.051 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-08T23:58:15.092 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-08T23:58:15.329 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T23:58:15.437 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T23:58:15.649 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-08T23:58:15.687 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-08T23:58:15.778 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-08T23:58:15.779 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T23:58:15.779 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T23:58:15.779 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:15.831 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-08T23:58:16.654 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T23:58:16.732 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T23:58:16.760 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T23:58:16.793 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T23:58:17.114 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T23:58:17.157 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-08T23:58:17.230 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-08T23:58:17.261 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-08T23:58:17.297 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T23:58:17.787 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T23:58:17.825 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-08T23:58:17.880 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-08T23:58:17.910 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-08T23:58:20.113 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-08T23:58:20.148 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-08T23:58:20.451 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-08T23:58:20.451 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-05-08T23:58:20.452 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-05-08T23:58:20.452 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-05-08T23:58:20.452 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-05-08T23:58:20.452 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-05-08T23:58:20.452 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /root 2024-05-08T23:58:20.452 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-05-08T23:58:20.452 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:20.638 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-08T23:58:20.656 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-08T23:58:20.656 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:20.656 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T23:58:20.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:20.707 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-08T23:58:20.752 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-08T23:58:20.867 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-08T23:58:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T23:58:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T23:58:20.868 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:20.902 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-08T23:58:22.291 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-08T23:58:22.320 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-08T23:58:22.320 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:22.320 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T23:58:22.320 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T23:58:22.321 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T23:58:22.321 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-08T23:58:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-08T23:58:22.592 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:22.592 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T23:58:22.592 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T23:58:22.592 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T23:58:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-08T23:58:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-08T23:58:23.101 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:23.101 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T23:58:23.101 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T23:58:23.101 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T23:58:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:23.207 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T23:58:23.257 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T23:58:23.320 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T23:58:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T23:58:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T23:58:23.641 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T23:58:23.738 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T23:58:23.881 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T23:58:23.927 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T23:58:23.959 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T23:58:23.985 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T23:58:24.458 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-08T23:58:24.493 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-08T23:58:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-08T23:58:24.590 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-08T23:58:24.615 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-08T23:58:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-08T23:58:25.750 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-08T23:58:26.771 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-08T23:58:26.792 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-08T23:58:26.914 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-08T23:58:27.091 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-08T23:58:27.110 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-08T23:58:27.110 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-05-08T23:58:27.111 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-05-08T23:58:27.111 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-05-08T23:58:27.111 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-05-08T23:58:27.111 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-05-08T23:58:27.111 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-05-08T23:58:27.111 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-05-08T23:58:27.111 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-08T23:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-08T23:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T23:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T23:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T23:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-08T23:58:27.208 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-08T23:58:27.281 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-08T23:58:27.299 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-08T23:58:27.299 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:27.299 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T23:58:27.300 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:27.310 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-08T23:58:27.368 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-08T23:58:28.357 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T23:58:28.390 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-08T23:58:28.415 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-08T23:58:28.415 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:28.415 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T23:58:28.415 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T23:58:28.416 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T23:58:28.416 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:28.457 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-08T23:58:28.482 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-08T23:58:28.482 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:28.482 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T23:58:28.482 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:28.533 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-08T23:58:28.558 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-08T23:58:28.558 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:28.558 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T23:58:28.558 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T23:58:28.558 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T23:58:28.558 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:28.875 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-08T23:58:28.883 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-08T23:58:28.901 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-08T23:58:28.901 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:28.901 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T23:58:28.901 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T23:58:28.901 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T23:58:28.901 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:28.909 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-08T23:58:28.909 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:28.909 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T23:58:28.909 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T23:58:28.909 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T23:58:28.910 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:29.126 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-08T23:58:29.148 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-08T23:58:29.148 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:29.148 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T23:58:29.148 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T23:58:29.148 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T23:58:29.148 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:29.610 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-08T23:58:29.632 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-08T23:58:29.632 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:29.633 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T23:58:29.633 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T23:58:29.633 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T23:58:29.633 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:29.740 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T23:58:29.798 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T23:58:29.853 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T23:58:30.041 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T23:58:30.094 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T23:58:30.173 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T23:58:30.262 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T23:58:30.412 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T23:58:30.470 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T23:58:30.502 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T23:58:30.529 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T23:58:30.990 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-08T23:58:31.020 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-08T23:58:31.053 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-08T23:58:31.113 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-08T23:58:31.143 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-08T23:58:32.184 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-08T23:58:32.220 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-08T23:58:32.275 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-08T23:58:32.275 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-05-08T23:58:32.275 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-05-08T23:58:32.275 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-05-08T23:58:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-05-08T23:58:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-05-08T23:58:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2024-05-08T23:58:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-05-08T23:58:32.276 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:32.447 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-08T23:58:32.462 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-08T23:58:32.462 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:32.462 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T23:58:32.462 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:33.182 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-08T23:58:33.212 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-08T23:58:33.322 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-08T23:58:33.412 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-08T23:58:33.454 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-08T23:58:33.487 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-08T23:58:33.505 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-08T23:58:33.506 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-08T23:58:33.529 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-08T23:58:33.529 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:33.529 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T23:58:33.529 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T23:58:33.529 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T23:58:33.529 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:33.534 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T23:58:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T23:58:33.573 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-08T23:58:33.584 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-08T23:58:33.682 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-08T23:58:33.730 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-08T23:58:33.775 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-08T23:58:33.824 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T23:58:33.855 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-08T23:58:33.885 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T23:58:33.922 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T23:58:34.033 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-08T23:58:34.059 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-08T23:58:34.059 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:34.059 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T23:58:34.059 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T23:58:34.059 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T23:58:34.060 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:34.283 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-08T23:58:34.306 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-08T23:58:34.306 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:34.307 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T23:58:34.307 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T23:58:34.309 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T23:58:34.309 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:34.510 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T23:58:34.553 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T23:58:34.575 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T23:58:34.662 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T23:58:34.692 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-08T23:58:34.715 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-08T23:58:34.715 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:34.715 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T23:58:34.715 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T23:58:34.715 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T23:58:34.716 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:34.760 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-08T23:58:34.767 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-08T23:58:34.781 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-08T23:58:34.782 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:34.782 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T23:58:34.782 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:34.790 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-08T23:58:34.790 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:34.790 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T23:58:34.790 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T23:58:34.790 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T23:58:34.791 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:34.827 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-08T23:58:34.849 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-08T23:58:34.849 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:34.849 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T23:58:34.849 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T23:58:34.849 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T23:58:34.849 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:34.898 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T23:58:34.955 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T23:58:35.009 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T23:58:35.153 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-08T23:58:35.175 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-08T23:58:35.175 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:35.175 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T23:58:35.175 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T23:58:35.175 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T23:58:35.176 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:35.195 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T23:58:35.245 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T23:58:35.323 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T23:58:35.411 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T23:58:35.495 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-08T23:58:35.541 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T23:58:35.574 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T23:58:35.578 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T23:58:35.610 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T23:58:35.622 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T23:58:35.638 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T23:58:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T23:58:35.752 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T23:58:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T23:58:35.778 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T23:58:35.863 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T23:58:35.895 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T23:58:35.896 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T23:58:35.980 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T23:58:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T23:58:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T23:58:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T23:58:36.082 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-08T23:58:36.333 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T23:58:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T23:58:36.334 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T23:58:36.334 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-08T23:58:36.334 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-08T23:58:36.334 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-08T23:58:36.334 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-08T23:58:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T23:58:37.268 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-08T23:58:37.303 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-08T23:58:38.264 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-08T23:58:38.286 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-08T23:58:38.395 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-08T23:58:38.570 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-08T23:58:38.596 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-08T23:58:38.619 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-08T23:58:38.619 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:38.619 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T23:58:38.619 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T23:58:38.619 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T23:58:38.619 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:38.664 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-08T23:58:38.675 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-08T23:58:38.773 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-08T23:58:38.820 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-08T23:58:39.457 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-08T23:58:39.499 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-08T23:58:39.539 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-08T23:58:39.571 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T23:58:39.598 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T23:58:39.757 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T23:58:39.783 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-08T23:58:39.790 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-08T23:58:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-08T23:58:39.806 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:39.806 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T23:58:39.806 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T23:58:39.806 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T23:58:39.806 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:39.836 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T23:58:39.859 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-08T23:58:39.867 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-08T23:58:39.881 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-08T23:58:39.881 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:39.881 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T23:58:39.882 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:39.896 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T23:58:39.926 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-08T23:58:39.943 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T23:58:39.948 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-08T23:58:39.948 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:39.949 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T23:58:39.949 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T23:58:39.949 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T23:58:39.949 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:40.260 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-08T23:58:40.282 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-08T23:58:40.283 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T23:58:40.283 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T23:58:40.283 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T23:58:40.283 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T23:58:40.283 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:40.553 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T23:58:40.590 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T23:58:40.610 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T23:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-08T23:58:41.539 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T23:58:41.580 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T23:58:41.664 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T23:58:41.695 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T23:58:41.717 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T23:58:41.718 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T23:58:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T23:58:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-08T23:58:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-08T23:58:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-08T23:58:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-08T23:58:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-08T23:58:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-08T23:58:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-08T23:58:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-08T23:58:41.753 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-08T23:58:41.754 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-08T23:58:41.755 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T23:58:41.756 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T23:58:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T23:58:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T23:58:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T23:58:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T23:58:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T23:58:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T23:58:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T23:58:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T23:58:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T23:58:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T23:58:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T23:58:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T23:58:41.758 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T23:58:41.759 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T23:58:41.760 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T23:58:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T23:58:41.762 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T23:58:41.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T23:58:41.764 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T23:58:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T23:58:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T23:58:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T23:58:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T23:58:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T23:58:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-08T23:58:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T23:58:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-08T23:58:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T23:58:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T23:58:41.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T23:58:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T23:58:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T23:58:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T23:58:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T23:58:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T23:58:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T23:58:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T23:58:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T23:58:41.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T23:58:41.802 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T23:58:41.835 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T23:58:41.835 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T23:58:41.928 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T23:58:41.953 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T23:58:41.957 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T23:58:41.957 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T23:58:42.137 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T23:58:42.138 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T23:58:42.222 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T23:58:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T23:58:44.547 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-08T23:58:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-08T23:58:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T23:58:44.602 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T23:58:44.603 INFO:teuthology.orchestra.run.smithi046.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:44.604 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T23:58:44.605 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T23:58:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T23:58:44.607 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T23:58:44.608 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T23:58:44.609 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T23:58:44.610 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T23:58:44.611 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T23:58:44.612 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T23:58:44.613 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T23:58:44.614 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T23:58:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-08T23:58:44.661 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T23:58:44.688 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T23:58:44.869 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-08T23:58:44.917 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T23:58:44.948 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-08T23:58:44.977 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T23:58:45.015 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T23:58:45.021 DEBUG:teuthology.parallel:result is None 2024-05-08T23:58:45.595 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T23:58:45.638 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T23:58:45.657 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T23:58:46.479 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-08T23:58:46.567 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T23:58:46.608 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T23:58:46.689 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T23:58:46.720 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T23:58:46.746 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T23:58:46.746 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T23:58:46.814 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T23:58:46.846 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T23:58:46.847 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T23:58:46.940 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T23:58:46.965 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T23:58:46.970 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T23:58:46.970 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T23:58:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T23:58:47.152 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T23:58:47.217 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T23:58:47.281 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T23:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T23:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-08T23:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-08T23:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-08T23:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-08T23:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-08T23:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-08T23:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-08T23:58:47.996 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-08T23:58:47.997 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-08T23:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-08T23:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-08T23:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-08T23:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-08T23:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-08T23:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-08T23:58:47.998 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-08T23:58:48.016 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-08T23:58:48.016 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-08T23:58:48.016 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-08T23:58:48.016 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-08T23:58:48.016 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-08T23:58:48.016 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-08T23:58:48.017 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-08T23:58:48.017 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-08T23:58:48.017 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-08T23:58:48.017 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-08T23:58:48.017 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T23:58:48.017 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T23:58:48.017 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T23:58:48.018 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T23:58:48.018 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T23:58:48.018 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T23:58:48.018 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T23:58:48.018 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T23:58:48.018 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T23:58:48.019 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T23:58:48.019 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T23:58:48.019 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T23:58:48.019 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T23:58:48.019 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T23:58:48.020 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T23:58:48.020 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T23:58:48.020 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T23:58:48.020 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T23:58:48.020 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T23:58:48.020 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T23:58:48.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T23:58:48.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T23:58:48.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T23:58:48.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T23:58:48.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T23:58:48.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T23:58:48.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T23:58:48.021 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T23:58:48.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T23:58:48.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T23:58:48.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T23:58:48.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T23:58:48.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T23:58:48.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T23:58:48.022 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T23:58:48.023 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T23:58:48.023 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T23:58:48.023 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T23:58:48.023 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T23:58:48.023 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T23:58:48.023 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T23:58:48.023 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T23:58:48.024 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T23:58:48.024 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T23:58:48.024 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T23:58:48.024 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T23:58:48.024 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T23:58:48.024 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T23:58:48.024 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T23:58:48.025 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T23:58:48.025 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T23:58:48.025 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T23:58:48.025 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T23:58:48.025 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T23:58:48.025 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T23:58:48.025 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T23:58:48.026 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T23:58:48.026 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T23:58:48.026 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T23:58:48.026 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T23:58:48.026 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T23:58:48.026 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T23:58:48.027 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T23:58:48.027 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T23:58:48.027 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T23:58:48.027 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T23:58:48.027 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T23:58:48.027 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T23:58:48.028 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T23:58:48.028 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T23:58:48.028 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T23:58:48.028 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T23:58:48.028 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T23:58:48.028 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T23:58:48.029 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T23:58:48.029 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T23:58:48.029 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T23:58:48.029 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T23:58:48.029 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T23:58:48.029 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T23:58:48.029 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T23:58:48.030 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T23:58:48.030 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T23:58:48.030 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T23:58:48.030 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T23:58:48.030 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T23:58:48.030 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T23:58:48.030 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T23:58:48.030 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T23:58:48.030 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T23:58:48.030 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T23:58:48.030 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T23:58:48.030 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T23:58:48.031 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T23:58:48.031 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T23:58:48.031 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T23:58:48.031 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T23:58:48.031 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T23:58:48.031 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T23:58:48.031 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T23:58:48.031 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T23:58:48.031 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T23:58:48.032 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T23:58:48.032 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T23:58:48.032 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T23:58:48.032 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T23:58:48.032 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T23:58:48.032 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T23:58:48.032 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T23:58:48.032 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-08T23:58:48.032 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T23:58:48.032 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-08T23:58:48.032 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T23:58:48.032 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T23:58:48.033 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T23:58:48.033 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T23:58:48.033 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T23:58:48.033 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T23:58:48.033 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T23:58:48.033 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T23:58:48.033 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T23:58:48.033 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T23:58:48.033 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T23:58:48.033 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T23:58:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T23:58:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:50.430 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-05-08T23:58:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T23:58:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T23:58:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T23:58:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T23:58:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:50.433 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:50.433 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:50.433 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:50.434 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.434 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T23:58:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T23:58:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T23:58:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T23:58:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T23:58:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T23:58:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T23:58:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T23:58:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T23:58:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T23:58:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T23:58:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T23:58:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T23:58:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T23:58:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T23:58:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T23:58:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T23:58:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T23:58:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T23:58:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T23:58:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T23:58:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T23:58:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T23:58:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T23:58:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T23:58:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T23:58:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T23:58:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T23:58:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T23:58:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T23:58:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T23:58:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T23:58:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T23:58:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T23:58:50.447 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T23:58:50.447 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T23:58:50.447 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-08T23:58:50.447 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-05-08T23:58:50.862 DEBUG:teuthology.parallel:result is None 2024-05-08T23:58:52.818 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T23:58:52.818 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-08T23:58:52.818 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-08T23:58:52.818 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-08T23:58:52.818 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-08T23:58:52.819 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-08T23:58:52.820 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T23:58:52.821 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T23:58:52.822 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T23:58:52.823 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T23:58:52.824 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T23:58:52.825 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T23:58:52.826 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T23:58:52.827 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T23:58:52.828 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T23:58:52.829 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T23:58:52.830 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T23:58:52.830 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T23:58:52.830 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T23:58:52.830 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T23:58:52.830 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T23:58:55.322 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T23:58:55.322 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:55.322 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-05-08T23:58:55.323 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T23:58:55.323 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T23:58:55.323 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T23:58:55.323 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T23:58:55.323 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.323 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.323 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T23:58:55.324 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T23:58:55.324 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-05-08T23:58:55.324 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T23:58:55.324 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T23:58:55.324 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T23:58:55.324 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T23:58:55.324 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.324 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.324 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.324 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.324 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:55.325 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.325 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.325 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.325 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:55.325 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:55.325 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:55.325 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:55.325 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:55.325 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.326 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.326 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:55.326 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.326 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.326 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.326 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:55.326 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:55.326 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.326 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.326 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-08T23:58:55.326 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T23:58:55.326 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T23:58:55.327 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T23:58:55.328 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T23:58:55.329 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T23:58:55.330 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T23:58:55.330 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T23:58:55.330 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.330 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.330 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.330 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T23:58:55.330 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T23:58:55.330 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T23:58:55.330 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T23:58:55.330 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T23:58:55.331 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T23:58:55.331 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T23:58:55.331 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T23:58:55.331 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T23:58:55.331 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T23:58:55.331 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T23:58:55.331 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T23:58:55.331 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T23:58:55.331 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T23:58:55.331 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T23:58:55.331 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T23:58:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T23:58:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T23:58:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T23:58:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T23:58:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T23:58:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T23:58:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T23:58:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T23:58:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T23:58:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T23:58:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T23:58:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T23:58:55.333 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T23:58:55.333 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T23:58:55.333 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T23:58:55.333 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T23:58:55.333 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T23:58:55.333 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T23:58:55.333 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T23:58:55.333 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T23:58:55.333 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T23:58:55.333 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T23:58:55.334 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T23:58:55.334 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T23:58:55.334 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.334 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.334 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T23:58:55.334 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T23:58:55.334 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T23:58:55.334 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.334 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T23:58:55.334 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T23:58:55.334 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T23:58:55.334 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T23:58:55.335 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T23:58:55.335 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T23:58:55.335 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T23:58:55.335 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T23:58:55.335 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T23:58:55.335 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T23:58:55.335 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T23:58:55.335 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T23:58:55.335 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T23:58:55.335 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T23:58:55.336 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T23:58:55.336 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.336 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.336 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-08T23:58:55.336 INFO:teuthology.orchestra.run.smithi142.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T23:58:55.336 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T23:58:55.336 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T23:58:55.336 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T23:58:55.336 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T23:58:55.336 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T23:58:55.336 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T23:58:55.336 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-05-08T23:58:55.337 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-05-08T23:58:55.742 DEBUG:teuthology.parallel:result is None 2024-05-08T23:58:55.743 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-08T23:58:55.964 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:58:55.992 INFO:teuthology.orchestra.run.smithi046.stdout:19.0.0-3594.gc9da4049.el9 2024-05-08T23:58:55.993 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-08T23:58:55.993 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-08T23:58:55.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-08T23:58:56.141 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:58:56.163 INFO:teuthology.orchestra.run.smithi142.stdout:19.0.0-3594.gc9da4049.el9 2024-05-08T23:58:56.163 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-08T23:58:56.164 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-08T23:58:56.165 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-08T23:58:56.306 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:58:56.331 INFO:teuthology.orchestra.run.smithi169.stdout:19.0.0-3594.gc9da4049.el9 2024-05-08T23:58:56.332 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-08T23:58:56.332 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-08T23:58:56.333 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T23:58:56.334 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:58:56.334 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:58:56.370 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-08T23:58:56.370 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:58:56.397 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-08T23:58:56.398 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:58:56.428 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T23:58:56.428 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:58:56.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:58:56.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:58:56.530 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-08T23:58:56.530 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:58:56.555 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:58:56.621 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-08T23:58:56.621 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:58:56.649 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:58:56.716 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T23:58:56.716 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:58:56.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:58:56.747 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:58:56.819 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-08T23:58:56.819 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:58:56.845 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:58:56.911 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-08T23:58:56.911 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:58:56.940 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:58:57.007 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T23:58:57.007 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:58:57.007 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:58:57.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:58:57.107 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-08T23:58:57.108 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:58:57.134 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:58:57.200 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-08T23:58:57.200 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:58:57.229 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:58:57.296 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-08T23:58:57.400 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-08T23:58:57.401 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-08T23:58:57.402 INFO:tasks.cephadm:Cluster fsid is eec67860-0d96-11ef-bc97-c7b262605968 2024-05-08T23:58:57.402 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-08T23:58:57.402 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.46', 'mon.b': '172.21.15.142', 'mon.c': '172.21.15.169'} 2024-05-08T23:58:57.402 INFO:tasks.cephadm:First mon is mon.a on smithi046 2024-05-08T23:58:57.402 INFO:tasks.cephadm:First mgr is x 2024-05-08T23:58:57.402 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-08T23:58:57.402 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname $(hostname -s) 2024-05-08T23:58:57.430 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-05-08T23:58:57.457 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname $(hostname -s) 2024-05-08T23:58:57.484 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-08T23:58:57.484 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-08T23:58:57.624 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79224/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3594-gc9da4049', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3594.gc9da4049'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:52:58.989794', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.183908-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-08T23:58:57.767 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-pdonnell-testing-20240508.183908-debug, sha1 c9da4049f638c0f84bdd9b4e33c42e86046ac554 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-08T23:58:57.768 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm 2024-05-08T23:58:57.768 DEBUG:teuthology.orchestra.run.smithi046:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T23:58:58.362 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r--. 1 ubuntu ubuntu 782809 May 8 23:58 /home/ubuntu/cephtest/cephadm 2024-05-08T23:58:58.362 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T23:58:58.835 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-r--r--. 1 ubuntu ubuntu 782809 May 8 23:58 /home/ubuntu/cephtest/cephadm 2024-05-08T23:58:58.836 DEBUG:teuthology.orchestra.run.smithi169:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T23:58:59.288 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r--. 1 ubuntu ubuntu 782809 May 8 23:58 /home/ubuntu/cephtest/cephadm 2024-05-08T23:58:59.289 DEBUG:teuthology.orchestra.run.smithi046:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T23:58:59.310 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T23:58:59.328 DEBUG:teuthology.orchestra.run.smithi169:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T23:58:59.363 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 on all hosts... 2024-05-08T23:58:59.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-08T23:58:59.366 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-08T23:58:59.371 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-08T23:59:00.892 INFO:teuthology.orchestra.run.smithi169.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-08T23:59:00.934 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-08T23:59:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-08T23:59:40.568 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T23:59:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-08T23:59:40.568 INFO:teuthology.orchestra.run.smithi046.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-08T23:59:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: "repo_digests": [ 2024-05-08T23:59:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-08T23:59:40.569 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-08T23:59:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T23:59:58.360 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-05-08T23:59:58.361 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-08T23:59:58.361 INFO:teuthology.orchestra.run.smithi169.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-08T23:59:58.361 INFO:teuthology.orchestra.run.smithi169.stdout: "repo_digests": [ 2024-05-08T23:59:58.361 INFO:teuthology.orchestra.run.smithi169.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-08T23:59:58.361 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2024-05-08T23:59:58.361 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-05-08T23:59:58.762 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-08T23:59:58.763 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-08T23:59:58.763 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-08T23:59:58.763 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-05-08T23:59:58.763 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-08T23:59:58.763 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-05-08T23:59:58.763 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-08T23:59:58.788 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph 2024-05-08T23:59:58.823 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-05-08T23:59:58.849 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph 2024-05-08T23:59:58.876 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /etc/ceph 2024-05-08T23:59:58.905 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-05-08T23:59:58.929 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /etc/ceph 2024-05-08T23:59:58.953 INFO:tasks.cephadm:Writing seed config... 2024-05-08T23:59:58.967 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-08T23:59:58.967 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-08T23:59:58.967 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-08T23:59:58.967 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-08T23:59:58.967 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-08T23:59:58.967 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-08T23:59:58.967 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-08T23:59:58.967 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-08T23:59:58.967 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-08T23:59:58.967 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-08T23:59:58.967 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-08T23:59:58.968 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-08T23:59:58.969 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-08T23:59:58.969 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-08T23:59:58.969 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-08T23:59:58.969 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-08T23:59:58.969 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-08T23:59:58.969 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-08T23:59:58.969 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-08T23:59:58.969 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-08T23:59:58.969 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-08T23:59:58.969 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-08T23:59:58.970 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-08T23:59:58.970 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-08T23:59:58.970 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-08T23:59:58.970 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-08T23:59:58.970 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-08T23:59:58.970 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-08T23:59:58.970 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-08T23:59:58.970 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-08T23:59:58.971 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T23:59:58.971 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-08T23:59:58.989 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = eec67860-0d96-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-08T23:59:58.989 DEBUG:teuthology.orchestra.run.smithi046:mon.a> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.a.service 2024-05-08T23:59:59.031 DEBUG:teuthology.orchestra.run.smithi046:mgr.x> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.x.service 2024-05-08T23:59:59.076 INFO:tasks.cephadm:Bootstrapping... 2024-05-08T23:59:59.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 -v bootstrap --fsid eec67860-0d96-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.46 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:59:59.278 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:59:59.278 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554', '-v', 'bootstrap', '--fsid', 'eec67860-0d96-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.46', '--skip-admin-label'] 2024-05-08T23:59:59.279 INFO:teuthology.orchestra.run.smithi046.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-08T23:59:59.279 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying podman|docker is present... 2024-05-08T23:59:59.305 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-08T23:59:59.305 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying lvm2 is present... 2024-05-08T23:59:59.305 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying time synchronization is in place... 2024-05-08T23:59:59.313 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T23:59:59.313 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T23:59:59.319 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T23:59:59.319 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-08T23:59:59.326 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-05-08T23:59:59.333 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-05-08T23:59:59.333 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-05-08T23:59:59.333 INFO:teuthology.orchestra.run.smithi046.stdout:Repeating the final host check... 2024-05-08T23:59:59.357 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-08T23:59:59.357 INFO:teuthology.orchestra.run.smithi046.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-08T23:59:59.357 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl is present 2024-05-08T23:59:59.357 INFO:teuthology.orchestra.run.smithi046.stdout:lvcreate is present 2024-05-08T23:59:59.364 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T23:59:59.364 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T23:59:59.370 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T23:59:59.370 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-08T23:59:59.377 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-05-08T23:59:59.384 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-05-08T23:59:59.384 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-05-08T23:59:59.384 INFO:teuthology.orchestra.run.smithi046.stdout:Host looks OK 2024-05-08T23:59:59.384 INFO:teuthology.orchestra.run.smithi046.stdout:Cluster fsid: eec67860-0d96-11ef-bc97-c7b262605968 2024-05-08T23:59:59.384 INFO:teuthology.orchestra.run.smithi046.stdout:Acquiring lock 140250636734176 on /run/cephadm/eec67860-0d96-11ef-bc97-c7b262605968.lock 2024-05-08T23:59:59.384 INFO:teuthology.orchestra.run.smithi046.stdout:Lock 140250636734176 acquired on /run/cephadm/eec67860-0d96-11ef-bc97-c7b262605968.lock 2024-05-08T23:59:59.385 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 3300 ... 2024-05-08T23:59:59.385 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 6789 ... 2024-05-08T23:59:59.385 INFO:teuthology.orchestra.run.smithi046.stdout:Base mon IP(s) is [172.21.15.46:3300, 172.21.15.46:6789], mon addrv is [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-05-08T23:59:59.388 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.46 metric 100 2024-05-08T23:59:59.388 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.46 metric 100 2024-05-08T23:59:59.392 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-08T23:59:59.392 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-08T23:59:59.392 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-08T23:59:59.394 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-08T23:59:59.395 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-08T23:59:59.395 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T23:59:59.395 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-08T23:59:59.395 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-05-08T23:59:59.395 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T23:59:59.395 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-05-08T23:59:59.395 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-05-08T23:59:59.396 INFO:teuthology.orchestra.run.smithi046.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-08T23:59:59.396 INFO:teuthology.orchestra.run.smithi046.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-08T23:59:59.396 INFO:teuthology.orchestra.run.smithi046.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-08T23:59:59.859 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-08T23:59:59.859 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-08T23:59:59.859 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Getting image source signatures 2024-05-08T23:59:59.859 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:0411e0fd7a5d45f8ab31b23cec89f97c70494de39461f644c7cdc77de3b9cbca 2024-05-08T23:59:59.859 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-08T23:59:59.859 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying config sha256:db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-08T23:59:59.859 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-09T00:00:04.916 INFO:teuthology.orchestra.run.smithi046.stdout:ceph: stdout ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T00:00:04.916 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph version: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T00:00:04.916 INFO:teuthology.orchestra.run.smithi046.stdout:Extracting ceph user uid/gid from container image... 2024-05-09T00:00:07.493 INFO:teuthology.orchestra.run.smithi046.stdout:stat: stdout 167 167 2024-05-09T00:00:07.493 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial keys... 2024-05-09T00:00:08.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQAHEjxmaKLaORAAhiYyCtInlyHhQGBVXqp9Jg== 2024-05-09T00:00:09.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQAIEjxmPXqYNhAAETw66wuUu0elI7rhndnnUQ== 2024-05-09T00:00:10.827 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQAJEjxmXoD1NBAAeoPEE/56ybBOZc8yOYKv7g== 2024-05-09T00:00:10.828 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial monmap... 2024-05-09T00:00:11.934 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T00:00:11.935 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-09T00:00:11.936 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:00:11.936 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T00:00:11.936 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool for a [v2:172.21.15.46:3300,v1:172.21.15.46:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T00:00:11.936 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = quincy 2024-05-09T00:00:11.936 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: set fsid to eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:00:11.936 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T00:00:11.936 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:00:11.936 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mon... 2024-05-09T00:00:12.848 INFO:teuthology.orchestra.run.smithi046.stdout:create mon.a on 2024-05-09T00:00:13.093 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-09T00:00:13.240 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-09T00:00:13.416 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-eec67860-0d96-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-eec67860-0d96-11ef-bc97-c7b262605968.target. 2024-05-09T00:00:13.416 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-eec67860-0d96-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-eec67860-0d96-11ef-bc97-c7b262605968.target. 2024-05-09T00:00:13.654 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.a 2024-05-09T00:00:13.654 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-09T00:00:13.826 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-eec67860-0d96-11ef-bc97-c7b262605968.target.wants/ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-eec67860-0d96-11ef-bc97-c7b262605968@.service. 2024-05-09T00:00:14.374 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T00:00:14.374 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-05-09T00:00:14.381 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T00:00:14.381 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-09T00:00:14.381 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-05-09T00:00:14.381 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T00:00:14.382 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon to start... 2024-05-09T00:00:14.382 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon... 2024-05-09T00:00:14.503 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:14 smithi046 systemd[1]: Started Ceph mon.a for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:00:15.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout cluster: 2024-05-09T00:00:15.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout id: eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:00:15.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-09T00:00:15.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:15.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout services: 2024-05-09T00:00:15.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.535708s) 2024-05-09T00:00:15.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-09T00:00:15.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-09T00:00:15.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:15.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout data: 2024-05-09T00:00:15.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pgs: 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.809+0000 7f68a0cde700 1 Processor -- start 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.809+0000 7f68a0cde700 1 -- start start 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.809+0000 7f68a0cde700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 0x7f689c108d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.809+0000 7f68a0cde700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f689c109330 con 0x7f689c108930 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.810+0000 7f689a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 0x7f689c108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.810+0000 7f689a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 0x7f689c108d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54068/0 (socket says 172.21.15.46:54068) 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.810+0000 7f689a59c700 1 -- 172.21.15.46:0/3991598060 learned_addr learned my addr 172.21.15.46:0/3991598060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.810+0000 7f689a59c700 1 -- 172.21.15.46:0/3991598060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c109b00 con 0x7f689c108930 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.810+0000 7f689a59c700 1 --2- 172.21.15.46:0/3991598060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 0x7f689c108d40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f68840098a0 tx=0x7f6884009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332e5377801afed7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.810+0000 7f689959a700 1 -- 172.21.15.46:0/3991598060 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6884004160 con 0x7f689c108930 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.810+0000 7f689959a700 1 -- 172.21.15.46:0/3991598060 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6884004300 con 0x7f689c108930 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.810+0000 7f689959a700 1 -- 172.21.15.46:0/3991598060 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68840036a0 con 0x7f689c108930 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.811+0000 7f68a0cde700 1 -- 172.21.15.46:0/3991598060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 msgr2=0x7f689c108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.811+0000 7f68a0cde700 1 --2- 172.21.15.46:0/3991598060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 0x7f689c108d40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f68840098a0 tx=0x7f6884009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.811+0000 7f68a0cde700 1 -- 172.21.15.46:0/3991598060 shutdown_connections 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.811+0000 7f68a0cde700 1 --2- 172.21.15.46:0/3991598060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 0x7f689c108d40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.811+0000 7f68a0cde700 1 -- 172.21.15.46:0/3991598060 >> 172.21.15.46:0/3991598060 conn(0x7f689c103fd0 msgr2=0x7f689c106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.811+0000 7f68a0cde700 1 -- 172.21.15.46:0/3991598060 shutdown_connections 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.811+0000 7f68a0cde700 1 -- 172.21.15.46:0/3991598060 wait complete. 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.812+0000 7f68a0cde700 1 Processor -- start 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.812+0000 7f68a0cde700 1 -- start start 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.812+0000 7f68a0cde700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 0x7f689c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:15.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.813+0000 7f68a0cde700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f689c10a030 con 0x7f689c108930 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.813+0000 7f689a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 0x7f689c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.813+0000 7f689a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 0x7f689c19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54072/0 (socket says 172.21.15.46:54072) 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.813+0000 7f689a59c700 1 -- 172.21.15.46:0/4101977920 learned_addr learned my addr 172.21.15.46:0/4101977920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.813+0000 7f689a59c700 1 -- 172.21.15.46:0/4101977920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c19dc30 con 0x7f689c108930 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.813+0000 7f689a59c700 1 --2- 172.21.15.46:0/4101977920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 0x7f689c19d6f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6884000c00 tx=0x7f6884004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.813+0000 7f68937fe700 1 -- 172.21.15.46:0/4101977920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6884003db0 con 0x7f689c108930 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.813+0000 7f68937fe700 1 -- 172.21.15.46:0/4101977920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6884011630 con 0x7f689c108930 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.813+0000 7f68a0cde700 1 -- 172.21.15.46:0/4101977920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c19dec0 con 0x7f689c108930 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.813+0000 7f68937fe700 1 -- 172.21.15.46:0/4101977920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6884011950 con 0x7f689c108930 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.813+0000 7f68a0cde700 1 -- 172.21.15.46:0/4101977920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c1a0ad0 con 0x7f689c108930 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.814+0000 7f68937fe700 1 -- 172.21.15.46:0/4101977920 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f688401a460 con 0x7f689c108930 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.814+0000 7f68937fe700 1 -- 172.21.15.46:0/4101977920 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f688401b780 con 0x7f689c108930 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.814+0000 7f68a0cde700 1 -- 172.21.15.46:0/4101977920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689c19e320 con 0x7f689c108930 2024-05-09T00:00:15.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.817+0000 7f68937fe700 1 -- 172.21.15.46:0/4101977920 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f688401b980 con 0x7f689c108930 2024-05-09T00:00:15.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.890+0000 7f68a0cde700 1 -- 172.21.15.46:0/4101977920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f689c02e800 con 0x7f689c108930 2024-05-09T00:00:15.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.891+0000 7f68937fe700 1 -- 172.21.15.46:0/4101977920 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f688401bb60 con 0x7f689c108930 2024-05-09T00:00:15.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.891+0000 7f68a0cde700 1 -- 172.21.15.46:0/4101977920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 msgr2=0x7f689c19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:15.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.891+0000 7f68a0cde700 1 --2- 172.21.15.46:0/4101977920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 0x7f689c19d6f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6884000c00 tx=0x7f6884004750 comp rx=0 tx=0).stop 2024-05-09T00:00:15.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.891+0000 7f68a0cde700 1 -- 172.21.15.46:0/4101977920 shutdown_connections 2024-05-09T00:00:15.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.891+0000 7f68a0cde700 1 --2- 172.21.15.46:0/4101977920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f689c108930 0x7f689c19d6f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:15.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.891+0000 7f68a0cde700 1 -- 172.21.15.46:0/4101977920 >> 172.21.15.46:0/4101977920 conn(0x7f689c103fd0 msgr2=0x7f689c193c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:15.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.891+0000 7f68a0cde700 1 -- 172.21.15.46:0/4101977920 shutdown_connections 2024-05-09T00:00:15.446 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:14.891+0000 7f68a0cde700 1 -- 172.21.15.46:0/4101977920 wait complete. 2024-05-09T00:00:15.446 INFO:teuthology.orchestra.run.smithi046.stdout:mon is available 2024-05-09T00:00:15.446 INFO:teuthology.orchestra.run.smithi046.stdout:Assimilating anything we can from ceph.conf... 2024-05-09T00:00:15.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:15 smithi046 ceph-mon[29778]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T00:00:15.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:15 smithi046 ceph-mon[29778]: monmap epoch 1 2024-05-09T00:00:15.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:15 smithi046 ceph-mon[29778]: fsid eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:00:15.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:15 smithi046 ceph-mon[29778]: last_changed 2024-05-09T00:00:11.176337+0000 2024-05-09T00:00:15.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:15 smithi046 ceph-mon[29778]: created 2024-05-09T00:00:11.176337+0000 2024-05-09T00:00:15.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:15 smithi046 ceph-mon[29778]: min_mon_release 19 (squid) 2024-05-09T00:00:15.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:15 smithi046 ceph-mon[29778]: election_strategy: 1 2024-05-09T00:00:15.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:15 smithi046 ceph-mon[29778]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T00:00:15.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:15 smithi046 ceph-mon[29778]: fsmap 2024-05-09T00:00:15.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:15 smithi046 ceph-mon[29778]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T00:00:15.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:15 smithi046 ceph-mon[29778]: mgrmap e1: no daemons active 2024-05-09T00:00:15.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:15 smithi046 ceph-mon[29778]: from='client.? 172.21.15.46:0/4101977920' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-09T00:00:16.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:16 smithi046 ceph-mon[29778]: from='client.? 172.21.15.46:0/3367535075' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T00:00:16.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:16 smithi046 ceph-mon[29778]: from='client.? 172.21.15.46:0/3367535075' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-09T00:00:16.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:16.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T00:00:16.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:00:16.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T00:00:16.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-05-09T00:00:16.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T00:00:16.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T00:00:16.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T00:00:16.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T00:00:16.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.880+0000 7f52a2409700 1 Processor -- start 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.880+0000 7f52a2409700 1 -- start start 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.881+0000 7f52a2409700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 0x7f529c108790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.881+0000 7f52a2409700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f529c07ce30 con 0x7f529c108380 2024-05-09T00:00:16.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.881+0000 7f529bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 0x7f529c108790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.881+0000 7f529bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 0x7f529c108790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54078/0 (socket says 172.21.15.46:54078) 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.881+0000 7f529bfff700 1 -- 172.21.15.46:0/4228871514 learned_addr learned my addr 172.21.15.46:0/4228871514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.881+0000 7f529bfff700 1 -- 172.21.15.46:0/4228871514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529c108cd0 con 0x7f529c108380 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.881+0000 7f529bfff700 1 --2- 172.21.15.46:0/4228871514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 0x7f529c108790 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f528c0098a0 tx=0x7f528c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5e9abe868bfc2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.882+0000 7f529b7fe700 1 -- 172.21.15.46:0/4228871514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f528c004160 con 0x7f529c108380 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.882+0000 7f529b7fe700 1 -- 172.21.15.46:0/4228871514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f528c004300 con 0x7f529c108380 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.882+0000 7f529b7fe700 1 -- 172.21.15.46:0/4228871514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f528c0036a0 con 0x7f529c108380 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.882+0000 7f52a2409700 1 -- 172.21.15.46:0/4228871514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 msgr2=0x7f529c108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.882+0000 7f52a2409700 1 --2- 172.21.15.46:0/4228871514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 0x7f529c108790 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f528c0098a0 tx=0x7f528c009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.882+0000 7f52a2409700 1 -- 172.21.15.46:0/4228871514 shutdown_connections 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.882+0000 7f52a2409700 1 --2- 172.21.15.46:0/4228871514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 0x7f529c108790 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.882+0000 7f52a2409700 1 -- 172.21.15.46:0/4228871514 >> 172.21.15.46:0/4228871514 conn(0x7f529c104050 msgr2=0x7f529c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:16.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.882+0000 7f52a2409700 1 -- 172.21.15.46:0/4228871514 shutdown_connections 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.882+0000 7f52a2409700 1 -- 172.21.15.46:0/4228871514 wait complete. 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.883+0000 7f52a2409700 1 Processor -- start 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.883+0000 7f52a2409700 1 -- start start 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.883+0000 7f52a2409700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 0x7f529c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.883+0000 7f52a2409700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f529c109200 con 0x7f529c108380 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.884+0000 7f529bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 0x7f529c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.884+0000 7f529bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 0x7f529c19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54086/0 (socket says 172.21.15.46:54086) 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.884+0000 7f529bfff700 1 -- 172.21.15.46:0/3367535075 learned_addr learned my addr 172.21.15.46:0/3367535075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.884+0000 7f529bfff700 1 -- 172.21.15.46:0/3367535075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529c19fd50 con 0x7f529c108380 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.884+0000 7f529bfff700 1 --2- 172.21.15.46:0/3367535075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 0x7f529c19f810 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f528c000c00 tx=0x7f528c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.884+0000 7f5299ffb700 1 -- 172.21.15.46:0/3367535075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f528c003db0 con 0x7f529c108380 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.884+0000 7f5299ffb700 1 -- 172.21.15.46:0/3367535075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f528c011630 con 0x7f529c108380 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.884+0000 7f52a2409700 1 -- 172.21.15.46:0/3367535075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529c19ffe0 con 0x7f529c108380 2024-05-09T00:00:16.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.884+0000 7f5299ffb700 1 -- 172.21.15.46:0/3367535075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f528c011950 con 0x7f529c108380 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.884+0000 7f52a2409700 1 -- 172.21.15.46:0/3367535075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f529c1a2bf0 con 0x7f529c108380 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.884+0000 7f5299ffb700 1 -- 172.21.15.46:0/3367535075 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f528c01a460 con 0x7f529c108380 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.885+0000 7f5299ffb700 1 -- 172.21.15.46:0/3367535075 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f528c01b780 con 0x7f529c108380 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.885+0000 7f52a2409700 1 -- 172.21.15.46:0/3367535075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529c064e20 con 0x7f529c108380 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.888+0000 7f5299ffb700 1 -- 172.21.15.46:0/3367535075 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f528c042ec0 con 0x7f529c108380 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.959+0000 7f52a2409700 1 -- 172.21.15.46:0/3367535075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f529c1a2dc0 con 0x7f529c108380 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.963+0000 7f5299ffb700 1 -- 172.21.15.46:0/3367535075 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f528c031070 con 0x7f529c108380 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.964+0000 7f5299ffb700 1 -- 172.21.15.46:0/3367535075 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f528c01b980 con 0x7f529c108380 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.964+0000 7f52a2409700 1 -- 172.21.15.46:0/3367535075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 msgr2=0x7f529c19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.964+0000 7f52a2409700 1 --2- 172.21.15.46:0/3367535075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 0x7f529c19f810 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f528c000c00 tx=0x7f528c004750 comp rx=0 tx=0).stop 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.964+0000 7f52a2409700 1 -- 172.21.15.46:0/3367535075 shutdown_connections 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.964+0000 7f52a2409700 1 --2- 172.21.15.46:0/3367535075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f529c108380 0x7f529c19f810 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.964+0000 7f52a2409700 1 -- 172.21.15.46:0/3367535075 >> 172.21.15.46:0/3367535075 conn(0x7f529c104050 msgr2=0x7f529c195ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:16.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.964+0000 7f52a2409700 1 -- 172.21.15.46:0/3367535075 shutdown_connections 2024-05-09T00:00:16.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:15.964+0000 7f52a2409700 1 -- 172.21.15.46:0/3367535075 wait complete. 2024-05-09T00:00:16.555 INFO:teuthology.orchestra.run.smithi046.stdout:Generating new minimal ceph.conf... 2024-05-09T00:00:17.445 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:17 smithi046 ceph-mon[29778]: from='client.? 172.21.15.46:0/1187355420' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]: dispatch 2024-05-09T00:00:17.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.038+0000 7faa50398700 1 Processor -- start 2024-05-09T00:00:17.720 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.038+0000 7faa50398700 1 -- start start 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.038+0000 7faa50398700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 0x7faa48108790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.038+0000 7faa50398700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa4807d6a0 con 0x7faa48108380 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.038+0000 7faa4e134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 0x7faa48108790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.039+0000 7faa4e134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 0x7faa48108790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48708/0 (socket says 172.21.15.46:48708) 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.039+0000 7faa4e134700 1 -- 172.21.15.46:0/3033244299 learned_addr learned my addr 172.21.15.46:0/3033244299 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.039+0000 7faa4e134700 1 -- 172.21.15.46:0/3033244299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa48108cd0 con 0x7faa48108380 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.039+0000 7faa4e134700 1 --2- 172.21.15.46:0/3033244299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 0x7faa48108790 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faa440098a0 tx=0x7faa44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3841f3d082d4e8c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.039+0000 7faa4d132700 1 -- 172.21.15.46:0/3033244299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa44004160 con 0x7faa48108380 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.039+0000 7faa4d132700 1 -- 172.21.15.46:0/3033244299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faa44004300 con 0x7faa48108380 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.039+0000 7faa4d132700 1 -- 172.21.15.46:0/3033244299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa44011af0 con 0x7faa48108380 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.040+0000 7faa50398700 1 -- 172.21.15.46:0/3033244299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 msgr2=0x7faa48108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.040+0000 7faa50398700 1 --2- 172.21.15.46:0/3033244299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 0x7faa48108790 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faa440098a0 tx=0x7faa44009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.040+0000 7faa50398700 1 -- 172.21.15.46:0/3033244299 shutdown_connections 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.040+0000 7faa50398700 1 --2- 172.21.15.46:0/3033244299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 0x7faa48108790 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.040+0000 7faa50398700 1 -- 172.21.15.46:0/3033244299 >> 172.21.15.46:0/3033244299 conn(0x7faa48103f90 msgr2=0x7faa48106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.040+0000 7faa50398700 1 -- 172.21.15.46:0/3033244299 shutdown_connections 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.040+0000 7faa50398700 1 -- 172.21.15.46:0/3033244299 wait complete. 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.041+0000 7faa50398700 1 Processor -- start 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.041+0000 7faa50398700 1 -- start start 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.041+0000 7faa50398700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 0x7faa481a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.041+0000 7faa50398700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa48109d50 con 0x7faa48108380 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.041+0000 7faa4e134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 0x7faa481a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.041+0000 7faa4e134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 0x7faa481a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48722/0 (socket says 172.21.15.46:48722) 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.041+0000 7faa4e134700 1 -- 172.21.15.46:0/1187355420 learned_addr learned my addr 172.21.15.46:0/1187355420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.041+0000 7faa4e134700 1 -- 172.21.15.46:0/1187355420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa481a0730 con 0x7faa48108380 2024-05-09T00:00:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.042+0000 7faa4e134700 1 --2- 172.21.15.46:0/1187355420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 0x7faa481a01f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faa440117b0 tx=0x7faa44011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.042+0000 7faa3f7fe700 1 -- 172.21.15.46:0/1187355420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa440039c0 con 0x7faa48108380 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.042+0000 7faa3f7fe700 1 -- 172.21.15.46:0/1187355420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faa44003b60 con 0x7faa48108380 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.042+0000 7faa50398700 1 -- 172.21.15.46:0/1187355420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa481a09c0 con 0x7faa48108380 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.042+0000 7faa3f7fe700 1 -- 172.21.15.46:0/1187355420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa440196a0 con 0x7faa48108380 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.042+0000 7faa50398700 1 -- 172.21.15.46:0/1187355420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa481a1400 con 0x7faa48108380 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.042+0000 7faa3f7fe700 1 -- 172.21.15.46:0/1187355420 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faa44019ca0 con 0x7faa48108380 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.042+0000 7faa3f7fe700 1 -- 172.21.15.46:0/1187355420 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faa44024730 con 0x7faa48108380 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.043+0000 7faa50398700 1 -- 172.21.15.46:0/1187355420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa48064e20 con 0x7faa48108380 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.045+0000 7faa3f7fe700 1 -- 172.21.15.46:0/1187355420 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7faa44043ec0 con 0x7faa48108380 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.116+0000 7faa50398700 1 -- 172.21.15.46:0/1187355420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"} v 0) -- 0x7faa481a2110 con 0x7faa48108380 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.116+0000 7faa3f7fe700 1 -- 172.21.15.46:0/1187355420 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]=0 v2) ==== 126+0+175 (secure 0 0 0) 0x7faa44032020 con 0x7faa48108380 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.117+0000 7faa50398700 1 -- 172.21.15.46:0/1187355420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 msgr2=0x7faa481a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.117+0000 7faa50398700 1 --2- 172.21.15.46:0/1187355420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 0x7faa481a01f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faa440117b0 tx=0x7faa44011b70 comp rx=0 tx=0).stop 2024-05-09T00:00:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.117+0000 7faa50398700 1 -- 172.21.15.46:0/1187355420 shutdown_connections 2024-05-09T00:00:17.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.117+0000 7faa50398700 1 --2- 172.21.15.46:0/1187355420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa48108380 0x7faa481a01f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:17.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.117+0000 7faa50398700 1 -- 172.21.15.46:0/1187355420 >> 172.21.15.46:0/1187355420 conn(0x7faa48103f90 msgr2=0x7faa481967a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:17.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.117+0000 7faa50398700 1 -- 172.21.15.46:0/1187355420 shutdown_connections 2024-05-09T00:00:17.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:17.117+0000 7faa50398700 1 -- 172.21.15.46:0/1187355420 wait complete. 2024-05-09T00:00:17.724 INFO:teuthology.orchestra.run.smithi046.stdout:Restarting the monitor... 2024-05-09T00:00:17.967 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:17 smithi046 systemd[1]: Stopping Ceph mon.a for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:00:18.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:17 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a[29774]: 2024-05-09T00:00:17.972+0000 7f816b89b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:00:18.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:17 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a[29774]: 2024-05-09T00:00:17.972+0000 7f816b89b700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-09T00:00:18.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:18 smithi046 podman[30021]: 2024-05-09 00:00:18.137302648 +0000 UTC m=+0.244651370 container died 487bfa0e26d5ccfec99a4e141a5da84b0de8a3c5dabd467fedaa316fe111745e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:00:18.725 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:18 smithi046 podman[30021]: 2024-05-09 00:00:18.497116827 +0000 UTC m=+0.604465550 container cleanup 487bfa0e26d5ccfec99a4e141a5da84b0de8a3c5dabd467fedaa316fe111745e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-09T00:00:18.725 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:18 smithi046 bash[30021]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a 2024-05-09T00:00:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:18 smithi046 podman[30035]: 2024-05-09 00:00:18.731415761 +0000 UTC m=+0.596389067 container remove 487bfa0e26d5ccfec99a4e141a5da84b0de8a3c5dabd467fedaa316fe111745e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-09T00:00:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:18 smithi046 systemd[1]: ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-09T00:00:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:18 smithi046 systemd[1]: Stopped Ceph mon.a for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:00:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:18 smithi046 systemd[1]: Starting Ceph mon.a for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:00:19.323 INFO:teuthology.orchestra.run.smithi046.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-09T00:00:19.365 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 podman[30132]: 2024-05-09 00:00:19.115627267 +0000 UTC m=+0.112632559 container create 74e04927330a46af4264e25637bb122c894dd15339cd3ccfe10f0b968faa3272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 podman[30132]: 2024-05-09 00:00:19.021296704 +0000 UTC m=+0.018302015 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 podman[30132]: 2024-05-09 00:00:19.267697063 +0000 UTC m=+0.264702360 container init 74e04927330a46af4264e25637bb122c894dd15339cd3ccfe10f0b968faa3272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 podman[30132]: 2024-05-09 00:00:19.270733552 +0000 UTC m=+0.267738844 container start 74e04927330a46af4264e25637bb122c894dd15339cd3ccfe10f0b968faa3272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS) 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: pidfile_write: ignore empty --pid-file 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: load: jerasure load: lrc 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: RocksDB version: 7.9.2 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Git sha 0 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: DB SUMMARY 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: DB Session ID: 17HKWSOX2ECOKSE1PTGU 2024-05-09T00:00:19.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: CURRENT file: CURRENT 2024-05-09T00:00:19.367 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T00:00:19.367 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-09T00:00:19.367 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-09T00:00:19.367 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104266 ; 2024-05-09T00:00:19.368 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.error_if_exists: 0 2024-05-09T00:00:19.368 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.create_if_missing: 0 2024-05-09T00:00:19.368 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.paranoid_checks: 1 2024-05-09T00:00:19.368 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T00:00:19.368 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T00:00:19.368 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.env: 0x5589d0a5a500 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.info_log: 0x5589d2718020 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.statistics: (nil) 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.use_fsync: 0 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_log_file_size: 0 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.allow_fallocate: 1 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T00:00:19.369 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.use_direct_reads: 0 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.db_log_dir: 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.wal_dir: 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T00:00:19.370 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.write_buffer_manager: 0x5589d25e28c0 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.unordered_write: 0 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T00:00:19.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.row_cache: None 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.wal_filter: None 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.two_write_queues: 0 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.wal_compression: 0 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.atomic_flush: 0 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.log_readahead_size: 0 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T00:00:19.372 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_background_jobs: 2 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_background_compactions: -1 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_subcompactions: 1 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T00:00:19.373 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_open_files: -1 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_background_flushes: -1 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Compression algorithms supported: 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: kZSTD supported: 0 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: kXpressCompression supported: 0 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: kZlibCompression supported: 1 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: kSnappyCompression supported: 1 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: kLZ4Compression supported: 1 2024-05-09T00:00:19.374 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: kBZip2Compression supported: 0 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.merge_operator: 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_filter: None 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5589d27181e0) 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks: 1 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T00:00:19.375 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_top_level_index_and_filter: 1 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_type: 0 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_index_type: 0 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_shortening: 1 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: checksum: 4 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: no_block_cache: 0 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache: 0x5589d26151f0 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_name: BinnedLRUCache 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_options: 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: capacity : 536870912 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_shard_bits : 4 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: strict_capacity_limit : 0 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: high_pri_pool_ratio: 0.000 2024-05-09T00:00:19.376 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_compressed: (nil) 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: persistent_cache: (nil) 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size: 4096 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size_deviation: 10 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_restart_interval: 16 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_block_restart_interval: 1 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: metadata_block_size: 4096 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: partition_filters: 0 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: use_delta_encoding: 1 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: filter_policy: bloomfilter 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: whole_key_filtering: 1 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: verify_compression: 0 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: read_amp_bytes_per_bit: 0 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: format_version: 5 2024-05-09T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_index_compression: 1 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_align: 0 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout: max_auto_readahead_size: 262144 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout: prepopulate_block_cache: 0 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout: initial_auto_readahead_size: 8192 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compression: NoCompression 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.num_levels: 7 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T00:00:19.378 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T00:00:19.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T00:00:19.380 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T00:00:19.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.inplace_update_support: 0 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.bloom_locality: 0 2024-05-09T00:00:19.382 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.max_successive_merges: 0 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.ttl: 2592000 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.enable_blob_files: false 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.min_blob_size: 0 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T00:00:19.383 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: deb80c90-8614-40c5-9341-6c66663d066d 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715212819311308, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715212819312490, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99494, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97440, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13917, "raw_average_key_size": 49, "raw_value_size": 89834, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715212819, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "deb80c90-8614-40c5-9341-6c66663d066d", "db_session_id": "17HKWSOX2ECOKSE1PTGU", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715212819312574, "job": 1, "event": "recovery_finished"} 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T00:00:19.384 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5589d25f9c00 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: DB pointer 0x5589d73ec000 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** DB Stats ** 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T00:00:19.385 INFO:journalctl@ceph.mon.a.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: L0 2/0 99.02 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: Sum 2/0 99.02 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 153.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T00:00:19.386 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative compaction: 0.00 GB write, 28.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval compaction: 0.00 GB write, 28.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache BinnedLRUCache@0x5589d26151f0#2 capacity: 512.00 MB usage: 1.41 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.91 KB,0.000172853%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: starting mon.a rank 0 at public addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] at bind addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: mon.a@-1(???) e1 preinit fsid eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: mon.a@-1(???).mds e1 new map 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: mon.a@-1(???).mds e1 print_map 2024-05-09T00:00:19.387 INFO:journalctl@ceph.mon.a.smithi046.stdout: e1 2024-05-09T00:00:19.388 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T00:00:19.388 INFO:journalctl@ceph.mon.a.smithi046.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T00:00:19.388 INFO:journalctl@ceph.mon.a.smithi046.stdout: legacy client fscid: -1 2024-05-09T00:00:19.388 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-09T00:00:19.388 INFO:journalctl@ceph.mon.a.smithi046.stdout: No filesystems configured 2024-05-09T00:00:19.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T00:00:19.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:00:19.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:00:19.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:00:19.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-09T00:00:19.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 bash[30132]: 74e04927330a46af4264e25637bb122c894dd15339cd3ccfe10f0b968faa3272 2024-05-09T00:00:19.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 systemd[1]: Started Ceph mon.a for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:00:19.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T00:00:19.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: monmap epoch 1 2024-05-09T00:00:19.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: fsid eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:00:19.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: last_changed 2024-05-09T00:00:11.176337+0000 2024-05-09T00:00:19.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: created 2024-05-09T00:00:11.176337+0000 2024-05-09T00:00:19.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: min_mon_release 19 (squid) 2024-05-09T00:00:19.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: election_strategy: 1 2024-05-09T00:00:19.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T00:00:19.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: fsmap 2024-05-09T00:00:19.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T00:00:19.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:19 smithi046 ceph-mon[30151]: mgrmap e1: no daemons active 2024-05-09T00:00:20.437 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.838+0000 7fa1c2d96700 1 Processor -- start 2024-05-09T00:00:20.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.838+0000 7fa1c2d96700 1 -- start start 2024-05-09T00:00:20.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.839+0000 7fa1c2d96700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 0x7fa1bc108da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:20.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.839+0000 7fa1c2d96700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc109390 con 0x7fa1bc108990 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.839+0000 7fa1c0b32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 0x7fa1bc108da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.839+0000 7fa1c0b32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 0x7fa1bc108da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48734/0 (socket says 172.21.15.46:48734) 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.839+0000 7fa1c0b32700 1 -- 172.21.15.46:0/60473011 learned_addr learned my addr 172.21.15.46:0/60473011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.839+0000 7fa1c0b32700 1 -- 172.21.15.46:0/60473011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc109b60 con 0x7fa1bc108990 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.839+0000 7fa1c0b32700 1 --2- 172.21.15.46:0/60473011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 0x7fa1bc108da0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0098a0 tx=0x7fa1ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b7b4ac6690436f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.840+0000 7fa1bb7fe700 1 -- 172.21.15.46:0/60473011 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac004160 con 0x7fa1bc108990 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.840+0000 7fa1bb7fe700 1 -- 172.21.15.46:0/60473011 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa1ac004300 con 0x7fa1bc108990 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.840+0000 7fa1bb7fe700 1 -- 172.21.15.46:0/60473011 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac011af0 con 0x7fa1bc108990 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.840+0000 7fa1c2d96700 1 -- 172.21.15.46:0/60473011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 msgr2=0x7fa1bc108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.840+0000 7fa1c2d96700 1 --2- 172.21.15.46:0/60473011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 0x7fa1bc108da0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0098a0 tx=0x7fa1ac009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.840+0000 7fa1c2d96700 1 -- 172.21.15.46:0/60473011 shutdown_connections 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.840+0000 7fa1c2d96700 1 --2- 172.21.15.46:0/60473011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 0x7fa1bc108da0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.840+0000 7fa1c2d96700 1 -- 172.21.15.46:0/60473011 >> 172.21.15.46:0/60473011 conn(0x7fa1bc103ff0 msgr2=0x7fa1bc106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:20.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.841+0000 7fa1c2d96700 1 -- 172.21.15.46:0/60473011 shutdown_connections 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.841+0000 7fa1c2d96700 1 -- 172.21.15.46:0/60473011 wait complete. 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.841+0000 7fa1c2d96700 1 Processor -- start 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.841+0000 7fa1c2d96700 1 -- start start 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.841+0000 7fa1c2d96700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 0x7fa1bc19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.841+0000 7fa1c2d96700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc10cf10 con 0x7fa1bc108990 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.842+0000 7fa1c0b32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 0x7fa1bc19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.842+0000 7fa1c0b32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 0x7fa1bc19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48740/0 (socket says 172.21.15.46:48740) 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.842+0000 7fa1c0b32700 1 -- 172.21.15.46:0/2539898020 learned_addr learned my addr 172.21.15.46:0/2539898020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.842+0000 7fa1c0b32700 1 -- 172.21.15.46:0/2539898020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc19d370 con 0x7fa1bc108990 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.842+0000 7fa1c0b32700 1 --2- 172.21.15.46:0/2539898020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 0x7fa1bc19ce30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac011670 tx=0x7fa1ac004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.842+0000 7fa1b9ffb700 1 -- 172.21.15.46:0/2539898020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac0036a0 con 0x7fa1bc108990 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.842+0000 7fa1b9ffb700 1 -- 172.21.15.46:0/2539898020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa1ac003cd0 con 0x7fa1bc108990 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.842+0000 7fa1c2d96700 1 -- 172.21.15.46:0/2539898020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1bc19d600 con 0x7fa1bc108990 2024-05-09T00:00:20.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.842+0000 7fa1b9ffb700 1 -- 172.21.15.46:0/2539898020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac0193c0 con 0x7fa1bc108990 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.842+0000 7fa1c2d96700 1 -- 172.21.15.46:0/2539898020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1bc07c850 con 0x7fa1bc108990 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.843+0000 7fa1b9ffb700 1 -- 172.21.15.46:0/2539898020 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa1ac019a10 con 0x7fa1bc108990 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.843+0000 7fa1b9ffb700 1 -- 172.21.15.46:0/2539898020 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa1ac0243f0 con 0x7fa1bc108990 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.843+0000 7fa1c2d96700 1 -- 172.21.15.46:0/2539898020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1bc19da60 con 0x7fa1bc108990 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.846+0000 7fa1b9ffb700 1 -- 172.21.15.46:0/2539898020 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa1ac043ec0 con 0x7fa1bc108990 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.917+0000 7fa1c2d96700 1 -- 172.21.15.46:0/2539898020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fa1bc064e20 con 0x7fa1bc108990 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.919+0000 7fa1b9ffb700 1 -- 172.21.15.46:0/2539898020 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fa1ac032020 con 0x7fa1bc108990 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.920+0000 7fa1c2d96700 1 -- 172.21.15.46:0/2539898020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 msgr2=0x7fa1bc19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.920+0000 7fa1c2d96700 1 --2- 172.21.15.46:0/2539898020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 0x7fa1bc19ce30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac011670 tx=0x7fa1ac004280 comp rx=0 tx=0).stop 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.920+0000 7fa1c2d96700 1 -- 172.21.15.46:0/2539898020 shutdown_connections 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.920+0000 7fa1c2d96700 1 --2- 172.21.15.46:0/2539898020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1bc108990 0x7fa1bc19ce30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.920+0000 7fa1c2d96700 1 -- 172.21.15.46:0/2539898020 >> 172.21.15.46:0/2539898020 conn(0x7fa1bc103ff0 msgr2=0x7fa1bc104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.920+0000 7fa1c2d96700 1 -- 172.21.15.46:0/2539898020 shutdown_connections 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:19.920+0000 7fa1c2d96700 1 -- 172.21.15.46:0/2539898020 wait complete. 2024-05-09T00:00:20.445 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-09T00:00:20.468 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:00:20.468 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mgr... 2024-05-09T00:00:20.468 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-09T00:00:20.469 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-09T00:00:20.734 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.x 2024-05-09T00:00:20.734 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-09T00:00:20.893 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-eec67860-0d96-11ef-bc97-c7b262605968.target.wants/ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-eec67860-0d96-11ef-bc97-c7b262605968@.service. 2024-05-09T00:00:21.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:20 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2539898020' entity='client.admin' 2024-05-09T00:00:21.057 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:20 smithi046 systemd[1]: Starting Ceph mgr.x for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:00:21.423 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:21 smithi046 podman[30367]: 2024-05-09 00:00:21.044168659 +0000 UTC m=+0.018518050 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:00:21.423 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:21 smithi046 podman[30367]: 2024-05-09 00:00:21.17705841 +0000 UTC m=+0.151407822 container create 0ab4817301559982aba657317959817b69f3f40ce0365aa815183f3309723797 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-09T00:00:21.423 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:21 smithi046 podman[30367]: 2024-05-09 00:00:21.354486634 +0000 UTC m=+0.328836015 container init 0ab4817301559982aba657317959817b69f3f40ce0365aa815183f3309723797 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-09T00:00:21.423 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:21 smithi046 podman[30367]: 2024-05-09 00:00:21.357306636 +0000 UTC m=+0.331656016 container start 0ab4817301559982aba657317959817b69f3f40ce0365aa815183f3309723797 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True) 2024-05-09T00:00:21.423 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:21 smithi046 bash[30367]: 0ab4817301559982aba657317959817b69f3f40ce0365aa815183f3309723797 2024-05-09T00:00:21.438 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T00:00:21.438 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-05-09T00:00:21.446 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T00:00:21.446 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-09T00:00:21.447 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-05-09T00:00:21.447 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T00:00:21.455 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T00:00:21.456 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-05-09T00:00:21.463 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T00:00:21.463 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-09T00:00:21.463 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-05-09T00:00:21.463 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-09T00:00:21.464 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr to start... 2024-05-09T00:00:21.464 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr... 2024-05-09T00:00:21.709 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:21 smithi046 systemd[1]: Started Ceph mgr.x for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:00:21.709 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:21 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:21.528+0000 7fd698768200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T00:00:22.024 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:21 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:21.727+0000 7fd698768200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:00:22.024 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:21 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:21.834+0000 7fd698768200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T00:00:22.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:22 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2736909027' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T00:00:22.307 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:22 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:22.172+0000 7fd698768200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T00:00:22.307 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:22 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:22.266+0000 7fd698768200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T00:00:22.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:22.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:00:22.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "eec67860-0d96-11ef-bc97-c7b262605968", 2024-05-09T00:00:22.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T00:00:22.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T00:00:22.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T00:00:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T00:00:22.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T00:00:22.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T00:00:14.357151+0000", 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:22.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.947+0000 7ff69ba97700 1 Processor -- start 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.948+0000 7ff69ba97700 1 -- start start 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.948+0000 7ff69ba97700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 0x7ff69410c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.948+0000 7ff69ba97700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff69407ce70 con 0x7ff69410a550 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.948+0000 7ff699833700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 0x7ff69410c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.948+0000 7ff699833700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 0x7ff69410c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48762/0 (socket says 172.21.15.46:48762) 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.948+0000 7ff699833700 1 -- 172.21.15.46:0/3239783144 learned_addr learned my addr 172.21.15.46:0/3239783144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.949+0000 7ff699833700 1 -- 172.21.15.46:0/3239783144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69410ceb0 con 0x7ff69410a550 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.949+0000 7ff699833700 1 --2- 172.21.15.46:0/3239783144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 0x7ff69410c970 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff6900098a0 tx=0x7ff690009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2454ae1dd5941597 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.949+0000 7ff698831700 1 -- 172.21.15.46:0/3239783144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff690004160 con 0x7ff69410a550 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.949+0000 7ff698831700 1 -- 172.21.15.46:0/3239783144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff690004300 con 0x7ff69410a550 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.949+0000 7ff698831700 1 -- 172.21.15.46:0/3239783144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff690011af0 con 0x7ff69410a550 2024-05-09T00:00:22.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.950+0000 7ff69ba97700 1 -- 172.21.15.46:0/3239783144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 msgr2=0x7ff69410c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.950+0000 7ff69ba97700 1 --2- 172.21.15.46:0/3239783144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 0x7ff69410c970 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff6900098a0 tx=0x7ff690009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.950+0000 7ff69ba97700 1 -- 172.21.15.46:0/3239783144 shutdown_connections 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.950+0000 7ff69ba97700 1 --2- 172.21.15.46:0/3239783144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 0x7ff69410c970 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.950+0000 7ff69ba97700 1 -- 172.21.15.46:0/3239783144 >> 172.21.15.46:0/3239783144 conn(0x7ff694104050 msgr2=0x7ff6941064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.950+0000 7ff69ba97700 1 -- 172.21.15.46:0/3239783144 shutdown_connections 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.950+0000 7ff69ba97700 1 -- 172.21.15.46:0/3239783144 wait complete. 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.951+0000 7ff69ba97700 1 Processor -- start 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.951+0000 7ff69ba97700 1 -- start start 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.951+0000 7ff69ba97700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 0x7ff694109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.951+0000 7ff69ba97700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff69410de70 con 0x7ff69410a550 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.951+0000 7ff699833700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 0x7ff694109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.951+0000 7ff699833700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 0x7ff694109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48774/0 (socket says 172.21.15.46:48774) 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.951+0000 7ff699833700 1 -- 172.21.15.46:0/2736909027 learned_addr learned my addr 172.21.15.46:0/2736909027 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:22.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.952+0000 7ff699833700 1 -- 172.21.15.46:0/2736909027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69410a210 con 0x7ff69410a550 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.952+0000 7ff699833700 1 --2- 172.21.15.46:0/2736909027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 0x7ff694109cd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff690009fb0 tx=0x7ff690011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.952+0000 7ff68affd700 1 -- 172.21.15.46:0/2736909027 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff690011bc0 con 0x7ff69410a550 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.952+0000 7ff68affd700 1 -- 172.21.15.46:0/2736909027 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff690024440 con 0x7ff69410a550 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.952+0000 7ff68affd700 1 -- 172.21.15.46:0/2736909027 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff69001a5c0 con 0x7ff69410a550 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.952+0000 7ff69ba97700 1 -- 172.21.15.46:0/2736909027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff694108460 con 0x7ff69410a550 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.952+0000 7ff69ba97700 1 -- 172.21.15.46:0/2736909027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff694108940 con 0x7ff69410a550 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.952+0000 7ff68affd700 1 -- 172.21.15.46:0/2736909027 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff69001abc0 con 0x7ff69410a550 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.952+0000 7ff68affd700 1 -- 172.21.15.46:0/2736909027 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff690019700 con 0x7ff69410a550 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.953+0000 7ff69ba97700 1 -- 172.21.15.46:0/2736909027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff694064e20 con 0x7ff69410a550 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:21.955+0000 7ff68affd700 1 -- 172.21.15.46:0/2736909027 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff69002dd90 con 0x7ff69410a550 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:22.029+0000 7ff69ba97700 1 -- 172.21.15.46:0/2736909027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff694108bc0 con 0x7ff69410a550 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:22.030+0000 7ff68affd700 1 -- 172.21.15.46:0/2736909027 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff6900245f0 con 0x7ff69410a550 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:22.031+0000 7ff69ba97700 1 -- 172.21.15.46:0/2736909027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 msgr2=0x7ff694109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:22.031+0000 7ff69ba97700 1 --2- 172.21.15.46:0/2736909027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 0x7ff694109cd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff690009fb0 tx=0x7ff690011890 comp rx=0 tx=0).stop 2024-05-09T00:00:22.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:22.031+0000 7ff69ba97700 1 -- 172.21.15.46:0/2736909027 shutdown_connections 2024-05-09T00:00:22.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:22.031+0000 7ff69ba97700 1 --2- 172.21.15.46:0/2736909027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff69410a550 0x7ff694109cd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:22.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:22.031+0000 7ff69ba97700 1 -- 172.21.15.46:0/2736909027 >> 172.21.15.46:0/2736909027 conn(0x7ff694104050 msgr2=0x7ff694105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:22.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:22.031+0000 7ff69ba97700 1 -- 172.21.15.46:0/2736909027 shutdown_connections 2024-05-09T00:00:22.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:22.032+0000 7ff69ba97700 1 -- 172.21.15.46:0/2736909027 wait complete. 2024-05-09T00:00:22.667 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (1/15)... 2024-05-09T00:00:23.807 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:23 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:23.535+0000 7fd698768200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T00:00:23.807 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:23 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:23.712+0000 7fd698768200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T00:00:24.183 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:23 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:23.889+0000 7fd698768200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T00:00:24.183 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:24 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:24.189+0000 7fd698768200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T00:00:24.490 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:24 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:24.285+0000 7fd698768200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T00:00:24.490 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:24 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:24.496+0000 7fd698768200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T00:00:24.780 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:24 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:24.704+0000 7fd698768200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:00:25.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:25 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1441167893' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T00:00:25.807 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:25 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:25.349+0000 7fd698768200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T00:00:25.807 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:25 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:25.580+0000 7fd698768200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T00:00:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:00:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "eec67860-0d96-11ef-bc97-c7b262605968", 2024-05-09T00:00:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T00:00:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T00:00:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T00:00:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T00:00:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T00:00:25.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:25.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T00:00:25.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T00:00:25.893 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T00:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T00:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T00:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T00:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T00:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T00:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T00:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T00:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T00:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T00:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T00:00:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T00:00:14.357151+0000", 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.152+0000 7f56d6540700 1 Processor -- start 2024-05-09T00:00:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.152+0000 7f56d6540700 1 -- start start 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.152+0000 7f56d6540700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 0x7f56d0108da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.153+0000 7f56d6540700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0109390 con 0x7f56d0108990 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.153+0000 7f56cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 0x7f56d0108da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.153+0000 7f56cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 0x7f56d0108da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48778/0 (socket says 172.21.15.46:48778) 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.153+0000 7f56cffff700 1 -- 172.21.15.46:0/1305679308 learned_addr learned my addr 172.21.15.46:0/1305679308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.153+0000 7f56cffff700 1 -- 172.21.15.46:0/1305679308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d0109b60 con 0x7f56d0108990 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.153+0000 7f56cffff700 1 --2- 172.21.15.46:0/1305679308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 0x7f56d0108da0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56c00098a0 tx=0x7f56c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b32fd8614798a58 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.153+0000 7f56ceffd700 1 -- 172.21.15.46:0/1305679308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56c0004160 con 0x7f56d0108990 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.153+0000 7f56ceffd700 1 -- 172.21.15.46:0/1305679308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f56c0004300 con 0x7f56d0108990 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.153+0000 7f56ceffd700 1 -- 172.21.15.46:0/1305679308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56c0011af0 con 0x7f56d0108990 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.154+0000 7f56d6540700 1 -- 172.21.15.46:0/1305679308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 msgr2=0x7f56d0108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.154+0000 7f56d6540700 1 --2- 172.21.15.46:0/1305679308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 0x7f56d0108da0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56c00098a0 tx=0x7f56c0009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.154+0000 7f56d6540700 1 -- 172.21.15.46:0/1305679308 shutdown_connections 2024-05-09T00:00:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.154+0000 7f56d6540700 1 --2- 172.21.15.46:0/1305679308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 0x7f56d0108da0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.154+0000 7f56d6540700 1 -- 172.21.15.46:0/1305679308 >> 172.21.15.46:0/1305679308 conn(0x7f56d0103ff0 msgr2=0x7f56d0106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.154+0000 7f56d6540700 1 -- 172.21.15.46:0/1305679308 shutdown_connections 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.154+0000 7f56d6540700 1 -- 172.21.15.46:0/1305679308 wait complete. 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.155+0000 7f56d6540700 1 Processor -- start 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.155+0000 7f56d6540700 1 -- start start 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.156+0000 7f56d6540700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 0x7f56d019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.156+0000 7f56d6540700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f56d010cf10 con 0x7f56d0108990 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.156+0000 7f56cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 0x7f56d019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.156+0000 7f56cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 0x7f56d019dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48792/0 (socket says 172.21.15.46:48792) 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.156+0000 7f56cffff700 1 -- 172.21.15.46:0/1441167893 learned_addr learned my addr 172.21.15.46:0/1441167893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.156+0000 7f56cffff700 1 -- 172.21.15.46:0/1441167893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d019e4f0 con 0x7f56d0108990 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.156+0000 7f56cffff700 1 --2- 172.21.15.46:0/1441167893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 0x7f56d019dfb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f56c0011670 tx=0x7f56c0004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.156+0000 7f56cd7fa700 1 -- 172.21.15.46:0/1441167893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56c00036a0 con 0x7f56d0108990 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.156+0000 7f56cd7fa700 1 -- 172.21.15.46:0/1441167893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f56c0003cd0 con 0x7f56d0108990 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.156+0000 7f56d6540700 1 -- 172.21.15.46:0/1441167893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d019e780 con 0x7f56d0108990 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.157+0000 7f56cd7fa700 1 -- 172.21.15.46:0/1441167893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56c00193c0 con 0x7f56d0108990 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.157+0000 7f56d6540700 1 -- 172.21.15.46:0/1441167893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d01a1390 con 0x7f56d0108990 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.157+0000 7f56cd7fa700 1 -- 172.21.15.46:0/1441167893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f56c0019a10 con 0x7f56d0108990 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.157+0000 7f56cd7fa700 1 -- 172.21.15.46:0/1441167893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f56c00243f0 con 0x7f56d0108990 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.157+0000 7f56d6540700 1 -- 172.21.15.46:0/1441167893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d0064e20 con 0x7f56d0108990 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.160+0000 7f56cd7fa700 1 -- 172.21.15.46:0/1441167893 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f56c001a430 con 0x7f56d0108990 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.234+0000 7f56d6540700 1 -- 172.21.15.46:0/1441167893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f56d002e8f0 con 0x7f56d0108990 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.235+0000 7f56cd7fa700 1 -- 172.21.15.46:0/1441167893 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f56c0032030 con 0x7f56d0108990 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.236+0000 7f56d6540700 1 -- 172.21.15.46:0/1441167893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 msgr2=0x7f56d019dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.236+0000 7f56d6540700 1 --2- 172.21.15.46:0/1441167893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 0x7f56d019dfb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f56c0011670 tx=0x7f56c0004280 comp rx=0 tx=0).stop 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.236+0000 7f56d6540700 1 -- 172.21.15.46:0/1441167893 shutdown_connections 2024-05-09T00:00:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.236+0000 7f56d6540700 1 --2- 172.21.15.46:0/1441167893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56d0108990 0x7f56d019dfb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:25.907 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.236+0000 7f56d6540700 1 -- 172.21.15.46:0/1441167893 >> 172.21.15.46:0/1441167893 conn(0x7f56d0103ff0 msgr2=0x7f56d0194530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:25.907 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.236+0000 7f56d6540700 1 -- 172.21.15.46:0/1441167893 shutdown_connections 2024-05-09T00:00:25.907 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:25.236+0000 7f56d6540700 1 -- 172.21.15.46:0/1441167893 wait complete. 2024-05-09T00:00:25.907 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (2/15)... 2024-05-09T00:00:26.121 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:26 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:26.025+0000 7fd698768200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T00:00:26.121 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:26 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:26.126+0000 7fd698768200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T00:00:26.806 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:26 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:26.511+0000 7fd698768200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T00:00:27.231 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:26 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:26.886+0000 7fd698768200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T00:00:27.232 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:26 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:26.983+0000 7fd698768200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T00:00:27.557 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:27 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:27.237+0000 7fd698768200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T00:00:27.557 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:27 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:27.377+0000 7fd698768200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T00:00:27.557 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:27 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:27.476+0000 7fd698768200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T00:00:28.114 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:27 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:27.834+0000 7fd698768200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T00:00:28.114 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:27 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:27.933+0000 7fd698768200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T00:00:28.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:28 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3663270556' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T00:00:28.807 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:28 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:28.584+0000 7fd698768200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T00:00:28.962 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:28.962 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:00:28.962 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "eec67860-0d96-11ef-bc97-c7b262605968", 2024-05-09T00:00:28.962 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T00:00:28.962 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T00:00:28.962 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T00:00:28.962 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T00:00:28.962 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:28.963 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T00:00:28.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T00:00:28.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T00:00:28.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T00:00:28.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:28.973 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:00:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T00:00:14.357151+0000", 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.316+0000 7f5608ac8700 1 Processor -- start 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.316+0000 7f5608ac8700 1 -- start start 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.316+0000 7f5608ac8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 0x7f5604109e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.316+0000 7f5608ac8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f560410a460 con 0x7f5604109a60 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.316+0000 7f560259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 0x7f5604109e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.316+0000 7f560259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 0x7f5604109e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42924/0 (socket says 172.21.15.46:42924) 2024-05-09T00:00:28.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.316+0000 7f560259c700 1 -- 172.21.15.46:0/2136525489 learned_addr learned my addr 172.21.15.46:0/2136525489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.317+0000 7f560259c700 1 -- 172.21.15.46:0/2136525489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560410ac30 con 0x7f5604109a60 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.317+0000 7f560259c700 1 --2- 172.21.15.46:0/2136525489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 0x7f5604109e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55f40098a0 tx=0x7f55f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe3abcf18bdfc91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.317+0000 7f560159a700 1 -- 172.21.15.46:0/2136525489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55f4004160 con 0x7f5604109a60 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.317+0000 7f560159a700 1 -- 172.21.15.46:0/2136525489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f55f4004300 con 0x7f5604109a60 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.317+0000 7f560159a700 1 -- 172.21.15.46:0/2136525489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55f4011af0 con 0x7f5604109a60 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.318+0000 7f5608ac8700 1 -- 172.21.15.46:0/2136525489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 msgr2=0x7f5604109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.318+0000 7f5608ac8700 1 --2- 172.21.15.46:0/2136525489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 0x7f5604109e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55f40098a0 tx=0x7f55f4009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.318+0000 7f5608ac8700 1 -- 172.21.15.46:0/2136525489 shutdown_connections 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.318+0000 7f5608ac8700 1 --2- 172.21.15.46:0/2136525489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 0x7f5604109e70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.318+0000 7f5608ac8700 1 -- 172.21.15.46:0/2136525489 >> 172.21.15.46:0/2136525489 conn(0x7f5604105120 msgr2=0x7f5604107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.318+0000 7f5608ac8700 1 -- 172.21.15.46:0/2136525489 shutdown_connections 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.318+0000 7f5608ac8700 1 -- 172.21.15.46:0/2136525489 wait complete. 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.319+0000 7f5608ac8700 1 Processor -- start 2024-05-09T00:00:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.319+0000 7f5608ac8700 1 -- start start 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.319+0000 7f5608ac8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 0x7f560419df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.319+0000 7f5608ac8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f560410dfe0 con 0x7f5604109a60 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.319+0000 7f560259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 0x7f560419df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.319+0000 7f560259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 0x7f560419df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42932/0 (socket says 172.21.15.46:42932) 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.319+0000 7f560259c700 1 -- 172.21.15.46:0/3663270556 learned_addr learned my addr 172.21.15.46:0/3663270556 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.320+0000 7f560259c700 1 -- 172.21.15.46:0/3663270556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560419e440 con 0x7f5604109a60 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.320+0000 7f560259c700 1 --2- 172.21.15.46:0/3663270556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 0x7f560419df00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f55f40090e0 tx=0x7f55f4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.320+0000 7f55f37fe700 1 -- 172.21.15.46:0/3663270556 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55f40039e0 con 0x7f5604109a60 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.320+0000 7f5608ac8700 1 -- 172.21.15.46:0/3663270556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560419e6d0 con 0x7f5604109a60 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.320+0000 7f55f37fe700 1 -- 172.21.15.46:0/3663270556 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f55f4003b80 con 0x7f5604109a60 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.320+0000 7f55f37fe700 1 -- 172.21.15.46:0/3663270556 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55f4024690 con 0x7f5604109a60 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.320+0000 7f5608ac8700 1 -- 172.21.15.46:0/3663270556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56041a12e0 con 0x7f5604109a60 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.321+0000 7f55f37fe700 1 -- 172.21.15.46:0/3663270556 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f55f4022020 con 0x7f5604109a60 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.321+0000 7f55f37fe700 1 -- 172.21.15.46:0/3663270556 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f55f4019dd0 con 0x7f5604109a60 2024-05-09T00:00:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.321+0000 7f5608ac8700 1 -- 172.21.15.46:0/3663270556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5604064e20 con 0x7f5604109a60 2024-05-09T00:00:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.324+0000 7f55f37fe700 1 -- 172.21.15.46:0/3663270556 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f55f4024cd0 con 0x7f5604109a60 2024-05-09T00:00:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.399+0000 7f5608ac8700 1 -- 172.21.15.46:0/3663270556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f56041a14b0 con 0x7f5604109a60 2024-05-09T00:00:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.399+0000 7f55f37fe700 1 -- 172.21.15.46:0/3663270556 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f55f4032090 con 0x7f5604109a60 2024-05-09T00:00:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.400+0000 7f5608ac8700 1 -- 172.21.15.46:0/3663270556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 msgr2=0x7f560419df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.400+0000 7f5608ac8700 1 --2- 172.21.15.46:0/3663270556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 0x7f560419df00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f55f40090e0 tx=0x7f55f4011940 comp rx=0 tx=0).stop 2024-05-09T00:00:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.400+0000 7f5608ac8700 1 -- 172.21.15.46:0/3663270556 shutdown_connections 2024-05-09T00:00:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.400+0000 7f5608ac8700 1 --2- 172.21.15.46:0/3663270556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5604109a60 0x7f560419df00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.400+0000 7f5608ac8700 1 -- 172.21.15.46:0/3663270556 >> 172.21.15.46:0/3663270556 conn(0x7f5604105120 msgr2=0x7f5604106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.401+0000 7f5608ac8700 1 -- 172.21.15.46:0/3663270556 shutdown_connections 2024-05-09T00:00:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:28.401+0000 7f5608ac8700 1 -- 172.21.15.46:0/3663270556 wait complete. 2024-05-09T00:00:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (3/15)... 2024-05-09T00:00:29.307 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:28 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:28.814+0000 7fd698768200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T00:00:29.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: Activating manager daemon x 2024-05-09T00:00:29.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: mgrmap e2: x(active, starting, since 0.00366253s) 2024-05-09T00:00:29.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: from='mgr.14100 172.21.15.46:0/131978291' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T00:00:29.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: from='mgr.14100 172.21.15.46:0/131978291' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T00:00:29.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: from='mgr.14100 172.21.15.46:0/131978291' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T00:00:29.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: from='mgr.14100 172.21.15.46:0/131978291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:00:29.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: from='mgr.14100 172.21.15.46:0/131978291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T00:00:29.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: Manager daemon x is now available 2024-05-09T00:00:29.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: from='mgr.14100 172.21.15.46:0/131978291' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T00:00:29.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: from='mgr.14100 172.21.15.46:0/131978291' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T00:00:29.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: from='mgr.14100 172.21.15.46:0/131978291' entity='mgr.x' 2024-05-09T00:00:29.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: from='mgr.14100 172.21.15.46:0/131978291' entity='mgr.x' 2024-05-09T00:00:29.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:29 smithi046 ceph-mon[30151]: from='mgr.14100 172.21.15.46:0/131978291' entity='mgr.x' 2024-05-09T00:00:31.093 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:30 smithi046 ceph-mon[30151]: mgrmap e3: x(active, since 1.00813s) 2024-05-09T00:00:32.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:31 smithi046 ceph-mon[30151]: mgrmap e4: x(active, since 2s) 2024-05-09T00:00:32.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:31 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/14378585' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T00:00:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:00:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "eec67860-0d96-11ef-bc97-c7b262605968", 2024-05-09T00:00:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T00:00:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T00:00:32.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T00:00:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T00:00:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T00:00:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T00:00:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T00:00:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T00:00:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T00:00:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T00:00:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:32.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T00:00:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T00:00:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T00:00:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T00:00:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T00:00:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T00:00:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T00:00:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T00:00:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T00:00:14.357151+0000", 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.406+0000 7f4d1d4b7700 1 Processor -- start 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.406+0000 7f4d1d4b7700 1 -- start start 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.406+0000 7f4d1d4b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 0x7f4d1810c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:32.273 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.406+0000 7f4d1d4b7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1807ce70 con 0x7f4d1810a550 2024-05-09T00:00:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.407+0000 7f4d16ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 0x7f4d1810c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.407+0000 7f4d16ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 0x7f4d1810c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42990/0 (socket says 172.21.15.46:42990) 2024-05-09T00:00:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.407+0000 7f4d16ffd700 1 -- 172.21.15.46:0/1688984769 learned_addr learned my addr 172.21.15.46:0/1688984769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.407+0000 7f4d16ffd700 1 -- 172.21.15.46:0/1688984769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1810ceb0 con 0x7f4d1810a550 2024-05-09T00:00:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.407+0000 7f4d16ffd700 1 --2- 172.21.15.46:0/1688984769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 0x7f4d1810c970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4d080098a0 tx=0x7f4d08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9a7f1451b40156 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.407+0000 7f4d15ffb700 1 -- 172.21.15.46:0/1688984769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d08004160 con 0x7f4d1810a550 2024-05-09T00:00:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.407+0000 7f4d15ffb700 1 -- 172.21.15.46:0/1688984769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4d08004300 con 0x7f4d1810a550 2024-05-09T00:00:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.407+0000 7f4d15ffb700 1 -- 172.21.15.46:0/1688984769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d08011af0 con 0x7f4d1810a550 2024-05-09T00:00:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.408+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/1688984769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 msgr2=0x7f4d1810c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.408+0000 7f4d1d4b7700 1 --2- 172.21.15.46:0/1688984769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 0x7f4d1810c970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4d080098a0 tx=0x7f4d08009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:32.274 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.408+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/1688984769 shutdown_connections 2024-05-09T00:00:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.408+0000 7f4d1d4b7700 1 --2- 172.21.15.46:0/1688984769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 0x7f4d1810c970 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.408+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/1688984769 >> 172.21.15.46:0/1688984769 conn(0x7f4d18104050 msgr2=0x7f4d181064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.408+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/1688984769 shutdown_connections 2024-05-09T00:00:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.408+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/1688984769 wait complete. 2024-05-09T00:00:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.409+0000 7f4d1d4b7700 1 Processor -- start 2024-05-09T00:00:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.409+0000 7f4d1d4b7700 1 -- start start 2024-05-09T00:00:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.409+0000 7f4d1d4b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 0x7f4d1819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.409+0000 7f4d1d4b7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1810de70 con 0x7f4d1810a550 2024-05-09T00:00:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.409+0000 7f4d16ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 0x7f4d1819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.409+0000 7f4d16ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 0x7f4d1819dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42992/0 (socket says 172.21.15.46:42992) 2024-05-09T00:00:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.409+0000 7f4d16ffd700 1 -- 172.21.15.46:0/14378585 learned_addr learned my addr 172.21.15.46:0/14378585 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.410+0000 7f4d16ffd700 1 -- 172.21.15.46:0/14378585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1819e400 con 0x7f4d1810a550 2024-05-09T00:00:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.410+0000 7f4d16ffd700 1 --2- 172.21.15.46:0/14378585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 0x7f4d1819dec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009fb0 tx=0x7f4d08011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.410+0000 7f4cfffff700 1 -- 172.21.15.46:0/14378585 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d08011bc0 con 0x7f4d1810a550 2024-05-09T00:00:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.410+0000 7f4cfffff700 1 -- 172.21.15.46:0/14378585 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4d08024440 con 0x7f4d1810a550 2024-05-09T00:00:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.410+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/14378585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1819e690 con 0x7f4d1810a550 2024-05-09T00:00:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.410+0000 7f4cfffff700 1 -- 172.21.15.46:0/14378585 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d0801a5c0 con 0x7f4d1810a550 2024-05-09T00:00:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.410+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/14378585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d181a12a0 con 0x7f4d1810a550 2024-05-09T00:00:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.411+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/14378585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d18064e20 con 0x7f4d1810a550 2024-05-09T00:00:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.411+0000 7f4cfffff700 1 -- 172.21.15.46:0/14378585 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f4d0801a880 con 0x7f4d1810a550 2024-05-09T00:00:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.411+0000 7f4cfffff700 1 --2- 172.21.15.46:0/14378585 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7f4d0003e3d0 0x7f4d000408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:32.276 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.412+0000 7f4cfffff700 1 -- 172.21.15.46:0/14378585 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4d08053770 con 0x7f4d1810a550 2024-05-09T00:00:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.412+0000 7f4d167fc700 1 --2- 172.21.15.46:0/14378585 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7f4d0003e3d0 0x7f4d000408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.412+0000 7f4d167fc700 1 --2- 172.21.15.46:0/14378585 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7f4d0003e3d0 0x7f4d000408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c006fd0 tx=0x7f4d0c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.417+0000 7f4cfffff700 1 -- 172.21.15.46:0/14378585 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d0800b3e0 con 0x7f4d1810a550 2024-05-09T00:00:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.719+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/14378585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4d181a1590 con 0x7f4d1810a550 2024-05-09T00:00:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.719+0000 7f4cfffff700 1 -- 172.21.15.46:0/14378585 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4d08029df0 con 0x7f4d1810a550 2024-05-09T00:00:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.721+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/14378585 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7f4d0003e3d0 msgr2=0x7f4d000408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.721+0000 7f4d1d4b7700 1 --2- 172.21.15.46:0/14378585 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7f4d0003e3d0 0x7f4d000408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c006fd0 tx=0x7f4d0c006e40 comp rx=0 tx=0).stop 2024-05-09T00:00:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.721+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/14378585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 msgr2=0x7f4d1819dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.721+0000 7f4d1d4b7700 1 --2- 172.21.15.46:0/14378585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 0x7f4d1819dec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009fb0 tx=0x7f4d08011890 comp rx=0 tx=0).stop 2024-05-09T00:00:32.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.722+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/14378585 shutdown_connections 2024-05-09T00:00:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.722+0000 7f4d1d4b7700 1 --2- 172.21.15.46:0/14378585 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7f4d0003e3d0 0x7f4d000408e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.722+0000 7f4d1d4b7700 1 --2- 172.21.15.46:0/14378585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1810a550 0x7f4d1819dec0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.722+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/14378585 >> 172.21.15.46:0/14378585 conn(0x7f4d18104050 msgr2=0x7f4d18105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.722+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/14378585 shutdown_connections 2024-05-09T00:00:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:31.722+0000 7f4d1d4b7700 1 -- 172.21.15.46:0/14378585 wait complete. 2024-05-09T00:00:32.278 INFO:teuthology.orchestra.run.smithi046.stdout:mgr is available 2024-05-09T00:00:33.238 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:33 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1703670054' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T00:00:33.499 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:33.499 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T00:00:33.499 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:00:33.499 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T00:00:33.499 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T00:00:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T00:00:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T00:00:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T00:00:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T00:00:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T00:00:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T00:00:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T00:00:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T00:00:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T00:00:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T00:00:33.500 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.723+0000 7ff31cf90700 1 Processor -- start 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.723+0000 7ff31cf90700 1 -- start start 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.723+0000 7ff31cf90700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 0x7ff318108da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.723+0000 7ff31cf90700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff318109390 con 0x7ff318108990 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.723+0000 7ff31659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 0x7ff318108da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.723+0000 7ff31659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 0x7ff318108da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43002/0 (socket says 172.21.15.46:43002) 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.723+0000 7ff31659c700 1 -- 172.21.15.46:0/649449214 learned_addr learned my addr 172.21.15.46:0/649449214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.724+0000 7ff31659c700 1 -- 172.21.15.46:0/649449214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff318109b60 con 0x7ff318108990 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.724+0000 7ff31659c700 1 --2- 172.21.15.46:0/649449214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 0x7ff318108da0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3080098a0 tx=0x7ff308009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa8640891e05763 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.724+0000 7ff315d9b700 1 -- 172.21.15.46:0/649449214 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff308004160 con 0x7ff318108990 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.724+0000 7ff315d9b700 1 -- 172.21.15.46:0/649449214 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff308004300 con 0x7ff318108990 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.724+0000 7ff315d9b700 1 -- 172.21.15.46:0/649449214 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff308011af0 con 0x7ff318108990 2024-05-09T00:00:33.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.725+0000 7ff31cf90700 1 -- 172.21.15.46:0/649449214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 msgr2=0x7ff318108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.725+0000 7ff31cf90700 1 --2- 172.21.15.46:0/649449214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 0x7ff318108da0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3080098a0 tx=0x7ff308009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.725+0000 7ff31cf90700 1 -- 172.21.15.46:0/649449214 shutdown_connections 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.725+0000 7ff31cf90700 1 --2- 172.21.15.46:0/649449214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 0x7ff318108da0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.725+0000 7ff31cf90700 1 -- 172.21.15.46:0/649449214 >> 172.21.15.46:0/649449214 conn(0x7ff318104030 msgr2=0x7ff318106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.725+0000 7ff31cf90700 1 -- 172.21.15.46:0/649449214 shutdown_connections 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.725+0000 7ff31cf90700 1 -- 172.21.15.46:0/649449214 wait complete. 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.726+0000 7ff31cf90700 1 Processor -- start 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.726+0000 7ff31cf90700 1 -- start start 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.726+0000 7ff31cf90700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 0x7ff31807d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.726+0000 7ff31cf90700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff31810cd90 con 0x7ff318108990 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.726+0000 7ff31659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 0x7ff31807d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.726+0000 7ff31659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 0x7ff31807d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43010/0 (socket says 172.21.15.46:43010) 2024-05-09T00:00:33.502 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.726+0000 7ff31659c700 1 -- 172.21.15.46:0/1703670054 learned_addr learned my addr 172.21.15.46:0/1703670054 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.727+0000 7ff31659c700 1 -- 172.21.15.46:0/1703670054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31807d8d0 con 0x7ff318108990 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.727+0000 7ff31659c700 1 --2- 172.21.15.46:0/1703670054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 0x7ff31807d390 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff3080090e0 tx=0x7ff308011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.727+0000 7ff30f7fe700 1 -- 172.21.15.46:0/1703670054 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3080039e0 con 0x7ff318108990 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.727+0000 7ff31cf90700 1 -- 172.21.15.46:0/1703670054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3181a0cd0 con 0x7ff318108990 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.727+0000 7ff30f7fe700 1 -- 172.21.15.46:0/1703670054 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff308003b80 con 0x7ff318108990 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.727+0000 7ff30f7fe700 1 -- 172.21.15.46:0/1703670054 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff308024690 con 0x7ff318108990 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.727+0000 7ff31cf90700 1 -- 172.21.15.46:0/1703670054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3181a1150 con 0x7ff318108990 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.728+0000 7ff30f7fe700 1 -- 172.21.15.46:0/1703670054 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7ff308022020 con 0x7ff318108990 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.728+0000 7ff30f7fe700 1 --2- 172.21.15.46:0/1703670054 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7ff30403e420 0x7ff304040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.728+0000 7ff30f7fe700 1 -- 172.21.15.46:0/1703670054 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff308054330 con 0x7ff318108990 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.728+0000 7ff31cf90700 1 -- 172.21.15.46:0/1703670054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff318064e20 con 0x7ff318108990 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.728+0000 7ff30ffff700 1 --2- 172.21.15.46:0/1703670054 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7ff30403e420 0x7ff304040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.729+0000 7ff30ffff700 1 --2- 172.21.15.46:0/1703670054 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7ff30403e420 0x7ff304040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff300006fd0 tx=0x7ff300006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:33.503 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.734+0000 7ff30f7fe700 1 -- 172.21.15.46:0/1703670054 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff308054090 con 0x7ff318108990 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.948+0000 7ff31cf90700 1 -- 172.21.15.46:0/1703670054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff31802ea40 con 0x7ff318108990 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.949+0000 7ff30f7fe700 1 -- 172.21.15.46:0/1703670054 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7ff30800fdd0 con 0x7ff318108990 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.952+0000 7ff31cf90700 1 -- 172.21.15.46:0/1703670054 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7ff30403e420 msgr2=0x7ff304040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.952+0000 7ff31cf90700 1 --2- 172.21.15.46:0/1703670054 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7ff30403e420 0x7ff304040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff300006fd0 tx=0x7ff300006e40 comp rx=0 tx=0).stop 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.952+0000 7ff31cf90700 1 -- 172.21.15.46:0/1703670054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 msgr2=0x7ff31807d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.952+0000 7ff31cf90700 1 --2- 172.21.15.46:0/1703670054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 0x7ff31807d390 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff3080090e0 tx=0x7ff308011940 comp rx=0 tx=0).stop 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.952+0000 7ff31cf90700 1 -- 172.21.15.46:0/1703670054 shutdown_connections 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.952+0000 7ff31cf90700 1 --2- 172.21.15.46:0/1703670054 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7ff30403e420 0x7ff304040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.952+0000 7ff31cf90700 1 --2- 172.21.15.46:0/1703670054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff318108990 0x7ff31807d390 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.952+0000 7ff31cf90700 1 -- 172.21.15.46:0/1703670054 >> 172.21.15.46:0/1703670054 conn(0x7ff318104030 msgr2=0x7ff318104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.952+0000 7ff31cf90700 1 -- 172.21.15.46:0/1703670054 shutdown_connections 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:32.952+0000 7ff31cf90700 1 -- 172.21.15.46:0/1703670054 wait complete. 2024-05-09T00:00:33.504 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling cephadm module... 2024-05-09T00:00:34.488 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:34 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1721281294' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-09T00:00:34.488 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:34 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: ignoring --setuser ceph since I am not root 2024-05-09T00:00:34.488 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:34 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: ignoring --setgroup ceph since I am not root 2024-05-09T00:00:34.488 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:34 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:34.494+0000 7fe2b319c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T00:00:34.793 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:34 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:34.690+0000 7fe2b319c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:00:34.793 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:34 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:34.798+0000 7fe2b319c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T00:00:34.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.947+0000 7ff9c19d2700 1 Processor -- start 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.947+0000 7ff9c19d2700 1 -- start start 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.947+0000 7ff9c19d2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 0x7ff9bc108d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.947+0000 7ff9c19d2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc109370 con 0x7ff9bc108970 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.948+0000 7ff9baffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 0x7ff9bc108d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.948+0000 7ff9baffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 0x7ff9bc108d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43022/0 (socket says 172.21.15.46:43022) 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.948+0000 7ff9baffd700 1 -- 172.21.15.46:0/1895167523 learned_addr learned my addr 172.21.15.46:0/1895167523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.948+0000 7ff9baffd700 1 -- 172.21.15.46:0/1895167523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9bc109b40 con 0x7ff9bc108970 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.948+0000 7ff9baffd700 1 --2- 172.21.15.46:0/1895167523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 0x7ff9bc108d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0098a0 tx=0x7ff9ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df8e108851ad38a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.948+0000 7ff9b9ffb700 1 -- 172.21.15.46:0/1895167523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9ac004160 con 0x7ff9bc108970 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.948+0000 7ff9b9ffb700 1 -- 172.21.15.46:0/1895167523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff9ac004300 con 0x7ff9bc108970 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.948+0000 7ff9b9ffb700 1 -- 172.21.15.46:0/1895167523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9ac011af0 con 0x7ff9bc108970 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.949+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1895167523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 msgr2=0x7ff9bc108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:34.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.949+0000 7ff9c19d2700 1 --2- 172.21.15.46:0/1895167523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 0x7ff9bc108d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0098a0 tx=0x7ff9ac009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.949+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1895167523 shutdown_connections 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.949+0000 7ff9c19d2700 1 --2- 172.21.15.46:0/1895167523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 0x7ff9bc108d80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.949+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1895167523 >> 172.21.15.46:0/1895167523 conn(0x7ff9bc104050 msgr2=0x7ff9bc1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.949+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1895167523 shutdown_connections 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.949+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1895167523 wait complete. 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.950+0000 7ff9c19d2700 1 Processor -- start 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.950+0000 7ff9c19d2700 1 -- start start 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.950+0000 7ff9c19d2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 0x7ff9bc107330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.950+0000 7ff9c19d2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc10cef0 con 0x7ff9bc108970 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.950+0000 7ff9baffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 0x7ff9bc107330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.950+0000 7ff9baffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 0x7ff9bc107330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43030/0 (socket says 172.21.15.46:43030) 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.950+0000 7ff9baffd700 1 -- 172.21.15.46:0/1721281294 learned_addr learned my addr 172.21.15.46:0/1721281294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.951+0000 7ff9baffd700 1 -- 172.21.15.46:0/1721281294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9bc107870 con 0x7ff9bc108970 2024-05-09T00:00:34.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.951+0000 7ff9baffd700 1 --2- 172.21.15.46:0/1721281294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 0x7ff9bc107330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0090e0 tx=0x7ff9ac011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.951+0000 7ff9c09d0700 1 -- 172.21.15.46:0/1721281294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9ac0039e0 con 0x7ff9bc108970 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.951+0000 7ff9c09d0700 1 -- 172.21.15.46:0/1721281294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff9ac003b80 con 0x7ff9bc108970 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.951+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1721281294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9bc105ac0 con 0x7ff9bc108970 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.951+0000 7ff9c09d0700 1 -- 172.21.15.46:0/1721281294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9ac024690 con 0x7ff9bc108970 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.951+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1721281294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9bc105fa0 con 0x7ff9bc108970 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.952+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1721281294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9bc064e20 con 0x7ff9bc108970 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.952+0000 7ff9c09d0700 1 -- 172.21.15.46:0/1721281294 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7ff9ac022020 con 0x7ff9bc108970 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.953+0000 7ff9c09d0700 1 --2- 172.21.15.46:0/1721281294 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7ff9a803e470 0x7ff9a8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.953+0000 7ff9c09d0700 1 -- 172.21.15.46:0/1721281294 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff9ac053350 con 0x7ff9bc108970 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.953+0000 7ff9ba7fc700 1 --2- 172.21.15.46:0/1721281294 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7ff9a803e470 0x7ff9a8040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.953+0000 7ff9ba7fc700 1 --2- 172.21.15.46:0/1721281294 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7ff9a803e470 0x7ff9a8040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4006fd0 tx=0x7ff9a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:33.958+0000 7ff9c09d0700 1 -- 172.21.15.46:0/1721281294 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9ac0249f0 con 0x7ff9bc108970 2024-05-09T00:00:34.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.233+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1721281294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff9bc1970f0 con 0x7ff9bc108970 2024-05-09T00:00:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.284+0000 7ff9c09d0700 1 -- 172.21.15.46:0/1721281294 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff9ac0172b0 con 0x7ff9bc108970 2024-05-09T00:00:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.286+0000 7ff9c09d0700 1 -- 172.21.15.46:0/1721281294 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7ff9ac024c70 con 0x7ff9bc108970 2024-05-09T00:00:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.287+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1721281294 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7ff9a803e470 msgr2=0x7ff9a8040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.287+0000 7ff9c19d2700 1 --2- 172.21.15.46:0/1721281294 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7ff9a803e470 0x7ff9a8040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4006fd0 tx=0x7ff9a4006e40 comp rx=0 tx=0).stop 2024-05-09T00:00:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.287+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1721281294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 msgr2=0x7ff9bc107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.287+0000 7ff9c19d2700 1 --2- 172.21.15.46:0/1721281294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 0x7ff9bc107330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0090e0 tx=0x7ff9ac011940 comp rx=0 tx=0).stop 2024-05-09T00:00:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.287+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1721281294 shutdown_connections 2024-05-09T00:00:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.287+0000 7ff9c19d2700 1 --2- 172.21.15.46:0/1721281294 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7ff9a803e470 0x7ff9a8040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.287+0000 7ff9c19d2700 1 --2- 172.21.15.46:0/1721281294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9bc108970 0x7ff9bc107330 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.287+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1721281294 >> 172.21.15.46:0/1721281294 conn(0x7ff9bc104050 msgr2=0x7ff9bc194490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.288+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1721281294 shutdown_connections 2024-05-09T00:00:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:34.288+0000 7ff9c19d2700 1 -- 172.21.15.46:0/1721281294 wait complete. 2024-05-09T00:00:35.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:35 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1721281294' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-09T00:00:35.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:35 smithi046 ceph-mon[30151]: mgrmap e5: x(active, since 5s) 2024-05-09T00:00:35.552 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:35 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:35.136+0000 7fe2b319c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T00:00:35.552 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:35 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:35.228+0000 7fe2b319c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T00:00:36.287 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:00:36.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-09T00:00:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T00:00:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T00:00:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T00:00:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:00:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.351+0000 7fae87451700 1 Processor -- start 2024-05-09T00:00:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.351+0000 7fae87451700 1 -- start start 2024-05-09T00:00:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.351+0000 7fae87451700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 0x7fae8010c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.351+0000 7fae87451700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae8007cdf0 con 0x7fae8010a5d0 2024-05-09T00:00:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.352+0000 7fae851ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 0x7fae8010c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.352+0000 7fae851ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 0x7fae8010c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43054/0 (socket says 172.21.15.46:43054) 2024-05-09T00:00:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.352+0000 7fae851ed700 1 -- 172.21.15.46:0/3022473262 learned_addr learned my addr 172.21.15.46:0/3022473262 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.352+0000 7fae851ed700 1 -- 172.21.15.46:0/3022473262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8010cf30 con 0x7fae8010a5d0 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.352+0000 7fae851ed700 1 --2- 172.21.15.46:0/3022473262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 0x7fae8010c9f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0098a0 tx=0x7fae7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9acd8a69ed6e0ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.352+0000 7fae77fff700 1 -- 172.21.15.46:0/3022473262 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae7c004160 con 0x7fae8010a5d0 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.352+0000 7fae77fff700 1 -- 172.21.15.46:0/3022473262 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fae7c004300 con 0x7fae8010a5d0 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.352+0000 7fae77fff700 1 -- 172.21.15.46:0/3022473262 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae7c011af0 con 0x7fae8010a5d0 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.353+0000 7fae87451700 1 -- 172.21.15.46:0/3022473262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 msgr2=0x7fae8010c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.353+0000 7fae87451700 1 --2- 172.21.15.46:0/3022473262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 0x7fae8010c9f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0098a0 tx=0x7fae7c009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.353+0000 7fae87451700 1 -- 172.21.15.46:0/3022473262 shutdown_connections 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.353+0000 7fae87451700 1 --2- 172.21.15.46:0/3022473262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 0x7fae8010c9f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.353+0000 7fae87451700 1 -- 172.21.15.46:0/3022473262 >> 172.21.15.46:0/3022473262 conn(0x7fae80104050 msgr2=0x7fae801064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.353+0000 7fae87451700 1 -- 172.21.15.46:0/3022473262 shutdown_connections 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.353+0000 7fae87451700 1 -- 172.21.15.46:0/3022473262 wait complete. 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.354+0000 7fae87451700 1 Processor -- start 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.354+0000 7fae87451700 1 -- start start 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.354+0000 7fae87451700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 0x7fae8019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:36.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.354+0000 7fae87451700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae8010de90 con 0x7fae8010a5d0 2024-05-09T00:00:36.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.354+0000 7fae851ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 0x7fae8019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:36.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.354+0000 7fae851ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 0x7fae8019ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43066/0 (socket says 172.21.15.46:43066) 2024-05-09T00:00:36.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.354+0000 7fae851ed700 1 -- 172.21.15.46:0/4111444424 learned_addr learned my addr 172.21.15.46:0/4111444424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:36.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.355+0000 7fae851ed700 1 -- 172.21.15.46:0/4111444424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8019e410 con 0x7fae8010a5d0 2024-05-09T00:00:36.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.355+0000 7fae851ed700 1 --2- 172.21.15.46:0/4111444424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 0x7fae8019ded0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009fb0 tx=0x7fae7c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:36.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.355+0000 7fae767fc700 1 -- 172.21.15.46:0/4111444424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae7c011bc0 con 0x7fae8010a5d0 2024-05-09T00:00:36.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.355+0000 7fae87451700 1 -- 172.21.15.46:0/4111444424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8019e6a0 con 0x7fae8010a5d0 2024-05-09T00:00:36.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.355+0000 7fae767fc700 1 -- 172.21.15.46:0/4111444424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fae7c024440 con 0x7fae8010a5d0 2024-05-09T00:00:36.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.355+0000 7fae767fc700 1 -- 172.21.15.46:0/4111444424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae7c01a5c0 con 0x7fae8010a5d0 2024-05-09T00:00:36.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.355+0000 7fae87451700 1 -- 172.21.15.46:0/4111444424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae801a12b0 con 0x7fae8010a5d0 2024-05-09T00:00:36.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.356+0000 7fae767fc700 1 -- 172.21.15.46:0/4111444424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fae7c01a880 con 0x7fae8010a5d0 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.356+0000 7fae767fc700 1 --2- 172.21.15.46:0/4111444424 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7fae6c03e420 0x7fae6c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.356+0000 7fae87451700 1 -- 172.21.15.46:0/4111444424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae80064e20 con 0x7fae8010a5d0 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.357+0000 7fae767fc700 1 -- 172.21.15.46:0/4111444424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fae7c0537e0 con 0x7fae8010a5d0 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.356+0000 7fae849ec700 1 -- 172.21.15.46:0/4111444424 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7fae6c03e420 msgr2=0x7fae6c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/953818590 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.357+0000 7fae849ec700 1 --2- 172.21.15.46:0/4111444424 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7fae6c03e420 0x7fae6c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.362+0000 7fae767fc700 1 -- 172.21.15.46:0/4111444424 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae7c00b3e0 con 0x7fae8010a5d0 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.558+0000 7fae849ec700 1 -- 172.21.15.46:0/4111444424 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7fae6c03e420 msgr2=0x7fae6c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/953818590 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.558+0000 7fae849ec700 1 --2- 172.21.15.46:0/4111444424 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7fae6c03e420 0x7fae6c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.685+0000 7fae87451700 1 -- 172.21.15.46:0/4111444424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fae801a15a0 con 0x7fae8010a5d0 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.686+0000 7fae767fc700 1 -- 172.21.15.46:0/4111444424 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fae7c029df0 con 0x7fae8010a5d0 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.690+0000 7fae87451700 1 -- 172.21.15.46:0/4111444424 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7fae6c03e420 msgr2=0x7fae6c040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.690+0000 7fae87451700 1 --2- 172.21.15.46:0/4111444424 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7fae6c03e420 0x7fae6c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.690+0000 7fae87451700 1 -- 172.21.15.46:0/4111444424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 msgr2=0x7fae8019ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:36.297 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.690+0000 7fae87451700 1 --2- 172.21.15.46:0/4111444424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 0x7fae8019ded0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009fb0 tx=0x7fae7c011890 comp rx=0 tx=0).stop 2024-05-09T00:00:36.298 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.690+0000 7fae87451700 1 -- 172.21.15.46:0/4111444424 shutdown_connections 2024-05-09T00:00:36.298 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.690+0000 7fae87451700 1 --2- 172.21.15.46:0/4111444424 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7fae6c03e420 0x7fae6c040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:36.298 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.690+0000 7fae87451700 1 --2- 172.21.15.46:0/4111444424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8010a5d0 0x7fae8019ded0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:36.298 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.690+0000 7fae87451700 1 -- 172.21.15.46:0/4111444424 >> 172.21.15.46:0/4111444424 conn(0x7fae80104050 msgr2=0x7fae80105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:36.298 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.690+0000 7fae87451700 1 -- 172.21.15.46:0/4111444424 shutdown_connections 2024-05-09T00:00:36.298 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:35.690+0000 7fae87451700 1 -- 172.21.15.46:0/4111444424 wait complete. 2024-05-09T00:00:36.298 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-05-09T00:00:36.298 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 5... 2024-05-09T00:00:36.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:36 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/4111444424' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T00:00:36.680 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:36 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:36.515+0000 7fe2b319c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T00:00:37.004 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:36 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:36.700+0000 7fe2b319c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T00:00:37.004 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:36 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:36.885+0000 7fe2b319c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T00:00:37.283 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:37 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:37.190+0000 7fe2b319c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T00:00:37.284 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:37 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:37.288+0000 7fe2b319c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T00:00:37.556 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:37 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:37.501+0000 7fe2b319c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T00:00:38.057 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:37 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:37.704+0000 7fe2b319c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:00:38.556 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:38 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:38.337+0000 7fe2b319c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T00:00:39.012 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:38 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:38.571+0000 7fe2b319c200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T00:00:39.307 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:39 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:39.018+0000 7fe2b319c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T00:00:39.307 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:39 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:39.124+0000 7fe2b319c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T00:00:39.807 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:39 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:39.520+0000 7fe2b319c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T00:00:40.263 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:39 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:39.904+0000 7fe2b319c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T00:00:40.263 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:40.004+0000 7fe2b319c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T00:00:40.557 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:40.269+0000 7fe2b319c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T00:00:40.557 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:40.412+0000 7fe2b319c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T00:00:40.557 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:40.515+0000 7fe2b319c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T00:00:41.306 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:40.862+0000 7fe2b319c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T00:00:41.307 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:40.962+0000 7fe2b319c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T00:00:41.878 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:41 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:41.599+0000 7fe2b319c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T00:00:41.879 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:41 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:00:41.831+0000 7fe2b319c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T00:00:42.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: Active manager daemon x restarted 2024-05-09T00:00:42.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: Activating manager daemon x 2024-05-09T00:00:42.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: osdmap e2: 0 total, 0 up, 0 in 2024-05-09T00:00:42.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: mgrmap e6: x(active, starting, since 0.00433673s) 2024-05-09T00:00:42.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:00:42.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T00:00:42.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T00:00:42.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T00:00:42.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T00:00:42.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: Manager daemon x is now available 2024-05-09T00:00:42.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:00:42.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:00:42.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:00:42.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:41 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:00:43.200 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:42 smithi046 ceph-mon[30151]: Found migration_current of "None". Setting to last migration. 2024-05-09T00:00:43.200 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:42 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T00:00:43.200 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:42 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T00:00:43.200 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:42 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:00:43.200 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:42 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:00:43.200 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:42 smithi046 ceph-mon[30151]: mgrmap e7: x(active, since 1.00641s) 2024-05-09T00:00:43.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:00:43.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-09T00:00:43.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T00:00:43.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:00:43.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.803+0000 7feb74e0a700 1 Processor -- start 2024-05-09T00:00:43.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.803+0000 7feb74e0a700 1 -- start start 2024-05-09T00:00:43.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.803+0000 7feb74e0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 0x7feb70109e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:43.504 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.803+0000 7feb74e0a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb7010a440 con 0x7feb70109a40 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.804+0000 7feb6e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 0x7feb70109e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.804+0000 7feb6e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 0x7feb70109e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39936/0 (socket says 172.21.15.46:39936) 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.804+0000 7feb6e59c700 1 -- 172.21.15.46:0/4061516547 learned_addr learned my addr 172.21.15.46:0/4061516547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.804+0000 7feb6e59c700 1 -- 172.21.15.46:0/4061516547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7010ac10 con 0x7feb70109a40 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.804+0000 7feb6e59c700 1 --2- 172.21.15.46:0/4061516547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 0x7feb70109e50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb600098a0 tx=0x7feb60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca5df8fa95bbbda server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.804+0000 7feb6d59a700 1 -- 172.21.15.46:0/4061516547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb60004160 con 0x7feb70109a40 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.804+0000 7feb6d59a700 1 -- 172.21.15.46:0/4061516547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feb60004300 con 0x7feb70109a40 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.804+0000 7feb6d59a700 1 -- 172.21.15.46:0/4061516547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb60011af0 con 0x7feb70109a40 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.805+0000 7feb74e0a700 1 -- 172.21.15.46:0/4061516547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 msgr2=0x7feb70109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.806+0000 7feb74e0a700 1 --2- 172.21.15.46:0/4061516547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 0x7feb70109e50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb600098a0 tx=0x7feb60009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.806+0000 7feb74e0a700 1 -- 172.21.15.46:0/4061516547 shutdown_connections 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.806+0000 7feb74e0a700 1 --2- 172.21.15.46:0/4061516547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 0x7feb70109e50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.806+0000 7feb74e0a700 1 -- 172.21.15.46:0/4061516547 >> 172.21.15.46:0/4061516547 conn(0x7feb70105160 msgr2=0x7feb701075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:43.505 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.806+0000 7feb74e0a700 1 -- 172.21.15.46:0/4061516547 shutdown_connections 2024-05-09T00:00:43.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.806+0000 7feb74e0a700 1 -- 172.21.15.46:0/4061516547 wait complete. 2024-05-09T00:00:43.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.807+0000 7feb74e0a700 1 Processor -- start 2024-05-09T00:00:43.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.807+0000 7feb74e0a700 1 -- start start 2024-05-09T00:00:43.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.807+0000 7feb74e0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 0x7feb7019df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:43.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.807+0000 7feb74e0a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb7010dfc0 con 0x7feb70109a40 2024-05-09T00:00:43.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.807+0000 7feb6e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 0x7feb7019df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:43.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.807+0000 7feb6e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 0x7feb7019df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39950/0 (socket says 172.21.15.46:39950) 2024-05-09T00:00:43.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.807+0000 7feb6e59c700 1 -- 172.21.15.46:0/3832689820 learned_addr learned my addr 172.21.15.46:0/3832689820 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:43.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.807+0000 7feb6e59c700 1 -- 172.21.15.46:0/3832689820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7019e470 con 0x7feb70109a40 2024-05-09T00:00:43.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.808+0000 7feb6e59c700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 0x7feb7019df30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feb60011670 tx=0x7feb60004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:43.506 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.808+0000 7feb677fe700 1 -- 172.21.15.46:0/3832689820 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb600036a0 con 0x7feb70109a40 2024-05-09T00:00:43.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.808+0000 7feb74e0a700 1 -- 172.21.15.46:0/3832689820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7019e700 con 0x7feb70109a40 2024-05-09T00:00:43.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.808+0000 7feb677fe700 1 -- 172.21.15.46:0/3832689820 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feb60003cd0 con 0x7feb70109a40 2024-05-09T00:00:43.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.808+0000 7feb677fe700 1 -- 172.21.15.46:0/3832689820 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb600193c0 con 0x7feb70109a40 2024-05-09T00:00:43.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.808+0000 7feb74e0a700 1 -- 172.21.15.46:0/3832689820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb701a1310 con 0x7feb70109a40 2024-05-09T00:00:43.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.809+0000 7feb677fe700 1 -- 172.21.15.46:0/3832689820 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7feb60019680 con 0x7feb70109a40 2024-05-09T00:00:43.507 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.809+0000 7feb677fe700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 0x7feb5c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:43.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.809+0000 7feb677fe700 1 -- 172.21.15.46:0/3832689820 --> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feb5c041040 con 0x7feb5c03e420 2024-05-09T00:00:43.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.809+0000 7feb677fe700 1 -- 172.21.15.46:0/3832689820 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feb600536b0 con 0x7feb70109a40 2024-05-09T00:00:43.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.809+0000 7feb6dd9b700 1 -- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 msgr2=0x7feb5c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/953818590 2024-05-09T00:00:43.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:36.809+0000 7feb6dd9b700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 0x7feb5c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:00:43.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:37.010+0000 7feb6dd9b700 1 -- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 msgr2=0x7feb5c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/953818590 2024-05-09T00:00:43.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:37.010+0000 7feb6dd9b700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 0x7feb5c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T00:00:43.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:37.411+0000 7feb6dd9b700 1 -- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 msgr2=0x7feb5c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/953818590 2024-05-09T00:00:43.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:37.411+0000 7feb6dd9b700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 0x7feb5c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T00:00:43.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:38.212+0000 7feb6dd9b700 1 -- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 msgr2=0x7feb5c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/953818590 2024-05-09T00:00:43.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:38.212+0000 7feb6dd9b700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 0x7feb5c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T00:00:43.508 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:39.813+0000 7feb6dd9b700 1 -- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 msgr2=0x7feb5c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/953818590 2024-05-09T00:00:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:39.813+0000 7feb6dd9b700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 0x7feb5c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T00:00:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:41.835+0000 7feb677fe700 1 -- 172.21.15.46:0/3832689820 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7feb600199c0 con 0x7feb70109a40 2024-05-09T00:00:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:41.835+0000 7feb677fe700 1 -- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 msgr2=0x7feb5c040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:00:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:41.835+0000 7feb677fe700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 0x7feb5c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.836+0000 7feb677fe700 1 -- 172.21.15.46:0/3832689820 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7feb600558c0 con 0x7feb70109a40 2024-05-09T00:00:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.836+0000 7feb677fe700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7feb5c041d60 0x7feb5c044180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.836+0000 7feb677fe700 1 -- 172.21.15.46:0/3832689820 --> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feb5c041040 con 0x7feb5c041d60 2024-05-09T00:00:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.837+0000 7feb6dd9b700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7feb5c041d60 0x7feb5c044180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.837+0000 7feb6dd9b700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7feb5c041d60 0x7feb5c044180 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7feb58003a10 tx=0x7feb58009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.838+0000 7feb677fe700 1 -- 172.21.15.46:0/3832689820 <== mgr.14120 v2:172.21.15.46:6800/1620504773 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7feb5c041040 con 0x7feb5c041d60 2024-05-09T00:00:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb74e0a700 1 -- 172.21.15.46:0/3832689820 --> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7feb70064e20 con 0x7feb5c041d60 2024-05-09T00:00:43.509 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb677fe700 1 -- 172.21.15.46:0/3832689820 <== mgr.14120 v2:172.21.15.46:6800/1620504773 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7feb70064e20 con 0x7feb5c041d60 2024-05-09T00:00:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb74e0a700 1 -- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7feb5c041d60 msgr2=0x7feb5c044180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb74e0a700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7feb5c041d60 0x7feb5c044180 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7feb58003a10 tx=0x7feb58009290 comp rx=0 tx=0).stop 2024-05-09T00:00:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb74e0a700 1 -- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 msgr2=0x7feb7019df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb74e0a700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 0x7feb7019df30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feb60011670 tx=0x7feb60004280 comp rx=0 tx=0).stop 2024-05-09T00:00:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb74e0a700 1 -- 172.21.15.46:0/3832689820 shutdown_connections 2024-05-09T00:00:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb74e0a700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7feb5c041d60 0x7feb5c044180 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb74e0a700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb70109a40 0x7feb7019df30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb74e0a700 1 --2- 172.21.15.46:0/3832689820 >> [v2:172.21.15.46:6800/953818590,v1:172.21.15.46:6801/953818590] conn(0x7feb5c03e420 0x7feb5c040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb74e0a700 1 -- 172.21.15.46:0/3832689820 >> 172.21.15.46:0/3832689820 conn(0x7feb70105160 msgr2=0x7feb70106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb74e0a700 1 -- 172.21.15.46:0/3832689820 shutdown_connections 2024-05-09T00:00:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:42.847+0000 7feb74e0a700 1 -- 172.21.15.46:0/3832689820 wait complete. 2024-05-09T00:00:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 5 is available 2024-05-09T00:00:43.511 INFO:teuthology.orchestra.run.smithi046.stdout:Setting orchestrator backend to cephadm... 2024-05-09T00:00:44.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:43 smithi046 ceph-mon[30151]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T00:00:44.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:43 smithi046 ceph-mon[30151]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T00:00:44.732 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.955+0000 7f33deaa7700 1 Processor -- start 2024-05-09T00:00:44.732 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.956+0000 7f33deaa7700 1 -- start start 2024-05-09T00:00:44.732 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.956+0000 7f33deaa7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 0x7f33d810c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:44.732 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.956+0000 7f33deaa7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33d807ce30 con 0x7f33d810a590 2024-05-09T00:00:44.732 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.956+0000 7f33dc843700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 0x7f33d810c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:44.732 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.956+0000 7f33dc843700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 0x7f33d810c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40016/0 (socket says 172.21.15.46:40016) 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.956+0000 7f33dc843700 1 -- 172.21.15.46:0/3702084376 learned_addr learned my addr 172.21.15.46:0/3702084376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.956+0000 7f33dc843700 1 -- 172.21.15.46:0/3702084376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d810cef0 con 0x7f33d810a590 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.956+0000 7f33dc843700 1 --2- 172.21.15.46:0/3702084376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 0x7f33d810c9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33c80098a0 tx=0x7f33c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d27045924b7275 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.957+0000 7f33d77fe700 1 -- 172.21.15.46:0/3702084376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c8004160 con 0x7f33d810a590 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.957+0000 7f33d77fe700 1 -- 172.21.15.46:0/3702084376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f33c8004300 con 0x7f33d810a590 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.957+0000 7f33d77fe700 1 -- 172.21.15.46:0/3702084376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c8011af0 con 0x7f33d810a590 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.957+0000 7f33deaa7700 1 -- 172.21.15.46:0/3702084376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 msgr2=0x7f33d810c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.957+0000 7f33deaa7700 1 --2- 172.21.15.46:0/3702084376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 0x7f33d810c9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33c80098a0 tx=0x7f33c8009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.957+0000 7f33deaa7700 1 -- 172.21.15.46:0/3702084376 shutdown_connections 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.957+0000 7f33deaa7700 1 --2- 172.21.15.46:0/3702084376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 0x7f33d810c9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.957+0000 7f33deaa7700 1 -- 172.21.15.46:0/3702084376 >> 172.21.15.46:0/3702084376 conn(0x7f33d8104050 msgr2=0x7f33d81064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.958+0000 7f33deaa7700 1 -- 172.21.15.46:0/3702084376 shutdown_connections 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.958+0000 7f33deaa7700 1 -- 172.21.15.46:0/3702084376 wait complete. 2024-05-09T00:00:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.958+0000 7f33deaa7700 1 Processor -- start 2024-05-09T00:00:44.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.958+0000 7f33deaa7700 1 -- start start 2024-05-09T00:00:44.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.958+0000 7f33deaa7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 0x7f33d819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:44.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.958+0000 7f33deaa7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33d810de50 con 0x7f33d810a590 2024-05-09T00:00:44.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.959+0000 7f33dc843700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 0x7f33d819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:44.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.959+0000 7f33dc843700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 0x7f33d819dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40024/0 (socket says 172.21.15.46:40024) 2024-05-09T00:00:44.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.959+0000 7f33dc843700 1 -- 172.21.15.46:0/878363755 learned_addr learned my addr 172.21.15.46:0/878363755 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:44.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.959+0000 7f33dc843700 1 -- 172.21.15.46:0/878363755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d819e3e0 con 0x7f33d810a590 2024-05-09T00:00:44.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.959+0000 7f33dc843700 1 --2- 172.21.15.46:0/878363755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 0x7f33d819dea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f33c80117b0 tx=0x7f33c8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:44.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.959+0000 7f33d5ffb700 1 -- 172.21.15.46:0/878363755 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c80039c0 con 0x7f33d810a590 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.959+0000 7f33d5ffb700 1 -- 172.21.15.46:0/878363755 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f33c8003b60 con 0x7f33d810a590 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.959+0000 7f33deaa7700 1 -- 172.21.15.46:0/878363755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d819e670 con 0x7f33d810a590 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.959+0000 7f33d5ffb700 1 -- 172.21.15.46:0/878363755 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c80196a0 con 0x7f33d810a590 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.959+0000 7f33deaa7700 1 -- 172.21.15.46:0/878363755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d81a1280 con 0x7f33d810a590 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.960+0000 7f33d5ffb700 1 -- 172.21.15.46:0/878363755 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f33c8019960 con 0x7f33d810a590 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.961+0000 7f33d5ffb700 1 --2- 172.21.15.46:0/878363755 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f33c403e470 0x7f33c4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.961+0000 7f33deaa7700 1 -- 172.21.15.46:0/878363755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d8064e20 con 0x7f33d810a590 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.961+0000 7f33d5ffb700 1 -- 172.21.15.46:0/878363755 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f33c8054a40 con 0x7f33d810a590 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.961+0000 7f33d7fff700 1 --2- 172.21.15.46:0/878363755 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f33c403e470 0x7f33c4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.961+0000 7f33d7fff700 1 --2- 172.21.15.46:0/878363755 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f33c403e470 0x7f33c4040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f33c0006fd0 tx=0x7f33c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:43.966+0000 7f33d5ffb700 1 -- 172.21.15.46:0/878363755 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33c8017e90 con 0x7f33d810a590 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:44.199+0000 7f33deaa7700 1 -- 172.21.15.46:0/878363755 --> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f33d8106220 con 0x7f33c403e470 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:44.204+0000 7f33d5ffb700 1 -- 172.21.15.46:0/878363755 <== mgr.14120 v2:172.21.15.46:6800/1620504773 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f33d8106220 con 0x7f33c403e470 2024-05-09T00:00:44.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:44.207+0000 7f33deaa7700 1 -- 172.21.15.46:0/878363755 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f33c403e470 msgr2=0x7f33c4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:44.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:44.207+0000 7f33deaa7700 1 --2- 172.21.15.46:0/878363755 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f33c403e470 0x7f33c4040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f33c0006fd0 tx=0x7f33c0006e40 comp rx=0 tx=0).stop 2024-05-09T00:00:44.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:44.207+0000 7f33deaa7700 1 -- 172.21.15.46:0/878363755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 msgr2=0x7f33d819dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:44.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:44.207+0000 7f33deaa7700 1 --2- 172.21.15.46:0/878363755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 0x7f33d819dea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f33c80117b0 tx=0x7f33c8011b70 comp rx=0 tx=0).stop 2024-05-09T00:00:44.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:44.207+0000 7f33deaa7700 1 -- 172.21.15.46:0/878363755 shutdown_connections 2024-05-09T00:00:44.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:44.207+0000 7f33deaa7700 1 --2- 172.21.15.46:0/878363755 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f33c403e470 0x7f33c4040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:44.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:44.207+0000 7f33deaa7700 1 --2- 172.21.15.46:0/878363755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d810a590 0x7f33d819dea0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:44.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:44.207+0000 7f33deaa7700 1 -- 172.21.15.46:0/878363755 >> 172.21.15.46:0/878363755 conn(0x7f33d8104050 msgr2=0x7f33d8105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:44.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:44.207+0000 7f33deaa7700 1 -- 172.21.15.46:0/878363755 shutdown_connections 2024-05-09T00:00:44.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:44.207+0000 7f33deaa7700 1 -- 172.21.15.46:0/878363755 wait complete. 2024-05-09T00:00:45.126 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:44 smithi046 ceph-mon[30151]: [09/May/2024:00:00:43] ENGINE Bus STARTING 2024-05-09T00:00:45.126 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:44 smithi046 ceph-mon[30151]: [09/May/2024:00:00:43] ENGINE Serving on http://172.21.15.46:8765 2024-05-09T00:00:45.126 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:44 smithi046 ceph-mon[30151]: [09/May/2024:00:00:43] ENGINE Serving on https://172.21.15.46:7150 2024-05-09T00:00:45.127 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:44 smithi046 ceph-mon[30151]: [09/May/2024:00:00:43] ENGINE Bus STARTED 2024-05-09T00:00:45.127 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:44 smithi046 ceph-mon[30151]: mgrmap e8: x(active, since 2s) 2024-05-09T00:00:45.127 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:44 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:00:45.127 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:44 smithi046 ceph-mon[30151]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:00:45.127 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:44 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:00:45.127 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:44 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:00:45.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.173+0000 7fbadfad5700 1 Processor -- start 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.173+0000 7fbadfad5700 1 -- start start 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.174+0000 7fbadfad5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 0x7fbad810c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.174+0000 7fbadfad5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbad807cdf0 con 0x7fbad810a5d0 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.174+0000 7fbadd871700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 0x7fbad810c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.174+0000 7fbadd871700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 0x7fbad810c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40038/0 (socket says 172.21.15.46:40038) 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.174+0000 7fbadd871700 1 -- 172.21.15.46:0/2382558781 learned_addr learned my addr 172.21.15.46:0/2382558781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.174+0000 7fbadd871700 1 -- 172.21.15.46:0/2382558781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad810cf30 con 0x7fbad810a5d0 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.174+0000 7fbadd871700 1 --2- 172.21.15.46:0/2382558781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 0x7fbad810c9f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbad40098a0 tx=0x7fbad4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8424cd6807c1eb27 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.174+0000 7fbadc86f700 1 -- 172.21.15.46:0/2382558781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad4004160 con 0x7fbad810a5d0 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.174+0000 7fbadc86f700 1 -- 172.21.15.46:0/2382558781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbad4004300 con 0x7fbad810a5d0 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.174+0000 7fbadc86f700 1 -- 172.21.15.46:0/2382558781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad4011af0 con 0x7fbad810a5d0 2024-05-09T00:00:45.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.175+0000 7fbadfad5700 1 -- 172.21.15.46:0/2382558781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 msgr2=0x7fbad810c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.175+0000 7fbadfad5700 1 --2- 172.21.15.46:0/2382558781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 0x7fbad810c9f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbad40098a0 tx=0x7fbad4009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.175+0000 7fbadfad5700 1 -- 172.21.15.46:0/2382558781 shutdown_connections 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.175+0000 7fbadfad5700 1 --2- 172.21.15.46:0/2382558781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 0x7fbad810c9f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.175+0000 7fbadfad5700 1 -- 172.21.15.46:0/2382558781 >> 172.21.15.46:0/2382558781 conn(0x7fbad8104050 msgr2=0x7fbad81064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.175+0000 7fbadfad5700 1 -- 172.21.15.46:0/2382558781 shutdown_connections 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.175+0000 7fbadfad5700 1 -- 172.21.15.46:0/2382558781 wait complete. 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.176+0000 7fbadfad5700 1 Processor -- start 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.176+0000 7fbadfad5700 1 -- start start 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.176+0000 7fbadfad5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 0x7fbad819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.176+0000 7fbadfad5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbad810de90 con 0x7fbad810a5d0 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.176+0000 7fbadd871700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 0x7fbad819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.176+0000 7fbadd871700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 0x7fbad819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40050/0 (socket says 172.21.15.46:40050) 2024-05-09T00:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.176+0000 7fbadd871700 1 -- 172.21.15.46:0/2885550858 learned_addr learned my addr 172.21.15.46:0/2885550858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.177+0000 7fbadd871700 1 -- 172.21.15.46:0/2885550858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad819e410 con 0x7fbad810a5d0 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.177+0000 7fbadd871700 1 --2- 172.21.15.46:0/2885550858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 0x7fbad819ded0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009fb0 tx=0x7fbad4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.177+0000 7fbaceffd700 1 -- 172.21.15.46:0/2885550858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad4011bc0 con 0x7fbad810a5d0 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.177+0000 7fbaceffd700 1 -- 172.21.15.46:0/2885550858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbad4024440 con 0x7fbad810a5d0 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.177+0000 7fbadfad5700 1 -- 172.21.15.46:0/2885550858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad819e6a0 con 0x7fbad810a5d0 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.177+0000 7fbaceffd700 1 -- 172.21.15.46:0/2885550858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad401a5c0 con 0x7fbad810a5d0 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.177+0000 7fbadfad5700 1 -- 172.21.15.46:0/2885550858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad81a12b0 con 0x7fbad810a5d0 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.178+0000 7fbaceffd700 1 -- 172.21.15.46:0/2885550858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fbad401a830 con 0x7fbad810a5d0 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.178+0000 7fbadfad5700 1 -- 172.21.15.46:0/2885550858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad8064e20 con 0x7fbad810a5d0 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.178+0000 7fbaceffd700 1 --2- 172.21.15.46:0/2885550858 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fbac403e470 0x7fbac4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.178+0000 7fbaceffd700 1 -- 172.21.15.46:0/2885550858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbad40537b0 con 0x7fbad810a5d0 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.179+0000 7fbadd070700 1 --2- 172.21.15.46:0/2885550858 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fbac403e470 0x7fbac4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:45.941 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.179+0000 7fbadd070700 1 --2- 172.21.15.46:0/2885550858 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fbac403e470 0x7fbac4040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbac8006fd0 tx=0x7fbac8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.184+0000 7fbaceffd700 1 -- 172.21.15.46:0/2885550858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbad400b3e0 con 0x7fbad810a5d0 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.401+0000 7fbadfad5700 1 -- 172.21.15.46:0/2885550858 --> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fbad8106230 con 0x7fbac403e470 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.401+0000 7fbaceffd700 1 -- 172.21.15.46:0/2885550858 <== mgr.14120 v2:172.21.15.46:6800/1620504773 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fbad8106230 con 0x7fbac403e470 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.404+0000 7fbadfad5700 1 -- 172.21.15.46:0/2885550858 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fbac403e470 msgr2=0x7fbac4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.404+0000 7fbadfad5700 1 --2- 172.21.15.46:0/2885550858 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fbac403e470 0x7fbac4040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbac8006fd0 tx=0x7fbac8006e40 comp rx=0 tx=0).stop 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.404+0000 7fbadfad5700 1 -- 172.21.15.46:0/2885550858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 msgr2=0x7fbad819ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.404+0000 7fbadfad5700 1 --2- 172.21.15.46:0/2885550858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 0x7fbad819ded0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009fb0 tx=0x7fbad4011890 comp rx=0 tx=0).stop 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.404+0000 7fbadfad5700 1 -- 172.21.15.46:0/2885550858 shutdown_connections 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.404+0000 7fbadfad5700 1 --2- 172.21.15.46:0/2885550858 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fbac403e470 0x7fbac4040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.404+0000 7fbadfad5700 1 --2- 172.21.15.46:0/2885550858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad810a5d0 0x7fbad819ded0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.404+0000 7fbadfad5700 1 -- 172.21.15.46:0/2885550858 >> 172.21.15.46:0/2885550858 conn(0x7fbad8104050 msgr2=0x7fbad8105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.404+0000 7fbadfad5700 1 -- 172.21.15.46:0/2885550858 shutdown_connections 2024-05-09T00:00:45.942 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:45.404+0000 7fbadfad5700 1 -- 172.21.15.46:0/2885550858 wait complete. 2024-05-09T00:00:45.943 INFO:teuthology.orchestra.run.smithi046.stdout:Generating ssh key... 2024-05-09T00:00:46.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:46 smithi046 ceph-mon[30151]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:00:47.057 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: Generating public/private rsa key pair. 2024-05-09T00:00:47.058 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: Your identification has been saved in /tmp/tmpufvcl4vh/key. 2024-05-09T00:00:47.058 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: Your public key has been saved in /tmp/tmpufvcl4vh/key.pub. 2024-05-09T00:00:47.058 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: The key fingerprint is: 2024-05-09T00:00:47.058 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: SHA256:tadY5nR9SbvlaESSdH9qiK1RcM/ORFihjrniz7ICpTk ceph-eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:00:47.058 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: The key's randomart image is: 2024-05-09T00:00:47.058 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: +---[RSA 3072]----+ 2024-05-09T00:00:47.058 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: | . oo=. | 2024-05-09T00:00:47.058 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: | +.B . | 2024-05-09T00:00:47.058 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: | . = =.o| 2024-05-09T00:00:47.059 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: | . . O O..+| 2024-05-09T00:00:47.059 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: | + S X * Boo| 2024-05-09T00:00:47.059 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: | E * B o +o| 2024-05-09T00:00:47.059 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: | o o = o..| 2024-05-09T00:00:47.059 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: | ...o . | 2024-05-09T00:00:47.059 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: | .o+o | 2024-05-09T00:00:47.059 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:00:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: +----[SHA256]-----+ 2024-05-09T00:00:47.419 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.424+0000 7f8babd6a700 1 Processor -- start 2024-05-09T00:00:47.419 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.424+0000 7f8babd6a700 1 -- start start 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.424+0000 7f8babd6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 0x7f8ba41087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.424+0000 7f8babd6a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba407cdf0 con 0x7f8ba41083c0 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.424+0000 7f8ba9b06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 0x7f8ba41087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.424+0000 7f8ba9b06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 0x7f8ba41087d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39104/0 (socket says 172.21.15.46:39104) 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.424+0000 7f8ba9b06700 1 -- 172.21.15.46:0/2935786555 learned_addr learned my addr 172.21.15.46:0/2935786555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.425+0000 7f8ba9b06700 1 -- 172.21.15.46:0/2935786555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba4108d10 con 0x7f8ba41083c0 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.425+0000 7f8ba9b06700 1 --2- 172.21.15.46:0/2935786555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 0x7f8ba41087d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00098a0 tx=0x7f8ba0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278f767259153ed4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.425+0000 7f8ba8b04700 1 -- 172.21.15.46:0/2935786555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ba0004160 con 0x7f8ba41083c0 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.425+0000 7f8ba8b04700 1 -- 172.21.15.46:0/2935786555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8ba0004300 con 0x7f8ba41083c0 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.425+0000 7f8ba8b04700 1 -- 172.21.15.46:0/2935786555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ba0011af0 con 0x7f8ba41083c0 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.426+0000 7f8babd6a700 1 -- 172.21.15.46:0/2935786555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 msgr2=0x7f8ba41087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.426+0000 7f8babd6a700 1 --2- 172.21.15.46:0/2935786555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 0x7f8ba41087d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00098a0 tx=0x7f8ba0009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.420 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.426+0000 7f8babd6a700 1 -- 172.21.15.46:0/2935786555 shutdown_connections 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.426+0000 7f8babd6a700 1 --2- 172.21.15.46:0/2935786555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 0x7f8ba41087d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.426+0000 7f8babd6a700 1 -- 172.21.15.46:0/2935786555 >> 172.21.15.46:0/2935786555 conn(0x7f8ba4104050 msgr2=0x7f8ba41064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.426+0000 7f8babd6a700 1 -- 172.21.15.46:0/2935786555 shutdown_connections 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.426+0000 7f8babd6a700 1 -- 172.21.15.46:0/2935786555 wait complete. 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.427+0000 7f8babd6a700 1 Processor -- start 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.427+0000 7f8babd6a700 1 -- start start 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.427+0000 7f8babd6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 0x7f8ba41a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.427+0000 7f8babd6a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba4109d90 con 0x7f8ba41083c0 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.427+0000 7f8ba9b06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 0x7f8ba41a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.427+0000 7f8ba9b06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 0x7f8ba41a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39110/0 (socket says 172.21.15.46:39110) 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.427+0000 7f8ba9b06700 1 -- 172.21.15.46:0/1068124838 learned_addr learned my addr 172.21.15.46:0/1068124838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.427+0000 7f8ba9b06700 1 -- 172.21.15.46:0/1068124838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba41a0700 con 0x7f8ba41083c0 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.428+0000 7f8ba9b06700 1 --2- 172.21.15.46:0/1068124838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 0x7f8ba41a01c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00117b0 tx=0x7f8ba0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.428+0000 7f8b9affd700 1 -- 172.21.15.46:0/1068124838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ba00039c0 con 0x7f8ba41083c0 2024-05-09T00:00:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.428+0000 7f8b9affd700 1 -- 172.21.15.46:0/1068124838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8ba0003b60 con 0x7f8ba41083c0 2024-05-09T00:00:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.428+0000 7f8b9affd700 1 -- 172.21.15.46:0/1068124838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ba00196a0 con 0x7f8ba41083c0 2024-05-09T00:00:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.428+0000 7f8babd6a700 1 -- 172.21.15.46:0/1068124838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba41a0990 con 0x7f8ba41083c0 2024-05-09T00:00:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.428+0000 7f8babd6a700 1 -- 172.21.15.46:0/1068124838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba41a13d0 con 0x7f8ba41083c0 2024-05-09T00:00:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.429+0000 7f8b9affd700 1 -- 172.21.15.46:0/1068124838 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f8ba001a430 con 0x7f8ba41083c0 2024-05-09T00:00:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.429+0000 7f8babd6a700 1 -- 172.21.15.46:0/1068124838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba4064e20 con 0x7f8ba41083c0 2024-05-09T00:00:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.429+0000 7f8b9affd700 1 --2- 172.21.15.46:0/1068124838 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f8b9003e4c0 0x7f8b900409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.429+0000 7f8b9affd700 1 -- 172.21.15.46:0/1068124838 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8ba0053aa0 con 0x7f8ba41083c0 2024-05-09T00:00:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.430+0000 7f8ba9305700 1 --2- 172.21.15.46:0/1068124838 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f8b9003e4c0 0x7f8b900409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.430+0000 7f8ba9305700 1 --2- 172.21.15.46:0/1068124838 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f8b9003e4c0 0x7f8b900409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b94006fd0 tx=0x7f8b94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.435+0000 7f8b9affd700 1 -- 172.21.15.46:0/1068124838 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ba000b3e0 con 0x7f8ba41083c0 2024-05-09T00:00:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.653+0000 7f8babd6a700 1 -- 172.21.15.46:0/1068124838 --> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8ba40008d0 con 0x7f8b9003e4c0 2024-05-09T00:00:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.879+0000 7f8b9affd700 1 -- 172.21.15.46:0/1068124838 <== mgr.14120 v2:172.21.15.46:6800/1620504773 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ba40008d0 con 0x7f8b9003e4c0 2024-05-09T00:00:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.882+0000 7f8babd6a700 1 -- 172.21.15.46:0/1068124838 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f8b9003e4c0 msgr2=0x7f8b900409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.882+0000 7f8babd6a700 1 --2- 172.21.15.46:0/1068124838 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f8b9003e4c0 0x7f8b900409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b94006fd0 tx=0x7f8b94006e40 comp rx=0 tx=0).stop 2024-05-09T00:00:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.882+0000 7f8babd6a700 1 -- 172.21.15.46:0/1068124838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 msgr2=0x7f8ba41a01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.882+0000 7f8babd6a700 1 --2- 172.21.15.46:0/1068124838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 0x7f8ba41a01c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00117b0 tx=0x7f8ba0011b70 comp rx=0 tx=0).stop 2024-05-09T00:00:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.882+0000 7f8babd6a700 1 -- 172.21.15.46:0/1068124838 shutdown_connections 2024-05-09T00:00:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.882+0000 7f8babd6a700 1 --2- 172.21.15.46:0/1068124838 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f8b9003e4c0 0x7f8b900409d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.882+0000 7f8babd6a700 1 --2- 172.21.15.46:0/1068124838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ba41083c0 0x7f8ba41a01c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.882+0000 7f8babd6a700 1 -- 172.21.15.46:0/1068124838 >> 172.21.15.46:0/1068124838 conn(0x7f8ba4104050 msgr2=0x7f8ba4196780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.882+0000 7f8babd6a700 1 -- 172.21.15.46:0/1068124838 shutdown_connections 2024-05-09T00:00:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:46.882+0000 7f8babd6a700 1 -- 172.21.15.46:0/1068124838 wait complete. 2024-05-09T00:00:48.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:47 smithi046 ceph-mon[30151]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:00:48.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:47 smithi046 ceph-mon[30151]: Generating ssh key... 2024-05-09T00:00:48.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:47 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:00:48.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:47 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:00:48.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhe7ydJJpCLyxp0v0o+1VxeqNwYHVhgTFx4AdHppeI9Zn2PcYVUwcTOSRjBwV2r5Ft1LniyErXoXoF84w0rGQxH1WDoTQWdQevwFN6shdzyEBMWHqsqrGPadDoIiYa4GdStrCHgEUUhooUzL+lfBFMWDrUmZLOdHorSkN98bCZSt2+W8iE0VunhRcnIJzVAvjOaxXtcMWuXcQou5OhKfxZHWSOMp1LZhxDDgB/6UzdL8s71s4DbJohJYuThZSjkDNMUTkMYlaEhYiypRMS3oO2gdilgqDqi1L1xXF/97v5+CHxw8eAWxrZ6I0wYtMfttoxDVmyizpbjOY7Gy26iZR0iNNVWDvd2PZHtEMDuUqxCz+qydSx3gd7iL9W7Zr6UuCNVQRnVDAq1rJDQ0fzsQSrTedBeIGIqcpP+WRH65JFzohvdLvbPNgSDp4T1jKIO6Oa+5DBZgs/PLSxKBVbV4Odyu16WqkBob+R26iAvDMAI29pZxHE05U+39jq9zVQ0fc= ceph-eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:00:48.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.822+0000 7fe9446d1700 1 Processor -- start 2024-05-09T00:00:48.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.822+0000 7fe9446d1700 1 -- start start 2024-05-09T00:00:48.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.823+0000 7fe9446d1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 0x7fe93c10c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.823+0000 7fe9446d1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c07d700 con 0x7fe93c10a570 2024-05-09T00:00:48.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.823+0000 7fe94246d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 0x7fe93c10c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.823+0000 7fe94246d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 0x7fe93c10c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39126/0 (socket says 172.21.15.46:39126) 2024-05-09T00:00:48.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.823+0000 7fe94246d700 1 -- 172.21.15.46:0/463412438 learned_addr learned my addr 172.21.15.46:0/463412438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:48.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.823+0000 7fe94246d700 1 -- 172.21.15.46:0/463412438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c10ced0 con 0x7fe93c10a570 2024-05-09T00:00:48.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.824+0000 7fe94246d700 1 --2- 172.21.15.46:0/463412438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 0x7fe93c10c990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe938009e80 tx=0x7fe93800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8e1a8eeeada7e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.824+0000 7fe94146b700 1 -- 172.21.15.46:0/463412438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe938017070 con 0x7fe93c10a570 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.824+0000 7fe94146b700 1 -- 172.21.15.46:0/463412438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe93800b8c0 con 0x7fe93c10a570 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.824+0000 7fe94146b700 1 -- 172.21.15.46:0/463412438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe938004490 con 0x7fe93c10a570 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.824+0000 7fe9446d1700 1 -- 172.21.15.46:0/463412438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 msgr2=0x7fe93c10c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.824+0000 7fe9446d1700 1 --2- 172.21.15.46:0/463412438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 0x7fe93c10c990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe938009e80 tx=0x7fe93800b330 comp rx=0 tx=0).stop 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.825+0000 7fe9446d1700 1 -- 172.21.15.46:0/463412438 shutdown_connections 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.825+0000 7fe9446d1700 1 --2- 172.21.15.46:0/463412438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 0x7fe93c10c990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.825+0000 7fe9446d1700 1 -- 172.21.15.46:0/463412438 >> 172.21.15.46:0/463412438 conn(0x7fe93c103ff0 msgr2=0x7fe93c106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.825+0000 7fe9446d1700 1 -- 172.21.15.46:0/463412438 shutdown_connections 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.825+0000 7fe9446d1700 1 -- 172.21.15.46:0/463412438 wait complete. 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.825+0000 7fe9446d1700 1 Processor -- start 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.825+0000 7fe9446d1700 1 -- start start 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.826+0000 7fe9446d1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 0x7fe93c19dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.826+0000 7fe9446d1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c10de90 con 0x7fe93c10a570 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.826+0000 7fe94246d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 0x7fe93c19dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.826+0000 7fe94246d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 0x7fe93c19dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39134/0 (socket says 172.21.15.46:39134) 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.826+0000 7fe94246d700 1 -- 172.21.15.46:0/3412480589 learned_addr learned my addr 172.21.15.46:0/3412480589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.826+0000 7fe94246d700 1 -- 172.21.15.46:0/3412480589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c19e420 con 0x7fe93c10a570 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.826+0000 7fe94246d700 1 --2- 172.21.15.46:0/3412480589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 0x7fe93c19dee0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe93800b7b0 tx=0x7fe93800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.826+0000 7fe9337fe700 1 -- 172.21.15.46:0/3412480589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe938020020 con 0x7fe93c10a570 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.827+0000 7fe9337fe700 1 -- 172.21.15.46:0/3412480589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe938012430 con 0x7fe93c10a570 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.827+0000 7fe9446d1700 1 -- 172.21.15.46:0/3412480589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c19e6b0 con 0x7fe93c10a570 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.827+0000 7fe9337fe700 1 -- 172.21.15.46:0/3412480589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe938017040 con 0x7fe93c10a570 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.827+0000 7fe9446d1700 1 -- 172.21.15.46:0/3412480589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93c1a12c0 con 0x7fe93c10a570 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.828+0000 7fe9446d1700 1 -- 172.21.15.46:0/3412480589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe93c064e20 con 0x7fe93c10a570 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.828+0000 7fe9337fe700 1 -- 172.21.15.46:0/3412480589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe938023020 con 0x7fe93c10a570 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.828+0000 7fe9337fe700 1 --2- 172.21.15.46:0/3412480589 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fe92803e470 0x7fe928040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.828+0000 7fe9337fe700 1 -- 172.21.15.46:0/3412480589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe93800f050 con 0x7fe93c10a570 2024-05-09T00:00:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.828+0000 7fe941c6c700 1 --2- 172.21.15.46:0/3412480589 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fe92803e470 0x7fe928040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.829+0000 7fe941c6c700 1 --2- 172.21.15.46:0/3412480589 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fe92803e470 0x7fe928040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe92c006fd0 tx=0x7fe92c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:47.833+0000 7fe9337fe700 1 -- 172.21.15.46:0/3412480589 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe93802a020 con 0x7fe93c10a570 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:48.050+0000 7fe9446d1700 1 -- 172.21.15.46:0/3412480589 --> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fe93c1050c0 con 0x7fe92803e470 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:48.050+0000 7fe9337fe700 1 -- 172.21.15.46:0/3412480589 <== mgr.14120 v2:172.21.15.46:6800/1620504773 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fe93c1050c0 con 0x7fe92803e470 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:48.052+0000 7fe9446d1700 1 -- 172.21.15.46:0/3412480589 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fe92803e470 msgr2=0x7fe928040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:48.052+0000 7fe9446d1700 1 --2- 172.21.15.46:0/3412480589 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fe92803e470 0x7fe928040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe92c006fd0 tx=0x7fe92c006e40 comp rx=0 tx=0).stop 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:48.052+0000 7fe9446d1700 1 -- 172.21.15.46:0/3412480589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 msgr2=0x7fe93c19dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:48.052+0000 7fe9446d1700 1 --2- 172.21.15.46:0/3412480589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 0x7fe93c19dee0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe93800b7b0 tx=0x7fe93800bd50 comp rx=0 tx=0).stop 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:48.053+0000 7fe9446d1700 1 -- 172.21.15.46:0/3412480589 shutdown_connections 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:48.053+0000 7fe9446d1700 1 --2- 172.21.15.46:0/3412480589 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fe92803e470 0x7fe928040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:48.053+0000 7fe9446d1700 1 --2- 172.21.15.46:0/3412480589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c10a570 0x7fe93c19dee0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:48.053+0000 7fe9446d1700 1 -- 172.21.15.46:0/3412480589 >> 172.21.15.46:0/3412480589 conn(0x7fe93c103ff0 msgr2=0x7fe93c104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:48.053+0000 7fe9446d1700 1 -- 172.21.15.46:0/3412480589 shutdown_connections 2024-05-09T00:00:48.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:48.053+0000 7fe9446d1700 1 -- 172.21.15.46:0/3412480589 wait complete. 2024-05-09T00:00:48.629 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-09T00:00:48.629 INFO:teuthology.orchestra.run.smithi046.stdout:Adding key to root@localhost authorized_keys... 2024-05-09T00:00:48.629 INFO:teuthology.orchestra.run.smithi046.stdout:Adding host smithi046... 2024-05-09T00:00:49.266 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:48 smithi046 ceph-mon[30151]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:00:50.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:49 smithi046 ceph-mon[30151]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:00:51.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:50 smithi046 ceph-mon[30151]: Deploying cephadm binary to smithi046 2024-05-09T00:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Added host 'smithi046' with addr '172.21.15.46' 2024-05-09T00:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.040+0000 7fee7ee66700 1 Processor -- start 2024-05-09T00:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.040+0000 7fee7ee66700 1 -- start start 2024-05-09T00:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.041+0000 7fee7ee66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 0x7fee7810c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.041+0000 7fee7ee66700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee7807ce30 con 0x7fee7810a590 2024-05-09T00:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.041+0000 7fee7cc02700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 0x7fee7810c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.041+0000 7fee7cc02700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 0x7fee7810c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39140/0 (socket says 172.21.15.46:39140) 2024-05-09T00:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.041+0000 7fee7cc02700 1 -- 172.21.15.46:0/972579751 learned_addr learned my addr 172.21.15.46:0/972579751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.041+0000 7fee7cc02700 1 -- 172.21.15.46:0/972579751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7810cef0 con 0x7fee7810a590 2024-05-09T00:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.041+0000 7fee7cc02700 1 --2- 172.21.15.46:0/972579751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 0x7fee7810c9b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0098a0 tx=0x7fee6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bf454ef1c223f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.042+0000 7fee777fe700 1 -- 172.21.15.46:0/972579751 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee6c004160 con 0x7fee7810a590 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.042+0000 7fee777fe700 1 -- 172.21.15.46:0/972579751 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fee6c004300 con 0x7fee7810a590 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.042+0000 7fee777fe700 1 -- 172.21.15.46:0/972579751 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee6c011af0 con 0x7fee7810a590 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.042+0000 7fee7ee66700 1 -- 172.21.15.46:0/972579751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 msgr2=0x7fee7810c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.042+0000 7fee7ee66700 1 --2- 172.21.15.46:0/972579751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 0x7fee7810c9b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0098a0 tx=0x7fee6c009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.043+0000 7fee7ee66700 1 -- 172.21.15.46:0/972579751 shutdown_connections 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.043+0000 7fee7ee66700 1 --2- 172.21.15.46:0/972579751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 0x7fee7810c9b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.043+0000 7fee7ee66700 1 -- 172.21.15.46:0/972579751 >> 172.21.15.46:0/972579751 conn(0x7fee78104050 msgr2=0x7fee781064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.043+0000 7fee7ee66700 1 -- 172.21.15.46:0/972579751 shutdown_connections 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.043+0000 7fee7ee66700 1 -- 172.21.15.46:0/972579751 wait complete. 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.043+0000 7fee7ee66700 1 Processor -- start 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.043+0000 7fee7ee66700 1 -- start start 2024-05-09T00:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.044+0000 7fee7ee66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 0x7fee7819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.044+0000 7fee7ee66700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee7810de50 con 0x7fee7810a590 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.044+0000 7fee7cc02700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 0x7fee7819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.044+0000 7fee7cc02700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 0x7fee7819dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39156/0 (socket says 172.21.15.46:39156) 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.044+0000 7fee7cc02700 1 -- 172.21.15.46:0/674894616 learned_addr learned my addr 172.21.15.46:0/674894616 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.044+0000 7fee7cc02700 1 -- 172.21.15.46:0/674894616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7819e3e0 con 0x7fee7810a590 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.044+0000 7fee7cc02700 1 --2- 172.21.15.46:0/674894616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 0x7fee7819dea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0117b0 tx=0x7fee6c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.044+0000 7fee75ffb700 1 -- 172.21.15.46:0/674894616 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee6c0039c0 con 0x7fee7810a590 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.044+0000 7fee75ffb700 1 -- 172.21.15.46:0/674894616 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fee6c003b60 con 0x7fee7810a590 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.044+0000 7fee75ffb700 1 -- 172.21.15.46:0/674894616 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee6c0196a0 con 0x7fee7810a590 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.044+0000 7fee7ee66700 1 -- 172.21.15.46:0/674894616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7819e670 con 0x7fee7810a590 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.045+0000 7fee7ee66700 1 -- 172.21.15.46:0/674894616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee781a1280 con 0x7fee7810a590 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.046+0000 7fee7ee66700 1 -- 172.21.15.46:0/674894616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee78064e20 con 0x7fee7810a590 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.046+0000 7fee75ffb700 1 -- 172.21.15.46:0/674894616 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fee6c01ab00 con 0x7fee7810a590 2024-05-09T00:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.046+0000 7fee75ffb700 1 --2- 172.21.15.46:0/674894616 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fee6803e4c0 0x7fee680409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.046+0000 7fee75ffb700 1 -- 172.21.15.46:0/674894616 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fee6c053b10 con 0x7fee7810a590 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.046+0000 7fee77fff700 1 --2- 172.21.15.46:0/674894616 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fee6803e4c0 0x7fee680409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.047+0000 7fee77fff700 1 --2- 172.21.15.46:0/674894616 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fee6803e4c0 0x7fee680409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fee64006fd0 tx=0x7fee64006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.051+0000 7fee75ffb700 1 -- 172.21.15.46:0/674894616 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee6c019840 con 0x7fee7810a590 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:49.272+0000 7fee7ee66700 1 -- 172.21.15.46:0/674894616 --> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}) -- 0x7fee78106220 con 0x7fee6803e4c0 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:53.565+0000 7fee75ffb700 1 -- 172.21.15.46:0/674894616 <== mgr.14120 v2:172.21.15.46:6800/1620504773 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fee78106220 con 0x7fee6803e4c0 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:53.569+0000 7fee7ee66700 1 -- 172.21.15.46:0/674894616 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fee6803e4c0 msgr2=0x7fee680409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:53.569+0000 7fee7ee66700 1 --2- 172.21.15.46:0/674894616 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fee6803e4c0 0x7fee680409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fee64006fd0 tx=0x7fee64006e40 comp rx=0 tx=0).stop 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:53.569+0000 7fee7ee66700 1 -- 172.21.15.46:0/674894616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 msgr2=0x7fee7819dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:53.569+0000 7fee7ee66700 1 --2- 172.21.15.46:0/674894616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 0x7fee7819dea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0117b0 tx=0x7fee6c011b70 comp rx=0 tx=0).stop 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:53.569+0000 7fee7ee66700 1 -- 172.21.15.46:0/674894616 shutdown_connections 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:53.569+0000 7fee7ee66700 1 --2- 172.21.15.46:0/674894616 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fee6803e4c0 0x7fee680409d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:53.569+0000 7fee7ee66700 1 --2- 172.21.15.46:0/674894616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee7810a590 0x7fee7819dea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:53.569+0000 7fee7ee66700 1 -- 172.21.15.46:0/674894616 >> 172.21.15.46:0/674894616 conn(0x7fee78104050 msgr2=0x7fee78105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:53.569+0000 7fee7ee66700 1 -- 172.21.15.46:0/674894616 shutdown_connections 2024-05-09T00:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:53.569+0000 7fee7ee66700 1 -- 172.21.15.46:0/674894616 wait complete. 2024-05-09T00:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mon service... 2024-05-09T00:00:54.729 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:54 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:00:54.729 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:54 smithi046 ceph-mon[30151]: Added host smithi046 2024-05-09T00:00:54.729 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:54 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:00:55.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-09T00:00:55.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.777+0000 7fd5d7515700 1 Processor -- start 2024-05-09T00:00:55.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.777+0000 7fd5d7515700 1 -- start start 2024-05-09T00:00:55.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.777+0000 7fd5d7515700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 0x7fd5d010a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:55.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.777+0000 7fd5d7515700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d010a920 con 0x7fd5d0109f00 2024-05-09T00:00:55.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.778+0000 7fd5d52b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 0x7fd5d010a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:55.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.778+0000 7fd5d52b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 0x7fd5d010a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39158/0 (socket says 172.21.15.46:39158) 2024-05-09T00:00:55.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.778+0000 7fd5d52b1700 1 -- 172.21.15.46:0/2937414114 learned_addr learned my addr 172.21.15.46:0/2937414114 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:55.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.778+0000 7fd5d52b1700 1 -- 172.21.15.46:0/2937414114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d0108b30 con 0x7fd5d0109f00 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.778+0000 7fd5d52b1700 1 --2- 172.21.15.46:0/2937414114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 0x7fd5d010a310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc0098a0 tx=0x7fd5cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e1a8b86e93910a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.778+0000 7fd5c7fff700 1 -- 172.21.15.46:0/2937414114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5cc004160 con 0x7fd5d0109f00 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.778+0000 7fd5c7fff700 1 -- 172.21.15.46:0/2937414114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd5cc004300 con 0x7fd5d0109f00 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.778+0000 7fd5c7fff700 1 -- 172.21.15.46:0/2937414114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5cc011af0 con 0x7fd5d0109f00 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.779+0000 7fd5d7515700 1 -- 172.21.15.46:0/2937414114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 msgr2=0x7fd5d010a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.779+0000 7fd5d7515700 1 --2- 172.21.15.46:0/2937414114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 0x7fd5d010a310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc0098a0 tx=0x7fd5cc009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.779+0000 7fd5d7515700 1 -- 172.21.15.46:0/2937414114 shutdown_connections 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.779+0000 7fd5d7515700 1 --2- 172.21.15.46:0/2937414114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 0x7fd5d010a310 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.779+0000 7fd5d7515700 1 -- 172.21.15.46:0/2937414114 >> 172.21.15.46:0/2937414114 conn(0x7fd5d0103770 msgr2=0x7fd5d0105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.779+0000 7fd5d7515700 1 -- 172.21.15.46:0/2937414114 shutdown_connections 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.779+0000 7fd5d7515700 1 -- 172.21.15.46:0/2937414114 wait complete. 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.780+0000 7fd5d7515700 1 Processor -- start 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.780+0000 7fd5d7515700 1 -- start start 2024-05-09T00:00:55.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.780+0000 7fd5d7515700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 0x7fd5d01083a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.780+0000 7fd5d7515700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d010cef0 con 0x7fd5d0109f00 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.780+0000 7fd5d52b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 0x7fd5d01083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.780+0000 7fd5d52b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 0x7fd5d01083a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39166/0 (socket says 172.21.15.46:39166) 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.780+0000 7fd5d52b1700 1 -- 172.21.15.46:0/1589075217 learned_addr learned my addr 172.21.15.46:0/1589075217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.781+0000 7fd5d52b1700 1 -- 172.21.15.46:0/1589075217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d01088e0 con 0x7fd5d0109f00 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.781+0000 7fd5d52b1700 1 --2- 172.21.15.46:0/1589075217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 0x7fd5d01083a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009fb0 tx=0x7fd5cc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.781+0000 7fd5c67fc700 1 -- 172.21.15.46:0/1589075217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5cc011bc0 con 0x7fd5d0109f00 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.781+0000 7fd5d7515700 1 -- 172.21.15.46:0/1589075217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d01a0c80 con 0x7fd5d0109f00 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.781+0000 7fd5d7515700 1 -- 172.21.15.46:0/1589075217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d01a1100 con 0x7fd5d0109f00 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.781+0000 7fd5c67fc700 1 -- 172.21.15.46:0/1589075217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd5cc024440 con 0x7fd5d0109f00 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.781+0000 7fd5c67fc700 1 -- 172.21.15.46:0/1589075217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5cc01a5c0 con 0x7fd5d0109f00 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.782+0000 7fd5c67fc700 1 -- 172.21.15.46:0/1589075217 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd5cc01a760 con 0x7fd5d0109f00 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.782+0000 7fd5d7515700 1 -- 172.21.15.46:0/1589075217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b4005360 con 0x7fd5d0109f00 2024-05-09T00:00:55.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.782+0000 7fd5c67fc700 1 --2- 172.21.15.46:0/1589075217 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fd5bc03e420 0x7fd5bc040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:55.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.783+0000 7fd5c67fc700 1 -- 172.21.15.46:0/1589075217 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd5cc053720 con 0x7fd5d0109f00 2024-05-09T00:00:55.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.783+0000 7fd5d4ab0700 1 --2- 172.21.15.46:0/1589075217 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fd5bc03e420 0x7fd5bc040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:55.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.783+0000 7fd5d4ab0700 1 --2- 172.21.15.46:0/1589075217 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fd5bc03e420 0x7fd5bc040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0006fd0 tx=0x7fd5c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:55.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:54.788+0000 7fd5c67fc700 1 -- 172.21.15.46:0/1589075217 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd5cc02c350 con 0x7fd5d0109f00 2024-05-09T00:00:55.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:55.027+0000 7fd5d7515700 1 -- 172.21.15.46:0/1589075217 --> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd5b4000bf0 con 0x7fd5bc03e420 2024-05-09T00:00:55.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:55.032+0000 7fd5c67fc700 1 -- 172.21.15.46:0/1589075217 <== mgr.14120 v2:172.21.15.46:6800/1620504773 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd5b4000bf0 con 0x7fd5bc03e420 2024-05-09T00:00:55.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:55.035+0000 7fd5d7515700 1 -- 172.21.15.46:0/1589075217 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fd5bc03e420 msgr2=0x7fd5bc040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:55.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:55.035+0000 7fd5d7515700 1 --2- 172.21.15.46:0/1589075217 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fd5bc03e420 0x7fd5bc040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0006fd0 tx=0x7fd5c0006e40 comp rx=0 tx=0).stop 2024-05-09T00:00:55.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:55.035+0000 7fd5d7515700 1 -- 172.21.15.46:0/1589075217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 msgr2=0x7fd5d01083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:55.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:55.035+0000 7fd5d7515700 1 --2- 172.21.15.46:0/1589075217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 0x7fd5d01083a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009fb0 tx=0x7fd5cc011890 comp rx=0 tx=0).stop 2024-05-09T00:00:55.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:55.035+0000 7fd5d7515700 1 -- 172.21.15.46:0/1589075217 shutdown_connections 2024-05-09T00:00:55.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:55.035+0000 7fd5d7515700 1 --2- 172.21.15.46:0/1589075217 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fd5bc03e420 0x7fd5bc040930 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:55.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:55.035+0000 7fd5d7515700 1 --2- 172.21.15.46:0/1589075217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5d0109f00 0x7fd5d01083a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:55.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:55.035+0000 7fd5d7515700 1 -- 172.21.15.46:0/1589075217 >> 172.21.15.46:0/1589075217 conn(0x7fd5d0103770 msgr2=0x7fd5d0104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:55.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:55.035+0000 7fd5d7515700 1 -- 172.21.15.46:0/1589075217 shutdown_connections 2024-05-09T00:00:55.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:55.035+0000 7fd5d7515700 1 -- 172.21.15.46:0/1589075217 wait complete. 2024-05-09T00:00:55.740 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mgr service... 2024-05-09T00:00:56.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:56 smithi046 ceph-mon[30151]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:00:56.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:56 smithi046 ceph-mon[30151]: Saving service mon spec with placement count:5 2024-05-09T00:00:56.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:56 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:00:57.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-09T00:00:57.240 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.375+0000 7f2d4f660700 1 Processor -- start 2024-05-09T00:00:57.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.375+0000 7f2d4f660700 1 -- start start 2024-05-09T00:00:57.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.376+0000 7f2d4f660700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 0x7f2d4810c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:57.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.376+0000 7f2d4f660700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4807ce70 con 0x7f2d4810a550 2024-05-09T00:00:57.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.376+0000 7f2d4d3fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 0x7f2d4810c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:57.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.376+0000 7f2d4d3fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 0x7f2d4810c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49928/0 (socket says 172.21.15.46:49928) 2024-05-09T00:00:57.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.376+0000 7f2d4d3fc700 1 -- 172.21.15.46:0/1025497788 learned_addr learned my addr 172.21.15.46:0/1025497788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:57.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.376+0000 7f2d4d3fc700 1 -- 172.21.15.46:0/1025497788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4810ceb0 con 0x7f2d4810a550 2024-05-09T00:00:57.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.376+0000 7f2d4d3fc700 1 --2- 172.21.15.46:0/1025497788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 0x7f2d4810c970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d440098a0 tx=0x7f2d44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab0b090816f5513 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:57.241 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.376+0000 7f2d3ffff700 1 -- 172.21.15.46:0/1025497788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d44004160 con 0x7f2d4810a550 2024-05-09T00:00:57.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.376+0000 7f2d3ffff700 1 -- 172.21.15.46:0/1025497788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2d44004300 con 0x7f2d4810a550 2024-05-09T00:00:57.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.376+0000 7f2d3ffff700 1 -- 172.21.15.46:0/1025497788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d44011af0 con 0x7f2d4810a550 2024-05-09T00:00:57.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.377+0000 7f2d4f660700 1 -- 172.21.15.46:0/1025497788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 msgr2=0x7f2d4810c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:57.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.377+0000 7f2d4f660700 1 --2- 172.21.15.46:0/1025497788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 0x7f2d4810c970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d440098a0 tx=0x7f2d44009bb0 comp rx=0 tx=0).stop 2024-05-09T00:00:57.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.377+0000 7f2d4f660700 1 -- 172.21.15.46:0/1025497788 shutdown_connections 2024-05-09T00:00:57.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.377+0000 7f2d4f660700 1 --2- 172.21.15.46:0/1025497788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 0x7f2d4810c970 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:57.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.377+0000 7f2d4f660700 1 -- 172.21.15.46:0/1025497788 >> 172.21.15.46:0/1025497788 conn(0x7f2d48104050 msgr2=0x7f2d481064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:57.242 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.377+0000 7f2d4f660700 1 -- 172.21.15.46:0/1025497788 shutdown_connections 2024-05-09T00:00:57.243 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.377+0000 7f2d4f660700 1 -- 172.21.15.46:0/1025497788 wait complete. 2024-05-09T00:00:57.243 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.378+0000 7f2d4f660700 1 Processor -- start 2024-05-09T00:00:57.243 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.378+0000 7f2d4f660700 1 -- start start 2024-05-09T00:00:57.243 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.378+0000 7f2d4f660700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 0x7f2d4819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:57.243 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.378+0000 7f2d4f660700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4810de70 con 0x7f2d4810a550 2024-05-09T00:00:57.243 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.378+0000 7f2d4d3fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 0x7f2d4819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:57.243 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.378+0000 7f2d4d3fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 0x7f2d4819dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49934/0 (socket says 172.21.15.46:49934) 2024-05-09T00:00:57.243 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.379+0000 7f2d4d3fc700 1 -- 172.21.15.46:0/1576268612 learned_addr learned my addr 172.21.15.46:0/1576268612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:57.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.379+0000 7f2d4d3fc700 1 -- 172.21.15.46:0/1576268612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4819e400 con 0x7f2d4810a550 2024-05-09T00:00:57.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.379+0000 7f2d4d3fc700 1 --2- 172.21.15.46:0/1576268612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 0x7f2d4819dec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d440117b0 tx=0x7f2d44011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:57.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.379+0000 7f2d3e7fc700 1 -- 172.21.15.46:0/1576268612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d440039c0 con 0x7f2d4810a550 2024-05-09T00:00:57.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.379+0000 7f2d3e7fc700 1 -- 172.21.15.46:0/1576268612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2d44003b60 con 0x7f2d4810a550 2024-05-09T00:00:57.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.379+0000 7f2d4f660700 1 -- 172.21.15.46:0/1576268612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4819e690 con 0x7f2d4810a550 2024-05-09T00:00:57.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.379+0000 7f2d3e7fc700 1 -- 172.21.15.46:0/1576268612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d440196a0 con 0x7f2d4810a550 2024-05-09T00:00:57.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.379+0000 7f2d4f660700 1 -- 172.21.15.46:0/1576268612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d481a12a0 con 0x7f2d4810a550 2024-05-09T00:00:57.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.380+0000 7f2d4f660700 1 -- 172.21.15.46:0/1576268612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d48064e20 con 0x7f2d4810a550 2024-05-09T00:00:57.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.380+0000 7f2d3e7fc700 1 -- 172.21.15.46:0/1576268612 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2d44019910 con 0x7f2d4810a550 2024-05-09T00:00:57.245 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.381+0000 7f2d3e7fc700 1 --2- 172.21.15.46:0/1576268612 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f2d3403e4c0 0x7f2d340409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:57.245 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.381+0000 7f2d3e7fc700 1 -- 172.21.15.46:0/1576268612 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2d44053c00 con 0x7f2d4810a550 2024-05-09T00:00:57.245 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.381+0000 7f2d4cbfb700 1 --2- 172.21.15.46:0/1576268612 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f2d3403e4c0 0x7f2d340409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:57.245 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.381+0000 7f2d4cbfb700 1 --2- 172.21.15.46:0/1576268612 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f2d3403e4c0 0x7f2d340409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2d38006fd0 tx=0x7f2d38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:57.245 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.386+0000 7f2d3e7fc700 1 -- 172.21.15.46:0/1576268612 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d4402ce20 con 0x7f2d4810a550 2024-05-09T00:00:57.245 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.621+0000 7f2d4f660700 1 -- 172.21.15.46:0/1576268612 --> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2d481061f0 con 0x7f2d3403e4c0 2024-05-09T00:00:57.245 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.626+0000 7f2d3e7fc700 1 -- 172.21.15.46:0/1576268612 <== mgr.14120 v2:172.21.15.46:6800/1620504773 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2d481061f0 con 0x7f2d3403e4c0 2024-05-09T00:00:57.245 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.628+0000 7f2d4f660700 1 -- 172.21.15.46:0/1576268612 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f2d3403e4c0 msgr2=0x7f2d340409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:57.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.628+0000 7f2d4f660700 1 --2- 172.21.15.46:0/1576268612 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f2d3403e4c0 0x7f2d340409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2d38006fd0 tx=0x7f2d38006e40 comp rx=0 tx=0).stop 2024-05-09T00:00:57.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.628+0000 7f2d4f660700 1 -- 172.21.15.46:0/1576268612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 msgr2=0x7f2d4819dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:57.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.628+0000 7f2d4f660700 1 --2- 172.21.15.46:0/1576268612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 0x7f2d4819dec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d440117b0 tx=0x7f2d44011b70 comp rx=0 tx=0).stop 2024-05-09T00:00:57.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.629+0000 7f2d4f660700 1 -- 172.21.15.46:0/1576268612 shutdown_connections 2024-05-09T00:00:57.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.629+0000 7f2d4f660700 1 --2- 172.21.15.46:0/1576268612 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f2d3403e4c0 0x7f2d340409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:57.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.629+0000 7f2d4f660700 1 --2- 172.21.15.46:0/1576268612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d4810a550 0x7f2d4819dec0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:57.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.629+0000 7f2d4f660700 1 -- 172.21.15.46:0/1576268612 >> 172.21.15.46:0/1576268612 conn(0x7f2d48104050 msgr2=0x7f2d48105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:57.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.629+0000 7f2d4f660700 1 -- 172.21.15.46:0/1576268612 shutdown_connections 2024-05-09T00:00:57.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:56.629+0000 7f2d4f660700 1 -- 172.21.15.46:0/1576268612 wait complete. 2024-05-09T00:00:57.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:57 smithi046 ceph-mon[30151]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:00:57.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:57 smithi046 ceph-mon[30151]: Saving service mgr spec with placement count:2 2024-05-09T00:00:57.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:57 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:00:58.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.680+0000 7f70b47c0700 1 Processor -- start 2024-05-09T00:00:58.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.680+0000 7f70b47c0700 1 -- start start 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.681+0000 7f70b47c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 0x7f70ac10c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.681+0000 7f70b47c0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac07d840 con 0x7f70ac10a590 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.681+0000 7f70b255c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 0x7f70ac10c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.681+0000 7f70b255c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 0x7f70ac10c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49948/0 (socket says 172.21.15.46:49948) 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.681+0000 7f70b255c700 1 -- 172.21.15.46:0/1785570069 learned_addr learned my addr 172.21.15.46:0/1785570069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.681+0000 7f70b255c700 1 -- 172.21.15.46:0/1785570069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac10cef0 con 0x7f70ac10a590 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.682+0000 7f70b255c700 1 --2- 172.21.15.46:0/1785570069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 0x7f70ac10c9b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009e80 tx=0x7f70a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa4249e1231c96d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.682+0000 7f70b155a700 1 -- 172.21.15.46:0/1785570069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70a8017070 con 0x7f70ac10a590 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.682+0000 7f70b155a700 1 -- 172.21.15.46:0/1785570069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f70a800b8c0 con 0x7f70ac10a590 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.682+0000 7f70b155a700 1 -- 172.21.15.46:0/1785570069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70a8004490 con 0x7f70ac10a590 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.682+0000 7f70b47c0700 1 -- 172.21.15.46:0/1785570069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 msgr2=0x7f70ac10c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.682+0000 7f70b47c0700 1 --2- 172.21.15.46:0/1785570069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 0x7f70ac10c9b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009e80 tx=0x7f70a800b330 comp rx=0 tx=0).stop 2024-05-09T00:00:58.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.683+0000 7f70b47c0700 1 -- 172.21.15.46:0/1785570069 shutdown_connections 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.683+0000 7f70b47c0700 1 --2- 172.21.15.46:0/1785570069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 0x7f70ac10c9b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.683+0000 7f70b47c0700 1 -- 172.21.15.46:0/1785570069 >> 172.21.15.46:0/1785570069 conn(0x7f70ac104010 msgr2=0x7f70ac106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.683+0000 7f70b47c0700 1 -- 172.21.15.46:0/1785570069 shutdown_connections 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.683+0000 7f70b47c0700 1 -- 172.21.15.46:0/1785570069 wait complete. 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.683+0000 7f70b47c0700 1 Processor -- start 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.683+0000 7f70b47c0700 1 -- start start 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.684+0000 7f70b47c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 0x7f70ac109d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.684+0000 7f70b47c0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac10de50 con 0x7f70ac10a590 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.684+0000 7f70b255c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 0x7f70ac109d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.684+0000 7f70b255c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 0x7f70ac109d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49950/0 (socket says 172.21.15.46:49950) 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.684+0000 7f70b255c700 1 -- 172.21.15.46:0/2097864263 learned_addr learned my addr 172.21.15.46:0/2097864263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.684+0000 7f70b255c700 1 -- 172.21.15.46:0/2097864263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac10a250 con 0x7f70ac10a590 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.684+0000 7f70b255c700 1 --2- 172.21.15.46:0/2097864263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 0x7f70ac109d10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f70a800b7b0 tx=0x7f70a800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.684+0000 7f70a37fe700 1 -- 172.21.15.46:0/2097864263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70a8020020 con 0x7f70ac10a590 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.684+0000 7f70a37fe700 1 -- 172.21.15.46:0/2097864263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f70a8012430 con 0x7f70ac10a590 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.684+0000 7f70b47c0700 1 -- 172.21.15.46:0/2097864263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ac1084a0 con 0x7f70ac10a590 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.684+0000 7f70a37fe700 1 -- 172.21.15.46:0/2097864263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70a8017040 con 0x7f70ac10a590 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.684+0000 7f70b47c0700 1 -- 172.21.15.46:0/2097864263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ac1089e0 con 0x7f70ac10a590 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.685+0000 7f70b47c0700 1 -- 172.21.15.46:0/2097864263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70ac064e20 con 0x7f70ac10a590 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.685+0000 7f70a37fe700 1 -- 172.21.15.46:0/2097864263 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f70a8023020 con 0x7f70ac10a590 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.686+0000 7f70a37fe700 1 --2- 172.21.15.46:0/2097864263 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f709803e470 0x7f7098040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.686+0000 7f70a37fe700 1 -- 172.21.15.46:0/2097864263 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f70a800f050 con 0x7f70ac10a590 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.686+0000 7f70b1d5b700 1 --2- 172.21.15.46:0/2097864263 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f709803e470 0x7f7098040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.686+0000 7f70b1d5b700 1 --2- 172.21.15.46:0/2097864263 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f709803e470 0x7f7098040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f709c006fd0 tx=0x7f709c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.691+0000 7f70a37fe700 1 -- 172.21.15.46:0/2097864263 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70a802a020 con 0x7f70ac10a590 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.908+0000 7f70b47c0700 1 -- 172.21.15.46:0/2097864263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f70ac1090d0 con 0x7f70ac10a590 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.911+0000 7f70a37fe700 1 -- 172.21.15.46:0/2097864263 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f70a8011390 con 0x7f70ac10a590 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.914+0000 7f70b47c0700 1 -- 172.21.15.46:0/2097864263 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f709803e470 msgr2=0x7f7098040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:58.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.914+0000 7f70b47c0700 1 --2- 172.21.15.46:0/2097864263 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f709803e470 0x7f7098040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f709c006fd0 tx=0x7f709c006e40 comp rx=0 tx=0).stop 2024-05-09T00:00:58.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.914+0000 7f70b47c0700 1 -- 172.21.15.46:0/2097864263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 msgr2=0x7f70ac109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:00:58.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.914+0000 7f70b47c0700 1 --2- 172.21.15.46:0/2097864263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 0x7f70ac109d10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f70a800b7b0 tx=0x7f70a800bd50 comp rx=0 tx=0).stop 2024-05-09T00:00:58.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.914+0000 7f70b47c0700 1 -- 172.21.15.46:0/2097864263 shutdown_connections 2024-05-09T00:00:58.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.914+0000 7f70b47c0700 1 --2- 172.21.15.46:0/2097864263 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f709803e470 0x7f7098040980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:58.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.914+0000 7f70b47c0700 1 --2- 172.21.15.46:0/2097864263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac10a590 0x7f70ac109d10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:00:58.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.914+0000 7f70b47c0700 1 -- 172.21.15.46:0/2097864263 >> 172.21.15.46:0/2097864263 conn(0x7f70ac104010 msgr2=0x7f70ac1058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:00:58.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.914+0000 7f70b47c0700 1 -- 172.21.15.46:0/2097864263 shutdown_connections 2024-05-09T00:00:58.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:57.914+0000 7f70b47c0700 1 -- 172.21.15.46:0/2097864263 wait complete. 2024-05-09T00:00:59.239 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:58 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2097864263' entity='client.admin' 2024-05-09T00:00:59.239 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:58 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:00:59.239 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:00:58 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:01:00.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.015+0000 7f63a2c77700 1 Processor -- start 2024-05-09T00:01:00.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.015+0000 7f63a2c77700 1 -- start start 2024-05-09T00:01:00.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.015+0000 7f63a2c77700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 0x7f639c108d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:00.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.015+0000 7f63a2c77700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f639c109370 con 0x7f639c108970 2024-05-09T00:01:00.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.015+0000 7f63a0a13700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 0x7f639c108d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:00.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.015+0000 7f63a0a13700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 0x7f639c108d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49954/0 (socket says 172.21.15.46:49954) 2024-05-09T00:01:00.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.015+0000 7f63a0a13700 1 -- 172.21.15.46:0/1410534579 learned_addr learned my addr 172.21.15.46:0/1410534579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:00.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.016+0000 7f63a0a13700 1 -- 172.21.15.46:0/1410534579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c109b40 con 0x7f639c108970 2024-05-09T00:01:00.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.016+0000 7f63a0a13700 1 --2- 172.21.15.46:0/1410534579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 0x7f639c108d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f63900098a0 tx=0x7f6390009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3256c6bd74c92236 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:00.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.016+0000 7f639b7fe700 1 -- 172.21.15.46:0/1410534579 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6390004160 con 0x7f639c108970 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.016+0000 7f639b7fe700 1 -- 172.21.15.46:0/1410534579 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6390004300 con 0x7f639c108970 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.016+0000 7f639b7fe700 1 -- 172.21.15.46:0/1410534579 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6390011a20 con 0x7f639c108970 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.017+0000 7f63a2c77700 1 -- 172.21.15.46:0/1410534579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 msgr2=0x7f639c108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.017+0000 7f63a2c77700 1 --2- 172.21.15.46:0/1410534579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 0x7f639c108d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f63900098a0 tx=0x7f6390009bb0 comp rx=0 tx=0).stop 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.017+0000 7f63a2c77700 1 -- 172.21.15.46:0/1410534579 shutdown_connections 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.017+0000 7f63a2c77700 1 --2- 172.21.15.46:0/1410534579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 0x7f639c108d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.017+0000 7f63a2c77700 1 -- 172.21.15.46:0/1410534579 >> 172.21.15.46:0/1410534579 conn(0x7f639c104050 msgr2=0x7f639c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.017+0000 7f63a2c77700 1 -- 172.21.15.46:0/1410534579 shutdown_connections 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.017+0000 7f63a2c77700 1 -- 172.21.15.46:0/1410534579 wait complete. 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.018+0000 7f63a2c77700 1 Processor -- start 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.018+0000 7f63a2c77700 1 -- start start 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.018+0000 7f63a2c77700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 0x7f639c07c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.018+0000 7f63a2c77700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f639c10cef0 con 0x7f639c108970 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.018+0000 7f63a0a13700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 0x7f639c07c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:00.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.018+0000 7f63a0a13700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 0x7f639c07c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49966/0 (socket says 172.21.15.46:49966) 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.018+0000 7f63a0a13700 1 -- 172.21.15.46:0/2694026694 learned_addr learned my addr 172.21.15.46:0/2694026694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.018+0000 7f63a0a13700 1 -- 172.21.15.46:0/2694026694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c07cea0 con 0x7f639c108970 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.019+0000 7f63a0a13700 1 --2- 172.21.15.46:0/2694026694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 0x7f639c07c960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6390009130 tx=0x7f6390011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.019+0000 7f6399ffb700 1 -- 172.21.15.46:0/2694026694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6390003aa0 con 0x7f639c108970 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.019+0000 7f6399ffb700 1 -- 172.21.15.46:0/2694026694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f639001a8e0 con 0x7f639c108970 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.019+0000 7f63a2c77700 1 -- 172.21.15.46:0/2694026694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639c1a0c80 con 0x7f639c108970 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.019+0000 7f6399ffb700 1 -- 172.21.15.46:0/2694026694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63900248f0 con 0x7f639c108970 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.019+0000 7f63a2c77700 1 -- 172.21.15.46:0/2694026694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639c1a1100 con 0x7f639c108970 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.020+0000 7f6399ffb700 1 -- 172.21.15.46:0/2694026694 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f6390022020 con 0x7f639c108970 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.020+0000 7f63a2c77700 1 -- 172.21.15.46:0/2694026694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639c064e20 con 0x7f639c108970 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.020+0000 7f6399ffb700 1 --2- 172.21.15.46:0/2694026694 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f638c03e470 0x7f638c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.021+0000 7f6399ffb700 1 -- 172.21.15.46:0/2694026694 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6390053860 con 0x7f639c108970 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.021+0000 7f639bfff700 1 --2- 172.21.15.46:0/2694026694 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f638c03e470 0x7f638c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:00.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.021+0000 7f639bfff700 1 --2- 172.21.15.46:0/2694026694 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f638c03e470 0x7f638c040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6388006fd0 tx=0x7f6388006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.026+0000 7f6399ffb700 1 -- 172.21.15.46:0/2694026694 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6390029df0 con 0x7f639c108970 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.244+0000 7f63a2c77700 1 -- 172.21.15.46:0/2694026694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f639c1a1380 con 0x7f639c108970 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.247+0000 7f6399ffb700 1 -- 172.21.15.46:0/2694026694 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f639001aa90 con 0x7f639c108970 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.250+0000 7f63a2c77700 1 -- 172.21.15.46:0/2694026694 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f638c03e470 msgr2=0x7f638c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.250+0000 7f63a2c77700 1 --2- 172.21.15.46:0/2694026694 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f638c03e470 0x7f638c040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6388006fd0 tx=0x7f6388006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.250+0000 7f63a2c77700 1 -- 172.21.15.46:0/2694026694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 msgr2=0x7f639c07c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.250+0000 7f63a2c77700 1 --2- 172.21.15.46:0/2694026694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 0x7f639c07c960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6390009130 tx=0x7f6390011aa0 comp rx=0 tx=0).stop 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.250+0000 7f63a2c77700 1 -- 172.21.15.46:0/2694026694 shutdown_connections 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.250+0000 7f63a2c77700 1 --2- 172.21.15.46:0/2694026694 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f638c03e470 0x7f638c040980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.250+0000 7f63a2c77700 1 --2- 172.21.15.46:0/2694026694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c108970 0x7f639c07c960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.250+0000 7f63a2c77700 1 -- 172.21.15.46:0/2694026694 >> 172.21.15.46:0/2694026694 conn(0x7f639c104050 msgr2=0x7f639c105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.250+0000 7f63a2c77700 1 -- 172.21.15.46:0/2694026694 shutdown_connections 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:00:59.251+0000 7f63a2c77700 1 -- 172.21.15.46:0/2694026694 wait complete. 2024-05-09T00:01:00.167 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling the dashboard module... 2024-05-09T00:01:00.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:00 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2694026694' entity='client.admin' 2024-05-09T00:01:01.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:01 smithi046 ceph-mon[30151]: from='mgr.14120 172.21.15.46:0/774623222' entity='mgr.x' 2024-05-09T00:01:01.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:01 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1406910506' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-09T00:01:01.884 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:01 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: ignoring --setuser ceph since I am not root 2024-05-09T00:01:01.884 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:01 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: ignoring --setgroup ceph since I am not root 2024-05-09T00:01:01.884 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:01 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:01.808+0000 7f17ae77a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T00:01:02.281 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:02 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:02.011+0000 7f17ae77a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:01:02.281 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:02 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:02.121+0000 7f17ae77a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T00:01:02.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.925+0000 7f555bb80700 1 Processor -- start 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.925+0000 7f555bb80700 1 -- start start 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.925+0000 7f555bb80700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 0x7f555410c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.925+0000 7f555bb80700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f555407d6c0 con 0x7f555410a560 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.925+0000 7f555991c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 0x7f555410c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.925+0000 7f555991c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 0x7f555410c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49978/0 (socket says 172.21.15.46:49978) 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.925+0000 7f555991c700 1 -- 172.21.15.46:0/1713994913 learned_addr learned my addr 172.21.15.46:0/1713994913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.926+0000 7f555991c700 1 -- 172.21.15.46:0/1713994913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555410cec0 con 0x7f555410a560 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.926+0000 7f555991c700 1 --2- 172.21.15.46:0/1713994913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 0x7f555410c980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f55500098a0 tx=0x7f5550009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91fdd0d5153dbddd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.926+0000 7f555891a700 1 -- 172.21.15.46:0/1713994913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5550004160 con 0x7f555410a560 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.926+0000 7f555891a700 1 -- 172.21.15.46:0/1713994913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5550004300 con 0x7f555410a560 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.926+0000 7f555891a700 1 -- 172.21.15.46:0/1713994913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5550011a20 con 0x7f555410a560 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.927+0000 7f555bb80700 1 -- 172.21.15.46:0/1713994913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 msgr2=0x7f555410c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:02.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.927+0000 7f555bb80700 1 --2- 172.21.15.46:0/1713994913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 0x7f555410c980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f55500098a0 tx=0x7f5550009bb0 comp rx=0 tx=0).stop 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.927+0000 7f555bb80700 1 -- 172.21.15.46:0/1713994913 shutdown_connections 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.927+0000 7f555bb80700 1 --2- 172.21.15.46:0/1713994913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 0x7f555410c980 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.927+0000 7f555bb80700 1 -- 172.21.15.46:0/1713994913 >> 172.21.15.46:0/1713994913 conn(0x7f5554104000 msgr2=0x7f5554106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.927+0000 7f555bb80700 1 -- 172.21.15.46:0/1713994913 shutdown_connections 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.927+0000 7f555bb80700 1 -- 172.21.15.46:0/1713994913 wait complete. 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.928+0000 7f555bb80700 1 Processor -- start 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.928+0000 7f555bb80700 1 -- start start 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.928+0000 7f555bb80700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 0x7f5554109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.928+0000 7f555bb80700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f555410de60 con 0x7f555410a560 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.928+0000 7f555991c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 0x7f5554109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.928+0000 7f555991c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 0x7f5554109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49982/0 (socket says 172.21.15.46:49982) 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.928+0000 7f555991c700 1 -- 172.21.15.46:0/1406910506 learned_addr learned my addr 172.21.15.46:0/1406910506 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.928+0000 7f555991c700 1 -- 172.21.15.46:0/1406910506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555410a220 con 0x7f555410a560 2024-05-09T00:01:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.929+0000 7f555991c700 1 --2- 172.21.15.46:0/1406910506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 0x7f5554109ce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5550004660 tx=0x7f5550011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.929+0000 7f554affd700 1 -- 172.21.15.46:0/1406910506 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5550003a60 con 0x7f555410a560 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.929+0000 7f555bb80700 1 -- 172.21.15.46:0/1406910506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5554108470 con 0x7f555410a560 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.929+0000 7f554affd700 1 -- 172.21.15.46:0/1406910506 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f555001b8f0 con 0x7f555410a560 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.929+0000 7f554affd700 1 -- 172.21.15.46:0/1406910506 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5550023890 con 0x7f555410a560 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.929+0000 7f555bb80700 1 -- 172.21.15.46:0/1406910506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5554108950 con 0x7f555410a560 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.930+0000 7f554affd700 1 -- 172.21.15.46:0/1406910506 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f555002c3f0 con 0x7f555410a560 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.930+0000 7f555bb80700 1 -- 172.21.15.46:0/1406910506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5554064e20 con 0x7f555410a560 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.930+0000 7f554affd700 1 --2- 172.21.15.46:0/1406910506 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f554003e420 0x7f5540040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.930+0000 7f554affd700 1 -- 172.21.15.46:0/1406910506 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5550054ef0 con 0x7f555410a560 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.930+0000 7f555911b700 1 --2- 172.21.15.46:0/1406910506 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f554003e420 0x7f5540040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.931+0000 7f555911b700 1 --2- 172.21.15.46:0/1406910506 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f554003e420 0x7f5540040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5544006fd0 tx=0x7f5544006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:00.936+0000 7f554affd700 1 -- 172.21.15.46:0/1406910506 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f555001f030 con 0x7f555410a560 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.219+0000 7f555bb80700 1 -- 172.21.15.46:0/1406910506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f5554109000 con 0x7f555410a560 2024-05-09T00:01:02.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.587+0000 7f554affd700 1 -- 172.21.15.46:0/1406910506 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f555000f2e0 con 0x7f555410a560 2024-05-09T00:01:02.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.589+0000 7f554affd700 1 -- 172.21.15.46:0/1406910506 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f55500539a0 con 0x7f555410a560 2024-05-09T00:01:02.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.590+0000 7f555bb80700 1 -- 172.21.15.46:0/1406910506 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f554003e420 msgr2=0x7f5540040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:02.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.590+0000 7f555bb80700 1 --2- 172.21.15.46:0/1406910506 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f554003e420 0x7f5540040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5544006fd0 tx=0x7f5544006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:02.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.590+0000 7f555bb80700 1 -- 172.21.15.46:0/1406910506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 msgr2=0x7f5554109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:02.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.590+0000 7f555bb80700 1 --2- 172.21.15.46:0/1406910506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 0x7f5554109ce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5550004660 tx=0x7f5550011aa0 comp rx=0 tx=0).stop 2024-05-09T00:01:02.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.590+0000 7f555bb80700 1 -- 172.21.15.46:0/1406910506 shutdown_connections 2024-05-09T00:01:02.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.590+0000 7f555bb80700 1 --2- 172.21.15.46:0/1406910506 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f554003e420 0x7f5540040930 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:02.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.590+0000 7f555bb80700 1 --2- 172.21.15.46:0/1406910506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f555410a560 0x7f5554109ce0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:02.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.590+0000 7f555bb80700 1 -- 172.21.15.46:0/1406910506 >> 172.21.15.46:0/1406910506 conn(0x7f5554104000 msgr2=0x7f5554105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:02.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.590+0000 7f555bb80700 1 -- 172.21.15.46:0/1406910506 shutdown_connections 2024-05-09T00:01:02.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:01.590+0000 7f555bb80700 1 -- 172.21.15.46:0/1406910506 wait complete. 2024-05-09T00:01:02.543 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:02 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:02.453+0000 7f17ae77a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T00:01:02.543 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:02 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:02.548+0000 7f17ae77a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T00:01:02.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:02 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1406910506' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-09T00:01:02.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:02 smithi046 ceph-mon[30151]: mgrmap e9: x(active, since 19s) 2024-05-09T00:01:03.591 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:01:03.591 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-09T00:01:03.592 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T00:01:03.592 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T00:01:03.592 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T00:01:03.592 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:01:03.592 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.714+0000 7fde99abb700 1 Processor -- start 2024-05-09T00:01:03.592 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.714+0000 7fde99abb700 1 -- start start 2024-05-09T00:01:03.592 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.714+0000 7fde99abb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 0x7fde9407dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:03.592 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.714+0000 7fde99abb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde9407e250 con 0x7fde9407f850 2024-05-09T00:01:03.593 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.714+0000 7fde937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 0x7fde9407dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:03.593 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.714+0000 7fde937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 0x7fde9407dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50000/0 (socket says 172.21.15.46:50000) 2024-05-09T00:01:03.593 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.714+0000 7fde937fe700 1 -- 172.21.15.46:0/3344227337 learned_addr learned my addr 172.21.15.46:0/3344227337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:03.593 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.714+0000 7fde937fe700 1 -- 172.21.15.46:0/3344227337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9407e3d0 con 0x7fde9407f850 2024-05-09T00:01:03.593 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.715+0000 7fde937fe700 1 --2- 172.21.15.46:0/3344227337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 0x7fde9407dc80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde840098a0 tx=0x7fde84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf95c2de04050187 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:03.593 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.715+0000 7fde927fc700 1 -- 172.21.15.46:0/3344227337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde84004160 con 0x7fde9407f850 2024-05-09T00:01:03.593 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.715+0000 7fde927fc700 1 -- 172.21.15.46:0/3344227337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde84004300 con 0x7fde9407f850 2024-05-09T00:01:03.593 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.715+0000 7fde927fc700 1 -- 172.21.15.46:0/3344227337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde84011a20 con 0x7fde9407f850 2024-05-09T00:01:03.593 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.716+0000 7fde99abb700 1 -- 172.21.15.46:0/3344227337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 msgr2=0x7fde9407dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:03.594 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.716+0000 7fde99abb700 1 --2- 172.21.15.46:0/3344227337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 0x7fde9407dc80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde840098a0 tx=0x7fde84009bb0 comp rx=0 tx=0).stop 2024-05-09T00:01:03.594 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.716+0000 7fde99abb700 1 -- 172.21.15.46:0/3344227337 shutdown_connections 2024-05-09T00:01:03.594 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.716+0000 7fde99abb700 1 --2- 172.21.15.46:0/3344227337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 0x7fde9407dc80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:03.594 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.716+0000 7fde99abb700 1 -- 172.21.15.46:0/3344227337 >> 172.21.15.46:0/3344227337 conn(0x7fde94105160 msgr2=0x7fde941075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:03.594 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.716+0000 7fde99abb700 1 -- 172.21.15.46:0/3344227337 shutdown_connections 2024-05-09T00:01:03.594 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.716+0000 7fde99abb700 1 -- 172.21.15.46:0/3344227337 wait complete. 2024-05-09T00:01:03.594 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.717+0000 7fde99abb700 1 Processor -- start 2024-05-09T00:01:03.594 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.717+0000 7fde99abb700 1 -- start start 2024-05-09T00:01:03.595 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.717+0000 7fde99abb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 0x7fde941a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:03.595 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.717+0000 7fde99abb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde941100a0 con 0x7fde9407f850 2024-05-09T00:01:03.595 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.718+0000 7fde937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 0x7fde941a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:03.595 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.718+0000 7fde937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 0x7fde941a0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50010/0 (socket says 172.21.15.46:50010) 2024-05-09T00:01:03.595 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.718+0000 7fde937fe700 1 -- 172.21.15.46:0/4056271177 learned_addr learned my addr 172.21.15.46:0/4056271177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:03.595 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.718+0000 7fde937fe700 1 -- 172.21.15.46:0/4056271177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde941a0590 con 0x7fde9407f850 2024-05-09T00:01:03.595 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.718+0000 7fde937fe700 1 --2- 172.21.15.46:0/4056271177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 0x7fde941a0050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde84009520 tx=0x7fde84011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:03.595 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.718+0000 7fde90ff9700 1 -- 172.21.15.46:0/4056271177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde840039e0 con 0x7fde9407f850 2024-05-09T00:01:03.595 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.718+0000 7fde90ff9700 1 -- 172.21.15.46:0/4056271177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde8401b8f0 con 0x7fde9407f850 2024-05-09T00:01:03.596 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.718+0000 7fde99abb700 1 -- 172.21.15.46:0/4056271177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde941a0820 con 0x7fde9407f850 2024-05-09T00:01:03.596 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.718+0000 7fde90ff9700 1 -- 172.21.15.46:0/4056271177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde840197d0 con 0x7fde9407f850 2024-05-09T00:01:03.596 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.719+0000 7fde99abb700 1 -- 172.21.15.46:0/4056271177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde941a3430 con 0x7fde9407f850 2024-05-09T00:01:03.596 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.719+0000 7fde90ff9700 1 -- 172.21.15.46:0/4056271177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fde84019a90 con 0x7fde9407f850 2024-05-09T00:01:03.596 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.720+0000 7fde90ff9700 1 --2- 172.21.15.46:0/4056271177 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fde7c03e4c0 0x7fde7c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:03.596 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.720+0000 7fde99abb700 1 -- 172.21.15.46:0/4056271177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde94064e20 con 0x7fde9407f850 2024-05-09T00:01:03.596 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.720+0000 7fde92ffd700 1 -- 172.21.15.46:0/4056271177 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fde7c03e4c0 msgr2=0x7fde7c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1620504773 2024-05-09T00:01:03.596 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.720+0000 7fde92ffd700 1 --2- 172.21.15.46:0/4056271177 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fde7c03e4c0 0x7fde7c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:01:03.596 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.720+0000 7fde90ff9700 1 -- 172.21.15.46:0/4056271177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fde84054f60 con 0x7fde9407f850 2024-05-09T00:01:03.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.729+0000 7fde90ff9700 1 -- 172.21.15.46:0/4056271177 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fde8401f030 con 0x7fde9407f850 2024-05-09T00:01:03.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.921+0000 7fde92ffd700 1 -- 172.21.15.46:0/4056271177 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fde7c03e4c0 msgr2=0x7fde7c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1620504773 2024-05-09T00:01:03.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:02.921+0000 7fde92ffd700 1 --2- 172.21.15.46:0/4056271177 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fde7c03e4c0 0x7fde7c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T00:01:03.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:03.047+0000 7fde99abb700 1 -- 172.21.15.46:0/4056271177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fde941a3600 con 0x7fde9407f850 2024-05-09T00:01:03.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:03.047+0000 7fde90ff9700 1 -- 172.21.15.46:0/4056271177 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fde8400e260 con 0x7fde9407f850 2024-05-09T00:01:03.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:03.050+0000 7fde99abb700 1 -- 172.21.15.46:0/4056271177 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fde7c03e4c0 msgr2=0x7fde7c0409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:01:03.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:03.050+0000 7fde99abb700 1 --2- 172.21.15.46:0/4056271177 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fde7c03e4c0 0x7fde7c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:03.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:03.050+0000 7fde99abb700 1 -- 172.21.15.46:0/4056271177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 msgr2=0x7fde941a0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:03.597 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:03.050+0000 7fde99abb700 1 --2- 172.21.15.46:0/4056271177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 0x7fde941a0050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde84009520 tx=0x7fde84011aa0 comp rx=0 tx=0).stop 2024-05-09T00:01:03.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:03.050+0000 7fde99abb700 1 -- 172.21.15.46:0/4056271177 shutdown_connections 2024-05-09T00:01:03.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:03.050+0000 7fde99abb700 1 --2- 172.21.15.46:0/4056271177 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7fde7c03e4c0 0x7fde7c0409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:03.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:03.050+0000 7fde99abb700 1 --2- 172.21.15.46:0/4056271177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde9407f850 0x7fde941a0050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:03.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:03.050+0000 7fde99abb700 1 -- 172.21.15.46:0/4056271177 >> 172.21.15.46:0/4056271177 conn(0x7fde94105160 msgr2=0x7fde94106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:03.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:03.050+0000 7fde99abb700 1 -- 172.21.15.46:0/4056271177 shutdown_connections 2024-05-09T00:01:03.598 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:03.050+0000 7fde99abb700 1 -- 172.21.15.46:0/4056271177 wait complete. 2024-05-09T00:01:03.598 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-05-09T00:01:03.598 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 9... 2024-05-09T00:01:03.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:03 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/4056271177' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T00:01:04.024 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:03 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:03.843+0000 7f17ae77a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T00:01:04.307 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:04 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:04.033+0000 7f17ae77a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T00:01:04.307 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:04 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:04.215+0000 7f17ae77a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T00:01:04.807 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:04 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:04.526+0000 7f17ae77a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T00:01:04.807 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:04 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:04.625+0000 7f17ae77a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T00:01:05.307 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:04 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:04.839+0000 7f17ae77a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T00:01:05.307 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:05 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:05.045+0000 7f17ae77a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:01:05.806 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:05 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:05.685+0000 7f17ae77a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T00:01:06.306 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:05 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:05.923+0000 7f17ae77a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T00:01:06.806 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:06 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:06.378+0000 7f17ae77a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T00:01:06.807 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:06 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:06.482+0000 7f17ae77a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T00:01:07.251 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:06 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:06.873+0000 7f17ae77a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T00:01:07.251 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:07 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:07.256+0000 7f17ae77a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T00:01:07.556 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:07 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:07.356+0000 7f17ae77a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T00:01:08.057 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:07 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:07.618+0000 7f17ae77a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T00:01:08.057 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:07 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:07.763+0000 7f17ae77a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T00:01:08.057 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:07 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:07.867+0000 7f17ae77a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T00:01:08.557 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:08 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:08.209+0000 7f17ae77a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T00:01:08.557 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:08 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:08.310+0000 7f17ae77a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T00:01:09.236 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:08 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:08.955+0000 7f17ae77a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T00:01:09.236 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:01:09 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:01:09.188+0000 7f17ae77a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T00:01:09.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:09 smithi046 ceph-mon[30151]: Active manager daemon x restarted 2024-05-09T00:01:09.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:09 smithi046 ceph-mon[30151]: Activating manager daemon x 2024-05-09T00:01:09.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:09 smithi046 ceph-mon[30151]: osdmap e3: 0 total, 0 up, 0 in 2024-05-09T00:01:09.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:09 smithi046 ceph-mon[30151]: mgrmap e10: x(active, starting, since 0.0039525s) 2024-05-09T00:01:09.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:01:09.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T00:01:09.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T00:01:09.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T00:01:09.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T00:01:09.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:09 smithi046 ceph-mon[30151]: Manager daemon x is now available 2024-05-09T00:01:09.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:01:10.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T00:01:10.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T00:01:10.406 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:10 smithi046 ceph-mon[30151]: mgrmap e11: x(active, since 1.00792s) 2024-05-09T00:01:11.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T00:01:11.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-09T00:01:11.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T00:01:11.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T00:01:11.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.060+0000 7f76ceccc700 1 Processor -- start 2024-05-09T00:01:11.032 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.061+0000 7f76ceccc700 1 -- start start 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.061+0000 7f76ceccc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 0x7f76c8109e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.061+0000 7f76ceccc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76c810a440 con 0x7f76c8109a40 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.061+0000 7f76cca68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 0x7f76c8109e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.061+0000 7f76cca68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 0x7f76c8109e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50022/0 (socket says 172.21.15.46:50022) 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.061+0000 7f76cca68700 1 -- 172.21.15.46:0/2443732155 learned_addr learned my addr 172.21.15.46:0/2443732155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.061+0000 7f76cca68700 1 -- 172.21.15.46:0/2443732155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c810ac10 con 0x7f76c8109a40 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.062+0000 7f76cca68700 1 --2- 172.21.15.46:0/2443732155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 0x7f76c8109e50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0098a0 tx=0x7f76bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b06ce2d4b915a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.062+0000 7f76c77fe700 1 -- 172.21.15.46:0/2443732155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76bc004160 con 0x7f76c8109a40 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.062+0000 7f76c77fe700 1 -- 172.21.15.46:0/2443732155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76bc004300 con 0x7f76c8109a40 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.062+0000 7f76c77fe700 1 -- 172.21.15.46:0/2443732155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76bc011a20 con 0x7f76c8109a40 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.063+0000 7f76ceccc700 1 -- 172.21.15.46:0/2443732155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 msgr2=0x7f76c8109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.063+0000 7f76ceccc700 1 --2- 172.21.15.46:0/2443732155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 0x7f76c8109e50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0098a0 tx=0x7f76bc009bb0 comp rx=0 tx=0).stop 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.063+0000 7f76ceccc700 1 -- 172.21.15.46:0/2443732155 shutdown_connections 2024-05-09T00:01:11.033 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.063+0000 7f76ceccc700 1 --2- 172.21.15.46:0/2443732155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 0x7f76c8109e50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:11.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.063+0000 7f76ceccc700 1 -- 172.21.15.46:0/2443732155 >> 172.21.15.46:0/2443732155 conn(0x7f76c8105160 msgr2=0x7f76c81075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:11.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.063+0000 7f76ceccc700 1 -- 172.21.15.46:0/2443732155 shutdown_connections 2024-05-09T00:01:11.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.063+0000 7f76ceccc700 1 -- 172.21.15.46:0/2443732155 wait complete. 2024-05-09T00:01:11.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.064+0000 7f76ceccc700 1 Processor -- start 2024-05-09T00:01:11.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.064+0000 7f76ceccc700 1 -- start start 2024-05-09T00:01:11.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.064+0000 7f76ceccc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 0x7f76c819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:11.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.064+0000 7f76ceccc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76c810dfc0 con 0x7f76c8109a40 2024-05-09T00:01:11.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.064+0000 7f76cca68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 0x7f76c819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:11.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.064+0000 7f76cca68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 0x7f76c819dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50030/0 (socket says 172.21.15.46:50030) 2024-05-09T00:01:11.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.064+0000 7f76cca68700 1 -- 172.21.15.46:0/1677366529 learned_addr learned my addr 172.21.15.46:0/1677366529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:11.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.064+0000 7f76cca68700 1 -- 172.21.15.46:0/1677366529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c819e3e0 con 0x7f76c8109a40 2024-05-09T00:01:11.034 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.065+0000 7f76cca68700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 0x7f76c819dea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76bc011a20 tx=0x7f76bc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.065+0000 7f76c5ffb700 1 -- 172.21.15.46:0/1677366529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76bc0039c0 con 0x7f76c8109a40 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.065+0000 7f76c5ffb700 1 -- 172.21.15.46:0/1677366529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76bc003b60 con 0x7f76c8109a40 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.065+0000 7f76ceccc700 1 -- 172.21.15.46:0/1677366529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c819e670 con 0x7f76c8109a40 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.065+0000 7f76c5ffb700 1 -- 172.21.15.46:0/1677366529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76bc019700 con 0x7f76c8109a40 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.065+0000 7f76ceccc700 1 -- 172.21.15.46:0/1677366529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c81a1280 con 0x7f76c8109a40 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.066+0000 7f76c5ffb700 1 -- 172.21.15.46:0/1677366529 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f76bc0199b0 con 0x7f76c8109a40 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.066+0000 7f76c5ffb700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 0x7f76b80409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.066+0000 7f76c5ffb700 1 -- 172.21.15.46:0/1677366529 --> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f76b80410f0 con 0x7f76b803e4d0 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.066+0000 7f76c5ffb700 1 -- 172.21.15.46:0/1677366529 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f76bc0548f0 con 0x7f76c8109a40 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.066+0000 7f76c7fff700 1 -- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 msgr2=0x7f76b80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1620504773 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.066+0000 7f76c7fff700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 0x7f76b80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.267+0000 7f76c7fff700 1 -- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 msgr2=0x7f76b80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1620504773 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.267+0000 7f76c7fff700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 0x7f76b80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T00:01:11.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.668+0000 7f76c7fff700 1 -- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 msgr2=0x7f76b80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1620504773 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:04.668+0000 7f76c7fff700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 0x7f76b80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:05.469+0000 7f76c7fff700 1 -- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 msgr2=0x7f76b80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1620504773 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:05.469+0000 7f76c7fff700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 0x7f76b80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:07.070+0000 7f76c7fff700 1 -- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 msgr2=0x7f76b80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1620504773 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:07.070+0000 7f76c7fff700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 0x7f76b80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:09.192+0000 7f76c5ffb700 1 -- 172.21.15.46:0/1677366529 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f76bc02d430 con 0x7f76c8109a40 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:09.192+0000 7f76c5ffb700 1 -- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 msgr2=0x7f76b80409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:09.192+0000 7f76c5ffb700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 0x7f76b80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.194+0000 7f76c5ffb700 1 -- 172.21.15.46:0/1677366529 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f76bc026d70 con 0x7f76c8109a40 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.195+0000 7f76c5ffb700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f76bc02d020 0x7f76b8043fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.195+0000 7f76c5ffb700 1 -- 172.21.15.46:0/1677366529 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f76b80410f0 con 0x7f76bc02d020 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.196+0000 7f76c7fff700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f76bc02d020 0x7f76b8043fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.197+0000 7f76c7fff700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f76bc02d020 0x7f76b8043fd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f76b4003a10 tx=0x7f76b4009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.198+0000 7f76c5ffb700 1 -- 172.21.15.46:0/1677366529 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f76b80410f0 con 0x7f76bc02d020 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.214+0000 7f76ceccc700 1 -- 172.21.15.46:0/1677366529 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f76c8064e20 con 0x7f76bc02d020 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.215+0000 7f76c5ffb700 1 -- 172.21.15.46:0/1677366529 <== mgr.14152 v2:172.21.15.46:6800/286117730 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f76c8064e20 con 0x7f76bc02d020 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.215+0000 7f76ceccc700 1 -- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f76bc02d020 msgr2=0x7f76b8043fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.215+0000 7f76ceccc700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f76bc02d020 0x7f76b8043fd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f76b4003a10 tx=0x7f76b4009270 comp rx=0 tx=0).stop 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.215+0000 7f76ceccc700 1 -- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 msgr2=0x7f76c819dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.215+0000 7f76ceccc700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 0x7f76c819dea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76bc011a20 tx=0x7f76bc011b00 comp rx=0 tx=0).stop 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.215+0000 7f76ceccc700 1 -- 172.21.15.46:0/1677366529 shutdown_connections 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.215+0000 7f76ceccc700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f76bc02d020 0x7f76b8043fd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.215+0000 7f76ceccc700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c8109a40 0x7f76c819dea0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.215+0000 7f76ceccc700 1 --2- 172.21.15.46:0/1677366529 >> [v2:172.21.15.46:6800/1620504773,v1:172.21.15.46:6801/1620504773] conn(0x7f76b803e4d0 0x7f76b80409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.215+0000 7f76ceccc700 1 -- 172.21.15.46:0/1677366529 >> 172.21.15.46:0/1677366529 conn(0x7f76c8105160 msgr2=0x7f76c8106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.216+0000 7f76ceccc700 1 -- 172.21.15.46:0/1677366529 shutdown_connections 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:10.216+0000 7f76ceccc700 1 -- 172.21.15.46:0/1677366529 wait complete. 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 9 is available 2024-05-09T00:01:11.037 INFO:teuthology.orchestra.run.smithi046.stdout:Generating a dashboard self-signed certificate... 2024-05-09T00:01:11.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:11 smithi046 ceph-mon[30151]: [09/May/2024:00:01:10] ENGINE Bus STARTING 2024-05-09T00:01:11.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:11 smithi046 ceph-mon[30151]: [09/May/2024:00:01:10] ENGINE Serving on https://172.21.15.46:7150 2024-05-09T00:01:11.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:11 smithi046 ceph-mon[30151]: [09/May/2024:00:01:10] ENGINE Serving on http://172.21.15.46:8765 2024-05-09T00:01:11.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:11 smithi046 ceph-mon[30151]: [09/May/2024:00:01:10] ENGINE Bus STARTED 2024-05-09T00:01:11.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:11 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:12.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-09T00:01:12.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.529+0000 7fb14893e700 1 Processor -- start 2024-05-09T00:01:12.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.529+0000 7fb14893e700 1 -- start start 2024-05-09T00:01:12.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.529+0000 7fb14893e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 0x7fb144108d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:12.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.529+0000 7fb14893e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb144109370 con 0x7fb144108970 2024-05-09T00:01:12.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.529+0000 7fb14259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 0x7fb144108d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:12.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.530+0000 7fb14259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 0x7fb144108d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55242/0 (socket says 172.21.15.46:55242) 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.530+0000 7fb14259c700 1 -- 172.21.15.46:0/3658409501 learned_addr learned my addr 172.21.15.46:0/3658409501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.530+0000 7fb14259c700 1 -- 172.21.15.46:0/3658409501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb144109b40 con 0x7fb144108970 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.530+0000 7fb14259c700 1 --2- 172.21.15.46:0/3658409501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 0x7fb144108d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb13400bb40 tx=0x7fb13400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada069a2466f2e1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.530+0000 7fb14159a700 1 -- 172.21.15.46:0/3658409501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb13400dbe0 con 0x7fb144108970 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.530+0000 7fb14159a700 1 -- 172.21.15.46:0/3658409501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1340044e0 con 0x7fb144108970 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.530+0000 7fb14159a700 1 -- 172.21.15.46:0/3658409501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb134010470 con 0x7fb144108970 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.531+0000 7fb14893e700 1 -- 172.21.15.46:0/3658409501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 msgr2=0x7fb144108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.531+0000 7fb14893e700 1 --2- 172.21.15.46:0/3658409501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 0x7fb144108d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb13400bb40 tx=0x7fb13400be50 comp rx=0 tx=0).stop 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.531+0000 7fb14893e700 1 -- 172.21.15.46:0/3658409501 shutdown_connections 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.531+0000 7fb14893e700 1 --2- 172.21.15.46:0/3658409501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 0x7fb144108d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.531+0000 7fb14893e700 1 -- 172.21.15.46:0/3658409501 >> 172.21.15.46:0/3658409501 conn(0x7fb144104050 msgr2=0x7fb1441064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.531+0000 7fb14893e700 1 -- 172.21.15.46:0/3658409501 shutdown_connections 2024-05-09T00:01:12.477 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.531+0000 7fb14893e700 1 -- 172.21.15.46:0/3658409501 wait complete. 2024-05-09T00:01:12.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.532+0000 7fb14893e700 1 Processor -- start 2024-05-09T00:01:12.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.532+0000 7fb14893e700 1 -- start start 2024-05-09T00:01:12.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.533+0000 7fb14893e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 0x7fb14419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:12.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.533+0000 7fb14893e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb14407c140 con 0x7fb144108970 2024-05-09T00:01:12.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.533+0000 7fb14259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 0x7fb14419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:12.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.533+0000 7fb14259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 0x7fb14419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55248/0 (socket says 172.21.15.46:55248) 2024-05-09T00:01:12.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.533+0000 7fb14259c700 1 -- 172.21.15.46:0/1507650444 learned_addr learned my addr 172.21.15.46:0/1507650444 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:12.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.533+0000 7fb14259c700 1 -- 172.21.15.46:0/1507650444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14419c1b0 con 0x7fb144108970 2024-05-09T00:01:12.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.533+0000 7fb14259c700 1 --2- 172.21.15.46:0/1507650444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 0x7fb14419bc70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb134003b90 tx=0x7fb134003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:12.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.533+0000 7fb1337fe700 1 -- 172.21.15.46:0/1507650444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb134027070 con 0x7fb144108970 2024-05-09T00:01:12.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.533+0000 7fb1337fe700 1 -- 172.21.15.46:0/1507650444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb134010560 con 0x7fb144108970 2024-05-09T00:01:12.478 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.534+0000 7fb14893e700 1 -- 172.21.15.46:0/1507650444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14419c440 con 0x7fb144108970 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.534+0000 7fb1337fe700 1 -- 172.21.15.46:0/1507650444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb134022ac0 con 0x7fb144108970 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.534+0000 7fb14893e700 1 -- 172.21.15.46:0/1507650444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14419f050 con 0x7fb144108970 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.535+0000 7fb1337fe700 1 -- 172.21.15.46:0/1507650444 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fb13402a070 con 0x7fb144108970 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.535+0000 7fb1337fe700 1 --2- 172.21.15.46:0/1507650444 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb12c03e3a0 0x7fb12c0408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.535+0000 7fb1337fe700 1 -- 172.21.15.46:0/1507650444 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb13401f070 con 0x7fb144108970 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.535+0000 7fb141d9b700 1 --2- 172.21.15.46:0/1507650444 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb12c03e3a0 0x7fb12c0408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.535+0000 7fb14893e700 1 -- 172.21.15.46:0/1507650444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb144064e20 con 0x7fb144108970 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.535+0000 7fb141d9b700 1 --2- 172.21.15.46:0/1507650444 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb12c03e3a0 0x7fb12c0408b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb138006fd0 tx=0x7fb138006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.541+0000 7fb1337fe700 1 -- 172.21.15.46:0/1507650444 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb134015070 con 0x7fb144108970 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.771+0000 7fb14893e700 1 -- 172.21.15.46:0/1507650444 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb144063c00 con 0x7fb12c03e3a0 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.857+0000 7fb1337fe700 1 -- 172.21.15.46:0/1507650444 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb144063c00 con 0x7fb12c03e3a0 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.859+0000 7fb14893e700 1 -- 172.21.15.46:0/1507650444 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb12c03e3a0 msgr2=0x7fb12c0408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:12.479 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.859+0000 7fb14893e700 1 --2- 172.21.15.46:0/1507650444 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb12c03e3a0 0x7fb12c0408b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb138006fd0 tx=0x7fb138006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:12.480 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.859+0000 7fb14893e700 1 -- 172.21.15.46:0/1507650444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 msgr2=0x7fb14419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:12.480 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.859+0000 7fb14893e700 1 --2- 172.21.15.46:0/1507650444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 0x7fb14419bc70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb134003b90 tx=0x7fb134003ff0 comp rx=0 tx=0).stop 2024-05-09T00:01:12.480 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.859+0000 7fb14893e700 1 -- 172.21.15.46:0/1507650444 shutdown_connections 2024-05-09T00:01:12.480 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.859+0000 7fb14893e700 1 --2- 172.21.15.46:0/1507650444 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb12c03e3a0 0x7fb12c0408b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:12.480 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.859+0000 7fb14893e700 1 --2- 172.21.15.46:0/1507650444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb144108970 0x7fb14419bc70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:12.480 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.859+0000 7fb14893e700 1 -- 172.21.15.46:0/1507650444 >> 172.21.15.46:0/1507650444 conn(0x7fb144104050 msgr2=0x7fb144191f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:12.480 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.859+0000 7fb14893e700 1 -- 172.21.15.46:0/1507650444 shutdown_connections 2024-05-09T00:01:12.480 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:11.859+0000 7fb14893e700 1 -- 172.21.15.46:0/1507650444 wait complete. 2024-05-09T00:01:12.480 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial admin user... 2024-05-09T00:01:13.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:12 smithi046 ceph-mon[30151]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:01:13.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:12 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:13.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:12 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:13.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:12 smithi046 ceph-mon[30151]: mgrmap e12: x(active, since 2s) 2024-05-09T00:01:13.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:12 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:14.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$S4P5ZX87OAftV0G9Z.uvzuaSO6h.yZSx3chK3k4eTt6TmKZMHrWl6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715212873, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-09T00:01:14.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.889+0000 7fbe6220a700 1 Processor -- start 2024-05-09T00:01:14.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.889+0000 7fbe6220a700 1 -- start start 2024-05-09T00:01:14.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.889+0000 7fbe6220a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 0x7fbe5c108d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:14.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.889+0000 7fbe6220a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c109330 con 0x7fbe5c108930 2024-05-09T00:01:14.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.889+0000 7fbe5b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 0x7fbe5c108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.889+0000 7fbe5b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 0x7fbe5c108d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55260/0 (socket says 172.21.15.46:55260) 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.889+0000 7fbe5b7fe700 1 -- 172.21.15.46:0/3750394382 learned_addr learned my addr 172.21.15.46:0/3750394382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.890+0000 7fbe5b7fe700 1 -- 172.21.15.46:0/3750394382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c109b00 con 0x7fbe5c108930 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.890+0000 7fbe5b7fe700 1 --2- 172.21.15.46:0/3750394382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 0x7fbe5c108d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00bb40 tx=0x7fbe4c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c386acb7ef532c97 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.890+0000 7fbe5a7fc700 1 -- 172.21.15.46:0/3750394382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe4c00dbe0 con 0x7fbe5c108930 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.890+0000 7fbe5a7fc700 1 -- 172.21.15.46:0/3750394382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4c0044e0 con 0x7fbe5c108930 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.890+0000 7fbe5a7fc700 1 -- 172.21.15.46:0/3750394382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe4c010470 con 0x7fbe5c108930 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.891+0000 7fbe6220a700 1 -- 172.21.15.46:0/3750394382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 msgr2=0x7fbe5c108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.891+0000 7fbe6220a700 1 --2- 172.21.15.46:0/3750394382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 0x7fbe5c108d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00bb40 tx=0x7fbe4c00be50 comp rx=0 tx=0).stop 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.891+0000 7fbe6220a700 1 -- 172.21.15.46:0/3750394382 shutdown_connections 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.891+0000 7fbe6220a700 1 --2- 172.21.15.46:0/3750394382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 0x7fbe5c108d40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.891+0000 7fbe6220a700 1 -- 172.21.15.46:0/3750394382 >> 172.21.15.46:0/3750394382 conn(0x7fbe5c103fd0 msgr2=0x7fbe5c106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.891+0000 7fbe6220a700 1 -- 172.21.15.46:0/3750394382 shutdown_connections 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.891+0000 7fbe6220a700 1 -- 172.21.15.46:0/3750394382 wait complete. 2024-05-09T00:01:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.892+0000 7fbe6220a700 1 Processor -- start 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.892+0000 7fbe6220a700 1 -- start start 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.892+0000 7fbe6220a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 0x7fbe5c1a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.892+0000 7fbe6220a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c07c9d0 con 0x7fbe5c108930 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.893+0000 7fbe5b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 0x7fbe5c1a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.893+0000 7fbe5b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 0x7fbe5c1a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55276/0 (socket says 172.21.15.46:55276) 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.893+0000 7fbe5b7fe700 1 -- 172.21.15.46:0/2271810028 learned_addr learned my addr 172.21.15.46:0/2271810028 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.893+0000 7fbe5b7fe700 1 -- 172.21.15.46:0/2271810028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c1a0600 con 0x7fbe5c108930 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.893+0000 7fbe5b7fe700 1 --2- 172.21.15.46:0/2271810028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 0x7fbe5c1a00c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c003c30 tx=0x7fbe4c003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.893+0000 7fbe58ff9700 1 -- 172.21.15.46:0/2271810028 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe4c010720 con 0x7fbe5c108930 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.893+0000 7fbe58ff9700 1 -- 172.21.15.46:0/2271810028 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4c01d070 con 0x7fbe5c108930 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.893+0000 7fbe6220a700 1 -- 172.21.15.46:0/2271810028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5c1a0890 con 0x7fbe5c108930 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.893+0000 7fbe58ff9700 1 -- 172.21.15.46:0/2271810028 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe4c019a80 con 0x7fbe5c108930 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.893+0000 7fbe6220a700 1 -- 172.21.15.46:0/2271810028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5c1a34a0 con 0x7fbe5c108930 2024-05-09T00:01:14.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.894+0000 7fbe58ff9700 1 -- 172.21.15.46:0/2271810028 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fbe4c028060 con 0x7fbe5c108930 2024-05-09T00:01:14.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.895+0000 7fbe6220a700 1 -- 172.21.15.46:0/2271810028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe5c064e20 con 0x7fbe5c108930 2024-05-09T00:01:14.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.895+0000 7fbe58ff9700 1 --2- 172.21.15.46:0/2271810028 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbe4403e4c0 0x7fbe440409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:14.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.895+0000 7fbe58ff9700 1 -- 172.21.15.46:0/2271810028 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbe4c0548b0 con 0x7fbe5c108930 2024-05-09T00:01:14.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.895+0000 7fbe5affd700 1 --2- 172.21.15.46:0/2271810028 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbe4403e4c0 0x7fbe440409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:14.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.895+0000 7fbe5affd700 1 --2- 172.21.15.46:0/2271810028 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbe4403e4c0 0x7fbe440409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe50006fd0 tx=0x7fbe50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:14.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:12.901+0000 7fbe58ff9700 1 -- 172.21.15.46:0/2271810028 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe4c015070 con 0x7fbe5c108930 2024-05-09T00:01:14.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:13.144+0000 7fbe6220a700 1 -- 172.21.15.46:0/2271810028 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fbe5c193720 con 0x7fbe4403e4c0 2024-05-09T00:01:14.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:13.391+0000 7fbe58ff9700 1 -- 172.21.15.46:0/2271810028 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fbe5c193720 con 0x7fbe4403e4c0 2024-05-09T00:01:14.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:13.393+0000 7fbe6220a700 1 -- 172.21.15.46:0/2271810028 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbe4403e4c0 msgr2=0x7fbe440409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:14.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:13.393+0000 7fbe6220a700 1 --2- 172.21.15.46:0/2271810028 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbe4403e4c0 0x7fbe440409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe50006fd0 tx=0x7fbe50006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:14.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:13.394+0000 7fbe6220a700 1 -- 172.21.15.46:0/2271810028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 msgr2=0x7fbe5c1a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:14.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:13.394+0000 7fbe6220a700 1 --2- 172.21.15.46:0/2271810028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 0x7fbe5c1a00c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c003c30 tx=0x7fbe4c003d10 comp rx=0 tx=0).stop 2024-05-09T00:01:14.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:13.394+0000 7fbe6220a700 1 -- 172.21.15.46:0/2271810028 shutdown_connections 2024-05-09T00:01:14.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:13.394+0000 7fbe6220a700 1 --2- 172.21.15.46:0/2271810028 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbe4403e4c0 0x7fbe440409d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:14.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:13.394+0000 7fbe6220a700 1 --2- 172.21.15.46:0/2271810028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe5c108930 0x7fbe5c1a00c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:14.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:13.394+0000 7fbe6220a700 1 -- 172.21.15.46:0/2271810028 >> 172.21.15.46:0/2271810028 conn(0x7fbe5c103fd0 msgr2=0x7fbe5c191f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:14.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:13.394+0000 7fbe6220a700 1 -- 172.21.15.46:0/2271810028 shutdown_connections 2024-05-09T00:01:14.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:13.394+0000 7fbe6220a700 1 -- 172.21.15.46:0/2271810028 wait complete. 2024-05-09T00:01:14.084 INFO:teuthology.orchestra.run.smithi046.stdout:Fetching dashboard port number... 2024-05-09T00:01:14.725 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:14 smithi046 ceph-mon[30151]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:01:14.725 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:15.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:15 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3804051119' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-09T00:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 8443 2024-05-09T00:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.068+0000 7f62d6f54700 1 Processor -- start 2024-05-09T00:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.068+0000 7f62d6f54700 1 -- start start 2024-05-09T00:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.069+0000 7f62d6f54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 0x7f62d01087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.069+0000 7f62d6f54700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f62d007d680 con 0x7f62d0108390 2024-05-09T00:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.069+0000 7f62d4cf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 0x7f62d01087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.069+0000 7f62d4cf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 0x7f62d01087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55292/0 (socket says 172.21.15.46:55292) 2024-05-09T00:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.069+0000 7f62d4cf0700 1 -- 172.21.15.46:0/1384659714 learned_addr learned my addr 172.21.15.46:0/1384659714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.069+0000 7f62d4cf0700 1 -- 172.21.15.46:0/1384659714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d0108ce0 con 0x7f62d0108390 2024-05-09T00:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.069+0000 7f62d4cf0700 1 --2- 172.21.15.46:0/1384659714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 0x7f62d01087a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009e80 tx=0x7f62c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2553059049e890 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.069+0000 7f62cf7fe700 1 -- 172.21.15.46:0/1384659714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62c0017070 con 0x7f62d0108390 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.069+0000 7f62cf7fe700 1 -- 172.21.15.46:0/1384659714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c000b8c0 con 0x7f62d0108390 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.070+0000 7f62cf7fe700 1 -- 172.21.15.46:0/1384659714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62c0004490 con 0x7f62d0108390 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.070+0000 7f62d6f54700 1 -- 172.21.15.46:0/1384659714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 msgr2=0x7f62d01087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.070+0000 7f62d6f54700 1 --2- 172.21.15.46:0/1384659714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 0x7f62d01087a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009e80 tx=0x7f62c000b330 comp rx=0 tx=0).stop 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.070+0000 7f62d6f54700 1 -- 172.21.15.46:0/1384659714 shutdown_connections 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.070+0000 7f62d6f54700 1 --2- 172.21.15.46:0/1384659714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 0x7f62d01087a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.070+0000 7f62d6f54700 1 -- 172.21.15.46:0/1384659714 >> 172.21.15.46:0/1384659714 conn(0x7f62d0104000 msgr2=0x7f62d0106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.070+0000 7f62d6f54700 1 -- 172.21.15.46:0/1384659714 shutdown_connections 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.070+0000 7f62d6f54700 1 -- 172.21.15.46:0/1384659714 wait complete. 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.071+0000 7f62d6f54700 1 Processor -- start 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.071+0000 7f62d6f54700 1 -- start start 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.071+0000 7f62d6f54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 0x7f62d01a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.071+0000 7f62d6f54700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f62d0109da0 con 0x7f62d0108390 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.071+0000 7f62d4cf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 0x7f62d01a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.071+0000 7f62d4cf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 0x7f62d01a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55294/0 (socket says 172.21.15.46:55294) 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.071+0000 7f62d4cf0700 1 -- 172.21.15.46:0/3804051119 learned_addr learned my addr 172.21.15.46:0/3804051119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.072+0000 7f62d4cf0700 1 -- 172.21.15.46:0/3804051119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d01a06e0 con 0x7f62d0108390 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.072+0000 7f62d4cf0700 1 --2- 172.21.15.46:0/3804051119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 0x7f62d01a01a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62c00072a0 tx=0x7f62c000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.072+0000 7f62cdffb700 1 -- 172.21.15.46:0/3804051119 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62c0020020 con 0x7f62d0108390 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.072+0000 7f62cdffb700 1 -- 172.21.15.46:0/3804051119 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c0012430 con 0x7f62d0108390 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.072+0000 7f62d6f54700 1 -- 172.21.15.46:0/3804051119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d01a0970 con 0x7f62d0108390 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.072+0000 7f62cdffb700 1 -- 172.21.15.46:0/3804051119 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62c0017040 con 0x7f62d0108390 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.072+0000 7f62d6f54700 1 -- 172.21.15.46:0/3804051119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d01a3580 con 0x7f62d0108390 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.073+0000 7f62cdffb700 1 -- 172.21.15.46:0/3804051119 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f62c0023020 con 0x7f62d0108390 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.073+0000 7f62d6f54700 1 -- 172.21.15.46:0/3804051119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d0064e20 con 0x7f62d0108390 2024-05-09T00:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.073+0000 7f62cdffb700 1 --2- 172.21.15.46:0/3804051119 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f62bc03e4d0 0x7f62bc0409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.074+0000 7f62cdffb700 1 -- 172.21.15.46:0/3804051119 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f62c000f050 con 0x7f62d0108390 2024-05-09T00:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.074+0000 7f62cffff700 1 --2- 172.21.15.46:0/3804051119 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f62bc03e4d0 0x7f62bc0409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.074+0000 7f62cffff700 1 --2- 172.21.15.46:0/3804051119 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f62bc03e4d0 0x7f62bc0409e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62b8006fd0 tx=0x7f62b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.079+0000 7f62cdffb700 1 -- 172.21.15.46:0/3804051119 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62c002a020 con 0x7f62d0108390 2024-05-09T00:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.295+0000 7f62d6f54700 1 -- 172.21.15.46:0/3804051119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f62d002e950 con 0x7f62d0108390 2024-05-09T00:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.296+0000 7f62cdffb700 1 -- 172.21.15.46:0/3804051119 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f62c0012650 con 0x7f62d0108390 2024-05-09T00:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.298+0000 7f62d6f54700 1 -- 172.21.15.46:0/3804051119 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f62bc03e4d0 msgr2=0x7f62bc0409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.298+0000 7f62d6f54700 1 --2- 172.21.15.46:0/3804051119 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f62bc03e4d0 0x7f62bc0409e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62b8006fd0 tx=0x7f62b8006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.298+0000 7f62d6f54700 1 -- 172.21.15.46:0/3804051119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 msgr2=0x7f62d01a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.298+0000 7f62d6f54700 1 --2- 172.21.15.46:0/3804051119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 0x7f62d01a01a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62c00072a0 tx=0x7f62c000bd50 comp rx=0 tx=0).stop 2024-05-09T00:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.298+0000 7f62d6f54700 1 -- 172.21.15.46:0/3804051119 shutdown_connections 2024-05-09T00:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.298+0000 7f62d6f54700 1 --2- 172.21.15.46:0/3804051119 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f62bc03e4d0 0x7f62bc0409e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.298+0000 7f62d6f54700 1 --2- 172.21.15.46:0/3804051119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d0108390 0x7f62d01a01a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.298+0000 7f62d6f54700 1 -- 172.21.15.46:0/3804051119 >> 172.21.15.46:0/3804051119 conn(0x7f62d0104000 msgr2=0x7f62d0194140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.298+0000 7f62d6f54700 1 -- 172.21.15.46:0/3804051119 shutdown_connections 2024-05-09T00:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:15.298+0000 7f62d6f54700 1 -- 172.21.15.46:0/3804051119 wait complete. 2024-05-09T00:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T00:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-05-09T00:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T00:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-09T00:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-05-09T00:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-09T00:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph Dashboard is now available at: 2024-05-09T00:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: URL: https://smithi046.front.sepia.ceph.com:8443/ 2024-05-09T00:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: User: admin 2024-05-09T00:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: Password: t63jop6hfs 2024-05-09T00:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout:Saving cluster configuration to /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config directory 2024-05-09T00:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling autotune for osd_memory_target 2024-05-09T00:01:16.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:16 smithi046 ceph-mon[30151]: mgrmap e13: x(active, since 6s) 2024-05-09T00:01:17.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.460+0000 7f21d38e6700 1 Processor -- start 2024-05-09T00:01:17.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.461+0000 7f21d38e6700 1 -- start start 2024-05-09T00:01:17.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.461+0000 7f21d38e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 0x7f21cc07dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:17.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.461+0000 7f21d38e6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc07e020 con 0x7f21cc07f660 2024-05-09T00:01:17.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.461+0000 7f21d1682700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 0x7f21cc07dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:17.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.461+0000 7f21d1682700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 0x7f21cc07dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41216/0 (socket says 172.21.15.46:41216) 2024-05-09T00:01:17.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.461+0000 7f21d1682700 1 -- 172.21.15.46:0/1329042322 learned_addr learned my addr 172.21.15.46:0/1329042322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:17.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.462+0000 7f21d1682700 1 -- 172.21.15.46:0/1329042322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc07e1a0 con 0x7f21cc07f660 2024-05-09T00:01:17.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.462+0000 7f21d1682700 1 --2- 172.21.15.46:0/1329042322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 0x7f21cc07dae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f21c80098a0 tx=0x7f21c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77586db13fb34c79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:17.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.462+0000 7f21c3fff700 1 -- 172.21.15.46:0/1329042322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21c8004160 con 0x7f21cc07f660 2024-05-09T00:01:17.353 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.462+0000 7f21c3fff700 1 -- 172.21.15.46:0/1329042322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21c8004300 con 0x7f21cc07f660 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.462+0000 7f21c3fff700 1 -- 172.21.15.46:0/1329042322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21c8011a20 con 0x7f21cc07f660 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.463+0000 7f21d38e6700 1 -- 172.21.15.46:0/1329042322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 msgr2=0x7f21cc07dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.463+0000 7f21d38e6700 1 --2- 172.21.15.46:0/1329042322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 0x7f21cc07dae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f21c80098a0 tx=0x7f21c8009bb0 comp rx=0 tx=0).stop 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.463+0000 7f21d38e6700 1 -- 172.21.15.46:0/1329042322 shutdown_connections 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.463+0000 7f21d38e6700 1 --2- 172.21.15.46:0/1329042322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 0x7f21cc07dae0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.463+0000 7f21d38e6700 1 -- 172.21.15.46:0/1329042322 >> 172.21.15.46:0/1329042322 conn(0x7f21cc105100 msgr2=0x7f21cc107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.463+0000 7f21d38e6700 1 -- 172.21.15.46:0/1329042322 shutdown_connections 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.463+0000 7f21d38e6700 1 -- 172.21.15.46:0/1329042322 wait complete. 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.464+0000 7f21d38e6700 1 Processor -- start 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.464+0000 7f21d38e6700 1 -- start start 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.464+0000 7f21d38e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 0x7f21cc1a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.464+0000 7f21d38e6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc1100a0 con 0x7f21cc07f660 2024-05-09T00:01:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.464+0000 7f21d1682700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 0x7f21cc1a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.464+0000 7f21d1682700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 0x7f21cc1a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41226/0 (socket says 172.21.15.46:41226) 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.464+0000 7f21d1682700 1 -- 172.21.15.46:0/3037474867 learned_addr learned my addr 172.21.15.46:0/3037474867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.464+0000 7f21d1682700 1 -- 172.21.15.46:0/3037474867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc1a0600 con 0x7f21cc07f660 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.464+0000 7f21d1682700 1 --2- 172.21.15.46:0/3037474867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 0x7f21cc1a00c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009520 tx=0x7f21c8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.465+0000 7f21c27fc700 1 -- 172.21.15.46:0/3037474867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21c80039e0 con 0x7f21cc07f660 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.465+0000 7f21c27fc700 1 -- 172.21.15.46:0/3037474867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21c801b8f0 con 0x7f21cc07f660 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.465+0000 7f21c27fc700 1 -- 172.21.15.46:0/3037474867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21c80197d0 con 0x7f21cc07f660 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.465+0000 7f21d38e6700 1 -- 172.21.15.46:0/3037474867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc1a0890 con 0x7f21cc07f660 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.465+0000 7f21d38e6700 1 -- 172.21.15.46:0/3037474867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc1a34a0 con 0x7f21cc07f660 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.466+0000 7f21d38e6700 1 -- 172.21.15.46:0/3037474867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc064e20 con 0x7f21cc07f660 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.466+0000 7f21c27fc700 1 -- 172.21.15.46:0/3037474867 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f21c8019a90 con 0x7f21cc07f660 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.466+0000 7f21c27fc700 1 --2- 172.21.15.46:0/3037474867 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f21b803e560 0x7f21b8040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.466+0000 7f21c27fc700 1 -- 172.21.15.46:0/3037474867 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f21c80543f0 con 0x7f21cc07f660 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.466+0000 7f21d0e81700 1 --2- 172.21.15.46:0/3037474867 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f21b803e560 0x7f21b8040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:17.355 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.467+0000 7f21d0e81700 1 --2- 172.21.15.46:0/3037474867 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f21b803e560 0x7f21b8040a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f21bc006fd0 tx=0x7f21bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.471+0000 7f21c27fc700 1 -- 172.21.15.46:0/3037474867 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f21c801f030 con 0x7f21cc07f660 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.683+0000 7f21d38e6700 1 -- 172.21.15.46:0/3037474867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f21cc1a3670 con 0x7f21cc07f660 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.684+0000 7f21c27fc700 1 -- 172.21.15.46:0/3037474867 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f21c80105e0 con 0x7f21cc07f660 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.686+0000 7f21d38e6700 1 -- 172.21.15.46:0/3037474867 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f21b803e560 msgr2=0x7f21b8040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.686+0000 7f21d38e6700 1 --2- 172.21.15.46:0/3037474867 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f21b803e560 0x7f21b8040a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f21bc006fd0 tx=0x7f21bc006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.686+0000 7f21d38e6700 1 -- 172.21.15.46:0/3037474867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 msgr2=0x7f21cc1a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.686+0000 7f21d38e6700 1 --2- 172.21.15.46:0/3037474867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 0x7f21cc1a00c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009520 tx=0x7f21c8011aa0 comp rx=0 tx=0).stop 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.686+0000 7f21d38e6700 1 -- 172.21.15.46:0/3037474867 shutdown_connections 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.686+0000 7f21d38e6700 1 --2- 172.21.15.46:0/3037474867 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f21b803e560 0x7f21b8040a70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.686+0000 7f21d38e6700 1 --2- 172.21.15.46:0/3037474867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21cc07f660 0x7f21cc1a00c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.686+0000 7f21d38e6700 1 -- 172.21.15.46:0/3037474867 >> 172.21.15.46:0/3037474867 conn(0x7f21cc105100 msgr2=0x7f21cc105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.686+0000 7f21d38e6700 1 -- 172.21.15.46:0/3037474867 shutdown_connections 2024-05-09T00:01:17.356 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:16.686+0000 7f21d38e6700 1 -- 172.21.15.46:0/3037474867 wait complete. 2024-05-09T00:01:18.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.798+0000 7f154afc8700 1 Processor -- start 2024-05-09T00:01:18.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.798+0000 7f154afc8700 1 -- start start 2024-05-09T00:01:18.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.798+0000 7f154afc8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 0x7f154410a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.798+0000 7f154afc8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f154410a950 con 0x7f1544109f50 2024-05-09T00:01:18.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.798+0000 7f1548d64700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 0x7f154410a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.798+0000 7f1548d64700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 0x7f154410a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41232/0 (socket says 172.21.15.46:41232) 2024-05-09T00:01:18.659 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.798+0000 7f1548d64700 1 -- 172.21.15.46:0/1373231348 learned_addr learned my addr 172.21.15.46:0/1373231348 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.799+0000 7f1548d64700 1 -- 172.21.15.46:0/1373231348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1544108bd0 con 0x7f1544109f50 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.799+0000 7f1548d64700 1 --2- 172.21.15.46:0/1373231348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 0x7f154410a360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15380098a0 tx=0x7f1538009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d79f5186dadc910 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.799+0000 7f15437fe700 1 -- 172.21.15.46:0/1373231348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538004160 con 0x7f1544109f50 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.799+0000 7f15437fe700 1 -- 172.21.15.46:0/1373231348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1538004300 con 0x7f1544109f50 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.799+0000 7f15437fe700 1 -- 172.21.15.46:0/1373231348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538011a20 con 0x7f1544109f50 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.800+0000 7f154afc8700 1 -- 172.21.15.46:0/1373231348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 msgr2=0x7f154410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.800+0000 7f154afc8700 1 --2- 172.21.15.46:0/1373231348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 0x7f154410a360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15380098a0 tx=0x7f1538009bb0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.800+0000 7f154afc8700 1 -- 172.21.15.46:0/1373231348 shutdown_connections 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.800+0000 7f154afc8700 1 --2- 172.21.15.46:0/1373231348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 0x7f154410a360 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.800+0000 7f154afc8700 1 -- 172.21.15.46:0/1373231348 >> 172.21.15.46:0/1373231348 conn(0x7f15441037d0 msgr2=0x7f1544105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.800+0000 7f154afc8700 1 -- 172.21.15.46:0/1373231348 shutdown_connections 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.800+0000 7f154afc8700 1 -- 172.21.15.46:0/1373231348 wait complete. 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.801+0000 7f154afc8700 1 Processor -- start 2024-05-09T00:01:18.660 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.801+0000 7f154afc8700 1 -- start start 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.801+0000 7f154afc8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 0x7f154419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.801+0000 7f154afc8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f154410cf90 con 0x7f1544109f50 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.801+0000 7f1548d64700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 0x7f154419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.801+0000 7f1548d64700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 0x7f154419dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41244/0 (socket says 172.21.15.46:41244) 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.801+0000 7f1548d64700 1 -- 172.21.15.46:0/3383114149 learned_addr learned my addr 172.21.15.46:0/3383114149 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.801+0000 7f1548d64700 1 -- 172.21.15.46:0/3383114149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154419e3e0 con 0x7f1544109f50 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.801+0000 7f1548d64700 1 --2- 172.21.15.46:0/3383114149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 0x7f154419dea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1538011a20 tx=0x7f1538011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.802+0000 7f1541ffb700 1 -- 172.21.15.46:0/3383114149 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15380039e0 con 0x7f1544109f50 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.802+0000 7f1541ffb700 1 -- 172.21.15.46:0/3383114149 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f153801b8f0 con 0x7f1544109f50 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.802+0000 7f154afc8700 1 -- 172.21.15.46:0/3383114149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154419e670 con 0x7f1544109f50 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.802+0000 7f1541ffb700 1 -- 172.21.15.46:0/3383114149 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538023770 con 0x7f1544109f50 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.802+0000 7f154afc8700 1 -- 172.21.15.46:0/3383114149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15441a1280 con 0x7f1544109f50 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.803+0000 7f1541ffb700 1 -- 172.21.15.46:0/3383114149 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f153802c3f0 con 0x7f1544109f50 2024-05-09T00:01:18.661 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.803+0000 7f1541ffb700 1 --2- 172.21.15.46:0/3383114149 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f153403e510 0x7f1534040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.803+0000 7f154afc8700 1 -- 172.21.15.46:0/3383114149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1544064e20 con 0x7f1544109f50 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.803+0000 7f1541ffb700 1 -- 172.21.15.46:0/3383114149 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f153801b4d0 con 0x7f1544109f50 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.803+0000 7f1543fff700 1 --2- 172.21.15.46:0/3383114149 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f153403e510 0x7f1534040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.804+0000 7f1543fff700 1 --2- 172.21.15.46:0/3383114149 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f153403e510 0x7f1534040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1530006fd0 tx=0x7f1530006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:17.809+0000 7f1541ffb700 1 -- 172.21.15.46:0/3383114149 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f153801f030 con 0x7f1544109f50 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:18.143+0000 7f154afc8700 1 -- 172.21.15.46:0/3383114149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f154402e820 con 0x7f1544109f50 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:18.146+0000 7f1541ffb700 1 -- 172.21.15.46:0/3383114149 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f1538010bd0 con 0x7f1544109f50 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:18.148+0000 7f154afc8700 1 -- 172.21.15.46:0/3383114149 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f153403e510 msgr2=0x7f1534040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:18.148+0000 7f154afc8700 1 --2- 172.21.15.46:0/3383114149 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f153403e510 0x7f1534040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1530006fd0 tx=0x7f1530006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:18.148+0000 7f154afc8700 1 -- 172.21.15.46:0/3383114149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 msgr2=0x7f154419dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:18.148+0000 7f154afc8700 1 --2- 172.21.15.46:0/3383114149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 0x7f154419dea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1538011a20 tx=0x7f1538011b00 comp rx=0 tx=0).stop 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:18.148+0000 7f154afc8700 1 -- 172.21.15.46:0/3383114149 shutdown_connections 2024-05-09T00:01:18.662 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:18.148+0000 7f154afc8700 1 --2- 172.21.15.46:0/3383114149 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f153403e510 0x7f1534040a20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:18.148+0000 7f154afc8700 1 --2- 172.21.15.46:0/3383114149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1544109f50 0x7f154419dea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:18.148+0000 7f154afc8700 1 -- 172.21.15.46:0/3383114149 >> 172.21.15.46:0/3383114149 conn(0x7f15441037d0 msgr2=0x7f1544194420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:18.148+0000 7f154afc8700 1 -- 172.21.15.46:0/3383114149 shutdown_connections 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T00:01:18.148+0000 7f154afc8700 1 -- 172.21.15.46:0/3383114149 wait complete. 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout:Or, if you are only running a single cluster on this host: 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:01:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: ceph telemetry on 2024-05-09T00:01:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:01:18.664 INFO:teuthology.orchestra.run.smithi046.stdout:For more information see: 2024-05-09T00:01:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:01:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-09T00:01:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:01:18.664 INFO:teuthology.orchestra.run.smithi046.stdout:Bootstrap complete. 2024-05-09T00:01:18.700 INFO:tasks.cephadm:Fetching config... 2024-05-09T00:01:18.700 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T00:01:18.700 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-09T00:01:18.717 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-09T00:01:18.717 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T00:01:18.717 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-09T00:01:18.774 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-09T00:01:18.774 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T00:01:18.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-09T00:01:18.839 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-09T00:01:18.840 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T00:01:18.840 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-09T00:01:18.896 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-09T00:01:18.896 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhe7ydJJpCLyxp0v0o+1VxeqNwYHVhgTFx4AdHppeI9Zn2PcYVUwcTOSRjBwV2r5Ft1LniyErXoXoF84w0rGQxH1WDoTQWdQevwFN6shdzyEBMWHqsqrGPadDoIiYa4GdStrCHgEUUhooUzL+lfBFMWDrUmZLOdHorSkN98bCZSt2+W8iE0VunhRcnIJzVAvjOaxXtcMWuXcQou5OhKfxZHWSOMp1LZhxDDgB/6UzdL8s71s4DbJohJYuThZSjkDNMUTkMYlaEhYiypRMS3oO2gdilgqDqi1L1xXF/97v5+CHxw8eAWxrZ6I0wYtMfttoxDVmyizpbjOY7Gy26iZR0iNNVWDvd2PZHtEMDuUqxCz+qydSx3gd7iL9W7Zr6UuCNVQRnVDAq1rJDQ0fzsQSrTedBeIGIqcpP+WRH65JFzohvdLvbPNgSDp4T1jKIO6Oa+5DBZgs/PLSxKBVbV4Odyu16WqkBob+R26iAvDMAI29pZxHE05U+39jq9zVQ0fc= ceph-eec67860-0d96-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T00:01:18.975 INFO:teuthology.orchestra.run.smithi046.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhe7ydJJpCLyxp0v0o+1VxeqNwYHVhgTFx4AdHppeI9Zn2PcYVUwcTOSRjBwV2r5Ft1LniyErXoXoF84w0rGQxH1WDoTQWdQevwFN6shdzyEBMWHqsqrGPadDoIiYa4GdStrCHgEUUhooUzL+lfBFMWDrUmZLOdHorSkN98bCZSt2+W8iE0VunhRcnIJzVAvjOaxXtcMWuXcQou5OhKfxZHWSOMp1LZhxDDgB/6UzdL8s71s4DbJohJYuThZSjkDNMUTkMYlaEhYiypRMS3oO2gdilgqDqi1L1xXF/97v5+CHxw8eAWxrZ6I0wYtMfttoxDVmyizpbjOY7Gy26iZR0iNNVWDvd2PZHtEMDuUqxCz+qydSx3gd7iL9W7Zr6UuCNVQRnVDAq1rJDQ0fzsQSrTedBeIGIqcpP+WRH65JFzohvdLvbPNgSDp4T1jKIO6Oa+5DBZgs/PLSxKBVbV4Odyu16WqkBob+R26iAvDMAI29pZxHE05U+39jq9zVQ0fc= ceph-eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:01:18.987 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhe7ydJJpCLyxp0v0o+1VxeqNwYHVhgTFx4AdHppeI9Zn2PcYVUwcTOSRjBwV2r5Ft1LniyErXoXoF84w0rGQxH1WDoTQWdQevwFN6shdzyEBMWHqsqrGPadDoIiYa4GdStrCHgEUUhooUzL+lfBFMWDrUmZLOdHorSkN98bCZSt2+W8iE0VunhRcnIJzVAvjOaxXtcMWuXcQou5OhKfxZHWSOMp1LZhxDDgB/6UzdL8s71s4DbJohJYuThZSjkDNMUTkMYlaEhYiypRMS3oO2gdilgqDqi1L1xXF/97v5+CHxw8eAWxrZ6I0wYtMfttoxDVmyizpbjOY7Gy26iZR0iNNVWDvd2PZHtEMDuUqxCz+qydSx3gd7iL9W7Zr6UuCNVQRnVDAq1rJDQ0fzsQSrTedBeIGIqcpP+WRH65JFzohvdLvbPNgSDp4T1jKIO6Oa+5DBZgs/PLSxKBVbV4Odyu16WqkBob+R26iAvDMAI29pZxHE05U+39jq9zVQ0fc= ceph-eec67860-0d96-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T00:01:19.031 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhe7ydJJpCLyxp0v0o+1VxeqNwYHVhgTFx4AdHppeI9Zn2PcYVUwcTOSRjBwV2r5Ft1LniyErXoXoF84w0rGQxH1WDoTQWdQevwFN6shdzyEBMWHqsqrGPadDoIiYa4GdStrCHgEUUhooUzL+lfBFMWDrUmZLOdHorSkN98bCZSt2+W8iE0VunhRcnIJzVAvjOaxXtcMWuXcQou5OhKfxZHWSOMp1LZhxDDgB/6UzdL8s71s4DbJohJYuThZSjkDNMUTkMYlaEhYiypRMS3oO2gdilgqDqi1L1xXF/97v5+CHxw8eAWxrZ6I0wYtMfttoxDVmyizpbjOY7Gy26iZR0iNNVWDvd2PZHtEMDuUqxCz+qydSx3gd7iL9W7Zr6UuCNVQRnVDAq1rJDQ0fzsQSrTedBeIGIqcpP+WRH65JFzohvdLvbPNgSDp4T1jKIO6Oa+5DBZgs/PLSxKBVbV4Odyu16WqkBob+R26iAvDMAI29pZxHE05U+39jq9zVQ0fc= ceph-eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:01:19.042 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhe7ydJJpCLyxp0v0o+1VxeqNwYHVhgTFx4AdHppeI9Zn2PcYVUwcTOSRjBwV2r5Ft1LniyErXoXoF84w0rGQxH1WDoTQWdQevwFN6shdzyEBMWHqsqrGPadDoIiYa4GdStrCHgEUUhooUzL+lfBFMWDrUmZLOdHorSkN98bCZSt2+W8iE0VunhRcnIJzVAvjOaxXtcMWuXcQou5OhKfxZHWSOMp1LZhxDDgB/6UzdL8s71s4DbJohJYuThZSjkDNMUTkMYlaEhYiypRMS3oO2gdilgqDqi1L1xXF/97v5+CHxw8eAWxrZ6I0wYtMfttoxDVmyizpbjOY7Gy26iZR0iNNVWDvd2PZHtEMDuUqxCz+qydSx3gd7iL9W7Zr6UuCNVQRnVDAq1rJDQ0fzsQSrTedBeIGIqcpP+WRH65JFzohvdLvbPNgSDp4T1jKIO6Oa+5DBZgs/PLSxKBVbV4Odyu16WqkBob+R26iAvDMAI29pZxHE05U+39jq9zVQ0fc= ceph-eec67860-0d96-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T00:01:19.087 INFO:teuthology.orchestra.run.smithi169.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhe7ydJJpCLyxp0v0o+1VxeqNwYHVhgTFx4AdHppeI9Zn2PcYVUwcTOSRjBwV2r5Ft1LniyErXoXoF84w0rGQxH1WDoTQWdQevwFN6shdzyEBMWHqsqrGPadDoIiYa4GdStrCHgEUUhooUzL+lfBFMWDrUmZLOdHorSkN98bCZSt2+W8iE0VunhRcnIJzVAvjOaxXtcMWuXcQou5OhKfxZHWSOMp1LZhxDDgB/6UzdL8s71s4DbJohJYuThZSjkDNMUTkMYlaEhYiypRMS3oO2gdilgqDqi1L1xXF/97v5+CHxw8eAWxrZ6I0wYtMfttoxDVmyizpbjOY7Gy26iZR0iNNVWDvd2PZHtEMDuUqxCz+qydSx3gd7iL9W7Zr6UuCNVQRnVDAq1rJDQ0fzsQSrTedBeIGIqcpP+WRH65JFzohvdLvbPNgSDp4T1jKIO6Oa+5DBZgs/PLSxKBVbV4Odyu16WqkBob+R26iAvDMAI29pZxHE05U+39jq9zVQ0fc= ceph-eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:01:19.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-09T00:01:19.362 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:01:19.485 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:19 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3383114149' entity='client.admin' 2024-05-09T00:01:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.517+0000 7ff64bacf700 1 -- 172.21.15.46:0/3675548437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff644104700 msgr2=0x7ff644106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.518+0000 7ff64bacf700 1 --2- 172.21.15.46:0/3675548437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff644104700 0x7ff644106b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff640009910 tx=0x7ff640009c20 comp rx=0 tx=0).stop 2024-05-09T00:01:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.518+0000 7ff64bacf700 1 -- 172.21.15.46:0/3675548437 shutdown_connections 2024-05-09T00:01:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.518+0000 7ff64bacf700 1 --2- 172.21.15.46:0/3675548437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff644104700 0x7ff644106b20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.518+0000 7ff64bacf700 1 -- 172.21.15.46:0/3675548437 >> 172.21.15.46:0/3675548437 conn(0x7ff6440fe140 msgr2=0x7ff6441005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.518+0000 7ff64bacf700 1 -- 172.21.15.46:0/3675548437 shutdown_connections 2024-05-09T00:01:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.518+0000 7ff64bacf700 1 -- 172.21.15.46:0/3675548437 wait complete. 2024-05-09T00:01:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.519+0000 7ff64bacf700 1 Processor -- start 2024-05-09T00:01:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.519+0000 7ff64bacf700 1 -- start start 2024-05-09T00:01:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.519+0000 7ff64bacf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff644104700 0x7ff6441996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.519+0000 7ff64bacf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff644109bc0 con 0x7ff644104700 2024-05-09T00:01:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.519+0000 7ff64986b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff644104700 0x7ff6441996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.519+0000 7ff64986b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff644104700 0x7ff6441996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41262/0 (socket says 172.21.15.46:41262) 2024-05-09T00:01:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.519+0000 7ff64986b700 1 -- 172.21.15.46:0/1340110448 learned_addr learned my addr 172.21.15.46:0/1340110448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.519+0000 7ff64986b700 1 -- 172.21.15.46:0/1340110448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff644199c30 con 0x7ff644104700 2024-05-09T00:01:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.519+0000 7ff64986b700 1 --2- 172.21.15.46:0/1340110448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff644104700 0x7ff6441996f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff640006010 tx=0x7ff640005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.520+0000 7ff63affd700 1 -- 172.21.15.46:0/1340110448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff640026070 con 0x7ff644104700 2024-05-09T00:01:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.520+0000 7ff63affd700 1 -- 172.21.15.46:0/1340110448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6400044a0 con 0x7ff644104700 2024-05-09T00:01:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.520+0000 7ff64bacf700 1 -- 172.21.15.46:0/1340110448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff644199ec0 con 0x7ff644104700 2024-05-09T00:01:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.520+0000 7ff63affd700 1 -- 172.21.15.46:0/1340110448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6400039d0 con 0x7ff644104700 2024-05-09T00:01:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.520+0000 7ff64bacf700 1 -- 172.21.15.46:0/1340110448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64419cad0 con 0x7ff644104700 2024-05-09T00:01:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.521+0000 7ff64bacf700 1 -- 172.21.15.46:0/1340110448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64410a340 con 0x7ff644104700 2024-05-09T00:01:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.521+0000 7ff63affd700 1 -- 172.21.15.46:0/1340110448 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff640029070 con 0x7ff644104700 2024-05-09T00:01:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.521+0000 7ff63affd700 1 --2- 172.21.15.46:0/1340110448 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff63003e510 0x7ff630040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.521+0000 7ff63affd700 1 -- 172.21.15.46:0/1340110448 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff64001e070 con 0x7ff644104700 2024-05-09T00:01:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.521+0000 7ff64906a700 1 --2- 172.21.15.46:0/1340110448 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff63003e510 0x7ff630040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.522+0000 7ff64906a700 1 --2- 172.21.15.46:0/1340110448 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff63003e510 0x7ff630040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff634006fd0 tx=0x7ff634006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:21.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.527+0000 7ff63affd700 1 -- 172.21.15.46:0/1340110448 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff640013070 con 0x7ff644104700 2024-05-09T00:01:21.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.738+0000 7ff64bacf700 1 -- 172.21.15.46:0/1340110448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7ff64402e700 con 0x7ff644104700 2024-05-09T00:01:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.741+0000 7ff63affd700 1 -- 172.21.15.46:0/1340110448 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7ff640003c00 con 0x7ff644104700 2024-05-09T00:01:21.736 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:21.736 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:21.736 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi046", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:01:21.736 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:21.736 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:01:21.736 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.744+0000 7ff64bacf700 1 -- 172.21.15.46:0/1340110448 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff63003e510 msgr2=0x7ff630040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.744+0000 7ff64bacf700 1 --2- 172.21.15.46:0/1340110448 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff63003e510 0x7ff630040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff634006fd0 tx=0x7ff634006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.744+0000 7ff64bacf700 1 -- 172.21.15.46:0/1340110448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff644104700 msgr2=0x7ff6441996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.744+0000 7ff64bacf700 1 --2- 172.21.15.46:0/1340110448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff644104700 0x7ff6441996f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff640006010 tx=0x7ff640005ad0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.744+0000 7ff64bacf700 1 -- 172.21.15.46:0/1340110448 shutdown_connections 2024-05-09T00:01:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.744+0000 7ff64bacf700 1 --2- 172.21.15.46:0/1340110448 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff63003e510 0x7ff630040a20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.744+0000 7ff64bacf700 1 --2- 172.21.15.46:0/1340110448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff644104700 0x7ff6441996f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.744+0000 7ff64bacf700 1 -- 172.21.15.46:0/1340110448 >> 172.21.15.46:0/1340110448 conn(0x7ff6440fe140 msgr2=0x7ff6440fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.744+0000 7ff64bacf700 1 -- 172.21.15.46:0/1340110448 shutdown_connections 2024-05-09T00:01:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:21.744+0000 7ff64bacf700 1 -- 172.21.15.46:0/1340110448 wait complete. 2024-05-09T00:01:22.376 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-09T00:01:22.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-09T00:01:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:01:23.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:22 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1340110448' entity='client.admin' 2024-05-09T00:01:23.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:22 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:01:23.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:22 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.142+0000 7f638aec1700 1 -- 172.21.15.46:0/2143848783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6384102ae0 msgr2=0x7f6384102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.142+0000 7f638aec1700 1 --2- 172.21.15.46:0/2143848783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6384102ae0 0x7f6384102ef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6378009910 tx=0x7f6378009c20 comp rx=0 tx=0).stop 2024-05-09T00:01:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.142+0000 7f638aec1700 1 -- 172.21.15.46:0/2143848783 shutdown_connections 2024-05-09T00:01:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.142+0000 7f638aec1700 1 --2- 172.21.15.46:0/2143848783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6384102ae0 0x7f6384102ef0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.142+0000 7f638aec1700 1 -- 172.21.15.46:0/2143848783 >> 172.21.15.46:0/2143848783 conn(0x7f63840fe140 msgr2=0x7f63841005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.142+0000 7f638aec1700 1 -- 172.21.15.46:0/2143848783 shutdown_connections 2024-05-09T00:01:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.142+0000 7f638aec1700 1 -- 172.21.15.46:0/2143848783 wait complete. 2024-05-09T00:01:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.142+0000 7f638aec1700 1 Processor -- start 2024-05-09T00:01:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.143+0000 7f638aec1700 1 -- start start 2024-05-09T00:01:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.143+0000 7f638aec1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6384102ae0 0x7f638406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.143+0000 7f638aec1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6384108aa0 con 0x7f6384102ae0 2024-05-09T00:01:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.143+0000 7f6388c5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6384102ae0 0x7f638406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.143+0000 7f6388c5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6384102ae0 0x7f638406c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41286/0 (socket says 172.21.15.46:41286) 2024-05-09T00:01:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.143+0000 7f6388c5d700 1 -- 172.21.15.46:0/1798883419 learned_addr learned my addr 172.21.15.46:0/1798883419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.143+0000 7f6388c5d700 1 -- 172.21.15.46:0/1798883419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638406ca80 con 0x7f6384102ae0 2024-05-09T00:01:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.143+0000 7f6388c5d700 1 --2- 172.21.15.46:0/1798883419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6384102ae0 0x7f638406c540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f63780057d0 tx=0x7f6378005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.143+0000 7f6381ffb700 1 -- 172.21.15.46:0/1798883419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6378027070 con 0x7f6384102ae0 2024-05-09T00:01:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.144+0000 7f6381ffb700 1 -- 172.21.15.46:0/1798883419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f637800be10 con 0x7f6384102ae0 2024-05-09T00:01:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.144+0000 7f6381ffb700 1 -- 172.21.15.46:0/1798883419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6378022dd0 con 0x7f6384102ae0 2024-05-09T00:01:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.144+0000 7f638aec1700 1 -- 172.21.15.46:0/1798883419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638419c4b0 con 0x7f6384102ae0 2024-05-09T00:01:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.144+0000 7f638aec1700 1 -- 172.21.15.46:0/1798883419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638419c930 con 0x7f6384102ae0 2024-05-09T00:01:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.145+0000 7f6381ffb700 1 -- 172.21.15.46:0/1798883419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f637802a070 con 0x7f6384102ae0 2024-05-09T00:01:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.145+0000 7f638aec1700 1 -- 172.21.15.46:0/1798883419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6384109220 con 0x7f6384102ae0 2024-05-09T00:01:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.145+0000 7f6381ffb700 1 --2- 172.21.15.46:0/1798883419 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f637403e4c0 0x7f63740409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.145+0000 7f6381ffb700 1 -- 172.21.15.46:0/1798883419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f637801e030 con 0x7f6384102ae0 2024-05-09T00:01:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.145+0000 7f6383fff700 1 --2- 172.21.15.46:0/1798883419 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f637403e4c0 0x7f63740409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:24.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.146+0000 7f6383fff700 1 --2- 172.21.15.46:0/1798883419 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f637403e4c0 0x7f63740409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6370006fd0 tx=0x7f6370006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.151+0000 7f6381ffb700 1 -- 172.21.15.46:0/1798883419 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6378013070 con 0x7f6384102ae0 2024-05-09T00:01:24.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.374+0000 7f638aec1700 1 -- 172.21.15.46:0/1798883419 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f638402e7e0 con 0x7f637403e4c0 2024-05-09T00:01:24.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.378+0000 7f6381ffb700 1 -- 172.21.15.46:0/1798883419 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f638402e7e0 con 0x7f637403e4c0 2024-05-09T00:01:24.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.381+0000 7f638aec1700 1 -- 172.21.15.46:0/1798883419 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f637403e4c0 msgr2=0x7f63740409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:24.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.381+0000 7f638aec1700 1 --2- 172.21.15.46:0/1798883419 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f637403e4c0 0x7f63740409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6370006fd0 tx=0x7f6370006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:24.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.381+0000 7f638aec1700 1 -- 172.21.15.46:0/1798883419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6384102ae0 msgr2=0x7f638406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:24.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.381+0000 7f638aec1700 1 --2- 172.21.15.46:0/1798883419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6384102ae0 0x7f638406c540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f63780057d0 tx=0x7f6378005800 comp rx=0 tx=0).stop 2024-05-09T00:01:24.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.381+0000 7f638aec1700 1 -- 172.21.15.46:0/1798883419 shutdown_connections 2024-05-09T00:01:24.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.381+0000 7f638aec1700 1 --2- 172.21.15.46:0/1798883419 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f637403e4c0 0x7f63740409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:24.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.381+0000 7f638aec1700 1 --2- 172.21.15.46:0/1798883419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6384102ae0 0x7f638406c540 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:24.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.381+0000 7f638aec1700 1 -- 172.21.15.46:0/1798883419 >> 172.21.15.46:0/1798883419 conn(0x7f63840fe140 msgr2=0x7f63840fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:24.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.381+0000 7f638aec1700 1 -- 172.21.15.46:0/1798883419 shutdown_connections 2024-05-09T00:01:24.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:24.381+0000 7f638aec1700 1 -- 172.21.15.46:0/1798883419 wait complete. 2024-05-09T00:01:25.056 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2024-05-09T00:01:25.057 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-09T00:01:25.057 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2024-05-09T00:01:25.075 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-09T00:01:25.075 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:01:25.132 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2024-05-09T00:01:25.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch host add smithi142 2024-05-09T00:01:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:01:25.639 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:25 smithi046 ceph-mon[30151]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:01:25.639 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:25.639 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:01:25.639 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:01:25.639 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:01:25.639 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:25 smithi046 ceph-mon[30151]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-09T00:01:25.639 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:25 smithi046 ceph-mon[30151]: Updating smithi046:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:01:26.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:26 smithi046 ceph-mon[30151]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:01:26.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:26 smithi046 ceph-mon[30151]: Updating smithi046:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T00:01:26.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:26 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:26.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:26 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:26.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:26 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:26.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.947+0000 7f55415ea700 1 -- 172.21.15.46:0/449782851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553c108ac0 msgr2=0x7f553c108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.947+0000 7f55415ea700 1 --2- 172.21.15.46:0/449782851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553c108ac0 0x7f553c108ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f552c009910 tx=0x7f552c009c20 comp rx=0 tx=0).stop 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.947+0000 7f55415ea700 1 -- 172.21.15.46:0/449782851 shutdown_connections 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.947+0000 7f55415ea700 1 --2- 172.21.15.46:0/449782851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553c108ac0 0x7f553c108ed0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.947+0000 7f55415ea700 1 -- 172.21.15.46:0/449782851 >> 172.21.15.46:0/449782851 conn(0x7f553c0fe5b0 msgr2=0x7f553c100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.947+0000 7f55415ea700 1 -- 172.21.15.46:0/449782851 shutdown_connections 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.947+0000 7f55415ea700 1 -- 172.21.15.46:0/449782851 wait complete. 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.948+0000 7f55415ea700 1 Processor -- start 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.948+0000 7f55415ea700 1 -- start start 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.948+0000 7f55415ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553c108ac0 0x7f553c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.948+0000 7f55415ea700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f553c10d010 con 0x7f553c108ac0 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.948+0000 7f553affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553c108ac0 0x7f553c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.948+0000 7f553affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553c108ac0 0x7f553c19a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60050/0 (socket says 172.21.15.46:60050) 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.948+0000 7f553affd700 1 -- 172.21.15.46:0/3229596390 learned_addr learned my addr 172.21.15.46:0/3229596390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.949+0000 7f553affd700 1 -- 172.21.15.46:0/3229596390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c06ca10 con 0x7f553c108ac0 2024-05-09T00:01:26.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.949+0000 7f553affd700 1 --2- 172.21.15.46:0/3229596390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553c108ac0 0x7f553c19a800 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f552c0092e0 tx=0x7f552c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:26.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.949+0000 7f5523fff700 1 -- 172.21.15.46:0/3229596390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f552c026070 con 0x7f553c108ac0 2024-05-09T00:01:26.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.949+0000 7f55415ea700 1 -- 172.21.15.46:0/3229596390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c19ae00 con 0x7f553c108ac0 2024-05-09T00:01:26.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.949+0000 7f5523fff700 1 -- 172.21.15.46:0/3229596390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f552c00b750 con 0x7f553c108ac0 2024-05-09T00:01:26.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.949+0000 7f5523fff700 1 -- 172.21.15.46:0/3229596390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f552c0038a0 con 0x7f553c108ac0 2024-05-09T00:01:26.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.949+0000 7f55415ea700 1 -- 172.21.15.46:0/3229596390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c19b2c0 con 0x7f553c108ac0 2024-05-09T00:01:26.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.950+0000 7f55415ea700 1 -- 172.21.15.46:0/3229596390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553c1041d0 con 0x7f553c108ac0 2024-05-09T00:01:26.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.950+0000 7f5523fff700 1 -- 172.21.15.46:0/3229596390 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f552c029070 con 0x7f553c108ac0 2024-05-09T00:01:26.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.950+0000 7f5523fff700 1 --2- 172.21.15.46:0/3229596390 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f552403e510 0x7f5524040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:26.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.950+0000 7f5523fff700 1 -- 172.21.15.46:0/3229596390 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f552c01e070 con 0x7f553c108ac0 2024-05-09T00:01:26.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.950+0000 7f553a7fc700 1 --2- 172.21.15.46:0/3229596390 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f552403e510 0x7f5524040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:26.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.951+0000 7f553a7fc700 1 --2- 172.21.15.46:0/3229596390 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f552403e510 0x7f5524040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5530006fd0 tx=0x7f5530006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:26.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:26.956+0000 7f5523fff700 1 -- 172.21.15.46:0/3229596390 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f553c06ca10 con 0x7f553c108ac0 2024-05-09T00:01:27.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:27.175+0000 7f55415ea700 1 -- 172.21.15.46:0/3229596390 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}) -- 0x7f553c06c160 con 0x7f552403e510 2024-05-09T00:01:28.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:27 smithi046 ceph-mon[30151]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:01:29.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:28 smithi046 ceph-mon[30151]: Deploying cephadm binary to smithi142 2024-05-09T00:01:30.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:30 smithi046 ceph-mon[30151]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:31.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:31.429+0000 7f5523fff700 1 -- 172.21.15.46:0/3229596390 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f553c06c160 con 0x7f552403e510 2024-05-09T00:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi142' with addr '172.21.15.142' 2024-05-09T00:01:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:31.432+0000 7f55415ea700 1 -- 172.21.15.46:0/3229596390 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f552403e510 msgr2=0x7f5524040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:31.432+0000 7f55415ea700 1 --2- 172.21.15.46:0/3229596390 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f552403e510 0x7f5524040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5530006fd0 tx=0x7f5530006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:31.432+0000 7f55415ea700 1 -- 172.21.15.46:0/3229596390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553c108ac0 msgr2=0x7f553c19a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:31.432+0000 7f55415ea700 1 --2- 172.21.15.46:0/3229596390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553c108ac0 0x7f553c19a800 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f552c0092e0 tx=0x7f552c005780 comp rx=0 tx=0).stop 2024-05-09T00:01:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:31.432+0000 7f55415ea700 1 -- 172.21.15.46:0/3229596390 shutdown_connections 2024-05-09T00:01:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:31.432+0000 7f55415ea700 1 --2- 172.21.15.46:0/3229596390 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f552403e510 0x7f5524040a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:31.432+0000 7f55415ea700 1 --2- 172.21.15.46:0/3229596390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553c108ac0 0x7f553c19a800 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:31.432+0000 7f55415ea700 1 -- 172.21.15.46:0/3229596390 >> 172.21.15.46:0/3229596390 conn(0x7f553c0fe5b0 msgr2=0x7f553c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:31.432+0000 7f55415ea700 1 -- 172.21.15.46:0/3229596390 shutdown_connections 2024-05-09T00:01:31.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:31.432+0000 7f55415ea700 1 -- 172.21.15.46:0/3229596390 wait complete. 2024-05-09T00:01:32.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T00:01:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:01:32.710 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:32 smithi046 ceph-mon[30151]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:32.710 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:32.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:32 smithi046 ceph-mon[30151]: Added host smithi142 2024-05-09T00:01:32.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:01:32.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:33.591 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.614+0000 7ff81bbcf700 1 -- 172.21.15.46:0/3573691216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814108a90 msgr2=0x7ff814108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.614+0000 7ff81bbcf700 1 --2- 172.21.15.46:0/3573691216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814108a90 0x7ff814108ea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff81000b0a0 tx=0x7ff81000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.614+0000 7ff81bbcf700 1 -- 172.21.15.46:0/3573691216 shutdown_connections 2024-05-09T00:01:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.614+0000 7ff81bbcf700 1 --2- 172.21.15.46:0/3573691216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814108a90 0x7ff814108ea0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.614+0000 7ff81bbcf700 1 -- 172.21.15.46:0/3573691216 >> 172.21.15.46:0/3573691216 conn(0x7ff8140785b0 msgr2=0x7ff814078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.615+0000 7ff81bbcf700 1 -- 172.21.15.46:0/3573691216 shutdown_connections 2024-05-09T00:01:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.615+0000 7ff81bbcf700 1 -- 172.21.15.46:0/3573691216 wait complete. 2024-05-09T00:01:33.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.615+0000 7ff81bbcf700 1 Processor -- start 2024-05-09T00:01:33.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.616+0000 7ff81bbcf700 1 -- start start 2024-05-09T00:01:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.616+0000 7ff81bbcf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814108a90 0x7ff81419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.616+0000 7ff81bbcf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff81410cfe0 con 0x7ff814108a90 2024-05-09T00:01:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.616+0000 7ff81996b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814108a90 0x7ff81419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.616+0000 7ff81996b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814108a90 0x7ff81419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60058/0 (socket says 172.21.15.46:60058) 2024-05-09T00:01:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.616+0000 7ff81996b700 1 -- 172.21.15.46:0/1436277433 learned_addr learned my addr 172.21.15.46:0/1436277433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.616+0000 7ff81996b700 1 -- 172.21.15.46:0/1436277433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81406bd00 con 0x7ff814108a90 2024-05-09T00:01:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.616+0000 7ff81996b700 1 --2- 172.21.15.46:0/1436277433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814108a90 0x7ff81419a700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff810015040 tx=0x7ff810009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.617+0000 7ff80affd700 1 -- 172.21.15.46:0/1436277433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff810021020 con 0x7ff814108a90 2024-05-09T00:01:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.617+0000 7ff80affd700 1 -- 172.21.15.46:0/1436277433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff810007970 con 0x7ff814108a90 2024-05-09T00:01:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.617+0000 7ff81bbcf700 1 -- 172.21.15.46:0/1436277433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81419ad00 con 0x7ff814108a90 2024-05-09T00:01:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.617+0000 7ff80affd700 1 -- 172.21.15.46:0/1436277433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff810017040 con 0x7ff814108a90 2024-05-09T00:01:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.617+0000 7ff81bbcf700 1 -- 172.21.15.46:0/1436277433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81419b1c0 con 0x7ff814108a90 2024-05-09T00:01:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.618+0000 7ff81bbcf700 1 -- 172.21.15.46:0/1436277433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff81419b500 con 0x7ff814108a90 2024-05-09T00:01:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.618+0000 7ff80affd700 1 -- 172.21.15.46:0/1436277433 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff810024020 con 0x7ff814108a90 2024-05-09T00:01:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.618+0000 7ff80affd700 1 --2- 172.21.15.46:0/1436277433 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff80003e510 0x7ff800040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.618+0000 7ff80affd700 1 -- 172.21.15.46:0/1436277433 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff81000f050 con 0x7ff814108a90 2024-05-09T00:01:33.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.618+0000 7ff81916a700 1 --2- 172.21.15.46:0/1436277433 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff80003e510 0x7ff800040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:33.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.619+0000 7ff81916a700 1 --2- 172.21.15.46:0/1436277433 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff80003e510 0x7ff800040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff804006fd0 tx=0x7ff804006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.623+0000 7ff80affd700 1 -- 172.21.15.46:0/1436277433 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff81002b020 con 0x7ff814108a90 2024-05-09T00:01:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.848+0000 7ff81bbcf700 1 -- 172.21.15.46:0/1436277433 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff814064e20 con 0x7ff80003e510 2024-05-09T00:01:33.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.849+0000 7ff80affd700 1 -- 172.21.15.46:0/1436277433 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7ff814064e20 con 0x7ff80003e510 2024-05-09T00:01:33.844 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:01:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2024-05-09T00:01:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.852+0000 7ff81bbcf700 1 -- 172.21.15.46:0/1436277433 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff80003e510 msgr2=0x7ff800040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.852+0000 7ff81bbcf700 1 --2- 172.21.15.46:0/1436277433 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff80003e510 0x7ff800040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff804006fd0 tx=0x7ff804006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.852+0000 7ff81bbcf700 1 -- 172.21.15.46:0/1436277433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814108a90 msgr2=0x7ff81419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.852+0000 7ff81bbcf700 1 --2- 172.21.15.46:0/1436277433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814108a90 0x7ff81419a700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff810015040 tx=0x7ff810009790 comp rx=0 tx=0).stop 2024-05-09T00:01:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.852+0000 7ff81bbcf700 1 -- 172.21.15.46:0/1436277433 shutdown_connections 2024-05-09T00:01:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.852+0000 7ff81bbcf700 1 --2- 172.21.15.46:0/1436277433 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff80003e510 0x7ff800040a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.852+0000 7ff81bbcf700 1 --2- 172.21.15.46:0/1436277433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff814108a90 0x7ff81419a700 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.852+0000 7ff81bbcf700 1 -- 172.21.15.46:0/1436277433 >> 172.21.15.46:0/1436277433 conn(0x7ff8140785b0 msgr2=0x7ff814101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.852+0000 7ff81bbcf700 1 -- 172.21.15.46:0/1436277433 shutdown_connections 2024-05-09T00:01:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:33.852+0000 7ff81bbcf700 1 -- 172.21.15.46:0/1436277433 wait complete. 2024-05-09T00:01:34.527 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi169 2024-05-09T00:01:34.527 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T00:01:34.527 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.conf 2024-05-09T00:01:34.546 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T00:01:34.546 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:01:34.603 INFO:tasks.cephadm:Adding host smithi169 to orchestrator... 2024-05-09T00:01:34.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch host add smithi169 2024-05-09T00:01:34.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:34 smithi046 ceph-mon[30151]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:34.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:34 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:34.850 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:01:35.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:35 smithi046 ceph-mon[30151]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:01:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.235+0000 7fa1effed700 1 -- 172.21.15.46:0/3504524371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108a60 msgr2=0x7fa1e8108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.235+0000 7fa1effed700 1 --2- 172.21.15.46:0/3504524371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108a60 0x7fa1e8108e70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009910 tx=0x7fa1e4009c20 comp rx=0 tx=0).stop 2024-05-09T00:01:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.235+0000 7fa1effed700 1 -- 172.21.15.46:0/3504524371 shutdown_connections 2024-05-09T00:01:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.235+0000 7fa1effed700 1 --2- 172.21.15.46:0/3504524371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108a60 0x7fa1e8108e70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.235+0000 7fa1effed700 1 -- 172.21.15.46:0/3504524371 >> 172.21.15.46:0/3504524371 conn(0x7fa1e80fe5b0 msgr2=0x7fa1e8100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.235+0000 7fa1effed700 1 -- 172.21.15.46:0/3504524371 shutdown_connections 2024-05-09T00:01:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.235+0000 7fa1effed700 1 -- 172.21.15.46:0/3504524371 wait complete. 2024-05-09T00:01:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.236+0000 7fa1effed700 1 Processor -- start 2024-05-09T00:01:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.236+0000 7fa1effed700 1 -- start start 2024-05-09T00:01:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.236+0000 7fa1effed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108a60 0x7fa1e819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.236+0000 7fa1effed700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810cfb0 con 0x7fa1e8108a60 2024-05-09T00:01:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.236+0000 7fa1edd89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108a60 0x7fa1e819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.236+0000 7fa1edd89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108a60 0x7fa1e819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60080/0 (socket says 172.21.15.46:60080) 2024-05-09T00:01:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.236+0000 7fa1edd89700 1 -- 172.21.15.46:0/2626340715 learned_addr learned my addr 172.21.15.46:0/2626340715 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.237+0000 7fa1edd89700 1 -- 172.21.15.46:0/2626340715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e806c730 con 0x7fa1e8108a60 2024-05-09T00:01:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.237+0000 7fa1edd89700 1 --2- 172.21.15.46:0/2626340715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108a60 0x7fa1e819a7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40092e0 tx=0x7fa1e4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.237+0000 7fa1deffd700 1 -- 172.21.15.46:0/2626340715 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1e4026070 con 0x7fa1e8108a60 2024-05-09T00:01:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.237+0000 7fa1deffd700 1 -- 172.21.15.46:0/2626340715 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1e400b750 con 0x7fa1e8108a60 2024-05-09T00:01:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.237+0000 7fa1effed700 1 -- 172.21.15.46:0/2626340715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e819adc0 con 0x7fa1e8108a60 2024-05-09T00:01:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.237+0000 7fa1deffd700 1 -- 172.21.15.46:0/2626340715 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1e40038a0 con 0x7fa1e8108a60 2024-05-09T00:01:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.237+0000 7fa1effed700 1 -- 172.21.15.46:0/2626340715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e819b280 con 0x7fa1e8108a60 2024-05-09T00:01:36.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.238+0000 7fa1effed700 1 -- 172.21.15.46:0/2626340715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e819b5c0 con 0x7fa1e8108a60 2024-05-09T00:01:36.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.238+0000 7fa1deffd700 1 -- 172.21.15.46:0/2626340715 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa1e4029070 con 0x7fa1e8108a60 2024-05-09T00:01:36.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.239+0000 7fa1deffd700 1 --2- 172.21.15.46:0/2626340715 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa1d403e570 0x7fa1d4040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:36.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.239+0000 7fa1deffd700 1 -- 172.21.15.46:0/2626340715 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa1e401e070 con 0x7fa1e8108a60 2024-05-09T00:01:36.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.239+0000 7fa1ed588700 1 --2- 172.21.15.46:0/2626340715 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa1d403e570 0x7fa1d4040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:36.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.239+0000 7fa1ed588700 1 --2- 172.21.15.46:0/2626340715 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa1d403e570 0x7fa1d4040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8006fd0 tx=0x7fa1d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:36.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.244+0000 7fa1deffd700 1 -- 172.21.15.46:0/2626340715 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1e4013070 con 0x7fa1e8108a60 2024-05-09T00:01:36.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:36.464+0000 7fa1effed700 1 -- 172.21.15.46:0/2626340715 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}) -- 0x7fa1e8064e20 con 0x7fa1d403e570 2024-05-09T00:01:36.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:36 smithi046 ceph-mon[30151]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:37.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:37 smithi046 ceph-mon[30151]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:01:37.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:37 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:37.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:37 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:37.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:37 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:37.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:37 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:37.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:37 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:01:37.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:37 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:01:37.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:37 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:01:38.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:38 smithi046 ceph-mon[30151]: Deploying cephadm binary to smithi169 2024-05-09T00:01:38.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:38 smithi046 ceph-mon[30151]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:38.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:38 smithi046 ceph-mon[30151]: Updating smithi142:/etc/ceph/ceph.conf 2024-05-09T00:01:38.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:38 smithi046 ceph-mon[30151]: Updating smithi142:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:01:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:40.736+0000 7fa1deffd700 1 -- 172.21.15.46:0/2626340715 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa1e8064e20 con 0x7fa1d403e570 2024-05-09T00:01:40.731 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi169' with addr '172.21.15.169' 2024-05-09T00:01:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:40.739+0000 7fa1effed700 1 -- 172.21.15.46:0/2626340715 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa1d403e570 msgr2=0x7fa1d4040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:40.739+0000 7fa1effed700 1 --2- 172.21.15.46:0/2626340715 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa1d403e570 0x7fa1d4040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8006fd0 tx=0x7fa1d8006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:40.739+0000 7fa1effed700 1 -- 172.21.15.46:0/2626340715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108a60 msgr2=0x7fa1e819a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:40.739+0000 7fa1effed700 1 --2- 172.21.15.46:0/2626340715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108a60 0x7fa1e819a7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40092e0 tx=0x7fa1e4005780 comp rx=0 tx=0).stop 2024-05-09T00:01:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:40.739+0000 7fa1effed700 1 -- 172.21.15.46:0/2626340715 shutdown_connections 2024-05-09T00:01:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:40.739+0000 7fa1effed700 1 --2- 172.21.15.46:0/2626340715 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa1d403e570 0x7fa1d4040a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:40.739+0000 7fa1effed700 1 --2- 172.21.15.46:0/2626340715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108a60 0x7fa1e819a7c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:40.739+0000 7fa1effed700 1 -- 172.21.15.46:0/2626340715 >> 172.21.15.46:0/2626340715 conn(0x7fa1e80fe5b0 msgr2=0x7fa1e80ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:40.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:40.739+0000 7fa1effed700 1 -- 172.21.15.46:0/2626340715 shutdown_connections 2024-05-09T00:01:40.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:40.739+0000 7fa1effed700 1 -- 172.21.15.46:0/2626340715 wait complete. 2024-05-09T00:01:40.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:40 smithi046 ceph-mon[30151]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:41.316 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T00:01:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:01:41.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:41 smithi046 ceph-mon[30151]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:01:41.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:41 smithi046 ceph-mon[30151]: Updating smithi142:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T00:01:41.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:41.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:41.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:41.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:41.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:01:41.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:42.899 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:42 smithi046 ceph-mon[30151]: Added host smithi169 2024-05-09T00:01:42.899 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:42 smithi046 ceph-mon[30151]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:42.899 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:42 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.025+0000 7f3405b69700 1 -- 172.21.15.46:0/1265477583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400104d20 msgr2=0x7f3400105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.026+0000 7f3405b69700 1 --2- 172.21.15.46:0/1265477583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400104d20 0x7f3400105130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009910 tx=0x7f33f0009c20 comp rx=0 tx=0).stop 2024-05-09T00:01:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.026+0000 7f3405b69700 1 -- 172.21.15.46:0/1265477583 shutdown_connections 2024-05-09T00:01:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.026+0000 7f3405b69700 1 --2- 172.21.15.46:0/1265477583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400104d20 0x7f3400105130 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.026+0000 7f3405b69700 1 -- 172.21.15.46:0/1265477583 >> 172.21.15.46:0/1265477583 conn(0x7f34000fe5b0 msgr2=0x7f3400100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.026+0000 7f3405b69700 1 -- 172.21.15.46:0/1265477583 shutdown_connections 2024-05-09T00:01:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.026+0000 7f3405b69700 1 -- 172.21.15.46:0/1265477583 wait complete. 2024-05-09T00:01:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.026+0000 7f3405b69700 1 Processor -- start 2024-05-09T00:01:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.027+0000 7f3405b69700 1 -- start start 2024-05-09T00:01:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.027+0000 7f3405b69700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400104d20 0x7f340019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.027+0000 7f3405b69700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f340010e0c0 con 0x7f3400104d20 2024-05-09T00:01:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.027+0000 7f33ff7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400104d20 0x7f340019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.027+0000 7f33ff7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400104d20 0x7f340019f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34174/0 (socket says 172.21.15.46:34174) 2024-05-09T00:01:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.027+0000 7f33ff7fe700 1 -- 172.21.15.46:0/2839180102 learned_addr learned my addr 172.21.15.46:0/2839180102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.027+0000 7f33ff7fe700 1 -- 172.21.15.46:0/2839180102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340019faf0 con 0x7f3400104d20 2024-05-09T00:01:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.027+0000 7f33ff7fe700 1 --2- 172.21.15.46:0/2839180102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400104d20 0x7f340019f5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f33f00056d0 tx=0x7f33f00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.028+0000 7f33fcff9700 1 -- 172.21.15.46:0/2839180102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33f0026070 con 0x7f3400104d20 2024-05-09T00:01:43.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.028+0000 7f3405b69700 1 -- 172.21.15.46:0/2839180102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340019fde0 con 0x7f3400104d20 2024-05-09T00:01:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.028+0000 7f33fcff9700 1 -- 172.21.15.46:0/2839180102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f000bba0 con 0x7f3400104d20 2024-05-09T00:01:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.028+0000 7f33fcff9700 1 -- 172.21.15.46:0/2839180102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33f0003920 con 0x7f3400104d20 2024-05-09T00:01:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.028+0000 7f3405b69700 1 -- 172.21.15.46:0/2839180102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3400199910 con 0x7f3400104d20 2024-05-09T00:01:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.029+0000 7f3405b69700 1 -- 172.21.15.46:0/2839180102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f340006c100 con 0x7f3400104d20 2024-05-09T00:01:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.029+0000 7f33fcff9700 1 -- 172.21.15.46:0/2839180102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f33f0029070 con 0x7f3400104d20 2024-05-09T00:01:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.029+0000 7f33fcff9700 1 --2- 172.21.15.46:0/2839180102 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33e803e510 0x7f33e8040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.029+0000 7f33fcff9700 1 -- 172.21.15.46:0/2839180102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f33f001e070 con 0x7f3400104d20 2024-05-09T00:01:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.029+0000 7f33feffd700 1 --2- 172.21.15.46:0/2839180102 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33e803e510 0x7f33e8040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.030+0000 7f33feffd700 1 --2- 172.21.15.46:0/2839180102 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33e803e510 0x7f33e8040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f33f4006fd0 tx=0x7f33f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.035+0000 7f33fcff9700 1 -- 172.21.15.46:0/2839180102 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33f0013070 con 0x7f3400104d20 2024-05-09T00:01:43.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.257+0000 7f3405b69700 1 -- 172.21.15.46:0/2839180102 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f340019acc0 con 0x7f33e803e510 2024-05-09T00:01:43.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.259+0000 7f33fcff9700 1 -- 172.21.15.46:0/2839180102 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f340019acc0 con 0x7f33e803e510 2024-05-09T00:01:43.252 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:01:43.252 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}, {"addr": "172.21.15.169", "hostname": "smithi169", "labels": [], "status": ""}] 2024-05-09T00:01:43.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.261+0000 7f3405b69700 1 -- 172.21.15.46:0/2839180102 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33e803e510 msgr2=0x7f33e8040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:43.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.261+0000 7f3405b69700 1 --2- 172.21.15.46:0/2839180102 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33e803e510 0x7f33e8040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f33f4006fd0 tx=0x7f33f4006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:43.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.261+0000 7f3405b69700 1 -- 172.21.15.46:0/2839180102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400104d20 msgr2=0x7f340019f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:43.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.261+0000 7f3405b69700 1 --2- 172.21.15.46:0/2839180102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400104d20 0x7f340019f5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f33f00056d0 tx=0x7f33f00057b0 comp rx=0 tx=0).stop 2024-05-09T00:01:43.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.261+0000 7f3405b69700 1 -- 172.21.15.46:0/2839180102 shutdown_connections 2024-05-09T00:01:43.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.261+0000 7f3405b69700 1 --2- 172.21.15.46:0/2839180102 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33e803e510 0x7f33e8040a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:43.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.261+0000 7f3405b69700 1 --2- 172.21.15.46:0/2839180102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400104d20 0x7f340019f5b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:43.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.261+0000 7f3405b69700 1 -- 172.21.15.46:0/2839180102 >> 172.21.15.46:0/2839180102 conn(0x7f34000fe5b0 msgr2=0x7f3400105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:43.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.261+0000 7f3405b69700 1 -- 172.21.15.46:0/2839180102 shutdown_connections 2024-05-09T00:01:43.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:43.261+0000 7f3405b69700 1 -- 172.21.15.46:0/2839180102 wait complete. 2024-05-09T00:01:43.841 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-09T00:01:43.841 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-09T00:01:44.086 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:01:44.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:44 smithi046 ceph-mon[30151]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:44.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:44 smithi046 ceph-mon[30151]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:01:44.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.588+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3479482387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68103df0 msgr2=0x7f3a68104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.588+0000 7f3a6d85c700 1 --2- 172.21.15.46:0/3479482387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68103df0 0x7f3a68104200 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009910 tx=0x7f3a58009c20 comp rx=0 tx=0).stop 2024-05-09T00:01:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.589+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3479482387 shutdown_connections 2024-05-09T00:01:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.589+0000 7f3a6d85c700 1 --2- 172.21.15.46:0/3479482387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68103df0 0x7f3a68104200 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.589+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3479482387 >> 172.21.15.46:0/3479482387 conn(0x7f3a680ff6a0 msgr2=0x7f3a68101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.589+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3479482387 shutdown_connections 2024-05-09T00:01:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.589+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3479482387 wait complete. 2024-05-09T00:01:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.589+0000 7f3a6d85c700 1 Processor -- start 2024-05-09T00:01:45.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.590+0000 7f3a6d85c700 1 -- start start 2024-05-09T00:01:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.590+0000 7f3a6d85c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68103df0 0x7f3a68075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.590+0000 7f3a6d85c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6810e0e0 con 0x7f3a68103df0 2024-05-09T00:01:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.590+0000 7f3a66ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68103df0 0x7f3a68075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.590+0000 7f3a66ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68103df0 0x7f3a68075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34196/0 (socket says 172.21.15.46:34196) 2024-05-09T00:01:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.590+0000 7f3a66ffd700 1 -- 172.21.15.46:0/3842749664 learned_addr learned my addr 172.21.15.46:0/3842749664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:01:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.590+0000 7f3a66ffd700 1 -- 172.21.15.46:0/3842749664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a68077910 con 0x7f3a68103df0 2024-05-09T00:01:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.590+0000 7f3a66ffd700 1 --2- 172.21.15.46:0/3842749664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68103df0 0x7f3a68075ac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3a58006010 tx=0x7f3a58005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.590+0000 7f3a6c85a700 1 -- 172.21.15.46:0/3842749664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a58026070 con 0x7f3a68103df0 2024-05-09T00:01:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.591+0000 7f3a6c85a700 1 -- 172.21.15.46:0/3842749664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a5800bba0 con 0x7f3a68103df0 2024-05-09T00:01:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.591+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3842749664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a68076000 con 0x7f3a68103df0 2024-05-09T00:01:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.591+0000 7f3a6c85a700 1 -- 172.21.15.46:0/3842749664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a58003910 con 0x7f3a68103df0 2024-05-09T00:01:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.591+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3842749664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a680764c0 con 0x7f3a68103df0 2024-05-09T00:01:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.591+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3842749664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a681052a0 con 0x7f3a68103df0 2024-05-09T00:01:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.592+0000 7f3a6c85a700 1 -- 172.21.15.46:0/3842749664 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3a58029070 con 0x7f3a68103df0 2024-05-09T00:01:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.592+0000 7f3a6c85a700 1 --2- 172.21.15.46:0/3842749664 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3a5003e510 0x7f3a50040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.592+0000 7f3a6c85a700 1 -- 172.21.15.46:0/3842749664 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3a5801e070 con 0x7f3a68103df0 2024-05-09T00:01:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.592+0000 7f3a667fc700 1 --2- 172.21.15.46:0/3842749664 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3a5003e510 0x7f3a50040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:45.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.592+0000 7f3a667fc700 1 --2- 172.21.15.46:0/3842749664 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3a5003e510 0x7f3a50040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c006fd0 tx=0x7f3a5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:45.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.597+0000 7f3a6c85a700 1 -- 172.21.15.46:0/3842749664 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3a58013070 con 0x7f3a68103df0 2024-05-09T00:01:45.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:45.821+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3842749664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3a68077020 con 0x7f3a68103df0 2024-05-09T00:01:46.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:46.378+0000 7f3a6c85a700 1 -- 172.21.15.46:0/3842749664 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f3a5800bd50 con 0x7f3a68103df0 2024-05-09T00:01:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:46.381+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3842749664 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3a5003e510 msgr2=0x7f3a50040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:46.381+0000 7f3a6d85c700 1 --2- 172.21.15.46:0/3842749664 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3a5003e510 0x7f3a50040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c006fd0 tx=0x7f3a5c006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:46.381+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3842749664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68103df0 msgr2=0x7f3a68075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:46.381+0000 7f3a6d85c700 1 --2- 172.21.15.46:0/3842749664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68103df0 0x7f3a68075ac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3a58006010 tx=0x7f3a58005780 comp rx=0 tx=0).stop 2024-05-09T00:01:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:46.381+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3842749664 shutdown_connections 2024-05-09T00:01:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:46.381+0000 7f3a6d85c700 1 --2- 172.21.15.46:0/3842749664 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3a5003e510 0x7f3a50040a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:46.381+0000 7f3a6d85c700 1 --2- 172.21.15.46:0/3842749664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68103df0 0x7f3a68075ac0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:46.381+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3842749664 >> 172.21.15.46:0/3842749664 conn(0x7f3a680ff6a0 msgr2=0x7f3a68101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:46.381+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3842749664 shutdown_connections 2024-05-09T00:01:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:01:46.381+0000 7f3a6d85c700 1 -- 172.21.15.46:0/3842749664 wait complete. 2024-05-09T00:01:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-05-09T00:01:46.748 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:46 smithi046 ceph-mon[30151]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:46.749 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:46 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3842749664' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T00:01:46.749 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:46.749 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:46.749 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:46.749 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:46.749 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:01:46.749 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:01:46.749 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:01:47.064 INFO:tasks.cephadm:Adding mon.a on smithi046 2024-05-09T00:01:47.064 INFO:tasks.cephadm:Adding mon.b on smithi142 2024-05-09T00:01:47.064 INFO:tasks.cephadm:Adding mon.c on smithi169 2024-05-09T00:01:47.065 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi046:172.21.15.46=a;smithi142:172.21.15.142=b;smithi169:172.21.15.169=c' 2024-05-09T00:01:47.298 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:01:47.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:47 smithi046 ceph-mon[30151]: Updating smithi169:/etc/ceph/ceph.conf 2024-05-09T00:01:47.838 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:47 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3842749664' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T00:01:47.839 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:47 smithi046 ceph-mon[30151]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:01:47.839 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:47 smithi046 ceph-mon[30151]: Updating smithi169:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:01:47.839 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:47.839 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:47.839 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:48.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.538+0000 7f351e80c700 1 -- 172.21.15.169:0/139524394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3518103dd0 msgr2=0x7f35181041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:48.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.538+0000 7f351e80c700 1 --2- 172.21.15.169:0/139524394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3518103dd0 0x7f35181041e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3500009910 tx=0x7f3500009c20 comp rx=0 tx=0).stop 2024-05-09T00:01:48.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.538+0000 7f351e80c700 1 -- 172.21.15.169:0/139524394 shutdown_connections 2024-05-09T00:01:48.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.538+0000 7f351e80c700 1 --2- 172.21.15.169:0/139524394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3518103dd0 0x7f35181041e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:48.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.538+0000 7f351e80c700 1 -- 172.21.15.169:0/139524394 >> 172.21.15.169:0/139524394 conn(0x7f35180ff660 msgr2=0x7f3518101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:48.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.538+0000 7f351e80c700 1 -- 172.21.15.169:0/139524394 shutdown_connections 2024-05-09T00:01:48.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.538+0000 7f351e80c700 1 -- 172.21.15.169:0/139524394 wait complete. 2024-05-09T00:01:48.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.539+0000 7f351e80c700 1 Processor -- start 2024-05-09T00:01:48.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.539+0000 7f351e80c700 1 -- start start 2024-05-09T00:01:48.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.539+0000 7f351e80c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3518103dd0 0x7f3518075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:48.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.539+0000 7f351e80c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f351810e0c0 con 0x7f3518103dd0 2024-05-09T00:01:48.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.539+0000 7f3517fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3518103dd0 0x7f3518075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:48.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.539+0000 7f3517fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3518103dd0 0x7f3518075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.169:39484/0 (socket says 172.21.15.169:39484) 2024-05-09T00:01:48.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.539+0000 7f3517fff700 1 -- 172.21.15.169:0/880940322 learned_addr learned my addr 172.21.15.169:0/880940322 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T00:01:48.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.540+0000 7f3517fff700 1 -- 172.21.15.169:0/880940322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3518077910 con 0x7f3518103dd0 2024-05-09T00:01:48.536 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.540+0000 7f3517fff700 1 --2- 172.21.15.169:0/880940322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3518103dd0 0x7f3518075ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f35000056d0 tx=0x7f35000057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:48.536 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.540+0000 7f35157fa700 1 -- 172.21.15.169:0/880940322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3500026070 con 0x7f3518103dd0 2024-05-09T00:01:48.536 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.540+0000 7f351e80c700 1 -- 172.21.15.169:0/880940322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3518076000 con 0x7f3518103dd0 2024-05-09T00:01:48.536 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.540+0000 7f35157fa700 1 -- 172.21.15.169:0/880940322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350000bba0 con 0x7f3518103dd0 2024-05-09T00:01:48.536 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.540+0000 7f35157fa700 1 -- 172.21.15.169:0/880940322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3500003920 con 0x7f3518103dd0 2024-05-09T00:01:48.536 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.540+0000 7f351e80c700 1 -- 172.21.15.169:0/880940322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35180764c0 con 0x7f3518103dd0 2024-05-09T00:01:48.536 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.541+0000 7f351e80c700 1 -- 172.21.15.169:0/880940322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3518105280 con 0x7f3518103dd0 2024-05-09T00:01:48.537 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.541+0000 7f35157fa700 1 -- 172.21.15.169:0/880940322 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3500029070 con 0x7f3518103dd0 2024-05-09T00:01:48.537 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.542+0000 7f35157fa700 1 --2- 172.21.15.169:0/880940322 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f350403e510 0x7f3504040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:48.537 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.542+0000 7f35157fa700 1 -- 172.21.15.169:0/880940322 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f350001e070 con 0x7f3518103dd0 2024-05-09T00:01:48.537 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.542+0000 7f35177fe700 1 --2- 172.21.15.169:0/880940322 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f350403e510 0x7f3504040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:48.537 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.542+0000 7f35177fe700 1 --2- 172.21.15.169:0/880940322 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f350403e510 0x7f3504040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3508006fd0 tx=0x7f3508006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:48.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.547+0000 7f35157fa700 1 -- 172.21.15.169:0/880940322 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3500013070 con 0x7f3518103dd0 2024-05-09T00:01:48.755 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.760+0000 7f351e80c700 1 -- 172.21.15.169:0/880940322 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi142:172.21.15.142=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}) -- 0x7f3518077060 con 0x7f350403e510 2024-05-09T00:01:48.762 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.768+0000 7f35157fa700 1 -- 172.21.15.169:0/880940322 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3518077060 con 0x7f350403e510 2024-05-09T00:01:48.762 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mon update... 2024-05-09T00:01:48.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.770+0000 7f351e80c700 1 -- 172.21.15.169:0/880940322 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f350403e510 msgr2=0x7f3504040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:48.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.770+0000 7f351e80c700 1 --2- 172.21.15.169:0/880940322 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f350403e510 0x7f3504040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3508006fd0 tx=0x7f3508006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:48.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.770+0000 7f351e80c700 1 -- 172.21.15.169:0/880940322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3518103dd0 msgr2=0x7f3518075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:48.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.770+0000 7f351e80c700 1 --2- 172.21.15.169:0/880940322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3518103dd0 0x7f3518075ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f35000056d0 tx=0x7f35000057b0 comp rx=0 tx=0).stop 2024-05-09T00:01:48.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.770+0000 7f351e80c700 1 -- 172.21.15.169:0/880940322 shutdown_connections 2024-05-09T00:01:48.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.770+0000 7f351e80c700 1 --2- 172.21.15.169:0/880940322 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f350403e510 0x7f3504040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:48.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.770+0000 7f351e80c700 1 --2- 172.21.15.169:0/880940322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3518103dd0 0x7f3518075ac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:48.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.770+0000 7f351e80c700 1 -- 172.21.15.169:0/880940322 >> 172.21.15.169:0/880940322 conn(0x7f35180ff660 msgr2=0x7f3518109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:48.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.770+0000 7f351e80c700 1 -- 172.21.15.169:0/880940322 shutdown_connections 2024-05-09T00:01:48.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:48.770+0000 7f351e80c700 1 -- 172.21.15.169:0/880940322 wait complete. 2024-05-09T00:01:48.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:48 smithi046 ceph-mon[30151]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:01:48.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:48 smithi046 ceph-mon[30151]: Updating smithi169:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T00:01:48.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:48 smithi046 ceph-mon[30151]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:49.446 DEBUG:teuthology.orchestra.run.smithi142:mon.b> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.b.service 2024-05-09T00:01:49.449 DEBUG:teuthology.orchestra.run.smithi169:mon.c> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.c.service 2024-05-09T00:01:49.451 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T00:01:49.452 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T00:01:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:49 smithi046 ceph-mon[30151]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi142:172.21.15.142=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:01:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:49 smithi046 ceph-mon[30151]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi142:172.21.15.142=b;smithi169:172.21.15.169=c;count:3 2024-05-09T00:01:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:01:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:01:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:01:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:01:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:01:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:49 smithi046 ceph-mon[30151]: Deploying daemon mon.c on smithi169 2024-05-09T00:01:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:49 smithi046 ceph-mon[30151]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:50.107 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:01:52.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:52 smithi046 ceph-mon[30151]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:53.177 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.182+0000 7f46a78fc700 1 -- 172.21.15.169:0/1276815509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46a0102440 msgr2=0x7f46a0102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:53.177 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.182+0000 7f46a78fc700 1 --2- 172.21.15.169:0/1276815509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46a0102440 0x7f46a0102850 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f469c009910 tx=0x7f469c009c20 comp rx=0 tx=0).stop 2024-05-09T00:01:53.177 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.182+0000 7f46a78fc700 1 -- 172.21.15.169:0/1276815509 shutdown_connections 2024-05-09T00:01:53.177 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.182+0000 7f46a78fc700 1 --2- 172.21.15.169:0/1276815509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46a0102440 0x7f46a0102850 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.178 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.182+0000 7f46a78fc700 1 -- 172.21.15.169:0/1276815509 >> 172.21.15.169:0/1276815509 conn(0x7f46a00fde30 msgr2=0x7f46a0100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:53.178 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.182+0000 7f46a78fc700 1 -- 172.21.15.169:0/1276815509 shutdown_connections 2024-05-09T00:01:53.178 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.182+0000 7f46a78fc700 1 -- 172.21.15.169:0/1276815509 wait complete. 2024-05-09T00:01:53.180 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.183+0000 7f46a78fc700 1 Processor -- start 2024-05-09T00:01:53.180 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.183+0000 7f46a78fc700 1 -- start start 2024-05-09T00:01:53.180 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.184+0000 7f46a78fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46a0102440 0x7f46a019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.180 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.184+0000 7f46a78fc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46a010c730 con 0x7f46a0102440 2024-05-09T00:01:53.180 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.184+0000 7f46a5698700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46a0102440 0x7f46a019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.180 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.184+0000 7f46a5698700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46a0102440 0x7f46a019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.169:39506/0 (socket says 172.21.15.169:39506) 2024-05-09T00:01:53.180 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.184+0000 7f46a5698700 1 -- 172.21.15.169:0/1133717642 learned_addr learned my addr 172.21.15.169:0/1133717642 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T00:01:53.181 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.184+0000 7f46a5698700 1 -- 172.21.15.169:0/1133717642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a019ad40 con 0x7f46a0102440 2024-05-09T00:01:53.181 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.185+0000 7f46a5698700 1 --2- 172.21.15.169:0/1133717642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46a0102440 0x7f46a019a800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f469c0056d0 tx=0x7f469c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:53.181 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.185+0000 7f46967fc700 1 -- 172.21.15.169:0/1133717642 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f469c026070 con 0x7f46a0102440 2024-05-09T00:01:53.181 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.185+0000 7f46a78fc700 1 -- 172.21.15.169:0/1133717642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a019b030 con 0x7f46a0102440 2024-05-09T00:01:53.181 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.185+0000 7f46967fc700 1 -- 172.21.15.169:0/1133717642 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c00bba0 con 0x7f46a0102440 2024-05-09T00:01:53.181 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.185+0000 7f46967fc700 1 -- 172.21.15.169:0/1133717642 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f469c003920 con 0x7f46a0102440 2024-05-09T00:01:53.181 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.185+0000 7f46a78fc700 1 -- 172.21.15.169:0/1133717642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a019b4f0 con 0x7f46a0102440 2024-05-09T00:01:53.182 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.186+0000 7f46a78fc700 1 -- 172.21.15.169:0/1133717642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a01038f0 con 0x7f46a0102440 2024-05-09T00:01:53.182 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.186+0000 7f46967fc700 1 -- 172.21.15.169:0/1133717642 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f469c029070 con 0x7f46a0102440 2024-05-09T00:01:53.182 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.187+0000 7f46967fc700 1 --2- 172.21.15.169:0/1133717642 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f468c03e510 0x7f468c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:53.182 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.187+0000 7f46967fc700 1 -- 172.21.15.169:0/1133717642 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f469c01e070 con 0x7f46a0102440 2024-05-09T00:01:53.182 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.187+0000 7f46a4e97700 1 --2- 172.21.15.169:0/1133717642 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f468c03e510 0x7f468c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:53.182 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.187+0000 7f46a4e97700 1 --2- 172.21.15.169:0/1133717642 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f468c03e510 0x7f468c040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4690006fd0 tx=0x7f4690006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:53.187 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.193+0000 7f46967fc700 1 -- 172.21.15.169:0/1133717642 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f469c013070 con 0x7f46a0102440 2024-05-09T00:01:53.485 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.491+0000 7f46a78fc700 1 -- 172.21.15.169:0/1133717642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f46a0064e20 con 0x7f46a0102440 2024-05-09T00:01:53.486 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.492+0000 7f46967fc700 1 -- 172.21.15.169:0/1133717642 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f469c030300 con 0x7f46a0102440 2024-05-09T00:01:53.486 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T00:01:53.486 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","modified":"2024-05-09T00:00:11.176337Z","created":"2024-05-09T00:00:11.176337Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T00:01:53.488 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.494+0000 7f46a78fc700 1 -- 172.21.15.169:0/1133717642 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f468c03e510 msgr2=0x7f468c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:53.488 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.494+0000 7f46a78fc700 1 --2- 172.21.15.169:0/1133717642 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f468c03e510 0x7f468c040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4690006fd0 tx=0x7f4690006e40 comp rx=0 tx=0).stop 2024-05-09T00:01:53.488 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.494+0000 7f46a78fc700 1 -- 172.21.15.169:0/1133717642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46a0102440 msgr2=0x7f46a019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:53.488 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.494+0000 7f46a78fc700 1 --2- 172.21.15.169:0/1133717642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46a0102440 0x7f46a019a800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f469c0056d0 tx=0x7f469c0057b0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.494+0000 7f46a78fc700 1 -- 172.21.15.169:0/1133717642 shutdown_connections 2024-05-09T00:01:53.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.494+0000 7f46a78fc700 1 --2- 172.21.15.169:0/1133717642 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f468c03e510 0x7f468c040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.494+0000 7f46a78fc700 1 --2- 172.21.15.169:0/1133717642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46a0102440 0x7f46a019a800 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:53.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.494+0000 7f46a78fc700 1 -- 172.21.15.169:0/1133717642 >> 172.21.15.169:0/1133717642 conn(0x7f46a00fde30 msgr2=0x7f46a010a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:53.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.494+0000 7f46a78fc700 1 -- 172.21.15.169:0/1133717642 shutdown_connections 2024-05-09T00:01:53.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:53.494+0000 7f46a78fc700 1 -- 172.21.15.169:0/1133717642 wait complete. 2024-05-09T00:01:53.489 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-05-09T00:01:55.258 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T00:01:55.258 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T00:01:55.504 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:01:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:01:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:01:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: mon.a calling monitor election 2024-05-09T00:01:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:01:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: mon.c calling monitor election 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: monmap epoch 2 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: fsid eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: last_changed 2024-05-09T00:01:53.570585+0000 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: created 2024-05-09T00:00:11.176337+0000 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: min_mon_release 19 (squid) 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: election_strategy: 1 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T00:01:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-05-09T00:01:59.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: fsmap 2024-05-09T00:01:59.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:01:59.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: mgrmap e13: x(active, since 49s) 2024-05-09T00:01:59.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: overall HEALTH_OK 2024-05-09T00:01:59.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:59.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:59.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:01:59.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:01:59.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:01:59.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:01:58 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:01:59.831 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.836+0000 7fd736134700 1 -- 172.21.15.169:0/3413296782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c0035c0 msgr2=0x7fd71c005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:01:59.832 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.836+0000 7fd736134700 1 --2- 172.21.15.169:0/3413296782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c0035c0 0x7fd71c005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd730103b60 tx=0x7fd72000feb0 comp rx=0 tx=0).stop 2024-05-09T00:01:59.832 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.837+0000 7fd736134700 1 -- 172.21.15.169:0/3413296782 shutdown_connections 2024-05-09T00:01:59.832 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.837+0000 7fd736134700 1 --2- 172.21.15.169:0/3413296782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c0035c0 0x7fd71c005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:59.832 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.837+0000 7fd736134700 1 -- 172.21.15.169:0/3413296782 >> 172.21.15.169:0/3413296782 conn(0x7fd7300ff770 msgr2=0x7fd730101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:01:59.832 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.837+0000 7fd736134700 1 -- 172.21.15.169:0/3413296782 shutdown_connections 2024-05-09T00:01:59.832 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.837+0000 7fd736134700 1 -- 172.21.15.169:0/3413296782 wait complete. 2024-05-09T00:01:59.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.838+0000 7fd736134700 1 Processor -- start 2024-05-09T00:01:59.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.838+0000 7fd736134700 1 -- start start 2024-05-09T00:01:59.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.838+0000 7fd736134700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c0035c0 0x7fd73019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:59.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.838+0000 7fd736134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73019fba0 0x7fd730199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:59.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.838+0000 7fd736134700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd73010bbe0 con 0x7fd73019fba0 2024-05-09T00:01:59.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.838+0000 7fd736134700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd73010ba60 con 0x7fd71c0035c0 2024-05-09T00:01:59.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.839+0000 7fd72f7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c0035c0 0x7fd73019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:59.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.839+0000 7fd726ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73019fba0 0x7fd730199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:59.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.839+0000 7fd72f7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c0035c0 0x7fd73019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57606/0 (socket says 172.21.15.169:57606) 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.839+0000 7fd72f7fe700 1 -- 172.21.15.169:0/2955582299 learned_addr learned my addr 172.21.15.169:0/2955582299 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.839+0000 7fd72f7fe700 1 -- 172.21.15.169:0/2955582299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c0035c0 msgr2=0x7fd73019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.839+0000 7fd72f7fe700 1 -- 172.21.15.169:0/2955582299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c0035c0 msgr2=0x7fd73019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.839+0000 7fd72f7fe700 1 --2- 172.21.15.169:0/2955582299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c0035c0 0x7fd73019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.839+0000 7fd72f7fe700 1 --2- 172.21.15.169:0/2955582299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c0035c0 0x7fd73019f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.839+0000 7fd726ffd700 1 -- 172.21.15.169:0/2955582299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c0035c0 msgr2=0x7fd73019f660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.839+0000 7fd726ffd700 1 --2- 172.21.15.169:0/2955582299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c0035c0 0x7fd73019f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.839+0000 7fd726ffd700 1 -- 172.21.15.169:0/2955582299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7301a0170 con 0x7fd73019fba0 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.839+0000 7fd726ffd700 1 --2- 172.21.15.169:0/2955582299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73019fba0 0x7fd730199880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd71800b4d0 tx=0x7fd71800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.840+0000 7fd72d7fa700 1 -- 172.21.15.169:0/2955582299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd718011630 con 0x7fd73019fba0 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.840+0000 7fd72d7fa700 1 -- 172.21.15.169:0/2955582299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd718011c80 con 0x7fd73019fba0 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.840+0000 7fd72d7fa700 1 -- 172.21.15.169:0/2955582299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd718010da0 con 0x7fd73019fba0 2024-05-09T00:01:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.840+0000 7fd736134700 1 -- 172.21.15.169:0/2955582299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd730199e20 con 0x7fd73019fba0 2024-05-09T00:01:59.836 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.840+0000 7fd736134700 1 -- 172.21.15.169:0/2955582299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73019a3b0 con 0x7fd73019fba0 2024-05-09T00:01:59.836 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.841+0000 7fd736134700 1 -- 172.21.15.169:0/2955582299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73019a6c0 con 0x7fd73019fba0 2024-05-09T00:01:59.836 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.841+0000 7fd72d7fa700 1 -- 172.21.15.169:0/2955582299 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd718018020 con 0x7fd73019fba0 2024-05-09T00:01:59.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.841+0000 7fd72d7fa700 1 --2- 172.21.15.169:0/2955582299 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd71003e300 0x7fd710040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:01:59.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.841+0000 7fd72d7fa700 1 -- 172.21.15.169:0/2955582299 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd7180520f0 con 0x7fd73019fba0 2024-05-09T00:01:59.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.842+0000 7fd72f7fe700 1 --2- 172.21.15.169:0/2955582299 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd71003e300 0x7fd710040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:01:59.837 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.842+0000 7fd72f7fe700 1 --2- 172.21.15.169:0/2955582299 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd71003e300 0x7fd710040810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd72000fa60 tx=0x7fd720005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:01:59.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:01:59.848+0000 7fd72d7fa700 1 -- 172.21.15.169:0/2955582299 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd71801f020 con 0x7fd73019fba0 2024-05-09T00:02:00.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:59 smithi046 ceph-mon[30151]: Deploying daemon mon.b on smithi142 2024-05-09T00:02:00.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:01:59 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:02:00.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:01:59 smithi169 ceph-mon[34777]: Deploying daemon mon.b on smithi142 2024-05-09T00:02:00.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:01:59 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:02:00.133 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.139+0000 7fd736134700 1 -- 172.21.15.169:0/2955582299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd73002e5f0 con 0x7fd73019fba0 2024-05-09T00:02:00.133 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.139+0000 7fd72d7fa700 1 -- 172.21.15.169:0/2955582299 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fd7180187b0 con 0x7fd73019fba0 2024-05-09T00:02:00.134 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T00:02:00.134 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":2,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","modified":"2024-05-09T00:01:53.570585Z","created":"2024-05-09T00:00:11.176337Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T00:02:00.137 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.142+0000 7fd736134700 1 -- 172.21.15.169:0/2955582299 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd71003e300 msgr2=0x7fd710040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:00.137 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.142+0000 7fd736134700 1 --2- 172.21.15.169:0/2955582299 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd71003e300 0x7fd710040810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd72000fa60 tx=0x7fd720005960 comp rx=0 tx=0).stop 2024-05-09T00:02:00.137 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.142+0000 7fd736134700 1 -- 172.21.15.169:0/2955582299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73019fba0 msgr2=0x7fd730199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:00.137 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.142+0000 7fd736134700 1 --2- 172.21.15.169:0/2955582299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73019fba0 0x7fd730199880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd71800b4d0 tx=0x7fd71800b890 comp rx=0 tx=0).stop 2024-05-09T00:02:00.137 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.142+0000 7fd736134700 1 -- 172.21.15.169:0/2955582299 shutdown_connections 2024-05-09T00:02:00.137 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.142+0000 7fd736134700 1 --2- 172.21.15.169:0/2955582299 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd71003e300 0x7fd710040810 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.137 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.142+0000 7fd736134700 1 --2- 172.21.15.169:0/2955582299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c0035c0 0x7fd73019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.137 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.142+0000 7fd736134700 1 --2- 172.21.15.169:0/2955582299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73019fba0 0x7fd730199880 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:00.137 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.142+0000 7fd736134700 1 -- 172.21.15.169:0/2955582299 >> 172.21.15.169:0/2955582299 conn(0x7fd7300ff770 msgr2=0x7fd730107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:00.137 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.142+0000 7fd736134700 1 -- 172.21.15.169:0/2955582299 shutdown_connections 2024-05-09T00:02:00.137 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:00.142+0000 7fd736134700 1 -- 172.21.15.169:0/2955582299 wait complete. 2024-05-09T00:02:00.138 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 2 2024-05-09T00:02:00.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:00 smithi169 ceph-mon[34777]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:00.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:00 smithi169 ceph-mon[34777]: from='client.? 172.21.15.169:0/2955582299' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T00:02:01.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:00 smithi046 ceph-mon[30151]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:01.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:00 smithi046 ceph-mon[30151]: from='client.? 172.21.15.169:0/2955582299' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T00:02:01.577 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T00:02:01.578 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T00:02:01.821 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:02:02.233 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(???) e0 preinit fsid eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:02:02.233 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).mds e1 new map 2024-05-09T00:02:02.233 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-09T00:02:02.233 INFO:journalctl@ceph.mon.b.smithi142.stdout: e1 2024-05-09T00:02:02.233 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T00:02:02.234 INFO:journalctl@ceph.mon.b.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T00:02:02.234 INFO:journalctl@ceph.mon.b.smithi142.stdout: legacy client fscid: -1 2024-05-09T00:02:02.234 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-05-09T00:02:02.234 INFO:journalctl@ceph.mon.b.smithi142.stdout: No filesystems configured 2024-05-09T00:02:02.234 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T00:02:02.234 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T00:02:02.528 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T00:02:02.528 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T00:02:02.528 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T00:02:02.528 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T00:02:02.528 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T00:02:02.528 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:02:02.529 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:02.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:02.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Deploying cephadm binary to smithi169 2024-05-09T00:02:02.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Updating smithi142:/etc/ceph/ceph.conf 2024-05-09T00:02:02.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Updating smithi142:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:02:02.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:02:02.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Updating smithi142:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T00:02:02.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.530 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.531 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:02.531 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.531 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Added host smithi169 2024-05-09T00:02:02.531 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.531 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.531 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.531 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:02:02.531 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3842749664' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T00:02:02.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:02:02.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:02.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:02.533 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Updating smithi169:/etc/ceph/ceph.conf 2024-05-09T00:02:02.533 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3842749664' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T00:02:02.533 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:02:02.533 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Updating smithi169:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:02:02.533 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.533 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.533 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.533 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:02:02.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Updating smithi169:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T00:02:02.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi142:172.21.15.142=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:02:02.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi142:172.21.15.142=b;smithi169:172.21.15.169=c;count:3 2024-05-09T00:02:02.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:02.534 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:02.535 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:02.535 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.535 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Deploying daemon mon.c on smithi169 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.a calling monitor election 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.c calling monitor election 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:02:02.536 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: monmap epoch 2 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: fsid eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: last_changed 2024-05-09T00:01:53.570585+0000 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: created 2024-05-09T00:00:11.176337+0000 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: min_mon_release 19 (squid) 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: election_strategy: 1 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: fsmap 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:02:02.537 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mgrmap e13: x(active, since 49s) 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: overall HEALTH_OK 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: Deploying daemon mon.b on smithi142 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: from='client.? 172.21.15.169:0/2955582299' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 ceph-mon[34533]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 bash[34513]: 4de2356ef3de9b2893e65068ec6cd5264615116fbcae8161e9b2e4a8769f1c2b 2024-05-09T00:02:02.538 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:02 smithi142 systemd[1]: Started Ceph mon.b for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:02:07.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.593+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/4130144191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cfc0035c0 msgr2=0x7f6cfc005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:07.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.593+0000 7f6d1a0f5700 1 --2- 172.21.15.169:0/4130144191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cfc0035c0 0x7f6cfc005aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6d14103b70 tx=0x7f6d0800fe60 comp rx=0 tx=0).stop 2024-05-09T00:02:07.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.593+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/4130144191 shutdown_connections 2024-05-09T00:02:07.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.593+0000 7f6d1a0f5700 1 --2- 172.21.15.169:0/4130144191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cfc0035c0 0x7f6cfc005aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:07.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.593+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/4130144191 >> 172.21.15.169:0/4130144191 conn(0x7f6d140ff7a0 msgr2=0x7f6d14101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:07.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.593+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/4130144191 shutdown_connections 2024-05-09T00:02:07.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.593+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/4130144191 wait complete. 2024-05-09T00:02:07.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:02:07.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: mon.a calling monitor election 2024-05-09T00:02:07.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:02:07.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: mon.c calling monitor election 2024-05-09T00:02:07.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:07.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: mon.b calling monitor election 2024-05-09T00:02:07.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:07.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: monmap epoch 3 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: fsid eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: last_changed 2024-05-09T00:02:02.246265+0000 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: created 2024-05-09T00:00:11.176337+0000 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: min_mon_release 19 (squid) 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: election_strategy: 1 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: 2: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: fsmap 2024-05-09T00:02:07.590 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:02:07.591 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: mgrmap e13: x(active, since 58s) 2024-05-09T00:02:07.591 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: overall HEALTH_OK 2024-05-09T00:02:07.591 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:07.591 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:07.591 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:07.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.594+0000 7f6d1a0f5700 1 Processor -- start 2024-05-09T00:02:07.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.594+0000 7f6d1a0f5700 1 -- start start 2024-05-09T00:02:07.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d1a0f5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cfc0035c0 0x7f6d1419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:07.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d1a0f5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d1419fde0 0x7f6d14199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d1a0f5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d1419a020 0x7f6d1419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d1a0f5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1410bba0 con 0x7f6d1419fde0 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d1a0f5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1410ba20 con 0x7f6d1419a020 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d1a0f5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1410bd20 con 0x7f6cfc0035c0 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d137fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cfc0035c0 0x7f6d1419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d137fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cfc0035c0 0x7f6d1419f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37788/0 (socket says 172.21.15.169:37788) 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d137fe700 1 -- 172.21.15.169:0/1901991668 learned_addr learned my addr 172.21.15.169:0/1901991668 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d12ffd700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d1419fde0 0x7f6d14199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d13fff700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d1419a020 0x7f6d1419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d137fe700 1 -- 172.21.15.169:0/1901991668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cfc0035c0 msgr2=0x7f6d1419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d137fe700 1 -- 172.21.15.169:0/1901991668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cfc0035c0 msgr2=0x7f6d1419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d137fe700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cfc0035c0 0x7f6d1419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T00:02:07.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d137fe700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cfc0035c0 0x7f6d1419f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d13fff700 1 -- 172.21.15.169:0/1901991668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d1419a020 msgr2=0x7f6d1419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d13fff700 1 -- 172.21.15.169:0/1901991668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d1419a020 msgr2=0x7f6d1419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d13fff700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d1419a020 0x7f6d1419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.595+0000 7f6d13fff700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d1419a020 0x7f6d1419a430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.596+0000 7f6d12ffd700 1 -- 172.21.15.169:0/1901991668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cfc0035c0 msgr2=0x7f6d1419f8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.596+0000 7f6d12ffd700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cfc0035c0 0x7f6d1419f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.596+0000 7f6d12ffd700 1 -- 172.21.15.169:0/1901991668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d1419a020 msgr2=0x7f6d1419a430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.596+0000 7f6d12ffd700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d1419a020 0x7f6d1419a430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.596+0000 7f6d12ffd700 1 -- 172.21.15.169:0/1901991668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d141a6750 con 0x7f6d1419fde0 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.596+0000 7f6d12ffd700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d1419fde0 0x7f6d14199ae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400b550 tx=0x7f6d0400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.596+0000 7f6d10ff9700 1 -- 172.21.15.169:0/1901991668 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04004230 con 0x7f6d1419fde0 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.596+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/1901991668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d14075a50 con 0x7f6d1419fde0 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.597+0000 7f6d10ff9700 1 -- 172.21.15.169:0/1901991668 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d040043d0 con 0x7f6d1419fde0 2024-05-09T00:02:07.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.597+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/1901991668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d14075fe0 con 0x7f6d1419fde0 2024-05-09T00:02:07.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.597+0000 7f6d10ff9700 1 -- 172.21.15.169:0/1901991668 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04011b90 con 0x7f6d1419fde0 2024-05-09T00:02:07.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.598+0000 7f6d10ff9700 1 -- 172.21.15.169:0/1901991668 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6d0401a400 con 0x7f6d1419fde0 2024-05-09T00:02:07.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.598+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/1901991668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cfc002140 con 0x7f6d1419fde0 2024-05-09T00:02:07.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.598+0000 7f6d10ff9700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d0003e9a0 0x7f6d00040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:07.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.598+0000 7f6d10ff9700 1 -- 172.21.15.169:0/1901991668 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6d04052250 con 0x7f6d1419fde0 2024-05-09T00:02:07.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.598+0000 7f6d137fe700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d0003e9a0 0x7f6d00040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:07.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.599+0000 7f6d137fe700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d0003e9a0 0x7f6d00040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800f540 tx=0x7f6d0800f890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:07.598 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.604+0000 7f6d10ff9700 1 -- 172.21.15.169:0/1901991668 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d0401a890 con 0x7f6d1419fde0 2024-05-09T00:02:07.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: mon.a calling monitor election 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: mon.c calling monitor election 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: mon.b calling monitor election 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T00:02:07.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: monmap epoch 3 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: fsid eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: last_changed 2024-05-09T00:02:02.246265+0000 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: created 2024-05-09T00:00:11.176337+0000 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: min_mon_release 19 (squid) 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: election_strategy: 1 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: 2: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: fsmap 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: mgrmap e13: x(active, since 58s) 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: overall HEALTH_OK 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:07.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:07.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:02:07.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: mon.a calling monitor election 2024-05-09T00:02:07.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:02:07.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: mon.c calling monitor election 2024-05-09T00:02:07.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:07.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: mon.b calling monitor election 2024-05-09T00:02:07.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:07.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: monmap epoch 3 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: fsid eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: last_changed 2024-05-09T00:02:02.246265+0000 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: created 2024-05-09T00:00:11.176337+0000 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: min_mon_release 19 (squid) 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: election_strategy: 1 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: 2: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: fsmap 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T00:02:07.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: mgrmap e13: x(active, since 58s) 2024-05-09T00:02:07.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: overall HEALTH_OK 2024-05-09T00:02:07.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:07.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:07.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:07.900 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.905+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/1901991668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6cfc002c30 con 0x7f6d1419fde0 2024-05-09T00:02:07.900 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.906+0000 7f6d10ff9700 1 -- 172.21.15.169:0/1901991668 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f6d04017d50 con 0x7f6d1419fde0 2024-05-09T00:02:07.901 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-09T00:02:07.901 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":3,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","modified":"2024-05-09T00:02:02.246265Z","created":"2024-05-09T00:00:11.176337Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-09T00:02:07.904 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.910+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/1901991668 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d0003e9a0 msgr2=0x7f6d00040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:07.904 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.910+0000 7f6d1a0f5700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d0003e9a0 0x7f6d00040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800f540 tx=0x7f6d0800f890 comp rx=0 tx=0).stop 2024-05-09T00:02:07.904 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.910+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/1901991668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d1419fde0 msgr2=0x7f6d14199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:07.904 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.910+0000 7f6d1a0f5700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d1419fde0 0x7f6d14199ae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400b550 tx=0x7f6d0400b910 comp rx=0 tx=0).stop 2024-05-09T00:02:07.904 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.910+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/1901991668 shutdown_connections 2024-05-09T00:02:07.905 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.910+0000 7f6d1a0f5700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d0003e9a0 0x7f6d00040eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:07.905 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.910+0000 7f6d1a0f5700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d1419a020 0x7f6d1419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:07.905 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.910+0000 7f6d1a0f5700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cfc0035c0 0x7f6d1419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:07.905 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.910+0000 7f6d1a0f5700 1 --2- 172.21.15.169:0/1901991668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d1419fde0 0x7f6d14199ae0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:07.905 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.910+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/1901991668 >> 172.21.15.169:0/1901991668 conn(0x7f6d140ff7a0 msgr2=0x7f6d141074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:07.905 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.910+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/1901991668 shutdown_connections 2024-05-09T00:02:07.905 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:07.910+0000 7f6d1a0f5700 1 -- 172.21.15.169:0/1901991668 wait complete. 2024-05-09T00:02:07.905 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 3 2024-05-09T00:02:08.510 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:08 smithi169 ceph-mon[34777]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:08.510 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:08.510 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:08.510 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:08 smithi169 ceph-mon[34777]: from='client.? 172.21.15.169:0/1901991668' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T00:02:08.510 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:08.538 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-09T00:02:08.539 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-09T00:02:08.567 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:08 smithi046 ceph-mon[30151]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:08.567 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:08.567 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:08.568 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:08 smithi046 ceph-mon[30151]: from='client.? 172.21.15.169:0/1901991668' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T00:02:08.568 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:08.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:08 smithi142 ceph-mon[34533]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:08.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:08.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:08.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:08 smithi142 ceph-mon[34533]: from='client.? 172.21.15.169:0/1901991668' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T00:02:08.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:02:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:02:09.556 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:02:09 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:02:09.245+0000 7f17741f1700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-09T00:02:09.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:09.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:09.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:09.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:09.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:09.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:09.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:09.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:09.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-09T00:02:09.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: Updating smithi142:/etc/ceph/ceph.conf 2024-05-09T00:02:09.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: Updating smithi169:/etc/ceph/ceph.conf 2024-05-09T00:02:09.890 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: Updating smithi046:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:02:09.891 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: Updating smithi142:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:02:09.891 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:09 smithi046 ceph-mon[30151]: Updating smithi169:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:02:10.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:10.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:10.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:10.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:10.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:10.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:10.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:10.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:10.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-09T00:02:10.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: Updating smithi142:/etc/ceph/ceph.conf 2024-05-09T00:02:10.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: Updating smithi169:/etc/ceph/ceph.conf 2024-05-09T00:02:10.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: Updating smithi046:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:02:10.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: Updating smithi142:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:02:10.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:09 smithi169 ceph-mon[34777]: Updating smithi169:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:02:10.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:10.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-09T00:02:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: Updating smithi142:/etc/ceph/ceph.conf 2024-05-09T00:02:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: Updating smithi169:/etc/ceph/ceph.conf 2024-05-09T00:02:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: Updating smithi046:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:02:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: Updating smithi142:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:02:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:09 smithi142 ceph-mon[34533]: Updating smithi169:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T00:02:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.425+0000 7ff4a827c700 1 -- 172.21.15.46:0/427025507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0108a90 msgr2=0x7ff4a0108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.425+0000 7ff4a827c700 1 --2- 172.21.15.46:0/427025507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0108a90 0x7ff4a0108ea0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009910 tx=0x7ff49c009c20 comp rx=0 tx=0).stop 2024-05-09T00:02:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.425+0000 7ff4a827c700 1 -- 172.21.15.46:0/427025507 shutdown_connections 2024-05-09T00:02:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.425+0000 7ff4a827c700 1 --2- 172.21.15.46:0/427025507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a0108a90 0x7ff4a0108ea0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.425+0000 7ff4a827c700 1 -- 172.21.15.46:0/427025507 >> 172.21.15.46:0/427025507 conn(0x7ff4a00785b0 msgr2=0x7ff4a0078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.425+0000 7ff4a827c700 1 -- 172.21.15.46:0/427025507 shutdown_connections 2024-05-09T00:02:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.425+0000 7ff4a827c700 1 -- 172.21.15.46:0/427025507 wait complete. 2024-05-09T00:02:10.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.426+0000 7ff4a827c700 1 Processor -- start 2024-05-09T00:02:10.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.426+0000 7ff4a827c700 1 -- start start 2024-05-09T00:02:10.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.426+0000 7ff4a827c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0108a90 0x7ff4a00ff920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:10.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.426+0000 7ff4a827c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a00ffe60 0x7ff4a0100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:10.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.426+0000 7ff4a827c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a01a5e20 0x7ff4a01a8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.426+0000 7ff4a827c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a010cfe0 con 0x7ff4a00ffe60 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.426+0000 7ff4a827c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a010ce60 con 0x7ff4a0108a90 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.426+0000 7ff4a827c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a010d160 con 0x7ff4a01a5e20 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.426+0000 7ff4a5817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a00ffe60 0x7ff4a0100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.426+0000 7ff4a5817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a00ffe60 0x7ff4a0100270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50190/0 (socket says 172.21.15.46:50190) 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.426+0000 7ff4a5817700 1 -- 172.21.15.46:0/3751791936 learned_addr learned my addr 172.21.15.46:0/3751791936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.426+0000 7ff4a6018700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0108a90 0x7ff4a00ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.427+0000 7ff4a6819700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a01a5e20 0x7ff4a01a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.427+0000 7ff4a5817700 1 -- 172.21.15.46:0/3751791936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a01a5e20 msgr2=0x7ff4a01a8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.427+0000 7ff4a5817700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a01a5e20 0x7ff4a01a8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.427+0000 7ff4a5817700 1 -- 172.21.15.46:0/3751791936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0108a90 msgr2=0x7ff4a00ff920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.427+0000 7ff4a5817700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0108a90 0x7ff4a00ff920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:10.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.427+0000 7ff4a5817700 1 -- 172.21.15.46:0/3751791936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a01a88c0 con 0x7ff4a00ffe60 2024-05-09T00:02:10.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.427+0000 7ff4a5817700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a00ffe60 0x7ff4a0100270 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff49000b780 tx=0x7ff49000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:10.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.427+0000 7ff4977fe700 1 -- 172.21.15.46:0/3751791936 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490004230 con 0x7ff4a00ffe60 2024-05-09T00:02:10.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.427+0000 7ff4a827c700 1 -- 172.21.15.46:0/3751791936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a01a8bb0 con 0x7ff4a00ffe60 2024-05-09T00:02:10.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.427+0000 7ff4977fe700 1 -- 172.21.15.46:0/3751791936 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4900043d0 con 0x7ff4a00ffe60 2024-05-09T00:02:10.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.427+0000 7ff4977fe700 1 -- 172.21.15.46:0/3751791936 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490011b20 con 0x7ff4a00ffe60 2024-05-09T00:02:10.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.427+0000 7ff4a827c700 1 -- 172.21.15.46:0/3751791936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a01a90f0 con 0x7ff4a00ffe60 2024-05-09T00:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.428+0000 7ff4977fe700 1 -- 172.21.15.46:0/3751791936 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff490011cc0 con 0x7ff4a00ffe60 2024-05-09T00:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.428+0000 7ff4977fe700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff48c03e9a0 0x7ff48c040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.428+0000 7ff4977fe700 1 -- 172.21.15.46:0/3751791936 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff490052300 con 0x7ff4a00ffe60 2024-05-09T00:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.429+0000 7ff4a6018700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff48c03e9a0 0x7ff48c040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.429+0000 7ff4a827c700 1 -- 172.21.15.46:0/3751791936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a0064e20 con 0x7ff4a00ffe60 2024-05-09T00:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.429+0000 7ff4a6018700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff48c03e9a0 0x7ff48c040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009750 tx=0x7ff49c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.434+0000 7ff4977fe700 1 -- 172.21.15.46:0/3751791936 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff490020740 con 0x7ff4a00ffe60 2024-05-09T00:02:10.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.674+0000 7ff4a827c700 1 -- 172.21.15.46:0/3751791936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff4a01a9320 con 0x7ff4a00ffe60 2024-05-09T00:02:10.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.675+0000 7ff4977fe700 1 -- 172.21.15.46:0/3751791936 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7ff4900151c0 con 0x7ff4a00ffe60 2024-05-09T00:02:10.669 INFO:teuthology.orchestra.run.smithi046.stdout:# minimal ceph.conf for eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:02:10.669 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-05-09T00:02:10.669 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:02:10.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon_host = [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] 2024-05-09T00:02:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.677+0000 7ff4a827c700 1 -- 172.21.15.46:0/3751791936 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff48c03e9a0 msgr2=0x7ff48c040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.677+0000 7ff4a827c700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff48c03e9a0 0x7ff48c040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009750 tx=0x7ff49c01a040 comp rx=0 tx=0).stop 2024-05-09T00:02:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.677+0000 7ff4a827c700 1 -- 172.21.15.46:0/3751791936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a00ffe60 msgr2=0x7ff4a0100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.677+0000 7ff4a827c700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a00ffe60 0x7ff4a0100270 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff49000b780 tx=0x7ff49000ba90 comp rx=0 tx=0).stop 2024-05-09T00:02:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.677+0000 7ff4a827c700 1 -- 172.21.15.46:0/3751791936 shutdown_connections 2024-05-09T00:02:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.678+0000 7ff4a827c700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff48c03e9a0 0x7ff48c040eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.678+0000 7ff4a827c700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a01a5e20 0x7ff4a01a8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.678+0000 7ff4a827c700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0108a90 0x7ff4a00ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.678+0000 7ff4a827c700 1 --2- 172.21.15.46:0/3751791936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a00ffe60 0x7ff4a0100270 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.678+0000 7ff4a827c700 1 -- 172.21.15.46:0/3751791936 >> 172.21.15.46:0/3751791936 conn(0x7ff4a00785b0 msgr2=0x7ff4a0106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.678+0000 7ff4a827c700 1 -- 172.21.15.46:0/3751791936 shutdown_connections 2024-05-09T00:02:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:10.678+0000 7ff4a827c700 1 -- 172.21.15.46:0/3751791936 wait complete. 2024-05-09T00:02:11.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T00:02:11.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:02:11.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:02:11.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:11.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: Reconfiguring daemon mon.a on smithi046 2024-05-09T00:02:11.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:10 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3751791936' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:11.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T00:02:11.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:02:11.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:02:11.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:11.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: Reconfiguring daemon mon.a on smithi046 2024-05-09T00:02:11.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:10 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3751791936' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:11.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: Reconfiguring daemon mon.a on smithi046 2024-05-09T00:02:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:10 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3751791936' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:11.313 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-09T00:02:11.313 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T00:02:11.313 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T00:02:11.384 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T00:02:11.384 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:02:11.451 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-09T00:02:11.451 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T00:02:11.484 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-09T00:02:11.484 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:02:11.552 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T00:02:11.552 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T00:02:11.584 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T00:02:11.584 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T00:02:11.652 INFO:tasks.cephadm:Adding mgr.x on smithi046 2024-05-09T00:02:11.652 INFO:tasks.cephadm:Adding mgr.y on smithi142 2024-05-09T00:02:11.652 INFO:tasks.cephadm:Adding mgr.z on smithi169 2024-05-09T00:02:11.652 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi046=x;smithi142=y;smithi169=z' 2024-05-09T00:02:11.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:11 smithi169 ceph-mon[34777]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:11.929 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:02:12.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:11 smithi046 ceph-mon[30151]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:12.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:11 smithi142 ceph-mon[34533]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:13.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.425+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/221403912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d64103cf0 msgr2=0x7f6d64104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.425+0000 7f6d6b6a1700 1 --2- 172.21.15.169:0/221403912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d64103cf0 0x7f6d64104100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009910 tx=0x7f6d54009c20 comp rx=0 tx=0).stop 2024-05-09T00:02:13.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.425+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/221403912 shutdown_connections 2024-05-09T00:02:13.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.425+0000 7f6d6b6a1700 1 --2- 172.21.15.169:0/221403912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d64103cf0 0x7f6d64104100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.425+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/221403912 >> 172.21.15.169:0/221403912 conn(0x7f6d640ff350 msgr2=0x7f6d641017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:13.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.425+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/221403912 shutdown_connections 2024-05-09T00:02:13.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.425+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/221403912 wait complete. 2024-05-09T00:02:13.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d6b6a1700 1 Processor -- start 2024-05-09T00:02:13.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d6b6a1700 1 -- start start 2024-05-09T00:02:13.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d6b6a1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d64103cf0 0x7f6d64199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d6b6a1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d64199ed0 0x7f6d6419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d6b6a1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6419f3a0 0x7f6d641a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.424 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d6b6a1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64109cb0 con 0x7f6d64103cf0 2024-05-09T00:02:13.424 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d6b6a1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64109b30 con 0x7f6d64199ed0 2024-05-09T00:02:13.424 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d6b6a1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64109e30 con 0x7f6d6419f3a0 2024-05-09T00:02:13.424 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d69c3e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6419f3a0 0x7f6d641a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.424 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d69c3e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6419f3a0 0x7f6d641a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37802/0 (socket says 172.21.15.169:37802) 2024-05-09T00:02:13.424 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d69c3e700 1 -- 172.21.15.169:0/755664276 learned_addr learned my addr 172.21.15.169:0/755664276 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T00:02:13.424 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d68c3c700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d64199ed0 0x7f6d6419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.424 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d6943d700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d64103cf0 0x7f6d64199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.424 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d69c3e700 1 -- 172.21.15.169:0/755664276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d64199ed0 msgr2=0x7f6d6419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d69c3e700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d64199ed0 0x7f6d6419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d69c3e700 1 -- 172.21.15.169:0/755664276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d64103cf0 msgr2=0x7f6d64199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d69c3e700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d64103cf0 0x7f6d64199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.426+0000 7f6d69c3e700 1 -- 172.21.15.169:0/755664276 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d641a1d30 con 0x7f6d6419f3a0 2024-05-09T00:02:13.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.427+0000 7f6d69c3e700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6419f3a0 0x7f6d641a17c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00ba50 tx=0x7f6d5c00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:13.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.427+0000 7f6d5a7fc700 1 -- 172.21.15.169:0/755664276 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c019070 con 0x7f6d6419f3a0 2024-05-09T00:02:13.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.427+0000 7f6d5a7fc700 1 -- 172.21.15.169:0/755664276 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d5c00cde0 con 0x7f6d6419f3a0 2024-05-09T00:02:13.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.427+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/755664276 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d641a1fc0 con 0x7f6d6419f3a0 2024-05-09T00:02:13.425 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.427+0000 7f6d5a7fc700 1 -- 172.21.15.169:0/755664276 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c007350 con 0x7f6d6419f3a0 2024-05-09T00:02:13.426 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.427+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/755664276 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d641a24f0 con 0x7f6d6419f3a0 2024-05-09T00:02:13.426 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.428+0000 7f6d5a7fc700 1 -- 172.21.15.169:0/755664276 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6d5c01c070 con 0x7f6d6419f3a0 2024-05-09T00:02:13.426 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.428+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/755664276 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d64064e20 con 0x7f6d6419f3a0 2024-05-09T00:02:13.426 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.428+0000 7f6d5a7fc700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d5003e9a0 0x7f6d50040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:13.426 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.428+0000 7f6d5a7fc700 1 -- 172.21.15.169:0/755664276 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6d5c051d30 con 0x7f6d6419f3a0 2024-05-09T00:02:13.427 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.428+0000 7f6d6943d700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d5003e9a0 0x7f6d50040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:13.427 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.429+0000 7f6d6943d700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d5003e9a0 0x7f6d50040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006010 tx=0x7f6d5401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:13.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.433+0000 7f6d5a7fc700 1 -- 172.21.15.169:0/755664276 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d5c01e440 con 0x7f6d6419f3a0 2024-05-09T00:02:13.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:13 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:13.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:13 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:13.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:13 smithi046 ceph-mon[30151]: Reconfiguring mon.b (monmap changed)... 2024-05-09T00:02:13.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:13 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:02:13.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:13 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:02:13.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:13 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:13.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:13 smithi046 ceph-mon[30151]: Reconfiguring daemon mon.b on smithi142 2024-05-09T00:02:13.571 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:13 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:13.571 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:13 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:13.571 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:13 smithi142 ceph-mon[34533]: Reconfiguring mon.b (monmap changed)... 2024-05-09T00:02:13.571 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:13 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:02:13.571 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:13 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:02:13.572 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:13 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:13.572 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:13 smithi142 ceph-mon[34533]: Reconfiguring daemon mon.b on smithi142 2024-05-09T00:02:13.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:13 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:13.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:13 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:13.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:13 smithi169 ceph-mon[34777]: Reconfiguring mon.b (monmap changed)... 2024-05-09T00:02:13.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:13 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:02:13.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:13 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:02:13.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:13 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:13.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:13 smithi169 ceph-mon[34777]: Reconfiguring daemon mon.b on smithi142 2024-05-09T00:02:13.642 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.648+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/755664276 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi142=y;smithi169=z", "target": ["mon-mgr", ""]}) -- 0x7f6d641011a0 con 0x7f6d5003e9a0 2024-05-09T00:02:13.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.654+0000 7f6d5a7fc700 1 -- 172.21.15.169:0/755664276 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6d641011a0 con 0x7f6d5003e9a0 2024-05-09T00:02:13.649 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mgr update... 2024-05-09T00:02:13.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.657+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/755664276 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d5003e9a0 msgr2=0x7f6d50040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.657+0000 7f6d6b6a1700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d5003e9a0 0x7f6d50040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006010 tx=0x7f6d5401a040 comp rx=0 tx=0).stop 2024-05-09T00:02:13.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.657+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/755664276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6419f3a0 msgr2=0x7f6d641a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:13.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.657+0000 7f6d6b6a1700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6419f3a0 0x7f6d641a17c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00ba50 tx=0x7f6d5c00bd60 comp rx=0 tx=0).stop 2024-05-09T00:02:13.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.657+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/755664276 shutdown_connections 2024-05-09T00:02:13.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.657+0000 7f6d6b6a1700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d5003e9a0 0x7f6d50040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.657+0000 7f6d6b6a1700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d6419f3a0 0x7f6d641a17c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.657+0000 7f6d6b6a1700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d64103cf0 0x7f6d64199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.657+0000 7f6d6b6a1700 1 --2- 172.21.15.169:0/755664276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d64199ed0 0x7f6d6419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:13.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.657+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/755664276 >> 172.21.15.169:0/755664276 conn(0x7f6d640ff350 msgr2=0x7f6d64100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:13.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.657+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/755664276 shutdown_connections 2024-05-09T00:02:13.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:02:13.657+0000 7f6d6b6a1700 1 -- 172.21.15.169:0/755664276 wait complete. 2024-05-09T00:02:14.246 DEBUG:teuthology.orchestra.run.smithi142:mgr.y> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.y.service 2024-05-09T00:02:14.248 DEBUG:teuthology.orchestra.run.smithi169:mgr.z> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.z.service 2024-05-09T00:02:14.250 INFO:tasks.cephadm:Deploying OSDs... 2024-05-09T00:02:14.251 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T00:02:14.251 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T00:02:14.271 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T00:02:14.272 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-05-09T00:02:14.330 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T00:02:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:02:14.331 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T00:02:14.331 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:02:14.331 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:02:14.331 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-09 00:01:19.643759733 +0000 2024-05-09T00:02:14.331 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 23:56:26.506615751 +0000 2024-05-09T00:02:14.331 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 23:56:26.506615751 +0000 2024-05-09T00:02:14.331 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 23:56:26.506615751 +0000 2024-05-09T00:02:14.331 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T00:02:14.399 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-09T00:02:14.400 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-09T00:02:14.400 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000109921 s, 4.7 MB/s 2024-05-09T00:02:14.401 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T00:02:14.462 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-05-09T00:02:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T00:02:14.519 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:02:14.519 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T00:02:14.519 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:02:14.519 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:02:14.519 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-09 00:01:19.679759374 +0000 2024-05-09T00:02:14.519 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 23:56:26.843612708 +0000 2024-05-09T00:02:14.519 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 23:56:26.843612708 +0000 2024-05-09T00:02:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 23:56:26.843612708 +0000 2024-05-09T00:02:14.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T00:02:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-09T00:02:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-09T00:02:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000115497 s, 4.4 MB/s 2024-05-09T00:02:14.587 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T00:02:14.645 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-05-09T00:02:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T00:02:14.702 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:02:14.702 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T00:02:14.702 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:02:14.702 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:02:14.702 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-09 00:01:19.713759036 +0000 2024-05-09T00:02:14.702 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 23:56:27.181609657 +0000 2024-05-09T00:02:14.702 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 23:56:27.181609657 +0000 2024-05-09T00:02:14.703 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 23:56:27.181609657 +0000 2024-05-09T00:02:14.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T00:02:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-09T00:02:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-09T00:02:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 9.1605e-05 s, 5.6 MB/s 2024-05-09T00:02:14.769 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T00:02:14.827 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-05-09T00:02:14.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:14 smithi169 ceph-mon[34777]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:14.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:14 smithi169 ceph-mon[34777]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi142=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:02:14.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:14 smithi169 ceph-mon[34777]: Saving service mgr spec with placement smithi046=x;smithi142=y;smithi169=z;count:3 2024-05-09T00:02:14.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:14.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:14.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:14.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:02:14.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:02:14.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T00:02:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:02:14.885 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-09T00:02:14.885 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:02:14.885 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:02:14.885 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-09 00:01:19.750758667 +0000 2024-05-09T00:02:14.885 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 23:56:27.512606668 +0000 2024-05-09T00:02:14.885 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 23:56:27.512606668 +0000 2024-05-09T00:02:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 23:56:27.512606668 +0000 2024-05-09T00:02:14.885 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T00:02:14.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:14 smithi142 ceph-mon[34533]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:14.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:14 smithi142 ceph-mon[34533]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi142=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:02:14.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:14 smithi142 ceph-mon[34533]: Saving service mgr spec with placement smithi046=x;smithi142=y;smithi169=z;count:3 2024-05-09T00:02:14.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:14.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:14.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:14.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:02:14.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:02:14.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:14.947 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:14 smithi046 ceph-mon[30151]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:14.947 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:14 smithi046 ceph-mon[30151]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi142=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:02:14.948 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:14 smithi046 ceph-mon[30151]: Saving service mgr spec with placement smithi046=x;smithi142=y;smithi169=z;count:3 2024-05-09T00:02:14.948 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:14.948 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:14.948 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:14.948 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T00:02:14.948 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T00:02:14.948 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-09T00:02:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-09T00:02:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000123858 s, 4.1 MB/s 2024-05-09T00:02:14.951 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T00:02:15.009 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-05-09T00:02:15.009 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T00:02:15.028 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T00:02:15.028 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-05-09T00:02:15.087 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T00:02:15.087 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:02:15.087 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-09T00:02:15.087 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:02:15.087 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:02:15.088 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-05-09 00:02:01.689498097 +0000 2024-05-09T00:02:15.088 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-05-08 23:56:28.918442612 +0000 2024-05-09T00:02:15.088 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-05-08 23:56:28.918442612 +0000 2024-05-09T00:02:15.088 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-05-08 23:56:28.918442612 +0000 2024-05-09T00:02:15.088 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T00:02:15.153 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-05-09T00:02:15.153 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-05-09T00:02:15.153 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000303793 s, 1.7 MB/s 2024-05-09T00:02:15.154 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T00:02:15.211 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-05-09T00:02:15.267 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T00:02:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:02:15.268 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-09T00:02:15.268 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:02:15.268 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:02:15.268 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-05-09 00:02:01.690498079 +0000 2024-05-09T00:02:15.268 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-05-08 23:56:29.279435920 +0000 2024-05-09T00:02:15.268 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-05-08 23:56:29.279435920 +0000 2024-05-09T00:02:15.268 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-05-08 23:56:29.279435920 +0000 2024-05-09T00:02:15.269 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T00:02:15.333 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-05-09T00:02:15.333 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-05-09T00:02:15.333 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000199437 s, 2.6 MB/s 2024-05-09T00:02:15.335 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T00:02:15.391 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-05-09T00:02:15.447 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T00:02:15.447 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:02:15.447 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-09T00:02:15.447 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:02:15.447 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:02:15.447 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-05-09 00:02:01.690498079 +0000 2024-05-09T00:02:15.447 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-05-08 23:56:29.614429711 +0000 2024-05-09T00:02:15.447 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-05-08 23:56:29.614429711 +0000 2024-05-09T00:02:15.447 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-05-08 23:56:29.614429711 +0000 2024-05-09T00:02:15.448 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T00:02:15.511 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-05-09T00:02:15.511 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-05-09T00:02:15.511 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000201047 s, 2.5 MB/s 2024-05-09T00:02:15.512 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T00:02:15.568 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-05-09T00:02:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T00:02:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:02:15.624 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-09T00:02:15.624 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:02:15.624 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:02:15.624 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-05-09 00:02:01.691498061 +0000 2024-05-09T00:02:15.624 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-05-08 23:56:29.939423686 +0000 2024-05-09T00:02:15.624 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-05-08 23:56:29.939423686 +0000 2024-05-09T00:02:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-05-08 23:56:29.939423686 +0000 2024-05-09T00:02:15.624 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T00:02:15.687 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-05-09T00:02:15.687 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-05-09T00:02:15.687 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000225381 s, 2.3 MB/s 2024-05-09T00:02:15.688 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T00:02:15.743 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-09T00:02:15.743 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T00:02:15.761 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T00:02:15.761 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-05-09T00:02:15.817 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T00:02:15.817 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:02:15.817 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T00:02:15.818 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:02:15.818 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:02:15.818 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-05-09 00:01:52.478646816 +0000 2024-05-09T00:02:15.818 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-05-08 23:56:41.889237990 +0000 2024-05-09T00:02:15.818 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-05-08 23:56:41.889237990 +0000 2024-05-09T00:02:15.818 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-05-08 23:56:41.889237990 +0000 2024-05-09T00:02:15.818 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T00:02:15.884 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-05-09T00:02:15.884 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-05-09T00:02:15.884 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000262204 s, 2.0 MB/s 2024-05-09T00:02:15.886 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T00:02:15.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:15 smithi142 ceph-mon[34533]: Reconfiguring mon.c (monmap changed)... 2024-05-09T00:02:15.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:15 smithi142 ceph-mon[34533]: Reconfiguring daemon mon.c on smithi169 2024-05-09T00:02:15.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:15.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:15.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:15.941 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-05-09T00:02:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T00:02:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:02:15.997 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-09T00:02:15.997 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:02:15.997 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:02:15.997 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-05-09 00:01:52.478646816 +0000 2024-05-09T00:02:15.997 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-05-08 23:56:42.213230786 +0000 2024-05-09T00:02:15.997 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-05-08 23:56:42.213230786 +0000 2024-05-09T00:02:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-05-08 23:56:42.213230786 +0000 2024-05-09T00:02:15.998 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T00:02:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:15 smithi046 ceph-mon[30151]: Reconfiguring mon.c (monmap changed)... 2024-05-09T00:02:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:15 smithi046 ceph-mon[30151]: Reconfiguring daemon mon.c on smithi169 2024-05-09T00:02:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:16.057 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:15 smithi169 ceph-mon[34777]: Reconfiguring mon.c (monmap changed)... 2024-05-09T00:02:16.058 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:15 smithi169 ceph-mon[34777]: Reconfiguring daemon mon.c on smithi169 2024-05-09T00:02:16.058 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:16.058 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:16.058 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:16.060 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-05-09T00:02:16.060 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-05-09T00:02:16.060 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000174347 s, 2.9 MB/s 2024-05-09T00:02:16.061 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T00:02:16.117 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-05-09T00:02:16.172 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T00:02:16.172 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:02:16.172 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T00:02:16.172 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:02:16.172 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:02:16.172 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-05-09 00:01:52.478646816 +0000 2024-05-09T00:02:16.172 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-05-08 23:56:42.553223226 +0000 2024-05-09T00:02:16.173 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-05-08 23:56:42.553223226 +0000 2024-05-09T00:02:16.173 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-05-08 23:56:42.553223226 +0000 2024-05-09T00:02:16.173 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T00:02:16.234 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-05-09T00:02:16.234 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-05-09T00:02:16.234 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.00021236 s, 2.4 MB/s 2024-05-09T00:02:16.236 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T00:02:16.292 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-05-09T00:02:16.348 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T00:02:16.348 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T00:02:16.348 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T00:02:16.348 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:02:16.348 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T00:02:16.348 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-05-09 00:01:52.479646794 +0000 2024-05-09T00:02:16.349 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-05-08 23:56:42.890215733 +0000 2024-05-09T00:02:16.349 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-05-08 23:56:42.890215733 +0000 2024-05-09T00:02:16.349 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-05-08 23:56:42.890215733 +0000 2024-05-09T00:02:16.349 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T00:02:16.410 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-05-09T00:02:16.410 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-05-09T00:02:16.410 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000218537 s, 2.3 MB/s 2024-05-09T00:02:16.411 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T00:02:16.467 INFO:tasks.cephadm:Deploying osd.0 on smithi046 with /dev/vg_nvme/lv_4... 2024-05-09T00:02:16.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T00:02:16.702 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:16 smithi169 ceph-mon[34777]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:16.702 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:16.702 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:16.702 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:16.702 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:02:16.702 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T00:02:16.702 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:02:16.702 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:16.722 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:02:16.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:16 smithi046 ceph-mon[30151]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:16.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:16.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:16.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:16.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:02:16.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T00:02:16.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:02:16.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:16.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:16 smithi142 ceph-mon[34533]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:16.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:16.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:16.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:16.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:02:16.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T00:02:16.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:02:16.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:17.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:17 smithi169 ceph-mon[34777]: Deploying daemon mgr.z on smithi169 2024-05-09T00:02:17.850 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:17 smithi169 systemd[1]: Starting Ceph mgr.z for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:02:17.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:17 smithi142 ceph-mon[34533]: Deploying daemon mgr.z on smithi169 2024-05-09T00:02:18.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:17 smithi046 ceph-mon[30151]: Deploying daemon mgr.z on smithi169 2024-05-09T00:02:18.159 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:17 smithi169 podman[36636]: 2024-05-09 00:02:17.89694557 +0000 UTC m=+0.110258831 container create d6dd060bda59e2c576716c94f3e66c403770da198f7a013592f0793a1153275f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T00:02:18.159 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:17 smithi169 podman[36636]: 2024-05-09 00:02:17.804409528 +0000 UTC m=+0.017722790 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:02:18.159 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:18 smithi169 podman[36636]: 2024-05-09 00:02:18.099229805 +0000 UTC m=+0.312543067 container init d6dd060bda59e2c576716c94f3e66c403770da198f7a013592f0793a1153275f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-09T00:02:18.159 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:18 smithi169 podman[36636]: 2024-05-09 00:02:18.101644087 +0000 UTC m=+0.314957341 container start d6dd060bda59e2c576716c94f3e66c403770da198f7a013592f0793a1153275f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:02:18.160 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:18 smithi169 ceph-mgr[36656]: -- 172.21.15.169:0/3062378130 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5569d994e5a0 con 0x5569de6d8480 2024-05-09T00:02:18.160 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:18 smithi169 bash[36636]: d6dd060bda59e2c576716c94f3e66c403770da198f7a013592f0793a1153275f 2024-05-09T00:02:18.444 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:18 smithi169 systemd[1]: Started Ceph mgr.z for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:02:18.444 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:18 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:18.264+0000 7f61358ee200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T00:02:18.444 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:18 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:18.449+0000 7f61358ee200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:02:18.716 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T00:02:18.716 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T00:02:18.716 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-05-09T00:02:18.716 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-05-09T00:02:18.716 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00833246 s, 1.3 GB/s 2024-05-09T00:02:18.716 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-05-09T00:02:18.741 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:18 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:18.551+0000 7f61358ee200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T00:02:18.742 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:18 smithi169 ceph-mon[34777]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:18.742 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:18.742 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:18.742 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:18.742 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:02:18.742 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T00:02:18.742 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:02:18.742 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:18.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:18 smithi142 ceph-mon[34533]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:18.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:18.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:18.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:18.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:02:18.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T00:02:18.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:02:18.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:18 smithi046 ceph-mon[30151]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:02:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T00:02:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:02:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:19.100 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:18 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:18.866+0000 7f61358ee200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T00:02:19.100 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:18 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:18.952+0000 7f61358ee200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T00:02:19.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_4 2024-05-09T00:02:19.714 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:02:19.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:19 smithi046 ceph-mon[30151]: Deploying daemon mgr.y on smithi142 2024-05-09T00:02:19.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:19 smithi142 ceph-mon[34533]: Deploying daemon mgr.y on smithi142 2024-05-09T00:02:20.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:19 smithi169 ceph-mon[34777]: Deploying daemon mgr.y on smithi142 2024-05-09T00:02:20.417 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:20 smithi142 systemd[1]: Starting Ceph mgr.y for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:02:20.532 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:20 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:20.187+0000 7f61358ee200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T00:02:20.532 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:20 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:20.361+0000 7f61358ee200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T00:02:20.532 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:20 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:20.537+0000 7f61358ee200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T00:02:20.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:20 smithi142 ceph-mon[34533]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:20.687 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:20 smithi142 podman[35908]: 2024-05-09 00:02:20.326249745 +0000 UTC m=+0.018074332 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:02:20.688 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:20 smithi142 podman[35908]: 2024-05-09 00:02:20.425445884 +0000 UTC m=+0.117270486 container create 9dcca95c5554f4eedae026a5a0f03679a8d99291bc78e6ed53844c8dd0dd8fc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-09T00:02:20.688 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:20 smithi142 podman[35908]: 2024-05-09 00:02:20.63652563 +0000 UTC m=+0.328350191 container init 9dcca95c5554f4eedae026a5a0f03679a8d99291bc78e6ed53844c8dd0dd8fc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-09T00:02:20.688 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:20 smithi142 podman[35908]: 2024-05-09 00:02:20.639046353 +0000 UTC m=+0.330870916 container start 9dcca95c5554f4eedae026a5a0f03679a8d99291bc78e6ed53844c8dd0dd8fc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T00:02:20.688 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:20 smithi142 ceph-mgr[35927]: -- 172.21.15.142:0/3267476795 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ca2d525a0 con 0x563ca7adc000 2024-05-09T00:02:20.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:20 smithi046 ceph-mon[30151]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:20.822 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:20 smithi169 ceph-mon[34777]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:20.991 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:20 smithi142 bash[35908]: 9dcca95c5554f4eedae026a5a0f03679a8d99291bc78e6ed53844c8dd0dd8fc7 2024-05-09T00:02:20.991 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:20 smithi142 systemd[1]: Started Ceph mgr.y for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:02:20.991 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:20 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:20.810+0000 7fc650a2b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T00:02:20.991 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:20 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:20.997+0000 7fc650a2b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:02:21.100 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:20 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:20.828+0000 7f61358ee200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T00:02:21.100 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:20 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:20.921+0000 7f61358ee200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T00:02:21.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.125+0000 7fe144dad700 1 -- 172.21.15.46:0/3893475370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1401077a0 msgr2=0x7fe140109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:21.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.125+0000 7fe144dad700 1 --2- 172.21.15.46:0/3893475370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1401077a0 0x7fe140109bc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe134009990 tx=0x7fe134009ca0 comp rx=0 tx=0).stop 2024-05-09T00:02:21.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.125+0000 7fe144dad700 1 -- 172.21.15.46:0/3893475370 shutdown_connections 2024-05-09T00:02:21.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.125+0000 7fe144dad700 1 --2- 172.21.15.46:0/3893475370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe14010a100 0x7fe14010c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:21.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.125+0000 7fe144dad700 1 --2- 172.21.15.46:0/3893475370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe140104e40 0x7fe140107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:21.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.125+0000 7fe144dad700 1 --2- 172.21.15.46:0/3893475370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1401077a0 0x7fe140109bc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:21.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.125+0000 7fe144dad700 1 -- 172.21.15.46:0/3893475370 >> 172.21.15.46:0/3893475370 conn(0x7fe1400fe880 msgr2=0x7fe140100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:21.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.125+0000 7fe144dad700 1 -- 172.21.15.46:0/3893475370 shutdown_connections 2024-05-09T00:02:21.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.126+0000 7fe144dad700 1 -- 172.21.15.46:0/3893475370 wait complete. 2024-05-09T00:02:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.126+0000 7fe144dad700 1 Processor -- start 2024-05-09T00:02:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe144dad700 1 -- start start 2024-05-09T00:02:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe144dad700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe140104e40 0x7fe14019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe144dad700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1401077a0 0x7fe14019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe144dad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14010a100 0x7fe1401a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe144dad700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe14010fd90 con 0x7fe14010a100 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe144dad700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe14010fc10 con 0x7fe1401077a0 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe144dad700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe14010ff10 con 0x7fe140104e40 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe13ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14010a100 0x7fe1401a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe13e59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe140104e40 0x7fe14019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe13ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14010a100 0x7fe1401a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36986/0 (socket says 172.21.15.46:36986) 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe13ed9d700 1 -- 172.21.15.46:0/2965826365 learned_addr learned my addr 172.21.15.46:0/2965826365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe13dd9b700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1401077a0 0x7fe14019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe13e59c700 1 -- 172.21.15.46:0/2965826365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1401077a0 msgr2=0x7fe14019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe13e59c700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1401077a0 0x7fe14019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe13e59c700 1 -- 172.21.15.46:0/2965826365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14010a100 msgr2=0x7fe1401a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.127+0000 7fe13e59c700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14010a100 0x7fe1401a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.128+0000 7fe13e59c700 1 -- 172.21.15.46:0/2965826365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1401a2440 con 0x7fe140104e40 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.128+0000 7fe13ed9d700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14010a100 0x7fe1401a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:02:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.128+0000 7fe13e59c700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe140104e40 0x7fe14019a8e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe13000d720 tx=0x7fe13000da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.128+0000 7fe12f7fe700 1 -- 172.21.15.46:0/2965826365 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130014030 con 0x7fe140104e40 2024-05-09T00:02:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.128+0000 7fe144dad700 1 -- 172.21.15.46:0/2965826365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe140102c90 con 0x7fe140104e40 2024-05-09T00:02:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.128+0000 7fe12f7fe700 1 -- 172.21.15.46:0/2965826365 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1300044d0 con 0x7fe140104e40 2024-05-09T00:02:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.128+0000 7fe12f7fe700 1 -- 172.21.15.46:0/2965826365 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130005400 con 0x7fe140104e40 2024-05-09T00:02:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.128+0000 7fe144dad700 1 -- 172.21.15.46:0/2965826365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe140103220 con 0x7fe140104e40 2024-05-09T00:02:21.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.129+0000 7fe12f7fe700 1 -- 172.21.15.46:0/2965826365 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe1300055a0 con 0x7fe140104e40 2024-05-09T00:02:21.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.129+0000 7fe144dad700 1 -- 172.21.15.46:0/2965826365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1400512c0 con 0x7fe140104e40 2024-05-09T00:02:21.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.130+0000 7fe12f7fe700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe12803e750 0x7fe128040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:21.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.130+0000 7fe12f7fe700 1 -- 172.21.15.46:0/2965826365 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe130052530 con 0x7fe140104e40 2024-05-09T00:02:21.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.130+0000 7fe13dd9b700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe12803e750 0x7fe128040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:21.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.130+0000 7fe13dd9b700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe12803e750 0x7fe128040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe13400b5c0 tx=0x7fe13401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.140+0000 7fe12f7fe700 1 -- 172.21.15.46:0/2965826365 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe1300213a0 con 0x7fe140104e40 2024-05-09T00:02:21.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:21.366+0000 7fe144dad700 1 -- 172.21.15.46:0/2965826365 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe140063c00 con 0x7fe12803e750 2024-05-09T00:02:21.416 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:21 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:21.102+0000 7fc650a2b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T00:02:21.416 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:21 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:21.422+0000 7fc650a2b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T00:02:21.600 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:21 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:21.135+0000 7f61358ee200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T00:02:21.600 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:21 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:21.333+0000 7f61358ee200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:02:21.687 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:21 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:21.526+0000 7fc650a2b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T00:02:22.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:22.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:22.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:22.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:22.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:22.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:21 smithi046 ceph-mon[30151]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:22.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:21 smithi046 ceph-mon[30151]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:02:22.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:02:22.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:02:22.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:22.100 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:21 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:21.972+0000 7f61358ee200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T00:02:22.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:22.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:22.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:22.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:22.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:22.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:21 smithi169 ceph-mon[34777]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:22.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:21 smithi169 ceph-mon[34777]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:02:22.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:02:22.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:02:22.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:22.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:22.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:22.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:22.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:22.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:22.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:21 smithi142 ceph-mon[34533]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:22.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:21 smithi142 ceph-mon[34533]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:02:22.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:02:22.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:02:22.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:22.600 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:22.192+0000 7f61358ee200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T00:02:23.092 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:22.630+0000 7f61358ee200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T00:02:23.092 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:22.726+0000 7f61358ee200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T00:02:23.092 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:23 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:23.096+0000 7f61358ee200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T00:02:23.104 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:22 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:22.770+0000 7fc650a2b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T00:02:23.104 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:22 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:22.940+0000 7fc650a2b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T00:02:23.104 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:23 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:23.110+0000 7fc650a2b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T00:02:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:23.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:23.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:23 smithi046 ceph-mon[30151]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T00:02:23.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:02:23.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:02:23.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:23.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:23 smithi046 ceph-mon[30151]: Reconfiguring daemon mgr.x on smithi046 2024-05-09T00:02:23.687 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:23 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:23.397+0000 7fc650a2b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T00:02:23.687 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:23 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:23.489+0000 7fc650a2b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T00:02:23.687 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:23 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:23.693+0000 7fc650a2b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T00:02:23.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:23.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:23.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:23 smithi142 ceph-mon[34533]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T00:02:23.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:02:23.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:02:23.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:23.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:23 smithi142 ceph-mon[34533]: Reconfiguring daemon mgr.x on smithi046 2024-05-09T00:02:23.777 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:23 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:23.448+0000 7f61358ee200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T00:02:23.777 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:23 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:23.541+0000 7f61358ee200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T00:02:23.777 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.777 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.777 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.777 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.777 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:23.777 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:23.777 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:23.777 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:23 smithi169 ceph-mon[34777]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T00:02:23.777 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T00:02:23.778 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T00:02:23.778 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:23.778 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:23 smithi169 ceph-mon[34777]: Reconfiguring daemon mgr.x on smithi046 2024-05-09T00:02:23.937 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:23 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:23.884+0000 7fc650a2b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T00:02:24.100 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:23 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:23.781+0000 7f61358ee200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T00:02:24.100 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:23 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:23.916+0000 7f61358ee200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T00:02:24.101 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:24 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:24.009+0000 7f61358ee200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T00:02:24.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:24 smithi169 ceph-mon[34777]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:24.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:24 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3272267909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "740d6388-5c1f-45eb-93b0-7583bfa5a225"}]: dispatch 2024-05-09T00:02:24.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:24 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3272267909' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "740d6388-5c1f-45eb-93b0-7583bfa5a225"}]': finished 2024-05-09T00:02:24.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:24 smithi169 ceph-mon[34777]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T00:02:24.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:24 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:24.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:24 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:24.601 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:24 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:24.336+0000 7f61358ee200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T00:02:24.601 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:24 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:24.428+0000 7f61358ee200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T00:02:24.687 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:24 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:24.487+0000 7fc650a2b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T00:02:24.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:24 smithi142 ceph-mon[34533]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:24.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:24 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3272267909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "740d6388-5c1f-45eb-93b0-7583bfa5a225"}]: dispatch 2024-05-09T00:02:24.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:24 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3272267909' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "740d6388-5c1f-45eb-93b0-7583bfa5a225"}]': finished 2024-05-09T00:02:24.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:24 smithi142 ceph-mon[34533]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T00:02:24.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:24 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:24.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:24 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:24.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:24 smithi046 ceph-mon[30151]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:24.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:24 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3272267909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "740d6388-5c1f-45eb-93b0-7583bfa5a225"}]: dispatch 2024-05-09T00:02:24.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:24 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3272267909' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "740d6388-5c1f-45eb-93b0-7583bfa5a225"}]': finished 2024-05-09T00:02:24.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:24 smithi046 ceph-mon[30151]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T00:02:24.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:24 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:24.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:24 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:25.124 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:24 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:24.707+0000 7fc650a2b200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T00:02:25.350 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:25.021+0000 7f61358ee200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T00:02:25.350 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:02:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z[36652]: 2024-05-09T00:02:25.236+0000 7f61358ee200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T00:02:25.380 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:25 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:25.130+0000 7fc650a2b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T00:02:25.380 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:25 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:25.228+0000 7fc650a2b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T00:02:25.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:25.390+0000 7fe12f7fe700 1 -- 172.21.15.46:0/2965826365 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7fe130020350 con 0x7fe140104e40 2024-05-09T00:02:25.687 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:25 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:25.598+0000 7fc650a2b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T00:02:25.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:25 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2141363589' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T00:02:25.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:25 smithi142 ceph-mon[34533]: Standby manager daemon z started 2024-05-09T00:02:25.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:25 smithi142 ceph-mon[34533]: from='mgr.? 172.21.15.169:0/97968487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T00:02:25.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:25 smithi142 ceph-mon[34533]: from='mgr.? 172.21.15.169:0/97968487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T00:02:25.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:25 smithi142 ceph-mon[34533]: from='mgr.? 172.21.15.169:0/97968487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T00:02:25.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:25 smithi142 ceph-mon[34533]: from='mgr.? 172.21.15.169:0/97968487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T00:02:25.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:25 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2141363589' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T00:02:25.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:25 smithi046 ceph-mon[30151]: Standby manager daemon z started 2024-05-09T00:02:25.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:25 smithi046 ceph-mon[30151]: from='mgr.? 172.21.15.169:0/97968487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T00:02:25.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:25 smithi046 ceph-mon[30151]: from='mgr.? 172.21.15.169:0/97968487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T00:02:25.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:25 smithi046 ceph-mon[30151]: from='mgr.? 172.21.15.169:0/97968487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T00:02:25.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:25 smithi046 ceph-mon[30151]: from='mgr.? 172.21.15.169:0/97968487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T00:02:25.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:25 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2141363589' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T00:02:25.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:25 smithi169 ceph-mon[34777]: Standby manager daemon z started 2024-05-09T00:02:25.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:25 smithi169 ceph-mon[34777]: from='mgr.? 172.21.15.169:0/97968487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T00:02:25.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:25 smithi169 ceph-mon[34777]: from='mgr.? 172.21.15.169:0/97968487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T00:02:25.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:25 smithi169 ceph-mon[34777]: from='mgr.? 172.21.15.169:0/97968487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T00:02:25.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:25 smithi169 ceph-mon[34777]: from='mgr.? 172.21.15.169:0/97968487' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T00:02:26.273 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:25 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:25.949+0000 7fc650a2b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T00:02:26.273 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:26 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:26.040+0000 7fc650a2b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T00:02:26.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:26 smithi142 ceph-mon[34533]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:26.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:26 smithi142 ceph-mon[34533]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-09T00:02:26.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:26 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T00:02:26.687 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:26 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:26.279+0000 7fc650a2b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T00:02:26.688 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:26 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:26.413+0000 7fc650a2b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T00:02:26.688 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:26 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:26.508+0000 7fc650a2b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T00:02:26.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:26 smithi046 ceph-mon[30151]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:26.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:26 smithi046 ceph-mon[30151]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-09T00:02:26.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:26 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T00:02:26.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:26 smithi169 ceph-mon[34777]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:26.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:26 smithi169 ceph-mon[34777]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-09T00:02:26.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:26 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T00:02:27.187 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:26 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:26.833+0000 7fc650a2b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T00:02:27.187 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:26 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:26.924+0000 7fc650a2b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T00:02:27.937 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:27 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:27.516+0000 7fc650a2b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T00:02:27.937 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:02:27 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y[35923]: 2024-05-09T00:02:27.730+0000 7fc650a2b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T00:02:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:28.402+0000 7fe12f7fe700 1 -- 172.21.15.46:0/2965826365 <== mon.1 v2:172.21.15.169:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fe130065730 con 0x7fe140104e40 2024-05-09T00:02:28.397 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:28 smithi046 ceph-mon[30151]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:28.397 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:28 smithi046 ceph-mon[30151]: Standby manager daemon y started 2024-05-09T00:02:28.397 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:28 smithi046 ceph-mon[30151]: from='mgr.? 172.21.15.142:0/395037287' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T00:02:28.398 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:28 smithi046 ceph-mon[30151]: from='mgr.? 172.21.15.142:0/395037287' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T00:02:28.398 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:28 smithi046 ceph-mon[30151]: from='mgr.? 172.21.15.142:0/395037287' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T00:02:28.398 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:28 smithi046 ceph-mon[30151]: from='mgr.? 172.21.15.142:0/395037287' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T00:02:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:28 smithi142 ceph-mon[34533]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:28 smithi142 ceph-mon[34533]: Standby manager daemon y started 2024-05-09T00:02:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:28 smithi142 ceph-mon[34533]: from='mgr.? 172.21.15.142:0/395037287' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T00:02:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:28 smithi142 ceph-mon[34533]: from='mgr.? 172.21.15.142:0/395037287' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T00:02:28.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:28 smithi142 ceph-mon[34533]: from='mgr.? 172.21.15.142:0/395037287' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T00:02:28.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:28 smithi142 ceph-mon[34533]: from='mgr.? 172.21.15.142:0/395037287' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T00:02:28.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:28 smithi169 ceph-mon[34777]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:28.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:28 smithi169 ceph-mon[34777]: Standby manager daemon y started 2024-05-09T00:02:28.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:28 smithi169 ceph-mon[34777]: from='mgr.? 172.21.15.142:0/395037287' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T00:02:28.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:28 smithi169 ceph-mon[34777]: from='mgr.? 172.21.15.142:0/395037287' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T00:02:28.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:28 smithi169 ceph-mon[34777]: from='mgr.? 172.21.15.142:0/395037287' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T00:02:28.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:28 smithi169 ceph-mon[34777]: from='mgr.? 172.21.15.142:0/395037287' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T00:02:29.428 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:29 smithi046 ceph-mon[30151]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-09T00:02:29.429 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:29 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T00:02:29.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:29 smithi142 ceph-mon[34533]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-09T00:02:29.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:29 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T00:02:29.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:29 smithi169 ceph-mon[34777]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-09T00:02:29.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:29 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T00:02:30.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:30 smithi046 ceph-mon[30151]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:30.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:30 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:30.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:30 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:30.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:30 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:30.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:30 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:30.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:30 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:30.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:30 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:30.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:30 smithi169 ceph-mon[34777]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:30.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:30 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:30.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:30 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:30.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:30 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:30.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:30 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:30.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:30 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:30.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:30 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:30.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:30 smithi142 ceph-mon[34533]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:30.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:30 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:30.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:30 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:30.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:30 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:30.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:30 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:30.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:30 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:30.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:30 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:32.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:32 smithi046 ceph-mon[30151]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:32.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T00:02:32.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:32.750 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:32 smithi142 ceph-mon[34533]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:32.751 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T00:02:32.751 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:32.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:32 smithi169 ceph-mon[34777]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:32.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T00:02:32.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:33.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:33 smithi046 ceph-mon[30151]: Deploying daemon osd.0 on smithi046 2024-05-09T00:02:33.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:33 smithi169 ceph-mon[34777]: Deploying daemon osd.0 on smithi046 2024-05-09T00:02:33.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:33 smithi142 ceph-mon[34533]: Deploying daemon osd.0 on smithi046 2024-05-09T00:02:34.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:34 smithi046 ceph-mon[30151]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:34.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:34 smithi169 ceph-mon[34777]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:34.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:34 smithi142 ceph-mon[34533]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:36.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:36 smithi046 ceph-mon[30151]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:36.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:36 smithi169 ceph-mon[34777]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:36.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:36 smithi142 ceph-mon[34533]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:38.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:38 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:38.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:38 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:38.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:38 smithi046 ceph-mon[30151]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:38.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:38 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:38.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:38 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:38.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:38 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:38.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:38 smithi142 ceph-mon[34533]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:38.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:38 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:38.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:38 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:38.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:38 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:38.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:38 smithi169 ceph-mon[34777]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:38.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:38 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:40.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:40 smithi046 ceph-mon[30151]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:40.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:40 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:40.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:40 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:40.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:40 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:40.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:40 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:40.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:40 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.588+0000 7fe12f7fe700 1 -- 172.21.15.46:0/2965826365 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe140063c00 con 0x7fe12803e750 2024-05-09T00:02:40.583 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 0 on host 'smithi046' 2024-05-09T00:02:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.591+0000 7fe144dad700 1 -- 172.21.15.46:0/2965826365 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe12803e750 msgr2=0x7fe128040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.591+0000 7fe144dad700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe12803e750 0x7fe128040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe13400b5c0 tx=0x7fe13401a040 comp rx=0 tx=0).stop 2024-05-09T00:02:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.591+0000 7fe144dad700 1 -- 172.21.15.46:0/2965826365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe140104e40 msgr2=0x7fe14019a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.591+0000 7fe144dad700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe140104e40 0x7fe14019a8e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe13000d720 tx=0x7fe13000da30 comp rx=0 tx=0).stop 2024-05-09T00:02:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.591+0000 7fe144dad700 1 -- 172.21.15.46:0/2965826365 shutdown_connections 2024-05-09T00:02:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.591+0000 7fe144dad700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe12803e750 0x7fe128040c60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.591+0000 7fe144dad700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14010a100 0x7fe1401a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.591+0000 7fe144dad700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe140104e40 0x7fe14019a8e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.591+0000 7fe144dad700 1 --2- 172.21.15.46:0/2965826365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1401077a0 0x7fe14019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.591+0000 7fe144dad700 1 -- 172.21.15.46:0/2965826365 >> 172.21.15.46:0/2965826365 conn(0x7fe1400fe880 msgr2=0x7fe140105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.591+0000 7fe144dad700 1 -- 172.21.15.46:0/2965826365 shutdown_connections 2024-05-09T00:02:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:02:40.591+0000 7fe144dad700 1 -- 172.21.15.46:0/2965826365 wait complete. 2024-05-09T00:02:40.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:40 smithi169 ceph-mon[34777]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:40.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:40 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:40.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:40 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:40.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:40 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:40.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:40 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:40.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:40 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:40.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:40 smithi142 ceph-mon[34533]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:40.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:40 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:40.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:40 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:40.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:40 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:40.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:40 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:40.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:40 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:41.145 DEBUG:teuthology.orchestra.run.smithi046:osd.0> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.0.service 2024-05-09T00:02:41.148 INFO:tasks.cephadm:Deploying osd.1 on smithi142 with /dev/vg_nvme/lv_4... 2024-05-09T00:02:41.148 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T00:02:41.400 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:02:41.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:41.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:41.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:41.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:41 smithi142 ceph-mon[34533]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T00:02:41.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:41.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:41.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:41.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:41 smithi046 ceph-mon[30151]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T00:02:41.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:41.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:02:41.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:41.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:41 smithi169 ceph-mon[34777]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T00:02:42.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:42 smithi046 ceph-mon[30151]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:42.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:42 smithi046 ceph-mon[30151]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T00:02:42.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:42 smithi046 ceph-mon[30151]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T00:02:42.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:42 smithi046 ceph-mon[30151]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T00:02:42.656 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:42 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:42.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:42 smithi142 ceph-mon[34533]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:42.794 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:42 smithi142 ceph-mon[34533]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T00:02:42.794 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:42 smithi142 ceph-mon[34533]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T00:02:42.794 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:42 smithi142 ceph-mon[34533]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T00:02:42.794 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:42 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:42.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:42 smithi169 ceph-mon[34777]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:42.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:42 smithi169 ceph-mon[34777]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T00:02:42.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:42 smithi169 ceph-mon[34777]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T00:02:42.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:42 smithi169 ceph-mon[34777]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T00:02:42.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:42 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:43.365 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T00:02:43.365 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T00:02:43.366 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-05-09T00:02:43.366 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-05-09T00:02:43.366 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113333 s, 925 MB/s 2024-05-09T00:02:43.366 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-05-09T00:02:43.558 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:02:43 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0[42715]: 2024-05-09T00:02:43.091+0000 7fc3c948c700 -1 osd.0 0 waiting for initial osdmap 2024-05-09T00:02:43.558 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:02:43 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0[42715]: 2024-05-09T00:02:43.095+0000 7fc3c1239700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:02:43.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:43 smithi169 ceph-mon[34777]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T00:02:43.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:43 smithi169 ceph-mon[34777]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T00:02:43.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:43 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:43.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:43 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:43.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:43 smithi169 ceph-mon[34777]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' 2024-05-09T00:02:43.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:43 smithi046 ceph-mon[30151]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T00:02:43.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:43 smithi046 ceph-mon[30151]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T00:02:43.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:43 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:43.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:43 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:43.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:43 smithi046 ceph-mon[30151]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' 2024-05-09T00:02:43.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:43 smithi142 ceph-mon[34533]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T00:02:43.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:43 smithi142 ceph-mon[34533]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T00:02:43.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:43 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:43.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:43 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:43.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:43 smithi142 ceph-mon[34533]: from='osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125]' entity='osd.0' 2024-05-09T00:02:43.973 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2024-05-09T00:02:44.212 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:02:44.621 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: purged_snaps scrub starts 2024-05-09T00:02:44.621 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: purged_snaps scrub ok 2024-05-09T00:02:44.621 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:44.622 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:44.622 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125] boot 2024-05-09T00:02:44.622 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T00:02:44.622 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:44.622 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:44.622 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:44.622 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:02:44.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:44.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:44.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:44.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:44.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: purged_snaps scrub starts 2024-05-09T00:02:44.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: purged_snaps scrub ok 2024-05-09T00:02:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125] boot 2024-05-09T00:02:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T00:02:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:02:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:44.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:45.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T00:02:45.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T00:02:45.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T00:02:45.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:45.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: osd.0 [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125] boot 2024-05-09T00:02:45.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T00:02:45.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T00:02:45.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:45.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:45.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:02:45.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:45.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:45.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:02:45.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:02:45.742 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.747+0000 7f535a2eb700 1 -- 172.21.15.142:0/3614382527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5354104d10 msgr2=0x7f535410b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:45.743 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.747+0000 7f535a2eb700 1 --2- 172.21.15.142:0/3614382527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5354104d10 0x7f535410b5c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f534c00b0a0 tx=0x7f534c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:02:45.743 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.747+0000 7f535a2eb700 1 -- 172.21.15.142:0/3614382527 shutdown_connections 2024-05-09T00:02:45.743 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.747+0000 7f535a2eb700 1 --2- 172.21.15.142:0/3614382527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5354104d10 0x7f535410b5c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:45.743 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.747+0000 7f535a2eb700 1 --2- 172.21.15.142:0/3614382527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5354103240 0x7f5354103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:45.743 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.747+0000 7f535a2eb700 1 --2- 172.21.15.142:0/3614382527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5354104370 0x7f5354104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:45.743 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.747+0000 7f535a2eb700 1 -- 172.21.15.142:0/3614382527 >> 172.21.15.142:0/3614382527 conn(0x7f53540fe880 msgr2=0x7f5354100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:02:45.743 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.747+0000 7f535a2eb700 1 -- 172.21.15.142:0/3614382527 shutdown_connections 2024-05-09T00:02:45.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.747+0000 7f535a2eb700 1 -- 172.21.15.142:0/3614382527 wait complete. 2024-05-09T00:02:45.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.748+0000 7f535a2eb700 1 Processor -- start 2024-05-09T00:02:45.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.748+0000 7f535a2eb700 1 -- start start 2024-05-09T00:02:45.746 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.748+0000 7f535a2eb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5354103240 0x7f535419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:45.746 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.748+0000 7f535a2eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5354104370 0x7f535419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:45.746 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.748+0000 7f535a2eb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5354104d10 0x7f53541a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:45.746 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.748+0000 7f535a2eb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f535410ed70 con 0x7f5354104370 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.748+0000 7f535a2eb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f535410ebf0 con 0x7f5354104d10 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.748+0000 7f535a2eb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f535410eef0 con 0x7f5354103240 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.748+0000 7f5358888700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5354104d10 0x7f53541a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f5358888700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5354104d10 0x7f53541a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:38058/0 (socket says 172.21.15.142:38058) 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f5358888700 1 -- 172.21.15.142:0/1204085773 learned_addr learned my addr 172.21.15.142:0/1204085773 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f5353fff700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5354103240 0x7f535419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f53537fe700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5354104370 0x7f535419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f5358888700 1 -- 172.21.15.142:0/1204085773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5354103240 msgr2=0x7f535419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f5358888700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5354103240 0x7f535419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f5358888700 1 -- 172.21.15.142:0/1204085773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5354104370 msgr2=0x7f535419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f5358888700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5354104370 0x7f535419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f5358888700 1 -- 172.21.15.142:0/1204085773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53541a24b0 con 0x7f5354104d10 2024-05-09T00:02:45.747 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f5358888700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5354104d10 0x7f53541a1e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f534c009fd0 tx=0x7f534c004da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:45.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f53517fa700 1 -- 172.21.15.142:0/1204085773 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c021070 con 0x7f5354104d10 2024-05-09T00:02:45.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f53517fa700 1 -- 172.21.15.142:0/1204085773 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f534c007c00 con 0x7f5354104d10 2024-05-09T00:02:45.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f53517fa700 1 -- 172.21.15.142:0/1204085773 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c005260 con 0x7f5354104d10 2024-05-09T00:02:45.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f535a2eb700 1 -- 172.21.15.142:0/1204085773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53541a2740 con 0x7f5354104d10 2024-05-09T00:02:45.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.749+0000 7f535a2eb700 1 -- 172.21.15.142:0/1204085773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53541a2c30 con 0x7f5354104d10 2024-05-09T00:02:45.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.750+0000 7f535a2eb700 1 -- 172.21.15.142:0/1204085773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53540512c0 con 0x7f5354104d10 2024-05-09T00:02:45.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.752+0000 7f53517fa700 1 -- 172.21.15.142:0/1204085773 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f534c024070 con 0x7f5354104d10 2024-05-09T00:02:45.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.752+0000 7f53517fa700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f533c0b27f0 0x7f533c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:02:45.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.753+0000 7f53517fa700 1 -- 172.21.15.142:0/1204085773 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f534c00f020 con 0x7f5354104d10 2024-05-09T00:02:45.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.753+0000 7f5353fff700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f533c0b27f0 0x7f533c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:02:45.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.753+0000 7f5353fff700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f533c0b27f0 0x7f533c0b4d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5354104270 tx=0x7f5344007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:02:45.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.757+0000 7f53517fa700 1 -- 172.21.15.142:0/1204085773 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f534c02b030 con 0x7f5354104d10 2024-05-09T00:02:45.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:45 smithi169 ceph-mon[34777]: Detected new or changed devices on smithi046 2024-05-09T00:02:45.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:45 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi046 to 17093M 2024-05-09T00:02:45.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:45 smithi169 ceph-mon[34777]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T00:02:45.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:45 smithi142 ceph-mon[34533]: Detected new or changed devices on smithi046 2024-05-09T00:02:45.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:45 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi046 to 17093M 2024-05-09T00:02:45.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:45 smithi142 ceph-mon[34533]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T00:02:45.962 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:02:45.968+0000 7f535a2eb700 1 -- 172.21.15.142:0/1204085773 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5354100830 con 0x7f533c0b27f0 2024-05-09T00:02:46.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:45 smithi046 ceph-mon[30151]: Detected new or changed devices on smithi046 2024-05-09T00:02:46.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:45 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi046 to 17093M 2024-05-09T00:02:46.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:45 smithi046 ceph-mon[30151]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T00:02:46.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:46 smithi169 ceph-mon[34777]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:46.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:46 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:02:46.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:46 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:02:46.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:46 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:46.879 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:46 smithi142 ceph-mon[34533]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:46.880 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:46 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:02:46.880 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:46 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:02:46.880 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:46 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:47.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:46 smithi046 ceph-mon[30151]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:47.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:02:47.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:02:47.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:47.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:47 smithi142 ceph-mon[34533]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:02:47.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:47 smithi169 ceph-mon[34777]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:02:48.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:47 smithi046 ceph-mon[30151]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:02:48.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:48 smithi169 ceph-mon[34777]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:48.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:48 smithi142 ceph-mon[34533]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:49.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:48 smithi046 ceph-mon[30151]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:49.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:49 smithi169 ceph-mon[34777]: from='client.? 172.21.15.142:0/3280648700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85fbf3ae-0be8-4504-84d6-66fae0d2169d"}]: dispatch 2024-05-09T00:02:49.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:49 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85fbf3ae-0be8-4504-84d6-66fae0d2169d"}]: dispatch 2024-05-09T00:02:49.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:49 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85fbf3ae-0be8-4504-84d6-66fae0d2169d"}]': finished 2024-05-09T00:02:49.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:49 smithi169 ceph-mon[34777]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T00:02:49.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:02:49.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:49 smithi169 ceph-mon[34777]: from='client.? 172.21.15.142:0/4118545775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T00:02:49.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:49 smithi142 ceph-mon[34533]: from='client.? 172.21.15.142:0/3280648700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85fbf3ae-0be8-4504-84d6-66fae0d2169d"}]: dispatch 2024-05-09T00:02:49.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:49 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85fbf3ae-0be8-4504-84d6-66fae0d2169d"}]: dispatch 2024-05-09T00:02:49.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:49 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85fbf3ae-0be8-4504-84d6-66fae0d2169d"}]': finished 2024-05-09T00:02:49.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:49 smithi142 ceph-mon[34533]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T00:02:49.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:02:49.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:49 smithi142 ceph-mon[34533]: from='client.? 172.21.15.142:0/4118545775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T00:02:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:49 smithi046 ceph-mon[30151]: from='client.? 172.21.15.142:0/3280648700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85fbf3ae-0be8-4504-84d6-66fae0d2169d"}]: dispatch 2024-05-09T00:02:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:49 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85fbf3ae-0be8-4504-84d6-66fae0d2169d"}]: dispatch 2024-05-09T00:02:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:49 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85fbf3ae-0be8-4504-84d6-66fae0d2169d"}]': finished 2024-05-09T00:02:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:49 smithi046 ceph-mon[30151]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T00:02:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:02:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:49 smithi046 ceph-mon[30151]: from='client.? 172.21.15.142:0/4118545775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T00:02:50.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:50 smithi169 ceph-mon[34777]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:50.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:50 smithi142 ceph-mon[34533]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:51.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:50 smithi046 ceph-mon[30151]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:52.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:52 smithi142 ceph-mon[34533]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:52.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:52 smithi169 ceph-mon[34777]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:53.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:52 smithi046 ceph-mon[30151]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:54.715 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:54 smithi142 ceph-mon[34533]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:55.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:54 smithi046 ceph-mon[30151]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:55.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:54 smithi169 ceph-mon[34777]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:56.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:56 smithi142 ceph-mon[34533]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:56.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T00:02:56.673 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:57.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:56 smithi046 ceph-mon[30151]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:57.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T00:02:57.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:57.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:56 smithi169 ceph-mon[34777]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:57.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T00:02:57.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:02:57.770 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:57 smithi142 ceph-mon[34533]: Deploying daemon osd.1 on smithi142 2024-05-09T00:02:58.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:57 smithi046 ceph-mon[30151]: Deploying daemon osd.1 on smithi142 2024-05-09T00:02:58.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:57 smithi169 ceph-mon[34777]: Deploying daemon osd.1 on smithi142 2024-05-09T00:02:58.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:02:58 smithi142 ceph-mon[34533]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:59.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:02:58 smithi046 ceph-mon[30151]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:02:59.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:02:58 smithi169 ceph-mon[34777]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:00.921 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:00 smithi142 ceph-mon[34533]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:01.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:00 smithi046 ceph-mon[30151]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:01.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:00 smithi169 ceph-mon[34777]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:02.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:02.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:02.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:02.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:02 smithi142 ceph-mon[34533]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:02.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:02.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:02.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:02.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:02 smithi046 ceph-mon[30151]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:02.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:02.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:02.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:02.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:02 smithi169 ceph-mon[34777]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:04.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:04 smithi046 ceph-mon[30151]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:04.583 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:04 smithi142 ceph-mon[34533]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:04.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:04 smithi169 ceph-mon[34777]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:04.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.692+0000 7f53517fa700 1 -- 172.21.15.142:0/1204085773 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5354100830 con 0x7f533c0b27f0 2024-05-09T00:03:04.687 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 1 on host 'smithi142' 2024-05-09T00:03:04.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.695+0000 7f535a2eb700 1 -- 172.21.15.142:0/1204085773 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f533c0b27f0 msgr2=0x7f533c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:04.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.695+0000 7f535a2eb700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f533c0b27f0 0x7f533c0b4d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5354104270 tx=0x7f5344007400 comp rx=0 tx=0).stop 2024-05-09T00:03:04.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.695+0000 7f535a2eb700 1 -- 172.21.15.142:0/1204085773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5354104d10 msgr2=0x7f53541a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:04.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.695+0000 7f535a2eb700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5354104d10 0x7f53541a1e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f534c009fd0 tx=0x7f534c004da0 comp rx=0 tx=0).stop 2024-05-09T00:03:04.690 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.696+0000 7f535a2eb700 1 -- 172.21.15.142:0/1204085773 shutdown_connections 2024-05-09T00:03:04.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.696+0000 7f535a2eb700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f533c0b27f0 0x7f533c0b4d00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:04.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.696+0000 7f535a2eb700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5354104d10 0x7f53541a1e30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:04.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.696+0000 7f535a2eb700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5354103240 0x7f535419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:04.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.696+0000 7f535a2eb700 1 --2- 172.21.15.142:0/1204085773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5354104370 0x7f535419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:04.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.696+0000 7f535a2eb700 1 -- 172.21.15.142:0/1204085773 >> 172.21.15.142:0/1204085773 conn(0x7f53540fe880 msgr2=0x7f5354100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:04.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.696+0000 7f535a2eb700 1 -- 172.21.15.142:0/1204085773 shutdown_connections 2024-05-09T00:03:04.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:04.696+0000 7f535a2eb700 1 -- 172.21.15.142:0/1204085773 wait complete. 2024-05-09T00:03:05.320 DEBUG:teuthology.orchestra.run.smithi142:osd.1> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.1.service 2024-05-09T00:03:05.322 INFO:tasks.cephadm:Deploying osd.2 on smithi169 with /dev/vg_nvme/lv_4... 2024-05-09T00:03:05.323 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T00:03:05.545 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.545 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.545 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:05.545 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:05.545 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.545 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.545 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.545 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:05.545 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:05 smithi142 ceph-mon[34533]: from='osd.1 [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T00:03:05.546 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:05 smithi142 ceph-mon[34533]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T00:03:05.568 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:03:05.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:05.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:05 smithi169 ceph-mon[34777]: from='osd.1 [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T00:03:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:05 smithi169 ceph-mon[34777]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T00:03:05.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:05.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:05.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:05.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:05.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:05 smithi046 ceph-mon[30151]: from='osd.1 [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T00:03:05.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:05 smithi046 ceph-mon[30151]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T00:03:06.715 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:06 smithi169 ceph-mon[34777]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:06.715 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:06 smithi169 ceph-mon[34777]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T00:03:06.715 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:06 smithi169 ceph-mon[34777]: from='osd.1 [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:03:06.716 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:06 smithi169 ceph-mon[34777]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T00:03:06.716 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:06 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:06.716 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:06 smithi169 ceph-mon[34777]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:03:06.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:06 smithi142 ceph-mon[34533]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:06.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:06 smithi142 ceph-mon[34533]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T00:03:06.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:06 smithi142 ceph-mon[34533]: from='osd.1 [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:03:06.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:06 smithi142 ceph-mon[34533]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T00:03:06.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:06 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:06.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:06 smithi142 ceph-mon[34533]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:03:07.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:06 smithi046 ceph-mon[30151]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:07.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:06 smithi046 ceph-mon[30151]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T00:03:07.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:06 smithi046 ceph-mon[30151]: from='osd.1 [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:03:07.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:06 smithi046 ceph-mon[30151]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T00:03:07.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:06 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:07.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:06 smithi046 ceph-mon[30151]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:03:07.268 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T00:03:07.268 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T00:03:07.268 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-05-09T00:03:07.268 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-05-09T00:03:07.268 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01217 s, 862 MB/s 2024-05-09T00:03:07.268 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-05-09T00:03:07.438 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:03:07 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1[39140]: 2024-05-09T00:03:07.142+0000 7f5a822e1700 -1 osd.1 0 waiting for initial osdmap 2024-05-09T00:03:07.438 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:03:07 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1[39140]: 2024-05-09T00:03:07.148+0000 7f5a7a08e700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:03:07.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:07 smithi169 ceph-mon[34777]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-09T00:03:07.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:07 smithi169 ceph-mon[34777]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T00:03:07.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:07.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:07 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:07.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:07 smithi169 ceph-mon[34777]: from='osd.1 ' entity='osd.1' 2024-05-09T00:03:07.909 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_4 2024-05-09T00:03:07.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:07 smithi142 ceph-mon[34533]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-09T00:03:07.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:07 smithi142 ceph-mon[34533]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T00:03:07.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:07.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:07 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:07.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:07 smithi142 ceph-mon[34533]: from='osd.1 ' entity='osd.1' 2024-05-09T00:03:08.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:07 smithi046 ceph-mon[30151]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-09T00:03:08.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:07 smithi046 ceph-mon[30151]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T00:03:08.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:08.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:07 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:08.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:07 smithi046 ceph-mon[30151]: from='osd.1 ' entity='osd.1' 2024-05-09T00:03:08.153 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:03:08.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: purged_snaps scrub starts 2024-05-09T00:03:08.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: purged_snaps scrub ok 2024-05-09T00:03:08.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:08.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:08.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: osd.1 [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469] boot 2024-05-09T00:03:08.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T00:03:08.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:08.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:08.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:08.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:03:08.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:08.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:08.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:08.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:08.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: purged_snaps scrub starts 2024-05-09T00:03:08.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: purged_snaps scrub ok 2024-05-09T00:03:08.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:08.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:08.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: osd.1 [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469] boot 2024-05-09T00:03:08.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T00:03:08.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:08.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:08.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:08.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:03:08.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:08.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:08.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:08.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:09.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T00:03:09.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T00:03:09.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T00:03:09.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:09.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: osd.1 [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469] boot 2024-05-09T00:03:09.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T00:03:09.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T00:03:09.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:09.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:09.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:03:09.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:09.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:09.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:09.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:09.621 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.626+0000 7f6c291cf700 1 -- 172.21.15.169:0/378945145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c24105470 msgr2=0x7f6c24105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:09.621 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.626+0000 7f6c291cf700 1 --2- 172.21.15.169:0/378945145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c24105470 0x7f6c24105880 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009990 tx=0x7f6c0c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:03:09.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.626+0000 7f6c291cf700 1 -- 172.21.15.169:0/378945145 shutdown_connections 2024-05-09T00:03:09.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.626+0000 7f6c291cf700 1 --2- 172.21.15.169:0/378945145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c24105e10 0x7f6c2410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:09.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.626+0000 7f6c291cf700 1 --2- 172.21.15.169:0/378945145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c24104340 0x7f6c24104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:09.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.626+0000 7f6c291cf700 1 --2- 172.21.15.169:0/378945145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c24105470 0x7f6c24105880 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:09.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.626+0000 7f6c291cf700 1 -- 172.21.15.169:0/378945145 >> 172.21.15.169:0/378945145 conn(0x7f6c240ff9c0 msgr2=0x7f6c24101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:09.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.626+0000 7f6c291cf700 1 -- 172.21.15.169:0/378945145 shutdown_connections 2024-05-09T00:03:09.622 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.626+0000 7f6c291cf700 1 -- 172.21.15.169:0/378945145 wait complete. 2024-05-09T00:03:09.624 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.627+0000 7f6c291cf700 1 Processor -- start 2024-05-09T00:03:09.624 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.627+0000 7f6c291cf700 1 -- start start 2024-05-09T00:03:09.624 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.627+0000 7f6c291cf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c24104340 0x7f6c2419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:09.624 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.627+0000 7f6c291cf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c24105470 0x7f6c2419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.627+0000 7f6c291cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c24105e10 0x7f6c241a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.627+0000 7f6c291cf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2410fe20 con 0x7f6c24105e10 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.627+0000 7f6c291cf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2410fca0 con 0x7f6c24105470 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.627+0000 7f6c291cf700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2410ffa0 con 0x7f6c24104340 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.627+0000 7f6c22d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c24104340 0x7f6c2419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.627+0000 7f6c22d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c24104340 0x7f6c2419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58696/0 (socket says 172.21.15.169:58696) 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.627+0000 7f6c22d9d700 1 -- 172.21.15.169:0/650758905 learned_addr learned my addr 172.21.15.169:0/650758905 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c2359e700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c24105e10 0x7f6c241a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c1a59c700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c24105470 0x7f6c2419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c22d9d700 1 -- 172.21.15.169:0/650758905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c24105470 msgr2=0x7f6c2419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c22d9d700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c24105470 0x7f6c2419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c22d9d700 1 -- 172.21.15.169:0/650758905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c24105e10 msgr2=0x7f6c241a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c22d9d700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c24105e10 0x7f6c241a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:09.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c22d9d700 1 -- 172.21.15.169:0/650758905 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c241a23d0 con 0x7f6c24104340 2024-05-09T00:03:09.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c1a59c700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c24105470 0x7f6c2419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:03:09.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c22d9d700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c24104340 0x7f6c2419a900 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400c970 tx=0x7f6c1400cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:09.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c20d99700 1 -- 172.21.15.169:0/650758905 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14007b00 con 0x7f6c24104340 2024-05-09T00:03:09.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c291cf700 1 -- 172.21.15.169:0/650758905 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c241a26c0 con 0x7f6c24104340 2024-05-09T00:03:09.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c20d99700 1 -- 172.21.15.169:0/650758905 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c14007ca0 con 0x7f6c24104340 2024-05-09T00:03:09.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c20d99700 1 -- 172.21.15.169:0/650758905 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14018660 con 0x7f6c24104340 2024-05-09T00:03:09.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.628+0000 7f6c291cf700 1 -- 172.21.15.169:0/650758905 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c241a2cd0 con 0x7f6c24104340 2024-05-09T00:03:09.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.629+0000 7f6c291cf700 1 -- 172.21.15.169:0/650758905 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c240512c0 con 0x7f6c24104340 2024-05-09T00:03:09.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.631+0000 7f6c20d99700 1 -- 172.21.15.169:0/650758905 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f6c14004020 con 0x7f6c24104340 2024-05-09T00:03:09.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.631+0000 7f6c20d99700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6c100b27f0 0x7f6c100b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:09.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.631+0000 7f6c20d99700 1 -- 172.21.15.169:0/650758905 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f6c140e0ab0 con 0x7f6c24104340 2024-05-09T00:03:09.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.632+0000 7f6c1a59c700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6c100b27f0 0x7f6c100b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:09.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.632+0000 7f6c1a59c700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6c100b27f0 0x7f6c100b4d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c005b40 tx=0x7f6c0c005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:09.630 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.635+0000 7f6c20d99700 1 -- 172.21.15.169:0/650758905 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c14053df0 con 0x7f6c24104340 2024-05-09T00:03:09.843 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:09.847+0000 7f6c291cf700 1 -- 172.21.15.169:0/650758905 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6c24063c00 con 0x7f6c100b27f0 2024-05-09T00:03:10.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:09 smithi046 ceph-mon[30151]: Detected new or changed devices on smithi142 2024-05-09T00:03:10.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:09 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi142 to 17088M 2024-05-09T00:03:10.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:09 smithi169 ceph-mon[34777]: Detected new or changed devices on smithi142 2024-05-09T00:03:10.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:09 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi142 to 17088M 2024-05-09T00:03:10.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:09 smithi142 ceph-mon[34533]: Detected new or changed devices on smithi142 2024-05-09T00:03:10.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:09 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi142 to 17088M 2024-05-09T00:03:10.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:10 smithi169 ceph-mon[34777]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:10.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:10 smithi169 ceph-mon[34777]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T00:03:10.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:03:10.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:03:10.752 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:10.958 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:10 smithi046 ceph-mon[30151]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:10.958 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:10 smithi046 ceph-mon[30151]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T00:03:10.958 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:03:10.958 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:03:10.958 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:10 smithi142 ceph-mon[34533]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:10 smithi142 ceph-mon[34533]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T00:03:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:03:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:03:11.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:12.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:11 smithi046 ceph-mon[30151]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:03:12.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:11 smithi169 ceph-mon[34777]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:03:12.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:11 smithi142 ceph-mon[34533]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:03:13.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:12 smithi046 ceph-mon[30151]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:13.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:12 smithi046 ceph-mon[30151]: from='client.? 172.21.15.169:0/489037222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05c070e7-83ad-4949-9834-4ce51ac0a2f1"}]: dispatch 2024-05-09T00:03:13.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:12 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05c070e7-83ad-4949-9834-4ce51ac0a2f1"}]: dispatch 2024-05-09T00:03:13.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:12 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05c070e7-83ad-4949-9834-4ce51ac0a2f1"}]': finished 2024-05-09T00:03:13.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:12 smithi046 ceph-mon[30151]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T00:03:13.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:12 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:12 smithi169 ceph-mon[34777]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:12 smithi169 ceph-mon[34777]: from='client.? 172.21.15.169:0/489037222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05c070e7-83ad-4949-9834-4ce51ac0a2f1"}]: dispatch 2024-05-09T00:03:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:12 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05c070e7-83ad-4949-9834-4ce51ac0a2f1"}]: dispatch 2024-05-09T00:03:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:12 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05c070e7-83ad-4949-9834-4ce51ac0a2f1"}]': finished 2024-05-09T00:03:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:12 smithi169 ceph-mon[34777]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T00:03:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:12 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:13.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:12 smithi142 ceph-mon[34533]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:13.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:12 smithi142 ceph-mon[34533]: from='client.? 172.21.15.169:0/489037222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05c070e7-83ad-4949-9834-4ce51ac0a2f1"}]: dispatch 2024-05-09T00:03:13.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:12 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05c070e7-83ad-4949-9834-4ce51ac0a2f1"}]: dispatch 2024-05-09T00:03:13.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:12 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05c070e7-83ad-4949-9834-4ce51ac0a2f1"}]': finished 2024-05-09T00:03:13.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:12 smithi142 ceph-mon[34533]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T00:03:13.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:12 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:14.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:13 smithi046 ceph-mon[30151]: from='client.? 172.21.15.169:0/1472297569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T00:03:14.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:13 smithi046 ceph-mon[30151]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:14.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:13 smithi169 ceph-mon[34777]: from='client.? 172.21.15.169:0/1472297569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T00:03:14.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:13 smithi169 ceph-mon[34777]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:14.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:13 smithi142 ceph-mon[34533]: from='client.? 172.21.15.169:0/1472297569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T00:03:14.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:13 smithi142 ceph-mon[34533]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:16.447 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:16 smithi169 ceph-mon[34777]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:16.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:16 smithi046 ceph-mon[30151]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:16.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:16 smithi142 ceph-mon[34533]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:18.463 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:18 smithi169 ceph-mon[34777]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:18.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:18 smithi046 ceph-mon[30151]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:18.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:18 smithi142 ceph-mon[34533]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:20.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:20 smithi046 ceph-mon[30151]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:20.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T00:03:20.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:20.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:20 smithi046 ceph-mon[30151]: Deploying daemon osd.2 on smithi169 2024-05-09T00:03:20.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:20 smithi169 ceph-mon[34777]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:20.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T00:03:20.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:20.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:20 smithi169 ceph-mon[34777]: Deploying daemon osd.2 on smithi169 2024-05-09T00:03:20.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:20 smithi142 ceph-mon[34533]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:20.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T00:03:20.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:20.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:20 smithi142 ceph-mon[34533]: Deploying daemon osd.2 on smithi169 2024-05-09T00:03:22.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:22 smithi046 ceph-mon[30151]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:22.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:22 smithi169 ceph-mon[34777]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:22.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:22 smithi142 ceph-mon[34533]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:24.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:24 smithi046 ceph-mon[30151]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:24.568 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:24 smithi169 ceph-mon[34777]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:24.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:24 smithi142 ceph-mon[34533]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:26.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:26.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:26.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:26.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:25 smithi046 ceph-mon[30151]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:26.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:26.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:26.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:26.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:25 smithi169 ceph-mon[34777]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:26.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:26.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:26.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:26.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:25 smithi142 ceph-mon[34533]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:27.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.813+0000 7f6c20d99700 1 -- 172.21.15.169:0/650758905 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6c24063c00 con 0x7f6c100b27f0 2024-05-09T00:03:27.808 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 2 on host 'smithi169' 2024-05-09T00:03:27.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.816+0000 7f6c291cf700 1 -- 172.21.15.169:0/650758905 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6c100b27f0 msgr2=0x7f6c100b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:27.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.816+0000 7f6c291cf700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6c100b27f0 0x7f6c100b4d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c005b40 tx=0x7f6c0c005a90 comp rx=0 tx=0).stop 2024-05-09T00:03:27.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.817+0000 7f6c291cf700 1 -- 172.21.15.169:0/650758905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c24104340 msgr2=0x7f6c2419a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:27.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.817+0000 7f6c291cf700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c24104340 0x7f6c2419a900 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400c970 tx=0x7f6c1400cd30 comp rx=0 tx=0).stop 2024-05-09T00:03:27.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.817+0000 7f6c291cf700 1 -- 172.21.15.169:0/650758905 shutdown_connections 2024-05-09T00:03:27.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.817+0000 7f6c291cf700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6c100b27f0 0x7f6c100b4d00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:27.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.817+0000 7f6c291cf700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c24105e10 0x7f6c241a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:27.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.817+0000 7f6c291cf700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c24104340 0x7f6c2419a900 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:27.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.817+0000 7f6c291cf700 1 --2- 172.21.15.169:0/650758905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c24105470 0x7f6c2419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:27.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.817+0000 7f6c291cf700 1 -- 172.21.15.169:0/650758905 >> 172.21.15.169:0/650758905 conn(0x7f6c240ff9c0 msgr2=0x7f6c241086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:27.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.817+0000 7f6c291cf700 1 -- 172.21.15.169:0/650758905 shutdown_connections 2024-05-09T00:03:27.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:03:27.817+0000 7f6c291cf700 1 -- 172.21.15.169:0/650758905 wait complete. 2024-05-09T00:03:28.372 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:28 smithi169 ceph-mon[34777]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:28.372 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:28 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:28.372 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:28 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:28.372 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:28 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:28.372 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:28 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:28.403 DEBUG:teuthology.orchestra.run.smithi169:osd.2> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.2.service 2024-05-09T00:03:28.406 INFO:tasks.cephadm:Deploying osd.3 on smithi046 with /dev/vg_nvme/lv_3... 2024-05-09T00:03:28.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T00:03:28.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:28 smithi046 ceph-mon[30151]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:28.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:28 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:28.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:28 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:28.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:28 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:28.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:28 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:28.664 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:03:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:28 smithi142 ceph-mon[34533]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:28 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:28 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:28 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:28.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:28 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:29.100 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:03:28 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2[39953]: 2024-05-09T00:03:28.628+0000 7fbbd08427c0 -1 osd.2 0 log_to_monitors true 2024-05-09T00:03:29.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:29 smithi046 ceph-mon[30151]: from='osd.2 [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T00:03:29.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:29 smithi046 ceph-mon[30151]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T00:03:29.567 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:29 smithi169 ceph-mon[34777]: from='osd.2 [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T00:03:29.567 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:29 smithi169 ceph-mon[34777]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T00:03:29.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:29 smithi142 ceph-mon[34533]: from='osd.2 [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T00:03:29.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:29 smithi142 ceph-mon[34533]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T00:03:30.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:30 smithi046 ceph-mon[30151]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:30.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:30 smithi046 ceph-mon[30151]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T00:03:30.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:30 smithi046 ceph-mon[30151]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T00:03:30.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:30 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:30.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:30 smithi046 ceph-mon[30151]: from='osd.2 [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:03:30.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:30 smithi046 ceph-mon[30151]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:03:30.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:30 smithi169 ceph-mon[34777]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:30.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:30 smithi169 ceph-mon[34777]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T00:03:30.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:30 smithi169 ceph-mon[34777]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T00:03:30.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:30 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:30.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:30 smithi169 ceph-mon[34777]: from='osd.2 [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:03:30.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:30 smithi169 ceph-mon[34777]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:03:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T00:03:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T00:03:30.668 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-05-09T00:03:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-05-09T00:03:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00833722 s, 1.3 GB/s 2024-05-09T00:03:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-05-09T00:03:30.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:30 smithi142 ceph-mon[34533]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:30.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:30 smithi142 ceph-mon[34533]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T00:03:30.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:30 smithi142 ceph-mon[34533]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T00:03:30.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:30 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:30.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:30 smithi142 ceph-mon[34533]: from='osd.2 [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:03:30.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:30 smithi142 ceph-mon[34533]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:03:30.943 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:03:30 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2[39953]: 2024-05-09T00:03:30.771+0000 7fbbc66a7700 -1 osd.2 0 waiting for initial osdmap 2024-05-09T00:03:30.943 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:03:30 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2[39953]: 2024-05-09T00:03:30.776+0000 7fbbbdc53700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:03:31.331 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_3 2024-05-09T00:03:31.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:31 smithi046 ceph-mon[30151]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T00:03:31.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:31 smithi046 ceph-mon[30151]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T00:03:31.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:31 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:31.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:31 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:31.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:31 smithi046 ceph-mon[30151]: from='osd.2 ' entity='osd.2' 2024-05-09T00:03:31.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:31 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:31.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:31 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:31.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:31 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:03:31.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:31 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:31.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:31 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:31.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:31 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:31.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:31 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:31.588 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:03:31.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:31 smithi169 ceph-mon[34777]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T00:03:31.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:31 smithi169 ceph-mon[34777]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T00:03:31.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:31 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:31.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:31 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:31.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:31 smithi169 ceph-mon[34777]: from='osd.2 ' entity='osd.2' 2024-05-09T00:03:31.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:31 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:31.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:31 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:31.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:31 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:03:31.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:31 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:31.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:31 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:31.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:31 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:31.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:31 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:31.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:31 smithi142 ceph-mon[34533]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T00:03:31.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:31 smithi142 ceph-mon[34533]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T00:03:31.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:31 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:31.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:31 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:31.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:31 smithi142 ceph-mon[34533]: from='osd.2 ' entity='osd.2' 2024-05-09T00:03:31.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:31 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:31.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:31 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:31.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:31 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:03:31.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:31 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:31.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:31 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:31.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:31 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:31.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:31 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:32.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T00:03:32.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T00:03:32.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:32.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: Detected new or changed devices on smithi169 2024-05-09T00:03:32.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi169 to 17088M 2024-05-09T00:03:32.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:32.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:32.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:32.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:32.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:32.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: osd.2 [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064] boot 2024-05-09T00:03:32.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T00:03:32.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: purged_snaps scrub starts 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: purged_snaps scrub ok 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: Detected new or changed devices on smithi169 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi169 to 17088M 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: osd.2 [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064] boot 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T00:03:32.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:32.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: purged_snaps scrub starts 2024-05-09T00:03:32.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: purged_snaps scrub ok 2024-05-09T00:03:32.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T00:03:32.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: Detected new or changed devices on smithi169 2024-05-09T00:03:32.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi169 to 17088M 2024-05-09T00:03:32.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:32.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:32.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:32.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:32.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:32.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: osd.2 [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064] boot 2024-05-09T00:03:32.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T00:03:32.603 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T00:03:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.220+0000 7fe5c2af4700 1 -- 172.21.15.46:0/2418365918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc1026a0 msgr2=0x7fe5bc102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.220+0000 7fe5c2af4700 1 --2- 172.21.15.46:0/2418365918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc1026a0 0x7fe5bc102ab0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009990 tx=0x7fe5b0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.220+0000 7fe5c2af4700 1 -- 172.21.15.46:0/2418365918 shutdown_connections 2024-05-09T00:03:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.220+0000 7fe5c2af4700 1 --2- 172.21.15.46:0/2418365918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5bc10a180 0x7fe5bc10c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.220+0000 7fe5c2af4700 1 --2- 172.21.15.46:0/2418365918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc1026a0 0x7fe5bc102ab0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.220+0000 7fe5c2af4700 1 --2- 172.21.15.46:0/2418365918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5bc107820 0x7fe5bc109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.220+0000 7fe5c2af4700 1 -- 172.21.15.46:0/2418365918 >> 172.21.15.46:0/2418365918 conn(0x7fe5bc0fe020 msgr2=0x7fe5bc1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.220+0000 7fe5c2af4700 1 -- 172.21.15.46:0/2418365918 shutdown_connections 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.220+0000 7fe5c2af4700 1 -- 172.21.15.46:0/2418365918 wait complete. 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5c2af4700 1 Processor -- start 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5c2af4700 1 -- start start 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5c2af4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5bc1026a0 0x7fe5bc19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5c2af4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc107820 0x7fe5bc19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5c2af4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5bc10a180 0x7fe5bc1a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5c2af4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc10fda0 con 0x7fe5bc107820 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5c2af4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc10fc20 con 0x7fe5bc1026a0 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5c2af4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc10ff20 con 0x7fe5bc10a180 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc107820 0x7fe5bc19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc107820 0x7fe5bc19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55646/0 (socket says 172.21.15.46:55646) 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5bbfff700 1 -- 172.21.15.46:0/166221249 learned_addr learned my addr 172.21.15.46:0/166221249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:03:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5c0890700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5bc1026a0 0x7fe5bc19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5c1091700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5bc10a180 0x7fe5bc1a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5bbfff700 1 -- 172.21.15.46:0/166221249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5bc10a180 msgr2=0x7fe5bc1a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5bbfff700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5bc10a180 0x7fe5bc1a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5bbfff700 1 -- 172.21.15.46:0/166221249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5bc1026a0 msgr2=0x7fe5bc19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.221+0000 7fe5bbfff700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5bc1026a0 0x7fe5bc19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.222+0000 7fe5bbfff700 1 -- 172.21.15.46:0/166221249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc1a2430 con 0x7fe5bc107820 2024-05-09T00:03:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.222+0000 7fe5bbfff700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc107820 0x7fe5bc19ae10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800d950 tx=0x7fe5a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.222+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/166221249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8016070 con 0x7fe5bc107820 2024-05-09T00:03:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.222+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/166221249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a800fd40 con 0x7fe5bc107820 2024-05-09T00:03:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.222+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/166221249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800e3b0 con 0x7fe5bc107820 2024-05-09T00:03:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.222+0000 7fe5c2af4700 1 -- 172.21.15.46:0/166221249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5bc1a2720 con 0x7fe5bc107820 2024-05-09T00:03:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.222+0000 7fe5c2af4700 1 -- 172.21.15.46:0/166221249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5bc1a2cb0 con 0x7fe5bc107820 2024-05-09T00:03:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.223+0000 7fe5c2af4700 1 -- 172.21.15.46:0/166221249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5bc0512c0 con 0x7fe5bc107820 2024-05-09T00:03:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.224+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/166221249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fe5a8019070 con 0x7fe5bc107820 2024-05-09T00:03:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.225+0000 7fe5b9ffb700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe5ac0b2870 0x7fe5ac0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.225+0000 7fe5c0890700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe5ac0b2870 0x7fe5ac0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.225+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/166221249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7fe5a80e0610 con 0x7fe5bc107820 2024-05-09T00:03:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.226+0000 7fe5c0890700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe5ac0b2870 0x7fe5ac0b4d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000b5c0 tx=0x7fe5b001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:33.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.230+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/166221249 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5a80a9960 con 0x7fe5bc107820 2024-05-09T00:03:33.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:33.458+0000 7fe5c2af4700 1 -- 172.21.15.46:0/166221249 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe5bc063c00 con 0x7fe5ac0b2870 2024-05-09T00:03:33.453 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:03:33.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:03:33.614 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:03:34.353 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 ceph-mon[30151]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:34.353 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T00:03:34.353 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 ceph-mon[30151]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T00:03:34.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:03:34.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 ceph-mon[30151]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:03:34.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:03:34.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:03:34.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:34.585 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 ceph-mon[34777]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:34.585 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T00:03:34.585 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 ceph-mon[34777]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T00:03:34.585 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:03:34.585 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 ceph-mon[34777]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:03:34.585 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:03:34.585 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:03:34.585 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:34.586 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44012]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:03:34.586 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44012]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:03:34.586 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44012]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:03:34.586 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44012]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:03:34.586 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44015]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:03:34.586 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44015]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:03:34.586 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44015]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:03:34.586 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44015]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:03:34.622 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42841]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:03:34.622 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42841]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:03:34.622 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42841]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:03:34.622 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42841]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:03:34.622 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42844]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:03:34.622 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42844]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:03:34.622 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42844]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:03:34.622 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42844]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:03:34.622 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 ceph-mon[34533]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:34.622 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T00:03:34.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 ceph-mon[34533]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T00:03:34.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:03:34.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 ceph-mon[34533]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:03:34.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:03:34.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:03:34.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:34.623 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42847]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:03:34.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47545]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:03:34.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47545]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:03:34.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47545]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:03:34.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47545]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:03:34.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47548]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:03:34.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47548]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:03:34.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47548]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:03:34.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47548]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:03:34.807 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47526]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:03:34.807 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47526]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:03:34.807 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47526]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:03:34.808 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47526]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:03:34.808 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47537]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:03:34.808 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47537]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:03:34.808 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47537]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:03:34.808 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:03:34 smithi046 sudo[47537]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:03:34.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44018]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T00:03:34.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44018]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:03:34.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44018]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:03:34.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44018]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:03:34.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44021]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:03:34.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44021]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:03:34.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44021]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:03:34.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:34 smithi169 sudo[44021]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:03:34.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42847]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:03:34.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42847]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:03:34.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42847]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:03:34.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42850]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T00:03:34.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42850]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T00:03:34.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42850]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T00:03:34.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:34 smithi142 sudo[42850]: pam_unix(sudo:session): session closed for user root 2024-05-09T00:03:35.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T00:03:35.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T00:03:35.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:35.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:35.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:03:35.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:03:35.409 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:35 smithi046 ceph-mon[30151]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:03:35.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:35.553+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/166221249 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe5a80a54f0 con 0x7fe5bc107820 2024-05-09T00:03:35.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T00:03:35.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T00:03:35.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:35.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:03:35.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:03:35.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:35 smithi169 ceph-mon[34777]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:03:35.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T00:03:35.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T00:03:35.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:35.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:35.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T00:03:35.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T00:03:35.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:35 smithi142 ceph-mon[34533]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T00:03:36.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:36 smithi046 ceph-mon[30151]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:36.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:36 smithi046 ceph-mon[30151]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T00:03:36.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:36 smithi046 ceph-mon[30151]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T00:03:36.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:36 smithi169 ceph-mon[34777]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:36.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:36 smithi169 ceph-mon[34777]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T00:03:36.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:36 smithi169 ceph-mon[34777]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T00:03:36.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:36 smithi142 ceph-mon[34533]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:36.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:36 smithi142 ceph-mon[34533]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T00:03:36.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:36 smithi142 ceph-mon[34533]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T00:03:37.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:37 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2939283378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d7bbe1-58b6-4407-bef2-920f179dfb9e"}]: dispatch 2024-05-09T00:03:37.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:37 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2939283378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d7bbe1-58b6-4407-bef2-920f179dfb9e"}]': finished 2024-05-09T00:03:37.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:37 smithi046 ceph-mon[30151]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T00:03:37.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:37 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:37.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:37 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1116700606' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T00:03:37.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:37 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2939283378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d7bbe1-58b6-4407-bef2-920f179dfb9e"}]: dispatch 2024-05-09T00:03:37.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:37 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2939283378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d7bbe1-58b6-4407-bef2-920f179dfb9e"}]': finished 2024-05-09T00:03:37.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:37 smithi169 ceph-mon[34777]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T00:03:37.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:37 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:37.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:37 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1116700606' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T00:03:37.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:37 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2939283378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d7bbe1-58b6-4407-bef2-920f179dfb9e"}]: dispatch 2024-05-09T00:03:37.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:37 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2939283378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d7bbe1-58b6-4407-bef2-920f179dfb9e"}]': finished 2024-05-09T00:03:37.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:37 smithi142 ceph-mon[34533]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T00:03:37.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:37 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:37.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:37 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1116700606' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T00:03:38.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:38 smithi142 ceph-mon[34533]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:38.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:38 smithi046 ceph-mon[30151]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:38.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:38 smithi169 ceph-mon[34777]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:40.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:40 smithi142 ceph-mon[34533]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:40.717 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:40 smithi169 ceph-mon[34777]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:40.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:40 smithi046 ceph-mon[30151]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:42.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:42 smithi142 ceph-mon[34533]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:42.759 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:42 smithi046 ceph-mon[30151]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:42.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:42 smithi169 ceph-mon[34777]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:44.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:44 smithi046 ceph-mon[30151]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:44.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T00:03:44.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:44.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:44 smithi142 ceph-mon[34533]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:44.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T00:03:44.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:44.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:44 smithi169 ceph-mon[34777]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:44.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T00:03:44.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:45.614 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:45 smithi046 ceph-mon[30151]: Deploying daemon osd.3 on smithi046 2024-05-09T00:03:45.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:45 smithi142 ceph-mon[34533]: Deploying daemon osd.3 on smithi046 2024-05-09T00:03:45.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:45 smithi169 ceph-mon[34777]: Deploying daemon osd.3 on smithi046 2024-05-09T00:03:46.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:46 smithi142 ceph-mon[34533]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:46.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:46 smithi046 ceph-mon[30151]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:46.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:46 smithi169 ceph-mon[34777]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:48.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:48 smithi142 ceph-mon[34533]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:48.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:48 smithi046 ceph-mon[30151]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:48.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:48 smithi169 ceph-mon[34777]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:50.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:50 smithi142 ceph-mon[34533]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:50.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:50.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:50.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:51.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:50 smithi046 ceph-mon[30151]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:51.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:51.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:51.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:51.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:50 smithi169 ceph-mon[34777]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:51.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:51.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:51.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:52.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:52 smithi046 ceph-mon[30151]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:52.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:52 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:52.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:52 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:52.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:52 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:52.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:52 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:52.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:52 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:52.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:52 smithi142 ceph-mon[34533]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:52.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:52 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:52.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:52 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:52.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:52 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:52.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:52 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:52.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:52 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:53.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:52 smithi169 ceph-mon[34777]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:53.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:52 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:53.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:52 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:53.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:52 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:53.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:52 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:53.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:52 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:53.674 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:53 smithi046 ceph-mon[30151]: from='osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T00:03:53.714 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 3 on host 'smithi046' 2024-05-09T00:03:53.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.721+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/166221249 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe5bc063c00 con 0x7fe5ac0b2870 2024-05-09T00:03:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.723+0000 7fe5c2af4700 1 -- 172.21.15.46:0/166221249 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe5ac0b2870 msgr2=0x7fe5ac0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.723+0000 7fe5c2af4700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe5ac0b2870 0x7fe5ac0b4d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000b5c0 tx=0x7fe5b001a040 comp rx=0 tx=0).stop 2024-05-09T00:03:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.723+0000 7fe5c2af4700 1 -- 172.21.15.46:0/166221249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc107820 msgr2=0x7fe5bc19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.723+0000 7fe5c2af4700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc107820 0x7fe5bc19ae10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800d950 tx=0x7fe5a800dc60 comp rx=0 tx=0).stop 2024-05-09T00:03:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.723+0000 7fe5c2af4700 1 -- 172.21.15.46:0/166221249 shutdown_connections 2024-05-09T00:03:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.723+0000 7fe5c2af4700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe5ac0b2870 0x7fe5ac0b4d80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.723+0000 7fe5c2af4700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5bc10a180 0x7fe5bc1a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.723+0000 7fe5c2af4700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5bc1026a0 0x7fe5bc19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.723+0000 7fe5c2af4700 1 --2- 172.21.15.46:0/166221249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc107820 0x7fe5bc19ae10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.723+0000 7fe5c2af4700 1 -- 172.21.15.46:0/166221249 >> 172.21.15.46:0/166221249 conn(0x7fe5bc0fe020 msgr2=0x7fe5bc108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.723+0000 7fe5c2af4700 1 -- 172.21.15.46:0/166221249 shutdown_connections 2024-05-09T00:03:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:03:53.723+0000 7fe5c2af4700 1 -- 172.21.15.46:0/166221249 wait complete. 2024-05-09T00:03:53.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:53 smithi142 ceph-mon[34533]: from='osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T00:03:54.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:53 smithi169 ceph-mon[34777]: from='osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T00:03:54.372 DEBUG:teuthology.orchestra.run.smithi046:osd.3> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.3.service 2024-05-09T00:03:54.374 INFO:tasks.cephadm:Deploying osd.4 on smithi142 with /dev/vg_nvme/lv_3... 2024-05-09T00:03:54.375 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T00:03:54.618 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:03:54.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:54 smithi142 ceph-mon[34533]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:54.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:54 smithi142 ceph-mon[34533]: from='osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T00:03:54.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:54 smithi142 ceph-mon[34533]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T00:03:54.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:54 smithi142 ceph-mon[34533]: from='osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T00:03:54.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:54.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:54.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:54.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:54.924 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:54 smithi046 ceph-mon[30151]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:54.924 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:54 smithi046 ceph-mon[30151]: from='osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T00:03:54.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:54 smithi046 ceph-mon[30151]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T00:03:54.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:54 smithi046 ceph-mon[30151]: from='osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T00:03:54.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:54.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:54.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:54.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:54.925 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:03:54 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3[49666]: 2024-05-09T00:03:54.655+0000 7f5923edb700 -1 osd.3 0 waiting for initial osdmap 2024-05-09T00:03:54.925 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:03:54 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3[49666]: 2024-05-09T00:03:54.660+0000 7f591fc90700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:03:55.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:54 smithi169 ceph-mon[34777]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:55.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:54 smithi169 ceph-mon[34777]: from='osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T00:03:55.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:54 smithi169 ceph-mon[34777]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T00:03:55.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:54 smithi169 ceph-mon[34777]: from='osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T00:03:55.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:55.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:55.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:55.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:03:55.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:55 smithi142 ceph-mon[34533]: from='osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T00:03:55.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:55 smithi142 ceph-mon[34533]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T00:03:55.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:55 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:55.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:55 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:55.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:55 smithi046 ceph-mon[30151]: from='osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T00:03:55.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:55 smithi046 ceph-mon[30151]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T00:03:55.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:55 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:55.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:55 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:56.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:55 smithi169 ceph-mon[34777]: from='osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T00:03:56.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:55 smithi169 ceph-mon[34777]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T00:03:56.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:55 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:56.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:55 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:56.694 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:56 smithi142 ceph-mon[34533]: purged_snaps scrub starts 2024-05-09T00:03:56.694 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:56 smithi142 ceph-mon[34533]: purged_snaps scrub ok 2024-05-09T00:03:56.694 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:56 smithi142 ceph-mon[34533]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:56.694 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:56.694 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:56 smithi142 ceph-mon[34533]: osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261] boot 2024-05-09T00:03:56.694 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:56 smithi142 ceph-mon[34533]: osdmap e25: 4 total, 4 up, 4 in 2024-05-09T00:03:56.695 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:56.695 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T00:03:56.695 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T00:03:56.695 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-05-09T00:03:56.695 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-05-09T00:03:56.695 INFO:teuthology.orchestra.run.smithi142.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011412 s, 919 MB/s 2024-05-09T00:03:56.695 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-05-09T00:03:57.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:56 smithi046 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T00:03:57.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:56 smithi046 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T00:03:57.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:56 smithi046 ceph-mon[30151]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:57.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:57.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:56 smithi046 ceph-mon[30151]: osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261] boot 2024-05-09T00:03:57.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:56 smithi046 ceph-mon[30151]: osdmap e25: 4 total, 4 up, 4 in 2024-05-09T00:03:57.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:57.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:56 smithi169 ceph-mon[34777]: purged_snaps scrub starts 2024-05-09T00:03:57.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:56 smithi169 ceph-mon[34777]: purged_snaps scrub ok 2024-05-09T00:03:57.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:56 smithi169 ceph-mon[34777]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-09T00:03:57.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:57.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:56 smithi169 ceph-mon[34777]: osd.3 [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261] boot 2024-05-09T00:03:57.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:56 smithi169 ceph-mon[34777]: osdmap e25: 4 total, 4 up, 4 in 2024-05-09T00:03:57.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T00:03:57.319 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2024-05-09T00:03:57.564 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:03:57.936 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:57 smithi142 ceph-mon[34533]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T00:03:57.936 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:57.936 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:57.936 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:03:57.936 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:03:57.936 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:57.936 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:57.936 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:57.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:58.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:57 smithi046 ceph-mon[30151]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T00:03:58.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:58.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:58.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:03:58.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:03:58.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:58.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:58.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:58.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:58.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:57 smithi169 ceph-mon[34777]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T00:03:58.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:58.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:58.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:03:58.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:03:58.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:58.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:03:58.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:03:58.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:03:58.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:58 smithi142 ceph-mon[34533]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:03:58.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:58 smithi142 ceph-mon[34533]: Detected new or changed devices on smithi046 2024-05-09T00:03:58.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:58 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi046 to 8546M 2024-05-09T00:03:58.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:58 smithi142 ceph-mon[34533]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T00:03:59.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:58 smithi046 ceph-mon[30151]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:03:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:58 smithi046 ceph-mon[30151]: Detected new or changed devices on smithi046 2024-05-09T00:03:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:58 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi046 to 8546M 2024-05-09T00:03:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:58 smithi046 ceph-mon[30151]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T00:03:59.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:58 smithi169 ceph-mon[34777]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:03:59.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:58 smithi169 ceph-mon[34777]: Detected new or changed devices on smithi046 2024-05-09T00:03:59.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:58 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi046 to 8546M 2024-05-09T00:03:59.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:58 smithi169 ceph-mon[34777]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T00:03:59.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.187+0000 7f2867f41700 1 -- 172.21.15.142:0/224059708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2860105080 msgr2=0x7f286010b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:59.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.187+0000 7f2867f41700 1 --2- 172.21.15.142:0/224059708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2860105080 0x7f286010b980 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f285800b0d0 tx=0x7f285800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:03:59.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.187+0000 7f2867f41700 1 -- 172.21.15.142:0/224059708 shutdown_connections 2024-05-09T00:03:59.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.187+0000 7f2867f41700 1 --2- 172.21.15.142:0/224059708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2860105080 0x7f286010b980 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:59.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.187+0000 7f2867f41700 1 --2- 172.21.15.142:0/224059708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2860103830 0x7f2860103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.187+0000 7f2867f41700 1 --2- 172.21.15.142:0/224059708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2860104730 0x7f2860104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.187+0000 7f2867f41700 1 -- 172.21.15.142:0/224059708 >> 172.21.15.142:0/224059708 conn(0x7f28600fe020 msgr2=0x7f28601004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.187+0000 7f2867f41700 1 -- 172.21.15.142:0/224059708 shutdown_connections 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.187+0000 7f2867f41700 1 -- 172.21.15.142:0/224059708 wait complete. 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.188+0000 7f2867f41700 1 Processor -- start 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.188+0000 7f2867f41700 1 -- start start 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.188+0000 7f2867f41700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2860103830 0x7f286019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.188+0000 7f2867f41700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2860104730 0x7f286019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.188+0000 7f2867f41700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2860105080 0x7f28601a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.188+0000 7f2867f41700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2860110050 con 0x7f2860104730 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.188+0000 7f2867f41700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f286010fed0 con 0x7f2860105080 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.188+0000 7f2867f41700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f28601101d0 con 0x7f2860103830 2024-05-09T00:03:59.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.188+0000 7f28664de700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2860105080 0x7f28601a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.188+0000 7f28664de700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2860105080 0x7f28601a1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:48166/0 (socket says 172.21.15.142:48166) 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f28664de700 1 -- 172.21.15.142:0/2884187411 learned_addr learned my addr 172.21.15.142:0/2884187411 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f28654dc700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2860104730 0x7f286019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f2865cdd700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2860103830 0x7f286019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f28664de700 1 -- 172.21.15.142:0/2884187411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2860103830 msgr2=0x7f286019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f28664de700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2860103830 0x7f286019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f28664de700 1 -- 172.21.15.142:0/2884187411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2860104730 msgr2=0x7f286019afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f28664de700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2860104730 0x7f286019afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f28664de700 1 -- 172.21.15.142:0/2884187411 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28601a25d0 con 0x7f2860105080 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f28664de700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2860105080 0x7f28601a1f50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2858009fd0 tx=0x7f2858004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f2856ffd700 1 -- 172.21.15.142:0/2884187411 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285801f030 con 0x7f2860105080 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f2867f41700 1 -- 172.21.15.142:0/2884187411 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28601a2860 con 0x7f2860105080 2024-05-09T00:03:59.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f2856ffd700 1 -- 172.21.15.142:0/2884187411 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2858003a00 con 0x7f2860105080 2024-05-09T00:03:59.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f2856ffd700 1 -- 172.21.15.142:0/2884187411 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858004670 con 0x7f2860105080 2024-05-09T00:03:59.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.189+0000 7f2867f41700 1 -- 172.21.15.142:0/2884187411 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28601a2cc0 con 0x7f2860105080 2024-05-09T00:03:59.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.190+0000 7f2867f41700 1 -- 172.21.15.142:0/2884187411 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28600512c0 con 0x7f2860105080 2024-05-09T00:03:59.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.192+0000 7f2856ffd700 1 -- 172.21.15.142:0/2884187411 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2858003bb0 con 0x7f2860105080 2024-05-09T00:03:59.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.192+0000 7f2856ffd700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f284c0b2880 0x7f284c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:03:59.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.193+0000 7f2856ffd700 1 -- 172.21.15.142:0/2884187411 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f285800f020 con 0x7f2860105080 2024-05-09T00:03:59.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.193+0000 7f2865cdd700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f284c0b2880 0x7f284c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:03:59.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.193+0000 7f2865cdd700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f284c0b2880 0x7f284c0b4d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f285c005950 tx=0x7f285c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:03:59.191 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.197+0000 7f2856ffd700 1 -- 172.21.15.142:0/2884187411 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2858029020 con 0x7f2860105080 2024-05-09T00:03:59.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:03:59.408+0000 7f2867f41700 1 -- 172.21.15.142:0/2884187411 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f286002e8b0 con 0x7f284c0b2880 2024-05-09T00:03:59.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:59 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:03:59.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:59 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:03:59.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:03:59 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:00.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:59 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:04:00.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:59 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:04:00.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:03:59 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:00.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:59 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:04:00.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:59 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:04:00.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:03:59 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:00.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:00 smithi142 ceph-mon[34533]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-09T00:04:00.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:00 smithi142 ceph-mon[34533]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:04:01.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:00 smithi046 ceph-mon[30151]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-09T00:04:01.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:00 smithi046 ceph-mon[30151]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:04:01.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:00 smithi169 ceph-mon[34777]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-05-09T00:04:01.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:00 smithi169 ceph-mon[34777]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:04:02.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:02 smithi142 ceph-mon[34533]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:04:02.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:02 smithi142 ceph-mon[34533]: from='client.? 172.21.15.142:0/2789898070' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66c9644b-6b0b-4fe7-aa63-b9d87bff7153"}]: dispatch 2024-05-09T00:04:02.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:02 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66c9644b-6b0b-4fe7-aa63-b9d87bff7153"}]: dispatch 2024-05-09T00:04:02.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:02 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66c9644b-6b0b-4fe7-aa63-b9d87bff7153"}]': finished 2024-05-09T00:04:02.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:02 smithi142 ceph-mon[34533]: osdmap e28: 5 total, 4 up, 5 in 2024-05-09T00:04:02.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:03.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:02 smithi046 ceph-mon[30151]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:04:03.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:02 smithi046 ceph-mon[30151]: from='client.? 172.21.15.142:0/2789898070' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66c9644b-6b0b-4fe7-aa63-b9d87bff7153"}]: dispatch 2024-05-09T00:04:03.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:02 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66c9644b-6b0b-4fe7-aa63-b9d87bff7153"}]: dispatch 2024-05-09T00:04:03.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:02 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66c9644b-6b0b-4fe7-aa63-b9d87bff7153"}]': finished 2024-05-09T00:04:03.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:02 smithi046 ceph-mon[30151]: osdmap e28: 5 total, 4 up, 5 in 2024-05-09T00:04:03.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:03.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:02 smithi169 ceph-mon[34777]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:04:03.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:02 smithi169 ceph-mon[34777]: from='client.? 172.21.15.142:0/2789898070' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66c9644b-6b0b-4fe7-aa63-b9d87bff7153"}]: dispatch 2024-05-09T00:04:03.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:02 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66c9644b-6b0b-4fe7-aa63-b9d87bff7153"}]: dispatch 2024-05-09T00:04:03.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:02 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66c9644b-6b0b-4fe7-aa63-b9d87bff7153"}]': finished 2024-05-09T00:04:03.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:02 smithi169 ceph-mon[34777]: osdmap e28: 5 total, 4 up, 5 in 2024-05-09T00:04:03.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:03.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:03 smithi142 ceph-mon[34533]: from='client.? 172.21.15.142:0/1957357496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T00:04:04.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:03 smithi046 ceph-mon[30151]: from='client.? 172.21.15.142:0/1957357496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T00:04:04.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:03 smithi169 ceph-mon[34777]: from='client.? 172.21.15.142:0/1957357496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T00:04:04.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:04 smithi142 ceph-mon[34533]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-09T00:04:05.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:04 smithi046 ceph-mon[30151]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-09T00:04:05.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:04 smithi169 ceph-mon[34777]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-09T00:04:06.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:06 smithi142 ceph-mon[34533]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:04:07.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:06 smithi046 ceph-mon[30151]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:04:07.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:06 smithi169 ceph-mon[34777]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:04:08.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:08 smithi142 ceph-mon[34533]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-09T00:04:09.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:08 smithi046 ceph-mon[30151]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-09T00:04:09.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:08 smithi169 ceph-mon[34777]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-09T00:04:10.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T00:04:10.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:10.097 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T00:04:10.097 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:10.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T00:04:10.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:10.861 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:10 smithi142 ceph-mon[34533]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:10.861 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:10 smithi142 ceph-mon[34533]: Deploying daemon osd.4 on smithi142 2024-05-09T00:04:10.958 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:10 smithi046 ceph-mon[30151]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:10.958 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:10 smithi046 ceph-mon[30151]: Deploying daemon osd.4 on smithi142 2024-05-09T00:04:11.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:10 smithi169 ceph-mon[34777]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:11.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:10 smithi169 ceph-mon[34777]: Deploying daemon osd.4 on smithi142 2024-05-09T00:04:12.872 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:12 smithi142 ceph-mon[34533]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:13.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:12 smithi046 ceph-mon[30151]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:12 smithi169 ceph-mon[34777]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:14.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:14 smithi142 ceph-mon[34533]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:15.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:14 smithi046 ceph-mon[30151]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:15.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:14 smithi169 ceph-mon[34777]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:16.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:16 smithi142 ceph-mon[34533]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:16.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:16.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:16.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:16.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:16 smithi169 ceph-mon[34777]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:16.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:16.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:16.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:16.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:16 smithi046 ceph-mon[30151]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:16.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:16.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:16.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:18.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:18 smithi169 ceph-mon[34777]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:18.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:18.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:18.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:18.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:18.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:18.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:18 smithi142 ceph-mon[34533]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:18.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:18.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:18.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:18.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:18.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:18.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:18 smithi046 ceph-mon[30151]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:18.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:18.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:18.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:18.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:18.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:19.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.178+0000 7f2856ffd700 1 -- 172.21.15.142:0/2884187411 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f286002e8b0 con 0x7f284c0b2880 2024-05-09T00:04:19.171 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 4 on host 'smithi142' 2024-05-09T00:04:19.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.180+0000 7f2867f41700 1 -- 172.21.15.142:0/2884187411 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f284c0b2880 msgr2=0x7f284c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:19.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.180+0000 7f2867f41700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f284c0b2880 0x7f284c0b4d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f285c005950 tx=0x7f285c00b410 comp rx=0 tx=0).stop 2024-05-09T00:04:19.173 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.180+0000 7f2867f41700 1 -- 172.21.15.142:0/2884187411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2860105080 msgr2=0x7f28601a1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:19.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.180+0000 7f2867f41700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2860105080 0x7f28601a1f50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2858009fd0 tx=0x7f2858004d00 comp rx=0 tx=0).stop 2024-05-09T00:04:19.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.180+0000 7f2867f41700 1 -- 172.21.15.142:0/2884187411 shutdown_connections 2024-05-09T00:04:19.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.180+0000 7f2867f41700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f284c0b2880 0x7f284c0b4d90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:19.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.180+0000 7f2867f41700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2860105080 0x7f28601a1f50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:19.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.180+0000 7f2867f41700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2860103830 0x7f286019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:19.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.180+0000 7f2867f41700 1 --2- 172.21.15.142:0/2884187411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2860104730 0x7f286019afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:19.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.180+0000 7f2867f41700 1 -- 172.21.15.142:0/2884187411 >> 172.21.15.142:0/2884187411 conn(0x7f28600fe020 msgr2=0x7f2860107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:19.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.180+0000 7f2867f41700 1 -- 172.21.15.142:0/2884187411 shutdown_connections 2024-05-09T00:04:19.174 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:04:19.180+0000 7f2867f41700 1 -- 172.21.15.142:0/2884187411 wait complete. 2024-05-09T00:04:19.851 DEBUG:teuthology.orchestra.run.smithi142:osd.4> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.4.service 2024-05-09T00:04:19.854 INFO:tasks.cephadm:Deploying osd.5 on smithi169 with /dev/vg_nvme/lv_3... 2024-05-09T00:04:19.854 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T00:04:20.098 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:04:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:20 smithi142 ceph-mon[34533]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:20 smithi142 ceph-mon[34533]: from='osd.4 [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T00:04:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:20 smithi142 ceph-mon[34533]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T00:04:20.510 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:20.510 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:20.510 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:20.510 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:20 smithi169 ceph-mon[34777]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:20.510 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:20 smithi169 ceph-mon[34777]: from='osd.4 [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T00:04:20.511 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:20 smithi169 ceph-mon[34777]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T00:04:20.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:20.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:20.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:20.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:20 smithi046 ceph-mon[30151]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:20.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:20 smithi046 ceph-mon[30151]: from='osd.4 [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T00:04:20.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:20 smithi046 ceph-mon[30151]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T00:04:21.384 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:21 smithi142 ceph-mon[34533]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T00:04:21.385 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:21 smithi142 ceph-mon[34533]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T00:04:21.385 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:21.385 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:21 smithi142 ceph-mon[34533]: from='osd.4 [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:04:21.385 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:21 smithi142 ceph-mon[34533]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:04:21.385 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:04:21 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4[45669]: 2024-05-09T00:04:21.185+0000 7f0c63f08700 -1 osd.4 0 waiting for initial osdmap 2024-05-09T00:04:21.385 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:04:21 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4[45669]: 2024-05-09T00:04:21.191+0000 7f0c5fcbd700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:04:21.522 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:21 smithi169 ceph-mon[34777]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T00:04:21.522 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:21 smithi169 ceph-mon[34777]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T00:04:21.522 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:21.523 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:21 smithi169 ceph-mon[34777]: from='osd.4 [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:04:21.523 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:21 smithi169 ceph-mon[34777]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:04:21.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:21 smithi046 ceph-mon[30151]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T00:04:21.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:21 smithi046 ceph-mon[30151]: osdmap e29: 5 total, 4 up, 5 in 2024-05-09T00:04:21.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:21.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:21 smithi046 ceph-mon[30151]: from='osd.4 [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:04:21.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:21 smithi046 ceph-mon[30151]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:04:22.103 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T00:04:22.103 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T00:04:22.103 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-05-09T00:04:22.104 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-05-09T00:04:22.104 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114369 s, 917 MB/s 2024-05-09T00:04:22.104 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 2024-05-09T00:04:22.104 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-05-09T00:04:22.414 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:22 smithi169 ceph-mon[34777]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-09T00:04:22.415 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:22 smithi169 ceph-mon[34777]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T00:04:22.415 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:22 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:22.415 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:22 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:22.415 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:22 smithi169 ceph-mon[34777]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:22.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:22 smithi142 ceph-mon[34533]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-09T00:04:22.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:22 smithi142 ceph-mon[34533]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T00:04:22.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:22 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:22.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:22 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:22.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:22 smithi142 ceph-mon[34533]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:22.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:22 smithi046 ceph-mon[30151]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-09T00:04:22.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:22 smithi046 ceph-mon[30151]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T00:04:22.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:22 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:22.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:22 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:22.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:22 smithi046 ceph-mon[30151]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T00:04:22.733 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_3 2024-05-09T00:04:22.980 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:04:23.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: purged_snaps scrub starts 2024-05-09T00:04:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: purged_snaps scrub ok 2024-05-09T00:04:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: osd.4 [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865] boot 2024-05-09T00:04:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: osdmap e31: 5 total, 5 up, 5 in 2024-05-09T00:04:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:04:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:04:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: purged_snaps scrub starts 2024-05-09T00:04:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: purged_snaps scrub ok 2024-05-09T00:04:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: osd.4 [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865] boot 2024-05-09T00:04:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: osdmap e31: 5 total, 5 up, 5 in 2024-05-09T00:04:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:23.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:04:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:04:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:23.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T00:04:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T00:04:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: osd.4 [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865] boot 2024-05-09T00:04:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: osdmap e31: 5 total, 5 up, 5 in 2024-05-09T00:04:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T00:04:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:04:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:04:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:23.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:23.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:24.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:24 smithi169 ceph-mon[34777]: Detected new or changed devices on smithi142 2024-05-09T00:04:24.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:24 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi142 to 8544M 2024-05-09T00:04:24.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:24 smithi169 ceph-mon[34777]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T00:04:24.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:24 smithi169 ceph-mon[34777]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:24.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:24 smithi046 ceph-mon[30151]: Detected new or changed devices on smithi142 2024-05-09T00:04:24.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:24 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi142 to 8544M 2024-05-09T00:04:24.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:24 smithi046 ceph-mon[30151]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T00:04:24.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:24 smithi046 ceph-mon[30151]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:24.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.611+0000 7fdc0a96d700 1 -- 172.21.15.169:0/843716551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc04104d20 msgr2=0x7fdc0410b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:24.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.611+0000 7fdc0a96d700 1 --2- 172.21.15.169:0/843716551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc04104d20 0x7fdc0410b5d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00b0a0 tx=0x7fdbfc00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:04:24.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.611+0000 7fdc0a96d700 1 -- 172.21.15.169:0/843716551 shutdown_connections 2024-05-09T00:04:24.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.611+0000 7fdc0a96d700 1 --2- 172.21.15.169:0/843716551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc04104d20 0x7fdc0410b5d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:24.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.611+0000 7fdc0a96d700 1 --2- 172.21.15.169:0/843716551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc04103250 0x7fdc04103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:24.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.611+0000 7fdc0a96d700 1 --2- 172.21.15.169:0/843716551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc04104380 0x7fdc04104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.611+0000 7fdc0a96d700 1 -- 172.21.15.169:0/843716551 >> 172.21.15.169:0/843716551 conn(0x7fdc040fe890 msgr2=0x7fdc04100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.611+0000 7fdc0a96d700 1 -- 172.21.15.169:0/843716551 shutdown_connections 2024-05-09T00:04:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.612+0000 7fdc0a96d700 1 -- 172.21.15.169:0/843716551 wait complete. 2024-05-09T00:04:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.612+0000 7fdc0a96d700 1 Processor -- start 2024-05-09T00:04:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.612+0000 7fdc0a96d700 1 -- start start 2024-05-09T00:04:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.612+0000 7fdc0a96d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc04103250 0x7fdc04076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.612+0000 7fdc0a96d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc04104380 0x7fdc0419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.612+0000 7fdc0a96d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc04104d20 0x7fdc041a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.612+0000 7fdc0a96d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0410ed30 con 0x7fdc04104380 2024-05-09T00:04:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.612+0000 7fdc0a96d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0410ebb0 con 0x7fdc04103250 2024-05-09T00:04:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.612+0000 7fdc0a96d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0410eeb0 con 0x7fdc04104d20 2024-05-09T00:04:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc08f0a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc04104d20 0x7fdc041a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc08f0a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc04104d20 0x7fdc041a1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48644/0 (socket says 172.21.15.169:48644) 2024-05-09T00:04:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc08f0a700 1 -- 172.21.15.169:0/2202367654 learned_addr learned my addr 172.21.15.169:0/2202367654 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T00:04:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdbfbfff700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc04104380 0x7fdc0419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc03fff700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc04103250 0x7fdc04076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc08f0a700 1 -- 172.21.15.169:0/2202367654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc04103250 msgr2=0x7fdc04076450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc08f0a700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc04103250 0x7fdc04076450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc08f0a700 1 -- 172.21.15.169:0/2202367654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc04104380 msgr2=0x7fdc0419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc08f0a700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc04104380 0x7fdc0419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc08f0a700 1 -- 172.21.15.169:0/2202367654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc041a2440 con 0x7fdc04104d20 2024-05-09T00:04:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc08f0a700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc04104d20 0x7fdc041a1c10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc006130 tx=0x7fdbfc004e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc01ffb700 1 -- 172.21.15.169:0/2202367654 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc021070 con 0x7fdc04104d20 2024-05-09T00:04:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc0a96d700 1 -- 172.21.15.169:0/2202367654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc041a26d0 con 0x7fdc04104d20 2024-05-09T00:04:24.611 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc01ffb700 1 -- 172.21.15.169:0/2202367654 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbfc007d00 con 0x7fdc04104d20 2024-05-09T00:04:24.611 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc01ffb700 1 -- 172.21.15.169:0/2202367654 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc01d450 con 0x7fdc04104d20 2024-05-09T00:04:24.611 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.613+0000 7fdc0a96d700 1 -- 172.21.15.169:0/2202367654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc041a2c90 con 0x7fdc04104d20 2024-05-09T00:04:24.612 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.615+0000 7fdc0a96d700 1 -- 172.21.15.169:0/2202367654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0402e5f0 con 0x7fdc04104d20 2024-05-09T00:04:24.612 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.616+0000 7fdc01ffb700 1 -- 172.21.15.169:0/2202367654 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdbfc003fb0 con 0x7fdc04104d20 2024-05-09T00:04:24.612 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.616+0000 7fdc01ffb700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdbf00b2880 0x7fdbf00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:24.612 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.616+0000 7fdc01ffb700 1 -- 172.21.15.169:0/2202367654 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7fdbfc00f020 con 0x7fdc04104d20 2024-05-09T00:04:24.612 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.617+0000 7fdc03fff700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdbf00b2880 0x7fdbf00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:24.612 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.617+0000 7fdc03fff700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdbf00b2880 0x7fdbf00b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4006fd0 tx=0x7fdbf4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:24.615 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.621+0000 7fdc01ffb700 1 -- 172.21.15.169:0/2202367654 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdbfc02a020 con 0x7fdc04104d20 2024-05-09T00:04:24.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:24 smithi142 ceph-mon[34533]: Detected new or changed devices on smithi142 2024-05-09T00:04:24.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:24 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi142 to 8544M 2024-05-09T00:04:24.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:24 smithi142 ceph-mon[34533]: osdmap e32: 5 total, 5 up, 5 in 2024-05-09T00:04:24.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:24 smithi142 ceph-mon[34533]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:24.823 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:24.829+0000 7fdc0a96d700 1 -- 172.21.15.169:0/2202367654 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdc04063c00 con 0x7fdbf00b2880 2024-05-09T00:04:25.450 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:04:25.450 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:04:25.451 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:25.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:04:25.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:04:25.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:25.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:04:25.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:04:25.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:26.529 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:26 smithi169 ceph-mon[34777]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:04:26.529 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:26 smithi169 ceph-mon[34777]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:26.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:26 smithi046 ceph-mon[30151]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:04:26.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:26 smithi046 ceph-mon[30151]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:26.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:26 smithi142 ceph-mon[34533]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:04:26.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:26 smithi142 ceph-mon[34533]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:28.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:28 smithi046 ceph-mon[30151]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:28.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:28 smithi046 ceph-mon[30151]: from='client.? 172.21.15.169:0/493512858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96a72e01-1c5f-418e-bb9b-b790d9b3059b"}]: dispatch 2024-05-09T00:04:28.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:28 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96a72e01-1c5f-418e-bb9b-b790d9b3059b"}]: dispatch 2024-05-09T00:04:28.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:28 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96a72e01-1c5f-418e-bb9b-b790d9b3059b"}]': finished 2024-05-09T00:04:28.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:28 smithi046 ceph-mon[30151]: osdmap e33: 6 total, 5 up, 6 in 2024-05-09T00:04:28.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:28 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:28.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:28 smithi046 ceph-mon[30151]: from='client.? 172.21.15.169:0/2254050108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T00:04:28.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:28 smithi169 ceph-mon[34777]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:28.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:28 smithi169 ceph-mon[34777]: from='client.? 172.21.15.169:0/493512858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96a72e01-1c5f-418e-bb9b-b790d9b3059b"}]: dispatch 2024-05-09T00:04:28.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:28 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96a72e01-1c5f-418e-bb9b-b790d9b3059b"}]: dispatch 2024-05-09T00:04:28.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:28 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96a72e01-1c5f-418e-bb9b-b790d9b3059b"}]': finished 2024-05-09T00:04:28.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:28 smithi169 ceph-mon[34777]: osdmap e33: 6 total, 5 up, 6 in 2024-05-09T00:04:28.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:28 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:28.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:28 smithi169 ceph-mon[34777]: from='client.? 172.21.15.169:0/2254050108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T00:04:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:28 smithi142 ceph-mon[34533]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:28 smithi142 ceph-mon[34533]: from='client.? 172.21.15.169:0/493512858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96a72e01-1c5f-418e-bb9b-b790d9b3059b"}]: dispatch 2024-05-09T00:04:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:28 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96a72e01-1c5f-418e-bb9b-b790d9b3059b"}]: dispatch 2024-05-09T00:04:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:28 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96a72e01-1c5f-418e-bb9b-b790d9b3059b"}]': finished 2024-05-09T00:04:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:28 smithi142 ceph-mon[34533]: osdmap e33: 6 total, 5 up, 6 in 2024-05-09T00:04:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:28 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:28.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:28 smithi142 ceph-mon[34533]: from='client.? 172.21.15.169:0/2254050108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T00:04:30.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:30 smithi046 ceph-mon[30151]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:30.599 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:30 smithi169 ceph-mon[34777]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:30.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:30 smithi142 ceph-mon[34533]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:32.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:32 smithi046 ceph-mon[30151]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:32.566 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:32 smithi169 ceph-mon[34777]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:32.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:32 smithi142 ceph-mon[34533]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:34.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:34 smithi169 ceph-mon[34777]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:34.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:34 smithi046 ceph-mon[30151]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:34.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:34 smithi142 ceph-mon[34533]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:35.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T00:04:35.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:35 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:35.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T00:04:35.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:35 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:35.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T00:04:35.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:35 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:36.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:36 smithi046 ceph-mon[30151]: Deploying daemon osd.5 on smithi169 2024-05-09T00:04:36.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:36 smithi046 ceph-mon[30151]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:36.569 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:36 smithi169 ceph-mon[34777]: Deploying daemon osd.5 on smithi169 2024-05-09T00:04:36.569 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:36 smithi169 ceph-mon[34777]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:36.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:36 smithi142 ceph-mon[34533]: Deploying daemon osd.5 on smithi169 2024-05-09T00:04:36.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:36 smithi142 ceph-mon[34533]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:38.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:38 smithi169 ceph-mon[34777]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:38.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:38 smithi046 ceph-mon[30151]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:38.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:38 smithi142 ceph-mon[34533]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:40.378 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:40 smithi169 ceph-mon[34777]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:40.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:40 smithi046 ceph-mon[30151]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:40.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:40 smithi142 ceph-mon[34533]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:41.383 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:41.383 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:41.384 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:41.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:41.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:41.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:41.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:41.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:41.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:42.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:42 smithi169 ceph-mon[34777]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:42.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:42 smithi142 ceph-mon[34533]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:42.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:42 smithi046 ceph-mon[30151]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:43.842 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.847+0000 7fdc01ffb700 1 -- 172.21.15.169:0/2202367654 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdc04063c00 con 0x7fdbf00b2880 2024-05-09T00:04:43.842 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 5 on host 'smithi169' 2024-05-09T00:04:43.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.849+0000 7fdc0a96d700 1 -- 172.21.15.169:0/2202367654 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdbf00b2880 msgr2=0x7fdbf00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:43.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.849+0000 7fdc0a96d700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdbf00b2880 0x7fdbf00b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4006fd0 tx=0x7fdbf4008040 comp rx=0 tx=0).stop 2024-05-09T00:04:43.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.849+0000 7fdc0a96d700 1 -- 172.21.15.169:0/2202367654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc04104d20 msgr2=0x7fdc041a1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:43.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.849+0000 7fdc0a96d700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc04104d20 0x7fdc041a1c10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc006130 tx=0x7fdbfc004e60 comp rx=0 tx=0).stop 2024-05-09T00:04:43.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.849+0000 7fdc0a96d700 1 -- 172.21.15.169:0/2202367654 shutdown_connections 2024-05-09T00:04:43.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.849+0000 7fdc0a96d700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdbf00b2880 0x7fdbf00b4d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:43.844 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.849+0000 7fdc0a96d700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc04104d20 0x7fdc041a1c10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:43.845 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.849+0000 7fdc0a96d700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc04103250 0x7fdc04076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:43.845 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.849+0000 7fdc0a96d700 1 --2- 172.21.15.169:0/2202367654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc04104380 0x7fdc0419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:43.845 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.849+0000 7fdc0a96d700 1 -- 172.21.15.169:0/2202367654 >> 172.21.15.169:0/2202367654 conn(0x7fdc040fe890 msgr2=0x7fdc041075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:43.845 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.849+0000 7fdc0a96d700 1 -- 172.21.15.169:0/2202367654 shutdown_connections 2024-05-09T00:04:43.845 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:04:43.849+0000 7fdc0a96d700 1 -- 172.21.15.169:0/2202367654 wait complete. 2024-05-09T00:04:44.427 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:44 smithi169 ceph-mon[34777]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:44.427 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:44.428 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:44.428 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:44.428 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:44.428 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:44 smithi169 ceph-mon[34777]: from='osd.5 [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T00:04:44.428 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:44 smithi169 ceph-mon[34777]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T00:04:44.452 DEBUG:teuthology.orchestra.run.smithi169:osd.5> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.5.service 2024-05-09T00:04:44.454 INFO:tasks.cephadm:Deploying osd.6 on smithi046 with /dev/vg_nvme/lv_2... 2024-05-09T00:04:44.455 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T00:04:44.691 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:44 smithi142 ceph-mon[34533]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:44.691 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:44.691 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:44.692 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:44.692 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:44.692 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:44 smithi142 ceph-mon[34533]: from='osd.5 [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T00:04:44.692 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:44 smithi142 ceph-mon[34533]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T00:04:44.715 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:04:44.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:44 smithi046 ceph-mon[30151]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:44.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:44.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:44.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:44.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:44.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:44 smithi046 ceph-mon[30151]: from='osd.5 [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T00:04:44.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:44 smithi046 ceph-mon[30151]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T00:04:46.211 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:45 smithi046 ceph-mon[30151]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T00:04:46.211 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:45 smithi046 ceph-mon[30151]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T00:04:46.211 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:45 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:46.211 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:45 smithi046 ceph-mon[30151]: from='osd.5 [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:04:46.211 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:45 smithi046 ceph-mon[30151]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:04:46.211 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:45 smithi046 ceph-mon[30151]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:46.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:45 smithi169 ceph-mon[34777]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T00:04:46.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:45 smithi169 ceph-mon[34777]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T00:04:46.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:45 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:46.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:45 smithi169 ceph-mon[34777]: from='osd.5 [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:04:46.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:45 smithi169 ceph-mon[34777]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:04:46.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:45 smithi169 ceph-mon[34777]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:46.351 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:04:45 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:04:45.958+0000 7fd83a99d700 -1 osd.5 0 waiting for initial osdmap 2024-05-09T00:04:46.351 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:04:45 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:04:45.965+0000 7fd836752700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:04:46.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:45 smithi142 ceph-mon[34533]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T00:04:46.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:45 smithi142 ceph-mon[34533]: osdmap e34: 6 total, 5 up, 6 in 2024-05-09T00:04:46.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:45 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:46.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:45 smithi142 ceph-mon[34533]: from='osd.5 [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:04:46.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:45 smithi142 ceph-mon[34533]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:04:46.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:45 smithi142 ceph-mon[34533]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T00:04:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T00:04:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T00:04:46.776 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-05-09T00:04:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-05-09T00:04:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00843346 s, 1.2 GB/s 2024-05-09T00:04:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-05-09T00:04:47.042 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:46 smithi169 ceph-mon[34777]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T00:04:47.042 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:46 smithi169 ceph-mon[34777]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T00:04:47.042 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:46 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:47.043 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:46 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:47.043 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:46 smithi169 ceph-mon[34777]: osd.5 [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713] boot 2024-05-09T00:04:47.043 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:46 smithi169 ceph-mon[34777]: osdmap e36: 6 total, 6 up, 6 in 2024-05-09T00:04:47.043 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:46 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:47.146 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:46 smithi046 ceph-mon[30151]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T00:04:47.146 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:46 smithi046 ceph-mon[30151]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T00:04:47.146 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:47.146 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:47.146 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:46 smithi046 ceph-mon[30151]: osd.5 [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713] boot 2024-05-09T00:04:47.146 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:46 smithi046 ceph-mon[30151]: osdmap e36: 6 total, 6 up, 6 in 2024-05-09T00:04:47.146 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:47.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:46 smithi142 ceph-mon[34533]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T00:04:47.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:46 smithi142 ceph-mon[34533]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T00:04:47.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:46 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:47.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:46 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:47.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:46 smithi142 ceph-mon[34533]: osd.5 [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713] boot 2024-05-09T00:04:47.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:46 smithi142 ceph-mon[34533]: osdmap e36: 6 total, 6 up, 6 in 2024-05-09T00:04:47.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:46 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T00:04:47.481 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_2 2024-05-09T00:04:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:04:48.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T00:04:48.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T00:04:48.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:04:48.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: Detected new or changed devices on smithi169 2024-05-09T00:04:48.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:04:48.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:04:48.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi169 to 8544M 2024-05-09T00:04:48.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:48.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:48.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:48.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:48.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:48.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:47 smithi046 ceph-mon[30151]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T00:04:48.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: purged_snaps scrub starts 2024-05-09T00:04:48.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: purged_snaps scrub ok 2024-05-09T00:04:48.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:04:48.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: Detected new or changed devices on smithi169 2024-05-09T00:04:48.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:04:48.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:04:48.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi169 to 8544M 2024-05-09T00:04:48.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:48.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:48.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:48.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:48.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:48.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:47 smithi169 ceph-mon[34777]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T00:04:48.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: purged_snaps scrub starts 2024-05-09T00:04:48.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: purged_snaps scrub ok 2024-05-09T00:04:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:04:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: Detected new or changed devices on smithi169 2024-05-09T00:04:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:04:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:04:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi169 to 8544M 2024-05-09T00:04:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:04:48.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:48.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:04:48.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:04:48.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:47 smithi142 ceph-mon[34533]: osdmap e37: 6 total, 6 up, 6 in 2024-05-09T00:04:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.433+0000 7fbbe7e39700 1 -- 172.21.15.46:0/4124197927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe0109ff0 msgr2=0x7fbbe010c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.433+0000 7fbbe7e39700 1 --2- 172.21.15.46:0/4124197927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe0109ff0 0x7fbbe010c500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009990 tx=0x7fbbd8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:04:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.433+0000 7fbbe7e39700 1 -- 172.21.15.46:0/4124197927 shutdown_connections 2024-05-09T00:04:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.433+0000 7fbbe7e39700 1 --2- 172.21.15.46:0/4124197927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe0109ff0 0x7fbbe010c500 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.433+0000 7fbbe7e39700 1 --2- 172.21.15.46:0/4124197927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe006c2b0 0x7fbbe0106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.433+0000 7fbbe7e39700 1 --2- 172.21.15.46:0/4124197927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe01074a0 0x7fbbe0109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.433+0000 7fbbe7e39700 1 -- 172.21.15.46:0/4124197927 >> 172.21.15.46:0/4124197927 conn(0x7fbbe00fe880 msgr2=0x7fbbe0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:04:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.433+0000 7fbbe7e39700 1 -- 172.21.15.46:0/4124197927 shutdown_connections 2024-05-09T00:04:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.433+0000 7fbbe7e39700 1 -- 172.21.15.46:0/4124197927 wait complete. 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.434+0000 7fbbe7e39700 1 Processor -- start 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.434+0000 7fbbe7e39700 1 -- start start 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe7e39700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe006c2b0 0x7fbbe0104720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe7e39700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe01074a0 0x7fbbe0102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe7e39700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe0109ff0 0x7fbbe0103310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe7e39700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe010fe90 con 0x7fbbe0109ff0 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe7e39700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe010fd10 con 0x7fbbe006c2b0 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe7e39700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0110010 con 0x7fbbe01074a0 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe63d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe0109ff0 0x7fbbe0103310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe63d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe0109ff0 0x7fbbe0103310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49500/0 (socket says 172.21.15.46:49500) 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe63d6700 1 -- 172.21.15.46:0/2491311166 learned_addr learned my addr 172.21.15.46:0/2491311166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe5bd5700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe006c2b0 0x7fbbe0104720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe53d4700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe01074a0 0x7fbbe0102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe63d6700 1 -- 172.21.15.46:0/2491311166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe01074a0 msgr2=0x7fbbe0102dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe63d6700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe01074a0 0x7fbbe0102dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe63d6700 1 -- 172.21.15.46:0/2491311166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe006c2b0 msgr2=0x7fbbe0104720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:04:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe63d6700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe006c2b0 0x7fbbe0104720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:04:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe63d6700 1 -- 172.21.15.46:0/2491311166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe0103a70 con 0x7fbbe0109ff0 2024-05-09T00:04:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe53d4700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe01074a0 0x7fbbe0102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:04:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.435+0000 7fbbe63d6700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe0109ff0 0x7fbbe0103310 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80038a0 tx=0x7fbbd80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.436+0000 7fbbdeffd700 1 -- 172.21.15.46:0/2491311166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8027070 con 0x7fbbe0109ff0 2024-05-09T00:04:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.436+0000 7fbbdeffd700 1 -- 172.21.15.46:0/2491311166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd8004100 con 0x7fbbe0109ff0 2024-05-09T00:04:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.436+0000 7fbbdeffd700 1 -- 172.21.15.46:0/2491311166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800f460 con 0x7fbbe0109ff0 2024-05-09T00:04:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.436+0000 7fbbe7e39700 1 -- 172.21.15.46:0/2491311166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe01a4a80 con 0x7fbbe0109ff0 2024-05-09T00:04:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.436+0000 7fbbe7e39700 1 -- 172.21.15.46:0/2491311166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe01a4e20 con 0x7fbbe0109ff0 2024-05-09T00:04:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.437+0000 7fbbe7e39700 1 -- 172.21.15.46:0/2491311166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe00512c0 con 0x7fbbe0109ff0 2024-05-09T00:04:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.438+0000 7fbbdeffd700 1 -- 172.21.15.46:0/2491311166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbbd802a070 con 0x7fbbe0109ff0 2024-05-09T00:04:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.439+0000 7fbbdeffd700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbbc80b2900 0x7fbbc80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:04:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.439+0000 7fbbe5bd5700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbbc80b2900 0x7fbbc80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:04:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.439+0000 7fbbdeffd700 1 -- 172.21.15.46:0/2491311166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7fbbd80e23c0 con 0x7fbbe0109ff0 2024-05-09T00:04:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.440+0000 7fbbe5bd5700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbbc80b2900 0x7fbbc80b4e10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbe006bfb0 tx=0x7fbbd000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:04:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.444+0000 7fbbdeffd700 1 -- 172.21.15.46:0/2491311166 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbbd8013070 con 0x7fbbe0109ff0 2024-05-09T00:04:49.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:04:49.669+0000 7fbbe7e39700 1 -- 172.21.15.46:0/2491311166 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fbbe0063c00 con 0x7fbbc80b2900 2024-05-09T00:04:50.302 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:49 smithi046 ceph-mon[30151]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T00:04:50.302 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:49 smithi046 ceph-mon[30151]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:04:50.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:49 smithi046 ceph-mon[30151]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:04:50.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:04:50.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:04:50.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:50.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:49 smithi169 ceph-mon[34777]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T00:04:50.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:49 smithi169 ceph-mon[34777]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:04:50.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:49 smithi169 ceph-mon[34777]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:04:50.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:04:50.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:04:50.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:50.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:49 smithi142 ceph-mon[34533]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T00:04:50.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:49 smithi142 ceph-mon[34533]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:04:50.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:49 smithi142 ceph-mon[34533]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:04:50.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:04:50.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:04:50.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:04:52.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:52 smithi046 ceph-mon[30151]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:04:52.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:52 smithi169 ceph-mon[34777]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:04:52.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:52 smithi142 ceph-mon[34533]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:04:53.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:53 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/340187874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b459542-0e56-4369-b542-e36c46a18f05"}]: dispatch 2024-05-09T00:04:53.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:53 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/340187874' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b459542-0e56-4369-b542-e36c46a18f05"}]': finished 2024-05-09T00:04:53.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:53 smithi046 ceph-mon[30151]: osdmap e39: 7 total, 6 up, 7 in 2024-05-09T00:04:53.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:53 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:04:53.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:53 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/340187874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b459542-0e56-4369-b542-e36c46a18f05"}]: dispatch 2024-05-09T00:04:53.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:53 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/340187874' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b459542-0e56-4369-b542-e36c46a18f05"}]': finished 2024-05-09T00:04:53.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:53 smithi169 ceph-mon[34777]: osdmap e39: 7 total, 6 up, 7 in 2024-05-09T00:04:53.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:53 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:04:53.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:53 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/340187874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b459542-0e56-4369-b542-e36c46a18f05"}]: dispatch 2024-05-09T00:04:53.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:53 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/340187874' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b459542-0e56-4369-b542-e36c46a18f05"}]': finished 2024-05-09T00:04:53.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:53 smithi142 ceph-mon[34533]: osdmap e39: 7 total, 6 up, 7 in 2024-05-09T00:04:53.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:53 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:04:54.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:54 smithi046 ceph-mon[30151]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:04:54.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:54 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3499418754' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T00:04:54.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:54 smithi169 ceph-mon[34777]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:04:54.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:54 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3499418754' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T00:04:54.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:54 smithi142 ceph-mon[34533]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:04:54.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:54 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3499418754' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T00:04:56.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:56 smithi046 ceph-mon[30151]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-09T00:04:56.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:56 smithi169 ceph-mon[34777]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-09T00:04:56.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:56 smithi142 ceph-mon[34533]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-09T00:04:58.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:04:58 smithi046 ceph-mon[30151]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-09T00:04:58.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:04:58 smithi169 ceph-mon[34777]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-09T00:04:58.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:04:58 smithi142 ceph-mon[34533]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-09T00:05:00.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:00 smithi046 ceph-mon[30151]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:05:00.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T00:05:00.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:00.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:00 smithi169 ceph-mon[34777]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:05:00.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T00:05:00.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:00.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:00 smithi142 ceph-mon[34533]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:05:00.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T00:05:00.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:01.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:01 smithi046 ceph-mon[30151]: Deploying daemon osd.6 on smithi046 2024-05-09T00:05:01.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:01 smithi169 ceph-mon[34777]: Deploying daemon osd.6 on smithi046 2024-05-09T00:05:01.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:01 smithi142 ceph-mon[34533]: Deploying daemon osd.6 on smithi046 2024-05-09T00:05:02.427 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:02 smithi046 ceph-mon[30151]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:05:02.588 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:02 smithi142 ceph-mon[34533]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:05:02.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:02 smithi169 ceph-mon[34777]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:05:04.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:04 smithi046 ceph-mon[30151]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-09T00:05:04.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:04 smithi169 ceph-mon[34777]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-09T00:05:04.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:04 smithi142 ceph-mon[34533]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-09T00:05:06.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:06 smithi046 ceph-mon[30151]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T00:05:06.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:06 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:06.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:06 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:06.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:06 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:06.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:06 smithi169 ceph-mon[34777]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T00:05:06.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:06 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:06.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:06 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:06.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:06 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:06.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:06 smithi142 ceph-mon[34533]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T00:05:06.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:06 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:06.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:06 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:06.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:06 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:08.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:08 smithi046 ceph-mon[30151]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:05:08.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:08 smithi169 ceph-mon[34777]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:05:08.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:08 smithi142 ceph-mon[34533]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:05:09.826 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 6 on host 'smithi046' 2024-05-09T00:05:09.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.832+0000 7fbbdeffd700 1 -- 172.21.15.46:0/2491311166 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbbe0063c00 con 0x7fbbc80b2900 2024-05-09T00:05:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.835+0000 7fbbe7e39700 1 -- 172.21.15.46:0/2491311166 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbbc80b2900 msgr2=0x7fbbc80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.835+0000 7fbbe7e39700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbbc80b2900 0x7fbbc80b4e10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbe006bfb0 tx=0x7fbbd000d040 comp rx=0 tx=0).stop 2024-05-09T00:05:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.835+0000 7fbbe7e39700 1 -- 172.21.15.46:0/2491311166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe0109ff0 msgr2=0x7fbbe0103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.835+0000 7fbbe7e39700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe0109ff0 0x7fbbe0103310 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80038a0 tx=0x7fbbd80047b0 comp rx=0 tx=0).stop 2024-05-09T00:05:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.835+0000 7fbbe7e39700 1 -- 172.21.15.46:0/2491311166 shutdown_connections 2024-05-09T00:05:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.835+0000 7fbbe7e39700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbbc80b2900 0x7fbbc80b4e10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.835+0000 7fbbe7e39700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe0109ff0 0x7fbbe0103310 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.835+0000 7fbbe7e39700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe006c2b0 0x7fbbe0104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.835+0000 7fbbe7e39700 1 --2- 172.21.15.46:0/2491311166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe01074a0 0x7fbbe0102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.835+0000 7fbbe7e39700 1 -- 172.21.15.46:0/2491311166 >> 172.21.15.46:0/2491311166 conn(0x7fbbe00fe880 msgr2=0x7fbbe01055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.835+0000 7fbbe7e39700 1 -- 172.21.15.46:0/2491311166 shutdown_connections 2024-05-09T00:05:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:05:09.835+0000 7fbbe7e39700 1 -- 172.21.15.46:0/2491311166 wait complete. 2024-05-09T00:05:10.533 DEBUG:teuthology.orchestra.run.smithi046:osd.6> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.6.service 2024-05-09T00:05:10.536 INFO:tasks.cephadm:Deploying osd.7 on smithi142 with /dev/vg_nvme/lv_2... 2024-05-09T00:05:10.536 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T00:05:10.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:10 smithi046 ceph-mon[30151]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:05:10.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:10.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:10.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:10 smithi046 ceph-mon[30151]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T00:05:10.563 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:10 smithi142 ceph-mon[34533]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:05:10.563 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:10.563 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:10.563 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:10 smithi142 ceph-mon[34533]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T00:05:10.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:10 smithi169 ceph-mon[34777]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:05:10.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:10.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:10.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:10 smithi169 ceph-mon[34777]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T00:05:10.783 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:05:11.901 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:11 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:11.901 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:11 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:11.901 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:11 smithi046 ceph-mon[30151]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T00:05:11.901 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:11 smithi046 ceph-mon[30151]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T00:05:11.901 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:11 smithi046 ceph-mon[30151]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T00:05:11.901 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:11 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:11.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:11 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:11.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:11 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:11.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:11 smithi142 ceph-mon[34533]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T00:05:11.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:11 smithi142 ceph-mon[34533]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T00:05:11.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:11 smithi142 ceph-mon[34533]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T00:05:11.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:11 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:12.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:11 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:12.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:11 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:12.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:11 smithi169 ceph-mon[34777]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T00:05:12.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:11 smithi169 ceph-mon[34777]: osdmap e40: 7 total, 6 up, 7 in 2024-05-09T00:05:12.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:11 smithi169 ceph-mon[34777]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T00:05:12.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:11 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:12.659 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:05:12 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:05:12.322+0000 7fb23d79f700 -1 osd.6 0 waiting for initial osdmap 2024-05-09T00:05:12.659 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:05:12 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:05:12.335+0000 7fb23554c700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:05:12.927 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:12 smithi046 ceph-mon[30151]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:05:12.927 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:12 smithi046 ceph-mon[30151]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T00:05:12.927 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:12 smithi046 ceph-mon[30151]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T00:05:12.927 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:12 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:12.927 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:12 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:12.927 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:12 smithi046 ceph-mon[30151]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' 2024-05-09T00:05:12.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:12 smithi142 ceph-mon[34533]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:05:12.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:12 smithi142 ceph-mon[34533]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T00:05:12.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:12 smithi142 ceph-mon[34533]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T00:05:12.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:12 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:12.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:12 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:12.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:12 smithi142 ceph-mon[34533]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' 2024-05-09T00:05:12.984 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T00:05:12.984 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T00:05:12.984 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-05-09T00:05:12.984 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-05-09T00:05:12.984 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112227 s, 934 MB/s 2024-05-09T00:05:12.984 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-05-09T00:05:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:12 smithi169 ceph-mon[34777]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:05:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:12 smithi169 ceph-mon[34777]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T00:05:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:12 smithi169 ceph-mon[34777]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T00:05:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:12 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:12 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:12 smithi169 ceph-mon[34777]: from='osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482]' entity='osd.6' 2024-05-09T00:05:13.628 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2024-05-09T00:05:13.906 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:05:13.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:13 smithi142 ceph-mon[34533]: purged_snaps scrub starts 2024-05-09T00:05:13.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:13 smithi142 ceph-mon[34533]: purged_snaps scrub ok 2024-05-09T00:05:13.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:13 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:13.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:13 smithi142 ceph-mon[34533]: osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482] boot 2024-05-09T00:05:13.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:13 smithi142 ceph-mon[34533]: osdmap e42: 7 total, 7 up, 7 in 2024-05-09T00:05:13.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:13 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:14.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:13 smithi046 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T00:05:14.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:13 smithi046 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T00:05:14.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:13 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:14.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:13 smithi046 ceph-mon[30151]: osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482] boot 2024-05-09T00:05:14.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:13 smithi046 ceph-mon[30151]: osdmap e42: 7 total, 7 up, 7 in 2024-05-09T00:05:14.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:13 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:14.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:13 smithi169 ceph-mon[34777]: purged_snaps scrub starts 2024-05-09T00:05:14.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:13 smithi169 ceph-mon[34777]: purged_snaps scrub ok 2024-05-09T00:05:14.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:13 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:14.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:13 smithi169 ceph-mon[34777]: osd.6 [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482] boot 2024-05-09T00:05:14.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:13 smithi169 ceph-mon[34777]: osdmap e42: 7 total, 7 up, 7 in 2024-05-09T00:05:14.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:13 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T00:05:14.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:14 smithi142 ceph-mon[34533]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:05:15.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:14 smithi046 ceph-mon[30151]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:05:15.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:14 smithi169 ceph-mon[34777]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-09T00:05:15.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.646+0000 7fd079efa700 1 -- 172.21.15.142:0/4209871736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd074103250 msgr2=0x7fd074103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:15.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.646+0000 7fd079efa700 1 --2- 172.21.15.142:0/4209871736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd074103250 0x7fd074103660 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd06400b0a0 tx=0x7fd06400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:05:15.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.646+0000 7fd079efa700 1 -- 172.21.15.142:0/4209871736 shutdown_connections 2024-05-09T00:05:15.640 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.646+0000 7fd079efa700 1 --2- 172.21.15.142:0/4209871736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd074104d20 0x7fd07410b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:15.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.646+0000 7fd079efa700 1 --2- 172.21.15.142:0/4209871736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd074103250 0x7fd074103660 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:15.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.646+0000 7fd079efa700 1 --2- 172.21.15.142:0/4209871736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd074104380 0x7fd074104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:15.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.646+0000 7fd079efa700 1 -- 172.21.15.142:0/4209871736 >> 172.21.15.142:0/4209871736 conn(0x7fd0740fe890 msgr2=0x7fd074100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:15.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.647+0000 7fd079efa700 1 -- 172.21.15.142:0/4209871736 shutdown_connections 2024-05-09T00:05:15.641 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.647+0000 7fd079efa700 1 -- 172.21.15.142:0/4209871736 wait complete. 2024-05-09T00:05:15.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.648+0000 7fd079efa700 1 Processor -- start 2024-05-09T00:05:15.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.648+0000 7fd079efa700 1 -- start start 2024-05-09T00:05:15.643 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.648+0000 7fd079efa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd074103250 0x7fd074076470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:15.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.648+0000 7fd079efa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd074104380 0x7fd07419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:15.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.648+0000 7fd079efa700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd074104d20 0x7fd0741a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:15.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.648+0000 7fd079efa700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd07410ec90 con 0x7fd074104380 2024-05-09T00:05:15.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.648+0000 7fd079efa700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd07410eb10 con 0x7fd074103250 2024-05-09T00:05:15.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.648+0000 7fd079efa700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd07410ee10 con 0x7fd074104d20 2024-05-09T00:05:15.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.648+0000 7fd0737fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd074103250 0x7fd074076470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:15.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.648+0000 7fd0737fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd074103250 0x7fd074076470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:34066/0 (socket says 172.21.15.142:34066) 2024-05-09T00:05:15.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.648+0000 7fd0737fe700 1 -- 172.21.15.142:0/838288943 learned_addr learned my addr 172.21.15.142:0/838288943 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-09T00:05:15.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd072ffd700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd074104380 0x7fd07419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:15.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd073fff700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd074104d20 0x7fd0741a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:15.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd0737fe700 1 -- 172.21.15.142:0/838288943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd074104d20 msgr2=0x7fd0741a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:15.644 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd0737fe700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd074104d20 0x7fd0741a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:15.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd0737fe700 1 -- 172.21.15.142:0/838288943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd074104380 msgr2=0x7fd07419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:15.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd0737fe700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd074104380 0x7fd07419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:15.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd0737fe700 1 -- 172.21.15.142:0/838288943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0741a2430 con 0x7fd074103250 2024-05-09T00:05:15.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd0737fe700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd074103250 0x7fd074076470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd06400b780 tx=0x7fd064004d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:15.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd070ff9700 1 -- 172.21.15.142:0/838288943 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064021070 con 0x7fd074103250 2024-05-09T00:05:15.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd070ff9700 1 -- 172.21.15.142:0/838288943 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd064007b80 con 0x7fd074103250 2024-05-09T00:05:15.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd079efa700 1 -- 172.21.15.142:0/838288943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0741a26c0 con 0x7fd074103250 2024-05-09T00:05:15.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd070ff9700 1 -- 172.21.15.142:0/838288943 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0640052c0 con 0x7fd074103250 2024-05-09T00:05:15.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.649+0000 7fd079efa700 1 -- 172.21.15.142:0/838288943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0741a2c00 con 0x7fd074103250 2024-05-09T00:05:15.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.650+0000 7fd079efa700 1 -- 172.21.15.142:0/838288943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0740512c0 con 0x7fd074103250 2024-05-09T00:05:15.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.652+0000 7fd070ff9700 1 -- 172.21.15.142:0/838288943 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd064024070 con 0x7fd074103250 2024-05-09T00:05:15.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.652+0000 7fd070ff9700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd05c0b2880 0x7fd05c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:15.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.652+0000 7fd070ff9700 1 -- 172.21.15.142:0/838288943 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7fd06400f020 con 0x7fd074103250 2024-05-09T00:05:15.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.652+0000 7fd072ffd700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd05c0b2880 0x7fd05c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:15.646 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.653+0000 7fd072ffd700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd05c0b2880 0x7fd05c0b4d90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd074075e20 tx=0x7fd068009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:15.649 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.656+0000 7fd070ff9700 1 -- 172.21.15.142:0/838288943 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd06402b020 con 0x7fd074103250 2024-05-09T00:05:15.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:15.871+0000 7fd079efa700 1 -- 172.21.15.142:0/838288943 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd074063c00 con 0x7fd05c0b2880 2024-05-09T00:05:15.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T00:05:15.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:15.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:15.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:15.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:15.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:15.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:15.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:15.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:15.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:15.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:15.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:15.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:15.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:16.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T00:05:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:16.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:16.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:16.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:16.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:16.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:16.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: osdmap e43: 7 total, 7 up, 7 in 2024-05-09T00:05:16.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:16.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:16.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:16.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:16.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:16.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:16.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:16.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:16.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:16.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:16.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:16.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:16.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:16.813 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:16 smithi142 ceph-mon[34533]: Detected new or changed devices on smithi046 2024-05-09T00:05:16.813 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:16 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi046 to 5697M 2024-05-09T00:05:16.813 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:16 smithi142 ceph-mon[34533]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:16.814 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:05:16.814 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:05:16.814 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:16 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:17.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:16 smithi046 ceph-mon[30151]: Detected new or changed devices on smithi046 2024-05-09T00:05:17.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:16 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi046 to 5697M 2024-05-09T00:05:17.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:16 smithi046 ceph-mon[30151]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:17.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:05:17.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:05:17.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:16 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:17.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:16 smithi169 ceph-mon[34777]: Detected new or changed devices on smithi046 2024-05-09T00:05:17.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:16 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi046 to 5697M 2024-05-09T00:05:17.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:16 smithi169 ceph-mon[34777]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:17.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:05:17.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:05:17.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:16 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:17.838 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:17 smithi142 ceph-mon[34533]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:05:18.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:17 smithi046 ceph-mon[30151]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:05:18.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:17 smithi169 ceph-mon[34777]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:05:18.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:18 smithi142 ceph-mon[34533]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:19.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:18 smithi046 ceph-mon[30151]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:19.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:18 smithi169 ceph-mon[34777]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:19.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:19 smithi142 ceph-mon[34533]: from='client.? 172.21.15.142:0/4197918918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "786b4517-4cbd-4fa3-b5a0-39cccf839893"}]: dispatch 2024-05-09T00:05:19.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:19 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "786b4517-4cbd-4fa3-b5a0-39cccf839893"}]: dispatch 2024-05-09T00:05:19.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:19 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "786b4517-4cbd-4fa3-b5a0-39cccf839893"}]': finished 2024-05-09T00:05:19.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:19 smithi142 ceph-mon[34533]: osdmap e44: 8 total, 7 up, 8 in 2024-05-09T00:05:19.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:19 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:19.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:19 smithi142 ceph-mon[34533]: from='client.? 172.21.15.142:0/3742401466' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T00:05:20.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:19 smithi046 ceph-mon[30151]: from='client.? 172.21.15.142:0/4197918918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "786b4517-4cbd-4fa3-b5a0-39cccf839893"}]: dispatch 2024-05-09T00:05:20.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:19 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "786b4517-4cbd-4fa3-b5a0-39cccf839893"}]: dispatch 2024-05-09T00:05:20.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:19 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "786b4517-4cbd-4fa3-b5a0-39cccf839893"}]': finished 2024-05-09T00:05:20.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:19 smithi046 ceph-mon[30151]: osdmap e44: 8 total, 7 up, 8 in 2024-05-09T00:05:20.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:19 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:20.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:19 smithi046 ceph-mon[30151]: from='client.? 172.21.15.142:0/3742401466' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T00:05:20.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:19 smithi169 ceph-mon[34777]: from='client.? 172.21.15.142:0/4197918918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "786b4517-4cbd-4fa3-b5a0-39cccf839893"}]: dispatch 2024-05-09T00:05:20.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:19 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "786b4517-4cbd-4fa3-b5a0-39cccf839893"}]: dispatch 2024-05-09T00:05:20.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:19 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "786b4517-4cbd-4fa3-b5a0-39cccf839893"}]': finished 2024-05-09T00:05:20.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:19 smithi169 ceph-mon[34777]: osdmap e44: 8 total, 7 up, 8 in 2024-05-09T00:05:20.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:19 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:20.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:19 smithi169 ceph-mon[34777]: from='client.? 172.21.15.142:0/3742401466' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T00:05:20.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:20 smithi142 ceph-mon[34533]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:21.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:20 smithi046 ceph-mon[30151]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:21.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:20 smithi169 ceph-mon[34777]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:22.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:22 smithi142 ceph-mon[34533]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:23.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:22 smithi046 ceph-mon[30151]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:23.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:22 smithi169 ceph-mon[34777]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:24.976 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:24 smithi142 ceph-mon[34533]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:25.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:24 smithi046 ceph-mon[30151]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:25.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:24 smithi169 ceph-mon[34777]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:26.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:26 smithi142 ceph-mon[34533]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:26.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:26 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T00:05:26.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:26 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:27.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:26 smithi046 ceph-mon[30151]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:27.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:26 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T00:05:27.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:26 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:27.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:26 smithi169 ceph-mon[34777]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:27.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:26 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T00:05:27.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:26 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:28.032 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:27 smithi142 ceph-mon[34533]: Deploying daemon osd.7 on smithi142 2024-05-09T00:05:28.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:27 smithi046 ceph-mon[30151]: Deploying daemon osd.7 on smithi142 2024-05-09T00:05:28.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:27 smithi169 ceph-mon[34777]: Deploying daemon osd.7 on smithi142 2024-05-09T00:05:29.003 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:28 smithi142 ceph-mon[34533]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:29.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:28 smithi046 ceph-mon[30151]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:29.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:28 smithi169 ceph-mon[34777]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:30.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:30 smithi142 ceph-mon[34533]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:31.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:30 smithi046 ceph-mon[30151]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:31.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:30 smithi169 ceph-mon[34777]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:32.862 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:32 smithi142 ceph-mon[34533]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:32.862 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:32.862 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:32.862 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:33.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:32 smithi046 ceph-mon[30151]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:33.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:33.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:33.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:33.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:32 smithi169 ceph-mon[34777]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:33.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:33.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:33.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:34.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:34 smithi142 ceph-mon[34533]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:35.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:34 smithi046 ceph-mon[30151]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:35.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:34 smithi169 ceph-mon[34777]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:35.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.895+0000 7fd070ff9700 1 -- 172.21.15.142:0/838288943 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd074063c00 con 0x7fd05c0b2880 2024-05-09T00:05:35.890 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 7 on host 'smithi142' 2024-05-09T00:05:35.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.897+0000 7fd079efa700 1 -- 172.21.15.142:0/838288943 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd05c0b2880 msgr2=0x7fd05c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:35.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.897+0000 7fd079efa700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd05c0b2880 0x7fd05c0b4d90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd074075e20 tx=0x7fd068009260 comp rx=0 tx=0).stop 2024-05-09T00:05:35.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.897+0000 7fd079efa700 1 -- 172.21.15.142:0/838288943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd074103250 msgr2=0x7fd074076470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:35.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.897+0000 7fd079efa700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd074103250 0x7fd074076470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd06400b780 tx=0x7fd064004d70 comp rx=0 tx=0).stop 2024-05-09T00:05:35.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.897+0000 7fd079efa700 1 -- 172.21.15.142:0/838288943 shutdown_connections 2024-05-09T00:05:35.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.897+0000 7fd079efa700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd05c0b2880 0x7fd05c0b4d90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:35.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.897+0000 7fd079efa700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd074104d20 0x7fd0741a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:35.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.898+0000 7fd079efa700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd074103250 0x7fd074076470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:35.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.898+0000 7fd079efa700 1 --2- 172.21.15.142:0/838288943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd074104380 0x7fd07419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:35.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.898+0000 7fd079efa700 1 -- 172.21.15.142:0/838288943 >> 172.21.15.142:0/838288943 conn(0x7fd0740fe890 msgr2=0x7fd074107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:35.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.898+0000 7fd079efa700 1 -- 172.21.15.142:0/838288943 shutdown_connections 2024-05-09T00:05:35.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:05:35.898+0000 7fd079efa700 1 -- 172.21.15.142:0/838288943 wait complete. 2024-05-09T00:05:36.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:36.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:36.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:36 smithi142 ceph-mon[34533]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:36.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:36 smithi142 ceph-mon[34533]: from='osd.7 [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T00:05:36.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:36 smithi142 ceph-mon[34533]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T00:05:36.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:36.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:36.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:36.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:36 smithi046 ceph-mon[30151]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:36.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:36 smithi046 ceph-mon[30151]: from='osd.7 [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T00:05:36.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:36 smithi046 ceph-mon[30151]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T00:05:36.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:36.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:36.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:36.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:36 smithi169 ceph-mon[34777]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:36.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:36 smithi169 ceph-mon[34777]: from='osd.7 [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T00:05:36.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:36 smithi169 ceph-mon[34777]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T00:05:36.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:36.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:36.555 DEBUG:teuthology.orchestra.run.smithi142:osd.7> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.7.service 2024-05-09T00:05:36.557 INFO:tasks.cephadm:Deploying osd.8 on smithi169 with /dev/vg_nvme/lv_2... 2024-05-09T00:05:36.558 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T00:05:36.808 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:05:38.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:37 smithi169 ceph-mon[34777]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T00:05:38.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:37 smithi169 ceph-mon[34777]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T00:05:38.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:37 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:38.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:37 smithi169 ceph-mon[34777]: from='osd.7 [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:05:38.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:37 smithi169 ceph-mon[34777]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:05:38.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:37 smithi169 ceph-mon[34777]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:38.132 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:37 smithi142 ceph-mon[34533]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T00:05:38.133 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:37 smithi142 ceph-mon[34533]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T00:05:38.133 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:37 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:38.133 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:37 smithi142 ceph-mon[34533]: from='osd.7 [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:05:38.133 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:37 smithi142 ceph-mon[34533]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:05:38.133 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:37 smithi142 ceph-mon[34533]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:38.133 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:05:37 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:05:37.854+0000 7f7d48d7f700 -1 osd.7 0 waiting for initial osdmap 2024-05-09T00:05:38.133 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:05:37 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:05:37.864+0000 7f7d44b34700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:05:38.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:37 smithi046 ceph-mon[30151]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T00:05:38.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:37 smithi046 ceph-mon[30151]: osdmap e45: 8 total, 7 up, 8 in 2024-05-09T00:05:38.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:37 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:38.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:37 smithi046 ceph-mon[30151]: from='osd.7 [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:05:38.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:37 smithi046 ceph-mon[30151]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:05:38.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:37 smithi046 ceph-mon[30151]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T00:05:38.850 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T00:05:38.851 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T00:05:38.851 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-05-09T00:05:38.851 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-05-09T00:05:38.851 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131698 s, 796 MB/s 2024-05-09T00:05:38.851 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-05-09T00:05:39.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:38 smithi142 ceph-mon[34533]: purged_snaps scrub starts 2024-05-09T00:05:39.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:38 smithi142 ceph-mon[34533]: purged_snaps scrub ok 2024-05-09T00:05:39.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:38 smithi142 ceph-mon[34533]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-09T00:05:39.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:38 smithi142 ceph-mon[34533]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T00:05:39.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:38 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:39.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:38 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:39.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:38 smithi142 ceph-mon[34533]: osd.7 [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607] boot 2024-05-09T00:05:39.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:38 smithi169 ceph-mon[34777]: purged_snaps scrub starts 2024-05-09T00:05:39.188 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:38 smithi169 ceph-mon[34777]: purged_snaps scrub ok 2024-05-09T00:05:39.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:38 smithi169 ceph-mon[34777]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-09T00:05:39.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:38 smithi169 ceph-mon[34777]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T00:05:39.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:38 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:39.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:38 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:39.189 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:38 smithi169 ceph-mon[34777]: osd.7 [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607] boot 2024-05-09T00:05:39.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:38 smithi046 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T00:05:39.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:38 smithi046 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T00:05:39.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:38 smithi046 ceph-mon[30151]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-09T00:05:39.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:38 smithi046 ceph-mon[30151]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T00:05:39.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:38 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:39.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:38 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:39.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:38 smithi046 ceph-mon[30151]: osd.7 [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607] boot 2024-05-09T00:05:39.509 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_2 2024-05-09T00:05:39.766 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:05:40.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: osdmap e47: 8 total, 8 up, 8 in 2024-05-09T00:05:40.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:40.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:40.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T00:05:40.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: Detected new or changed devices on smithi142 2024-05-09T00:05:40.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:40.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:40.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:40.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:40.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:40.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi142 to 5696M 2024-05-09T00:05:40.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:40.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:40.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:40.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:40.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: osdmap e47: 8 total, 8 up, 8 in 2024-05-09T00:05:40.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:40.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:40.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T00:05:40.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: Detected new or changed devices on smithi142 2024-05-09T00:05:40.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:40.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:40.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:40.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:40.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:40.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi142 to 5696M 2024-05-09T00:05:40.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:40.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:40.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:40.190 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:40.208 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: osdmap e47: 8 total, 8 up, 8 in 2024-05-09T00:05:40.208 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T00:05:40.208 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:40.208 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: osdmap e48: 8 total, 8 up, 8 in 2024-05-09T00:05:40.209 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: Detected new or changed devices on smithi142 2024-05-09T00:05:40.209 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:40.209 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:40.209 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:40.209 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:40.209 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:05:40.209 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi142 to 5696M 2024-05-09T00:05:40.209 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:40.209 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:40.209 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:05:40.209 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:41.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.370+0000 7f8ef4d51700 1 -- 172.21.15.169:0/3567198473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef0104350 msgr2=0x7f8ef0104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:41.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.370+0000 7f8ef4d51700 1 --2- 172.21.15.169:0/3567198473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef0104350 0x7f8ef0104760 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009990 tx=0x7f8ee0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:05:41.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.370+0000 7f8ef4d51700 1 -- 172.21.15.169:0/3567198473 shutdown_connections 2024-05-09T00:05:41.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.370+0000 7f8ef4d51700 1 --2- 172.21.15.169:0/3567198473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef0105e20 0x7f8ef010c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:41.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.370+0000 7f8ef4d51700 1 --2- 172.21.15.169:0/3567198473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef0104350 0x7f8ef0104760 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:41.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.370+0000 7f8ef4d51700 1 --2- 172.21.15.169:0/3567198473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0105480 0x7f8ef0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:41.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.370+0000 7f8ef4d51700 1 -- 172.21.15.169:0/3567198473 >> 172.21.15.169:0/3567198473 conn(0x7f8ef00ff990 msgr2=0x7f8ef0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:05:41.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.370+0000 7f8ef4d51700 1 -- 172.21.15.169:0/3567198473 shutdown_connections 2024-05-09T00:05:41.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.370+0000 7f8ef4d51700 1 -- 172.21.15.169:0/3567198473 wait complete. 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.371+0000 7f8ef4d51700 1 Processor -- start 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.371+0000 7f8ef4d51700 1 -- start start 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.371+0000 7f8ef4d51700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef0104350 0x7f8ef019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.371+0000 7f8ef4d51700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef0105480 0x7f8ef019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.371+0000 7f8ef4d51700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0105e20 0x7f8ef01a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.371+0000 7f8ef4d51700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef010fda0 con 0x7f8ef0104350 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.371+0000 7f8ef4d51700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef010fc20 con 0x7f8ef0105e20 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.371+0000 7f8ef4d51700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef010ff20 con 0x7f8ef0105480 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ee5d9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef0105480 0x7f8ef019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ee5d9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef0105480 0x7f8ef019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48592/0 (socket says 172.21.15.169:48592) 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ee5d9b700 1 -- 172.21.15.169:0/2796833723 learned_addr learned my addr 172.21.15.169:0/2796833723 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8eee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef0104350 0x7f8ef019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:41.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8eeed9d700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0105e20 0x7f8ef01a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ee5d9b700 1 -- 172.21.15.169:0/2796833723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0105e20 msgr2=0x7f8ef01a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ee5d9b700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0105e20 0x7f8ef01a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ee5d9b700 1 -- 172.21.15.169:0/2796833723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef0104350 msgr2=0x7f8ef019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:05:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ee5d9b700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef0104350 0x7f8ef019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:05:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ee5d9b700 1 -- 172.21.15.169:0/2796833723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef01a2470 con 0x7f8ef0105480 2024-05-09T00:05:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ee5d9b700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef0105480 0x7f8ef019ae50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800c920 tx=0x7f8ed800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ee7fff700 1 -- 172.21.15.169:0/2796833723 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8004a30 con 0x7f8ef0105480 2024-05-09T00:05:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ef4d51700 1 -- 172.21.15.169:0/2796833723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef01a2760 con 0x7f8ef0105480 2024-05-09T00:05:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ee7fff700 1 -- 172.21.15.169:0/2796833723 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ed80054d0 con 0x7f8ef0105480 2024-05-09T00:05:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ee7fff700 1 -- 172.21.15.169:0/2796833723 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8007bd0 con 0x7f8ef0105480 2024-05-09T00:05:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.372+0000 7f8ef4d51700 1 -- 172.21.15.169:0/2796833723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef01a2ca0 con 0x7f8ef0105480 2024-05-09T00:05:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.374+0000 7f8ef4d51700 1 -- 172.21.15.169:0/2796833723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef00512c0 con 0x7f8ef0105480 2024-05-09T00:05:41.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.375+0000 7f8ee7fff700 1 -- 172.21.15.169:0/2796833723 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8ed8005020 con 0x7f8ef0105480 2024-05-09T00:05:41.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.375+0000 7f8ee7fff700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8edc0b2880 0x7f8edc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:05:41.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.376+0000 7f8ee7fff700 1 -- 172.21.15.169:0/2796833723 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7f8ed80e1220 con 0x7f8ef0105480 2024-05-09T00:05:41.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.376+0000 7f8eee59c700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8edc0b2880 0x7f8edc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:05:41.372 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.376+0000 7f8eee59c700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8edc0b2880 0x7f8edc0b4d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00038a0 tx=0x7f8ee001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:05:41.374 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.381+0000 7f8ee7fff700 1 -- 172.21.15.169:0/2796833723 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ed8014030 con 0x7f8ef0105480 2024-05-09T00:05:41.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:05:41.592+0000 7f8ef4d51700 1 -- 172.21.15.169:0/2796833723 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8ef0063c00 con 0x7f8edc0b2880 2024-05-09T00:05:42.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:41 smithi142 ceph-mon[34533]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T00:05:42.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:41 smithi142 ceph-mon[34533]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:42.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:41 smithi142 ceph-mon[34533]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:05:42.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:05:42.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:05:42.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:42.192 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:41 smithi169 ceph-mon[34777]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T00:05:42.192 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:41 smithi169 ceph-mon[34777]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:42.192 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:41 smithi169 ceph-mon[34777]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:05:42.192 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:05:42.192 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:05:42.192 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:42.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:41 smithi046 ceph-mon[30151]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T00:05:42.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:41 smithi046 ceph-mon[30151]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:42.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:41 smithi046 ceph-mon[30151]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:05:42.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:05:42.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:05:42.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:44.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:44 smithi046 ceph-mon[30151]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:44.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:44 smithi046 ceph-mon[30151]: from='client.? 172.21.15.169:0/2221583621' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd"}]: dispatch 2024-05-09T00:05:44.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:44 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd"}]: dispatch 2024-05-09T00:05:44.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:44 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd"}]': finished 2024-05-09T00:05:44.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:44 smithi046 ceph-mon[30151]: osdmap e50: 9 total, 8 up, 9 in 2024-05-09T00:05:44.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:05:44.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:44 smithi169 ceph-mon[34777]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:44.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:44 smithi169 ceph-mon[34777]: from='client.? 172.21.15.169:0/2221583621' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd"}]: dispatch 2024-05-09T00:05:44.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:44 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd"}]: dispatch 2024-05-09T00:05:44.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:44 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd"}]': finished 2024-05-09T00:05:44.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:44 smithi169 ceph-mon[34777]: osdmap e50: 9 total, 8 up, 9 in 2024-05-09T00:05:44.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:05:44.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:44 smithi142 ceph-mon[34533]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:44.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:44 smithi142 ceph-mon[34533]: from='client.? 172.21.15.169:0/2221583621' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd"}]: dispatch 2024-05-09T00:05:44.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:44 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd"}]: dispatch 2024-05-09T00:05:44.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:44 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd"}]': finished 2024-05-09T00:05:44.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:44 smithi142 ceph-mon[34533]: osdmap e50: 9 total, 8 up, 9 in 2024-05-09T00:05:44.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:05:45.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:45 smithi046 ceph-mon[30151]: from='client.? 172.21.15.169:0/173401659' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T00:05:45.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:45 smithi169 ceph-mon[34777]: from='client.? 172.21.15.169:0/173401659' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T00:05:45.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:45 smithi142 ceph-mon[34533]: from='client.? 172.21.15.169:0/173401659' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T00:05:46.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:46 smithi046 ceph-mon[30151]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:05:46.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:46 smithi169 ceph-mon[34777]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:05:46.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:46 smithi142 ceph-mon[34533]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:05:48.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:48 smithi046 ceph-mon[30151]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-09T00:05:48.599 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:48 smithi169 ceph-mon[34777]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-09T00:05:48.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:48 smithi142 ceph-mon[34533]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-09T00:05:50.533 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:50 smithi169 ceph-mon[34777]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-09T00:05:50.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:50 smithi046 ceph-mon[30151]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-09T00:05:50.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:50 smithi142 ceph-mon[34533]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-09T00:05:52.537 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:52 smithi169 ceph-mon[34777]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:05:52.537 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:52 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T00:05:52.538 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:52 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:52.538 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:52 smithi169 ceph-mon[34777]: Deploying daemon osd.8 on smithi169 2024-05-09T00:05:52.571 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:52 smithi046 ceph-mon[30151]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:05:52.571 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:52 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T00:05:52.571 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:52 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:52.571 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:52 smithi046 ceph-mon[30151]: Deploying daemon osd.8 on smithi169 2024-05-09T00:05:52.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:52 smithi142 ceph-mon[34533]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:05:52.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:52 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T00:05:52.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:52 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:05:52.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:52 smithi142 ceph-mon[34533]: Deploying daemon osd.8 on smithi169 2024-05-09T00:05:54.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:54 smithi169 ceph-mon[34777]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:05:54.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:54 smithi142 ceph-mon[34533]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:05:54.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:54 smithi046 ceph-mon[30151]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:05:56.471 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:56 smithi169 ceph-mon[34777]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:56.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:56 smithi142 ceph-mon[34533]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:56.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:56 smithi046 ceph-mon[30151]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:58.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:58 smithi169 ceph-mon[34777]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:58.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:58 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:58.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:58 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:58.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:05:58 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:58.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:58 smithi142 ceph-mon[34533]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:58.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:58 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:58.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:58 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:58.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:05:58 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:05:58.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:58 smithi046 ceph-mon[30151]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:05:58.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:58.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:05:58.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:05:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:00.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:00 smithi169 ceph-mon[34777]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:06:00.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:00.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:00.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:00.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:06:00.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:00.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:00 smithi142 ceph-mon[34533]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:06:00.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:00.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:00.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:00.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:06:00.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:00.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:00 smithi046 ceph-mon[30151]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:06:00.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:00.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:00.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:00.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:06:00.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:01.158 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.161+0000 7f8ee7fff700 1 -- 172.21.15.169:0/2796833723 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8ef0063c00 con 0x7f8edc0b2880 2024-05-09T00:06:01.158 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 8 on host 'smithi169' 2024-05-09T00:06:01.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.163+0000 7f8ef4d51700 1 -- 172.21.15.169:0/2796833723 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8edc0b2880 msgr2=0x7f8edc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:01.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.163+0000 7f8ef4d51700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8edc0b2880 0x7f8edc0b4d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00038a0 tx=0x7f8ee001a040 comp rx=0 tx=0).stop 2024-05-09T00:06:01.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.163+0000 7f8ef4d51700 1 -- 172.21.15.169:0/2796833723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef0105480 msgr2=0x7f8ef019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:01.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.163+0000 7f8ef4d51700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef0105480 0x7f8ef019ae50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800c920 tx=0x7f8ed800cc30 comp rx=0 tx=0).stop 2024-05-09T00:06:01.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.163+0000 7f8ef4d51700 1 -- 172.21.15.169:0/2796833723 shutdown_connections 2024-05-09T00:06:01.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.163+0000 7f8ef4d51700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8edc0b2880 0x7f8edc0b4d90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:01.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.163+0000 7f8ef4d51700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef0105e20 0x7f8ef01a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:01.159 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.163+0000 7f8ef4d51700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef0104350 0x7f8ef019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:01.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.163+0000 7f8ef4d51700 1 --2- 172.21.15.169:0/2796833723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef0105480 0x7f8ef019ae50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:01.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.163+0000 7f8ef4d51700 1 -- 172.21.15.169:0/2796833723 >> 172.21.15.169:0/2796833723 conn(0x7f8ef00ff990 msgr2=0x7f8ef01086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:01.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.163+0000 7f8ef4d51700 1 -- 172.21.15.169:0/2796833723 shutdown_connections 2024-05-09T00:06:01.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:06:01.163+0000 7f8ef4d51700 1 -- 172.21.15.169:0/2796833723 wait complete. 2024-05-09T00:06:01.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:01 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:01.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:01 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:01.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:01 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:01.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:01 smithi169 ceph-mon[34777]: from='osd.8 [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T00:06:01.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:01 smithi169 ceph-mon[34777]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T00:06:01.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:01 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:01.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:01 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:01.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:01 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:01.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:01 smithi142 ceph-mon[34533]: from='osd.8 [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T00:06:01.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:01 smithi142 ceph-mon[34533]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T00:06:01.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:01 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:01.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:01 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:01.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:01 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:01.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:01 smithi046 ceph-mon[30151]: from='osd.8 [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T00:06:01.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:01 smithi046 ceph-mon[30151]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T00:06:01.853 DEBUG:teuthology.orchestra.run.smithi169:osd.8> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.8.service 2024-05-09T00:06:01.856 INFO:tasks.cephadm:Deploying osd.9 on smithi046 with /dev/vg_nvme/lv_1... 2024-05-09T00:06:01.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T00:06:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:06:02.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:02 smithi169 ceph-mon[34777]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:06:02.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:02 smithi169 ceph-mon[34777]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T00:06:02.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:02 smithi169 ceph-mon[34777]: from='osd.8 [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:06:02.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:02 smithi169 ceph-mon[34777]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T00:06:02.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:06:02.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:02 smithi169 ceph-mon[34777]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:06:02.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:02 smithi046 ceph-mon[30151]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:06:02.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:02 smithi046 ceph-mon[30151]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T00:06:02.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:02 smithi046 ceph-mon[30151]: from='osd.8 [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:06:02.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:02 smithi046 ceph-mon[30151]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T00:06:02.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:06:02.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:02 smithi046 ceph-mon[30151]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:06:02.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:02 smithi142 ceph-mon[34533]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:06:02.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:02 smithi142 ceph-mon[34533]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T00:06:02.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:02 smithi142 ceph-mon[34533]: from='osd.8 [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:06:02.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:02 smithi142 ceph-mon[34533]: osdmap e51: 9 total, 8 up, 9 in 2024-05-09T00:06:02.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:06:02.589 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:02 smithi142 ceph-mon[34533]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:06:03.338 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:06:03 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:06:03.137+0000 7f3ad7fdb700 -1 osd.8 0 waiting for initial osdmap 2024-05-09T00:06:03.338 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:06:03 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:06:03.152+0000 7f3ad3d90700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:06:04.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:04 smithi142 ceph-mon[34533]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T00:06:04.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:04 smithi142 ceph-mon[34533]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T00:06:04.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:04 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:06:04.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:04 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:06:04.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:04 smithi142 ceph-mon[34533]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:06:04.466 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T00:06:04.466 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T00:06:04.466 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-05-09T00:06:04.467 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-05-09T00:06:04.467 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0085396 s, 1.2 GB/s 2024-05-09T00:06:04.467 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-05-09T00:06:04.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:04 smithi046 ceph-mon[30151]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T00:06:04.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:04 smithi046 ceph-mon[30151]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T00:06:04.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:04 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:06:04.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:04 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:06:04.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:04 smithi046 ceph-mon[30151]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:06:04.474 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:04 smithi169 ceph-mon[34777]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T00:06:04.474 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:04 smithi169 ceph-mon[34777]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T00:06:04.474 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:04 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:06:04.474 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:04 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:06:04.474 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:04 smithi169 ceph-mon[34777]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-09T00:06:05.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_1 2024-05-09T00:06:05.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T00:06:05.389 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: osd.8 [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376] boot 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: osdmap e53: 9 total, 9 up, 9 in 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:06:05.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:05 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:05.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: purged_snaps scrub starts 2024-05-09T00:06:05.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: purged_snaps scrub ok 2024-05-09T00:06:05.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: osd.8 [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376] boot 2024-05-09T00:06:05.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: osdmap e53: 9 total, 9 up, 9 in 2024-05-09T00:06:05.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:06:05.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T00:06:05.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:05.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:05.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:05.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:05.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:05.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:05.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:05.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:06:05.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:05 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:05.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: purged_snaps scrub starts 2024-05-09T00:06:05.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: purged_snaps scrub ok 2024-05-09T00:06:05.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: osd.8 [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376] boot 2024-05-09T00:06:05.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: osdmap e53: 9 total, 9 up, 9 in 2024-05-09T00:06:05.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T00:06:05.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: osdmap e54: 9 total, 9 up, 9 in 2024-05-09T00:06:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:06:05.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:05 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:05.627 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:06:06.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:06 smithi142 ceph-mon[34533]: Detected new or changed devices on smithi169 2024-05-09T00:06:06.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:06 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi169 to 5696M 2024-05-09T00:06:06.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:06 smithi142 ceph-mon[34533]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:06.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:06 smithi142 ceph-mon[34533]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T00:06:06.771 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:06 smithi046 ceph-mon[30151]: Detected new or changed devices on smithi169 2024-05-09T00:06:06.771 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:06 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi169 to 5696M 2024-05-09T00:06:06.772 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:06 smithi046 ceph-mon[30151]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:06.772 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:06 smithi046 ceph-mon[30151]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T00:06:06.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:06 smithi169 ceph-mon[34777]: Detected new or changed devices on smithi169 2024-05-09T00:06:06.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:06 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi169 to 5696M 2024-05-09T00:06:06.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:06 smithi169 ceph-mon[34777]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:06.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:06 smithi169 ceph-mon[34777]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T00:06:07.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.330+0000 7ff204032700 1 -- 172.21.15.46:0/724560964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1fc1054a0 msgr2=0x7ff1fc1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:07.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.330+0000 7ff204032700 1 --2- 172.21.15.46:0/724560964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1fc1054a0 0x7ff1fc1058b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009990 tx=0x7ff1ec009ca0 comp rx=0 tx=0).stop 2024-05-09T00:06:07.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.330+0000 7ff204032700 1 -- 172.21.15.46:0/724560964 shutdown_connections 2024-05-09T00:06:07.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.330+0000 7ff204032700 1 --2- 172.21.15.46:0/724560964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc105e40 0x7ff1fc10c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:07.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.330+0000 7ff204032700 1 --2- 172.21.15.46:0/724560964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1fc104370 0x7ff1fc104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:07.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.330+0000 7ff204032700 1 --2- 172.21.15.46:0/724560964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1fc1054a0 0x7ff1fc1058b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:07.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.330+0000 7ff204032700 1 -- 172.21.15.46:0/724560964 >> 172.21.15.46:0/724560964 conn(0x7ff1fc0ff9d0 msgr2=0x7ff1fc101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:07.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.330+0000 7ff204032700 1 -- 172.21.15.46:0/724560964 shutdown_connections 2024-05-09T00:06:07.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.330+0000 7ff204032700 1 -- 172.21.15.46:0/724560964 wait complete. 2024-05-09T00:06:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.330+0000 7ff204032700 1 Processor -- start 2024-05-09T00:06:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff204032700 1 -- start start 2024-05-09T00:06:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff204032700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1fc104370 0x7ff1fc1a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff204032700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1fc1054a0 0x7ff1fc1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff204032700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc105e40 0x7ff1fc1a8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff204032700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc10fe30 con 0x7ff1fc104370 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff204032700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc10fcb0 con 0x7ff1fc1054a0 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff204032700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc10ffb0 con 0x7ff1fc105e40 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff201dce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1fc104370 0x7ff1fc1a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff201dce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1fc104370 0x7ff1fc1a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35022/0 (socket says 172.21.15.46:35022) 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff201dce700 1 -- 172.21.15.46:0/4145448261 learned_addr learned my addr 172.21.15.46:0/4145448261 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff2025cf700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc105e40 0x7ff1fc1a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff2015cd700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1fc1054a0 0x7ff1fc1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff201dce700 1 -- 172.21.15.46:0/4145448261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc105e40 msgr2=0x7ff1fc1a8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff201dce700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc105e40 0x7ff1fc1a8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff201dce700 1 -- 172.21.15.46:0/4145448261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1fc1054a0 msgr2=0x7ff1fc1a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff201dce700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1fc1054a0 0x7ff1fc1a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff201dce700 1 -- 172.21.15.46:0/4145448261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc1a8a20 con 0x7ff1fc104370 2024-05-09T00:06:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.331+0000 7ff201dce700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1fc104370 0x7ff1fc1a0f50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8007dd0 tx=0x7ff1f800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.332+0000 7ff1f2ffd700 1 -- 172.21.15.46:0/4145448261 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8016020 con 0x7ff1fc104370 2024-05-09T00:06:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.332+0000 7ff1f2ffd700 1 -- 172.21.15.46:0/4145448261 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1f800ad40 con 0x7ff1fc104370 2024-05-09T00:06:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.332+0000 7ff1f2ffd700 1 -- 172.21.15.46:0/4145448261 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8011670 con 0x7ff1fc104370 2024-05-09T00:06:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.332+0000 7ff204032700 1 -- 172.21.15.46:0/4145448261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1fc1a8d10 con 0x7ff1fc104370 2024-05-09T00:06:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.332+0000 7ff204032700 1 -- 172.21.15.46:0/4145448261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1fc1a91a0 con 0x7ff1fc104370 2024-05-09T00:06:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.333+0000 7ff204032700 1 -- 172.21.15.46:0/4145448261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1fc0512c0 con 0x7ff1fc104370 2024-05-09T00:06:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.335+0000 7ff1f2ffd700 1 -- 172.21.15.46:0/4145448261 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff1f8019020 con 0x7ff1fc104370 2024-05-09T00:06:07.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.336+0000 7ff1f2ffd700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff1e80b2880 0x7ff1e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:07.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.336+0000 7ff2015cd700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff1e80b2880 0x7ff1e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:07.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.336+0000 7ff1f2ffd700 1 -- 172.21.15.46:0/4145448261 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7ff1f80e1ed0 con 0x7ff1fc104370 2024-05-09T00:06:07.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.337+0000 7ff2015cd700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff1e80b2880 0x7ff1e80b4d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec006010 tx=0x7ff1ec003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.340+0000 7ff1f2ffd700 1 -- 172.21.15.46:0/4145448261 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff1f8020020 con 0x7ff1fc104370 2024-05-09T00:06:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:07.567+0000 7ff204032700 1 -- 172.21.15.46:0/4145448261 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff1fc063c00 con 0x7ff1e80b2880 2024-05-09T00:06:08.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:08 smithi046 ceph-mon[30151]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:08.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:08 smithi046 ceph-mon[30151]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:06:08.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:06:08.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:06:08.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:08 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:08.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:08 smithi169 ceph-mon[34777]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:08.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:08 smithi169 ceph-mon[34777]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:06:08.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:06:08.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:06:08.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:08 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:08.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:08 smithi142 ceph-mon[34533]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:08.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:08 smithi142 ceph-mon[34533]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:06:08.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:06:08.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:06:08.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:08 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:10.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:10 smithi142 ceph-mon[34533]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:06:10.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:10 smithi046 ceph-mon[30151]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:06:10.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:10 smithi169 ceph-mon[34777]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T00:06:11.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:11 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2579769694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32df0dab-5875-4beb-b733-60a915c94a9b"}]: dispatch 2024-05-09T00:06:11.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:11 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2579769694' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32df0dab-5875-4beb-b733-60a915c94a9b"}]': finished 2024-05-09T00:06:11.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:11 smithi142 ceph-mon[34533]: osdmap e56: 10 total, 9 up, 10 in 2024-05-09T00:06:11.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:11 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:11.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:11 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3548078194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T00:06:11.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:11 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2579769694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32df0dab-5875-4beb-b733-60a915c94a9b"}]: dispatch 2024-05-09T00:06:11.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:11 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2579769694' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32df0dab-5875-4beb-b733-60a915c94a9b"}]': finished 2024-05-09T00:06:11.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:11 smithi046 ceph-mon[30151]: osdmap e56: 10 total, 9 up, 10 in 2024-05-09T00:06:11.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:11 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:11.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:11 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3548078194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T00:06:11.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:11 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2579769694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32df0dab-5875-4beb-b733-60a915c94a9b"}]: dispatch 2024-05-09T00:06:11.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:11 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2579769694' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32df0dab-5875-4beb-b733-60a915c94a9b"}]': finished 2024-05-09T00:06:11.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:11 smithi169 ceph-mon[34777]: osdmap e56: 10 total, 9 up, 10 in 2024-05-09T00:06:11.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:11 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:11.850 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:11 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3548078194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T00:06:12.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:12 smithi142 ceph-mon[34533]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-09T00:06:13.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:12 smithi046 ceph-mon[30151]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-09T00:06:13.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:12 smithi169 ceph-mon[34777]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-09T00:06:14.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:14 smithi142 ceph-mon[34533]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:06:15.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:14 smithi046 ceph-mon[30151]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:06:15.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:14 smithi169 ceph-mon[34777]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:06:16.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:16 smithi142 ceph-mon[34533]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:06:16.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:16 smithi046 ceph-mon[30151]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:06:17.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:16 smithi169 ceph-mon[34777]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:06:18.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:18 smithi046 ceph-mon[30151]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:06:18.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T00:06:18.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:18 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:18.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:18 smithi142 ceph-mon[34533]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:06:18.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T00:06:18.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:18 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:19.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:18 smithi169 ceph-mon[34777]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:06:19.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T00:06:19.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:18 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:19.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:19 smithi046 ceph-mon[30151]: Deploying daemon osd.9 on smithi046 2024-05-09T00:06:19.932 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:19 smithi169 ceph-mon[34777]: Deploying daemon osd.9 on smithi046 2024-05-09T00:06:19.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:19 smithi142 ceph-mon[34533]: Deploying daemon osd.9 on smithi046 2024-05-09T00:06:20.758 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:20 smithi046 ceph-mon[30151]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:20.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:20 smithi142 ceph-mon[34533]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:21.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:20 smithi169 ceph-mon[34777]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:22.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:22 smithi046 ceph-mon[30151]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:22.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:22 smithi142 ceph-mon[34533]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:23.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:22 smithi169 ceph-mon[34777]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:24.937 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:24 smithi142 ceph-mon[34533]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:25.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:24 smithi046 ceph-mon[30151]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:25.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:24 smithi169 ceph-mon[34777]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:26.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:26.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:26.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:26.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:25 smithi142 ceph-mon[34533]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:26.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:26.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:26.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:26.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:25 smithi046 ceph-mon[30151]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:26.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:26.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:26.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:26.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:25 smithi169 ceph-mon[34777]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:28.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:28 smithi046 ceph-mon[30151]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:28.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:28 smithi169 ceph-mon[34777]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:28.671 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 9 on host 'smithi046' 2024-05-09T00:06:28.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.676+0000 7ff1f2ffd700 1 -- 172.21.15.46:0/4145448261 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff1fc063c00 con 0x7ff1e80b2880 2024-05-09T00:06:28.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.678+0000 7ff204032700 1 -- 172.21.15.46:0/4145448261 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff1e80b2880 msgr2=0x7ff1e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.678+0000 7ff204032700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff1e80b2880 0x7ff1e80b4d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec006010 tx=0x7ff1ec003860 comp rx=0 tx=0).stop 2024-05-09T00:06:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.678+0000 7ff204032700 1 -- 172.21.15.46:0/4145448261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1fc104370 msgr2=0x7ff1fc1a0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.678+0000 7ff204032700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1fc104370 0x7ff1fc1a0f50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8007dd0 tx=0x7ff1f800a340 comp rx=0 tx=0).stop 2024-05-09T00:06:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.679+0000 7ff204032700 1 -- 172.21.15.46:0/4145448261 shutdown_connections 2024-05-09T00:06:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.679+0000 7ff204032700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff1e80b2880 0x7ff1e80b4d90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.679+0000 7ff204032700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1fc105e40 0x7ff1fc1a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.679+0000 7ff204032700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1fc104370 0x7ff1fc1a0f50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.679+0000 7ff204032700 1 --2- 172.21.15.46:0/4145448261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1fc1054a0 0x7ff1fc1a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.679+0000 7ff204032700 1 -- 172.21.15.46:0/4145448261 >> 172.21.15.46:0/4145448261 conn(0x7ff1fc0ff9d0 msgr2=0x7ff1fc1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.679+0000 7ff204032700 1 -- 172.21.15.46:0/4145448261 shutdown_connections 2024-05-09T00:06:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:06:28.679+0000 7ff204032700 1 -- 172.21.15.46:0/4145448261 wait complete. 2024-05-09T00:06:28.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:28 smithi142 ceph-mon[34533]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:29.399 DEBUG:teuthology.orchestra.run.smithi046:osd.9> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.9.service 2024-05-09T00:06:29.442 INFO:tasks.cephadm:Deploying osd.10 on smithi142 with /dev/vg_nvme/lv_1... 2024-05-09T00:06:29.442 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T00:06:29.688 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:06:29.738 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:29 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:29.739 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:29 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:29.739 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:29 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:29.739 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:29 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:29.739 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:29 smithi046 ceph-mon[30151]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T00:06:29.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:29 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:29.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:29 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:29.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:29 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:29.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:29 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:29.811 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:29 smithi142 ceph-mon[34533]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T00:06:30.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:29 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:30.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:29 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:30.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:29 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:30.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:29 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:30.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:29 smithi169 ceph-mon[34777]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T00:06:30.892 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:30 smithi142 ceph-mon[34533]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:30.892 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:30 smithi142 ceph-mon[34533]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T00:06:30.892 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:30 smithi142 ceph-mon[34533]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T00:06:30.892 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:30 smithi142 ceph-mon[34533]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T00:06:30.892 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:30 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:30.899 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:30 smithi046 ceph-mon[30151]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:30.899 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:30 smithi046 ceph-mon[30151]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T00:06:30.899 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:30 smithi046 ceph-mon[30151]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T00:06:30.899 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:30 smithi046 ceph-mon[30151]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T00:06:30.900 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:30 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:31.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:30 smithi169 ceph-mon[34777]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:31.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:30 smithi169 ceph-mon[34777]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T00:06:31.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:30 smithi169 ceph-mon[34777]: osdmap e57: 10 total, 9 up, 10 in 2024-05-09T00:06:31.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:30 smithi169 ceph-mon[34777]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T00:06:31.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:30 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:31.557 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:06:31 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:06:31.309+0000 7f6779aa4700 -1 osd.9 0 waiting for initial osdmap 2024-05-09T00:06:31.557 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:06:31 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:06:31.321+0000 7f6771851700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:06:31.801 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T00:06:31.801 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T00:06:31.801 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-05-09T00:06:31.801 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-05-09T00:06:31.801 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113112 s, 927 MB/s 2024-05-09T00:06:31.801 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-05-09T00:06:32.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:31 smithi046 ceph-mon[30151]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T00:06:32.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:31 smithi046 ceph-mon[30151]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T00:06:32.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:31 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:32.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:31 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:32.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:31 smithi046 ceph-mon[30151]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:32.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:31 smithi046 ceph-mon[30151]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' 2024-05-09T00:06:32.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:31 smithi169 ceph-mon[34777]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T00:06:32.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:31 smithi169 ceph-mon[34777]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T00:06:32.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:31 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:32.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:31 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:32.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:31 smithi169 ceph-mon[34777]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:32.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:31 smithi169 ceph-mon[34777]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' 2024-05-09T00:06:32.131 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:31 smithi142 ceph-mon[34533]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T00:06:32.132 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:31 smithi142 ceph-mon[34533]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T00:06:32.132 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:31 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:32.132 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:31 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:32.132 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:31 smithi142 ceph-mon[34533]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-09T00:06:32.132 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:31 smithi142 ceph-mon[34533]: from='osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248]' entity='osd.9' 2024-05-09T00:06:32.493 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_1 2024-05-09T00:06:32.746 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:06:33.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T00:06:33.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T00:06:33.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:33.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248] boot 2024-05-09T00:06:33.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: osdmap e59: 10 total, 10 up, 10 in 2024-05-09T00:06:33.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:33.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: Detected new or changed devices on smithi046 2024-05-09T00:06:33.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:33.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:33.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:33.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:33.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:33.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:33.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi046 to 4273M 2024-05-09T00:06:33.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:33.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:33.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:06:33.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:32 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:33.077 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: purged_snaps scrub starts 2024-05-09T00:06:33.077 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: purged_snaps scrub ok 2024-05-09T00:06:33.077 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:33.077 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248] boot 2024-05-09T00:06:33.077 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: osdmap e59: 10 total, 10 up, 10 in 2024-05-09T00:06:33.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:33.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: Detected new or changed devices on smithi046 2024-05-09T00:06:33.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:33.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:33.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:33.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:33.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:33.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:33.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi046 to 4273M 2024-05-09T00:06:33.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:33.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:33.078 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:06:33.079 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:32 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:33.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: purged_snaps scrub starts 2024-05-09T00:06:33.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: purged_snaps scrub ok 2024-05-09T00:06:33.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:33.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: osd.9 [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248] boot 2024-05-09T00:06:33.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: osdmap e59: 10 total, 10 up, 10 in 2024-05-09T00:06:33.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T00:06:33.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: Detected new or changed devices on smithi046 2024-05-09T00:06:33.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:33.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:33.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:33.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:33.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:33.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:06:33.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi046 to 4273M 2024-05-09T00:06:33.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:33.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:33.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:06:33.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:32 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:34.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:34.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:33 smithi046 ceph-mon[30151]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:34.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:34.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:06:34.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:34.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:34.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:33 smithi169 ceph-mon[34777]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:34.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:34.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:06:34.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:34.128 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:34.128 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:33 smithi142 ceph-mon[34533]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:34.128 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:34.128 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:06:34.128 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:34.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.369+0000 7f64c9d8b700 1 -- 172.21.15.142:0/2100810908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c4103250 msgr2=0x7f64c4103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:34.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.369+0000 7f64c9d8b700 1 --2- 172.21.15.142:0/2100810908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c4103250 0x7f64c4103660 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009990 tx=0x7f64b4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:06:34.366 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.369+0000 7f64c9d8b700 1 -- 172.21.15.142:0/2100810908 shutdown_connections 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.369+0000 7f64c9d8b700 1 --2- 172.21.15.142:0/2100810908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64c4104d20 0x7f64c410b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.369+0000 7f64c9d8b700 1 --2- 172.21.15.142:0/2100810908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c4103250 0x7f64c4103660 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.369+0000 7f64c9d8b700 1 --2- 172.21.15.142:0/2100810908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c4104380 0x7f64c4104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.369+0000 7f64c9d8b700 1 -- 172.21.15.142:0/2100810908 >> 172.21.15.142:0/2100810908 conn(0x7f64c40fe890 msgr2=0x7f64c4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.369+0000 7f64c9d8b700 1 -- 172.21.15.142:0/2100810908 shutdown_connections 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.369+0000 7f64c9d8b700 1 -- 172.21.15.142:0/2100810908 wait complete. 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c9d8b700 1 Processor -- start 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c9d8b700 1 -- start start 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c9d8b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c4103250 0x7f64c4199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c9d8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c4104380 0x7f64c4199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c9d8b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64c4104d20 0x7f64c41a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c9d8b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64c410ecb0 con 0x7f64c4104380 2024-05-09T00:06:34.367 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c9d8b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f64c410eb30 con 0x7f64c4103250 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c9d8b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64c410ee30 con 0x7f64c4104d20 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c4103250 0x7f64c4199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c4103250 0x7f64c4199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57936/0 (socket says 172.21.15.142:57936) 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c37fe700 1 -- 172.21.15.142:0/3122714140 learned_addr learned my addr 172.21.15.142:0/3122714140 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c3fff700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64c4104d20 0x7f64c41a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c2ffd700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c4104380 0x7f64c4199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.370+0000 7f64c37fe700 1 -- 172.21.15.142:0/3122714140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64c4104d20 msgr2=0x7f64c41a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.371+0000 7f64c37fe700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64c4104d20 0x7f64c41a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.371+0000 7f64c37fe700 1 -- 172.21.15.142:0/3122714140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c4104380 msgr2=0x7f64c4199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.371+0000 7f64c37fe700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c4104380 0x7f64c4199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.371+0000 7f64c37fe700 1 -- 172.21.15.142:0/3122714140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c41a1380 con 0x7f64c4103250 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.371+0000 7f64c37fe700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c4103250 0x7f64c4199820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f64b4000c00 tx=0x7f64b40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.371+0000 7f64c0ff9700 1 -- 172.21.15.142:0/3122714140 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4004830 con 0x7f64c4103250 2024-05-09T00:06:34.368 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.371+0000 7f64c9d8b700 1 -- 172.21.15.142:0/3122714140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c41a1610 con 0x7f64c4103250 2024-05-09T00:06:34.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.371+0000 7f64c0ff9700 1 -- 172.21.15.142:0/3122714140 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b4005020 con 0x7f64c4103250 2024-05-09T00:06:34.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.371+0000 7f64c0ff9700 1 -- 172.21.15.142:0/3122714140 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4003d90 con 0x7f64c4103250 2024-05-09T00:06:34.369 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.371+0000 7f64c9d8b700 1 -- 172.21.15.142:0/3122714140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c41a1a60 con 0x7f64c4103250 2024-05-09T00:06:34.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.372+0000 7f64c9d8b700 1 -- 172.21.15.142:0/3122714140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c40512c0 con 0x7f64c4103250 2024-05-09T00:06:34.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.373+0000 7f64c0ff9700 1 -- 172.21.15.142:0/3122714140 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f64b40051d0 con 0x7f64c4103250 2024-05-09T00:06:34.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.374+0000 7f64c0ff9700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f64ac0b2880 0x7f64ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:06:34.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.374+0000 7f64c0ff9700 1 -- 172.21.15.142:0/3122714140 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7f64b40e2d30 con 0x7f64c4103250 2024-05-09T00:06:34.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.374+0000 7f64c2ffd700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f64ac0b2880 0x7f64ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:06:34.370 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.375+0000 7f64c2ffd700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f64ac0b2880 0x7f64ac0b4d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f64c419a8d0 tx=0x7f64b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:06:34.372 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.378+0000 7f64c0ff9700 1 -- 172.21.15.142:0/3122714140 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f64b4013070 con 0x7f64c4103250 2024-05-09T00:06:34.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:34.593+0000 7f64c9d8b700 1 -- 172.21.15.142:0/3122714140 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f64c4063c00 con 0x7f64ac0b2880 2024-05-09T00:06:34.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:34 smithi142 ceph-mon[34533]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T00:06:34.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:34 smithi142 ceph-mon[34533]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:06:34.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:34 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:06:34.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:34 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:06:34.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:34 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:35.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:34 smithi046 ceph-mon[30151]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T00:06:35.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:34 smithi046 ceph-mon[30151]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:06:35.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:34 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:06:35.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:34 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:06:35.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:34 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:35.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:34 smithi169 ceph-mon[34777]: osdmap e60: 10 total, 10 up, 10 in 2024-05-09T00:06:35.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:34 smithi169 ceph-mon[34777]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:06:35.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:34 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:06:35.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:34 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:06:35.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:34 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:36.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:35 smithi046 ceph-mon[30151]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:36.100 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:35 smithi169 ceph-mon[34777]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:36.187 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:35 smithi142 ceph-mon[34533]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:38.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:38 smithi046 ceph-mon[30151]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:38.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:38 smithi046 ceph-mon[30151]: from='client.? 172.21.15.142:0/2230710955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380a4f47-e528-4258-8836-32eac0e6e630"}]: dispatch 2024-05-09T00:06:38.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:38 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380a4f47-e528-4258-8836-32eac0e6e630"}]: dispatch 2024-05-09T00:06:38.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:38 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "380a4f47-e528-4258-8836-32eac0e6e630"}]': finished 2024-05-09T00:06:38.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:38 smithi046 ceph-mon[30151]: osdmap e61: 11 total, 10 up, 11 in 2024-05-09T00:06:38.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:38 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:38.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:38 smithi046 ceph-mon[30151]: from='client.? 172.21.15.142:0/419001448' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T00:06:38.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:38 smithi169 ceph-mon[34777]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:38.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:38 smithi169 ceph-mon[34777]: from='client.? 172.21.15.142:0/2230710955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380a4f47-e528-4258-8836-32eac0e6e630"}]: dispatch 2024-05-09T00:06:38.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:38 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380a4f47-e528-4258-8836-32eac0e6e630"}]: dispatch 2024-05-09T00:06:38.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:38 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "380a4f47-e528-4258-8836-32eac0e6e630"}]': finished 2024-05-09T00:06:38.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:38 smithi169 ceph-mon[34777]: osdmap e61: 11 total, 10 up, 11 in 2024-05-09T00:06:38.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:38 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:38.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:38 smithi169 ceph-mon[34777]: from='client.? 172.21.15.142:0/419001448' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T00:06:38.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:38 smithi142 ceph-mon[34533]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:38.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:38 smithi142 ceph-mon[34533]: from='client.? 172.21.15.142:0/2230710955' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380a4f47-e528-4258-8836-32eac0e6e630"}]: dispatch 2024-05-09T00:06:38.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:38 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "380a4f47-e528-4258-8836-32eac0e6e630"}]: dispatch 2024-05-09T00:06:38.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:38 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "380a4f47-e528-4258-8836-32eac0e6e630"}]': finished 2024-05-09T00:06:38.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:38 smithi142 ceph-mon[34533]: osdmap e61: 11 total, 10 up, 11 in 2024-05-09T00:06:38.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:38 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:38.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:38 smithi142 ceph-mon[34533]: from='client.? 172.21.15.142:0/419001448' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T00:06:40.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:40 smithi169 ceph-mon[34777]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:40.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:40 smithi046 ceph-mon[30151]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:40.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:40 smithi142 ceph-mon[34533]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:42.570 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:42 smithi142 ceph-mon[34533]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:42.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:42 smithi169 ceph-mon[34777]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:42.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:42 smithi046 ceph-mon[30151]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:44.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:44 smithi142 ceph-mon[34533]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:44.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:44 smithi169 ceph-mon[34777]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:44.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:44 smithi046 ceph-mon[30151]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:45.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:45 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T00:06:45.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:45 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:45.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:45 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T00:06:45.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:45 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:45.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:45 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T00:06:45.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:45 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:06:46.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:46 smithi169 ceph-mon[34777]: Deploying daemon osd.10 on smithi142 2024-05-09T00:06:46.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:46 smithi169 ceph-mon[34777]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:46.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:46 smithi142 ceph-mon[34533]: Deploying daemon osd.10 on smithi142 2024-05-09T00:06:46.662 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:46 smithi142 ceph-mon[34533]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:46.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:46 smithi046 ceph-mon[30151]: Deploying daemon osd.10 on smithi142 2024-05-09T00:06:46.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:46 smithi046 ceph-mon[30151]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:48.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:48 smithi142 ceph-mon[34533]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:48.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:48 smithi169 ceph-mon[34777]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:48.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:48 smithi046 ceph-mon[30151]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:50.521 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:50 smithi142 ceph-mon[34533]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:50.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:50 smithi169 ceph-mon[34777]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:50.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:50 smithi046 ceph-mon[30151]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:52.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:51 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:52.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:51 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:52.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:51 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:52.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:51 smithi046 ceph-mon[30151]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:52.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:51 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:52.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:51 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:52.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:51 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:52.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:51 smithi142 ceph-mon[34533]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:52.350 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:51 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:52.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:51 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:52.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:51 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:06:52.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:51 smithi169 ceph-mon[34777]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:54.592 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:54 smithi142 ceph-mon[34533]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:54.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:54 smithi169 ceph-mon[34777]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:54.659 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 10 on host 'smithi142' 2024-05-09T00:06:54.659 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.663+0000 7f64c0ff9700 1 -- 172.21.15.142:0/3122714140 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f64c4063c00 con 0x7f64ac0b2880 2024-05-09T00:06:54.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.666+0000 7f64c9d8b700 1 -- 172.21.15.142:0/3122714140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f64ac0b2880 msgr2=0x7f64ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.666+0000 7f64c9d8b700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f64ac0b2880 0x7f64ac0b4d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f64c419a8d0 tx=0x7f64b800b410 comp rx=0 tx=0).stop 2024-05-09T00:06:54.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.666+0000 7f64c9d8b700 1 -- 172.21.15.142:0/3122714140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c4103250 msgr2=0x7f64c4199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:06:54.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.666+0000 7f64c9d8b700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c4103250 0x7f64c4199820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f64b4000c00 tx=0x7f64b40047b0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.666+0000 7f64c9d8b700 1 -- 172.21.15.142:0/3122714140 shutdown_connections 2024-05-09T00:06:54.661 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.666+0000 7f64c9d8b700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f64ac0b2880 0x7f64ac0b4d90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.666+0000 7f64c9d8b700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64c4104d20 0x7f64c41a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.666+0000 7f64c9d8b700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64c4103250 0x7f64c4199820 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.666+0000 7f64c9d8b700 1 --2- 172.21.15.142:0/3122714140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c4104380 0x7f64c4199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:06:54.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.666+0000 7f64c9d8b700 1 -- 172.21.15.142:0/3122714140 >> 172.21.15.142:0/3122714140 conn(0x7f64c40fe890 msgr2=0x7f64c41075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:06:54.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.666+0000 7f64c9d8b700 1 -- 172.21.15.142:0/3122714140 shutdown_connections 2024-05-09T00:06:54.662 INFO:teuthology.orchestra.run.smithi142.stderr:2024-05-09T00:06:54.666+0000 7f64c9d8b700 1 -- 172.21.15.142:0/3122714140 wait complete. 2024-05-09T00:06:54.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:54 smithi046 ceph-mon[30151]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:55.291 DEBUG:teuthology.orchestra.run.smithi142:osd.10> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.10.service 2024-05-09T00:06:55.294 INFO:tasks.cephadm:Deploying osd.11 on smithi169 with /dev/vg_nvme/lv_1... 2024-05-09T00:06:55.294 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T00:06:55.556 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:06:55.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:55 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:55.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:55 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:55.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:55 smithi142 ceph-mon[34533]: from='osd.10 [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T00:06:55.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:55 smithi142 ceph-mon[34533]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T00:06:55.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:55 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:55.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:55 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:55.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:55 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:55.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:55 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:55.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:55 smithi169 ceph-mon[34777]: from='osd.10 [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T00:06:55.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:55 smithi169 ceph-mon[34777]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T00:06:55.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:55 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:55.967 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:55 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:56.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:55 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:56.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:55 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:56.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:55 smithi046 ceph-mon[30151]: from='osd.10 [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T00:06:56.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:55 smithi046 ceph-mon[30151]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T00:06:56.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:55 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:56.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:55 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:06:56.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:56 smithi142 ceph-mon[34533]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:56.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:56 smithi142 ceph-mon[34533]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T00:06:56.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:56 smithi142 ceph-mon[34533]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T00:06:56.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:56.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:56 smithi142 ceph-mon[34533]: from='osd.10 [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:06:56.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:56 smithi142 ceph-mon[34533]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:06:56.755 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:56 smithi169 ceph-mon[34777]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:56.755 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:56 smithi169 ceph-mon[34777]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T00:06:56.755 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:56 smithi169 ceph-mon[34777]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T00:06:56.755 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:56.755 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:56 smithi169 ceph-mon[34777]: from='osd.10 [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:06:56.756 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:56 smithi169 ceph-mon[34777]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:06:57.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:56 smithi046 ceph-mon[30151]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:57.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:56 smithi046 ceph-mon[30151]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T00:06:57.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:56 smithi046 ceph-mon[30151]: osdmap e62: 11 total, 10 up, 11 in 2024-05-09T00:06:57.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:57.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:56 smithi046 ceph-mon[30151]: from='osd.10 [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:06:57.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:56 smithi046 ceph-mon[30151]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-05-09T00:06:57.406 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:06:57 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:06:57.087+0000 7f13027c1700 -1 osd.10 0 waiting for initial osdmap 2024-05-09T00:06:57.406 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:06:57 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:06:57.099+0000 7f12fa56e700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-05-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-05-09T00:06:57.638 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129748 s, 808 MB/s 2024-05-09T00:06:57.639 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-05-09T00:06:57.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:57 smithi142 ceph-mon[34533]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-09T00:06:57.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:57 smithi142 ceph-mon[34533]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T00:06:57.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:57.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:57.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:57 smithi142 ceph-mon[34533]: from='osd.10 ' entity='osd.10' 2024-05-09T00:06:57.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:57 smithi169 ceph-mon[34777]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-09T00:06:57.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:57 smithi169 ceph-mon[34777]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T00:06:57.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:57.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:57.971 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:57 smithi169 ceph-mon[34777]: from='osd.10 ' entity='osd.10' 2024-05-09T00:06:58.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:57 smithi046 ceph-mon[30151]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-05-09T00:06:58.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:57 smithi046 ceph-mon[30151]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T00:06:58.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:58.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:58.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:57 smithi046 ceph-mon[30151]: from='osd.10 ' entity='osd.10' 2024-05-09T00:06:58.249 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_1 2024-05-09T00:06:58.503 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:06:58.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:58 smithi169 ceph-mon[34777]: purged_snaps scrub starts 2024-05-09T00:06:58.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:58 smithi169 ceph-mon[34777]: purged_snaps scrub ok 2024-05-09T00:06:58.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:58 smithi169 ceph-mon[34777]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:58.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:58 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:58.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:58 smithi169 ceph-mon[34777]: osd.10 [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982] boot 2024-05-09T00:06:58.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:58 smithi169 ceph-mon[34777]: osdmap e64: 11 total, 11 up, 11 in 2024-05-09T00:06:58.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:06:58 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:58.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:58 smithi142 ceph-mon[34533]: purged_snaps scrub starts 2024-05-09T00:06:58.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:58 smithi142 ceph-mon[34533]: purged_snaps scrub ok 2024-05-09T00:06:58.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:58 smithi142 ceph-mon[34533]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:58.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:58 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:58.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:58 smithi142 ceph-mon[34533]: osd.10 [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982] boot 2024-05-09T00:06:58.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:58 smithi142 ceph-mon[34533]: osdmap e64: 11 total, 11 up, 11 in 2024-05-09T00:06:58.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:06:58 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:58 smithi046 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T00:06:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:58 smithi046 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T00:06:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:58 smithi046 ceph-mon[30151]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-09T00:06:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:06:59.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:58 smithi046 ceph-mon[30151]: osd.10 [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982] boot 2024-05-09T00:06:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:58 smithi046 ceph-mon[30151]: osdmap e64: 11 total, 11 up, 11 in 2024-05-09T00:06:59.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:06:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T00:07:00.104 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.104+0000 7f79d96cc700 1 -- 172.21.15.169:0/1910354051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d4104340 msgr2=0x7f79d4104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:00.104 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.104+0000 7f79d96cc700 1 --2- 172.21.15.169:0/1910354051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d4104340 0x7f79d4104750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009990 tx=0x7f79c4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:07:00.104 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.104+0000 7f79d96cc700 1 -- 172.21.15.169:0/1910354051 shutdown_connections 2024-05-09T00:07:00.104 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.104+0000 7f79d96cc700 1 --2- 172.21.15.169:0/1910354051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79d4105e10 0x7f79d410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:00.104 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.104+0000 7f79d96cc700 1 --2- 172.21.15.169:0/1910354051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d4104340 0x7f79d4104750 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:00.104 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.104+0000 7f79d96cc700 1 --2- 172.21.15.169:0/1910354051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d4105470 0x7f79d4105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:00.104 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.104+0000 7f79d96cc700 1 -- 172.21.15.169:0/1910354051 >> 172.21.15.169:0/1910354051 conn(0x7f79d40ff9c0 msgr2=0x7f79d4101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:00.104 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.104+0000 7f79d96cc700 1 -- 172.21.15.169:0/1910354051 shutdown_connections 2024-05-09T00:07:00.104 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.104+0000 7f79d96cc700 1 -- 172.21.15.169:0/1910354051 wait complete. 2024-05-09T00:07:00.104 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.105+0000 7f79d96cc700 1 Processor -- start 2024-05-09T00:07:00.104 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.105+0000 7f79d96cc700 1 -- start start 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.105+0000 7f79d96cc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d4104340 0x7f79d419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.105+0000 7f79d96cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d4105470 0x7f79d419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.105+0000 7f79d96cc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79d4105e10 0x7f79d41a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.105+0000 7f79d96cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79d410fe80 con 0x7f79d4105470 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.105+0000 7f79d96cc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79d410fd00 con 0x7f79d4105e10 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.105+0000 7f79d96cc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79d4110000 con 0x7f79d4104340 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.105+0000 7f79d2ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d4104340 0x7f79d419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.105+0000 7f79d2ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d4104340 0x7f79d419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35080/0 (socket says 172.21.15.169:35080) 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.105+0000 7f79d2ffd700 1 -- 172.21.15.169:0/4130050617 learned_addr learned my addr 172.21.15.169:0/4130050617 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79ca7fc700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d4105470 0x7f79d419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79d37fe700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79d4105e10 0x7f79d41a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79d2ffd700 1 -- 172.21.15.169:0/4130050617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79d4105e10 msgr2=0x7f79d41a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79d2ffd700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79d4105e10 0x7f79d41a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:00.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79d2ffd700 1 -- 172.21.15.169:0/4130050617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d4105470 msgr2=0x7f79d419aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:00.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79d2ffd700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d4105470 0x7f79d419aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:00.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79d2ffd700 1 -- 172.21.15.169:0/4130050617 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d41a25c0 con 0x7f79d4104340 2024-05-09T00:07:00.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79d2ffd700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d4104340 0x7f79d419aab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f79c40057c0 tx=0x7f79c40057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:00.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79d0ff9700 1 -- 172.21.15.169:0/4130050617 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4027070 con 0x7f79d4104340 2024-05-09T00:07:00.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79d0ff9700 1 -- 172.21.15.169:0/4130050617 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c4004000 con 0x7f79d4104340 2024-05-09T00:07:00.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79d96cc700 1 -- 172.21.15.169:0/4130050617 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d41a2850 con 0x7f79d4104340 2024-05-09T00:07:00.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79d0ff9700 1 -- 172.21.15.169:0/4130050617 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4022c60 con 0x7f79d4104340 2024-05-09T00:07:00.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.106+0000 7f79d96cc700 1 -- 172.21.15.169:0/4130050617 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d41a2d20 con 0x7f79d4104340 2024-05-09T00:07:00.106 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.108+0000 7f79d96cc700 1 -- 172.21.15.169:0/4130050617 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d40512c0 con 0x7f79d4104340 2024-05-09T00:07:00.107 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.109+0000 7f79d0ff9700 1 -- 172.21.15.169:0/4130050617 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f79c402a070 con 0x7f79d4104340 2024-05-09T00:07:00.107 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.110+0000 7f79d0ff9700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f79c00b2880 0x7f79c00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:00.107 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.110+0000 7f79d0ff9700 1 -- 172.21.15.169:0/4130050617 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7f79c40e2ce0 con 0x7f79d4104340 2024-05-09T00:07:00.107 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.110+0000 7f79ca7fc700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f79c00b2880 0x7f79c00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:00.107 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.111+0000 7f79ca7fc700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f79c00b2880 0x7f79c00b4d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f79d419bb10 tx=0x7f79bc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:00.111 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.115+0000 7f79d0ff9700 1 -- 172.21.15.169:0/4130050617 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f79c4022e20 con 0x7f79d4104340 2024-05-09T00:07:00.233 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: Detected new or changed devices on smithi142 2024-05-09T00:07:00.233 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.233 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.233 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.233 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi142 to 4272M 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:00.234 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: Detected new or changed devices on smithi142 2024-05-09T00:07:00.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi142 to 4272M 2024-05-09T00:07:00.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:00.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:00.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T00:07:00.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:07:00.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:00.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:00.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:00.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.329 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:00.333+0000 7f79d96cc700 1 -- 172.21.15.169:0/4130050617 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f79d4063c00 con 0x7f79c00b2880 2024-05-09T00:07:00.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: Detected new or changed devices on smithi142 2024-05-09T00:07:00.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:00.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi142 to 4272M 2024-05-09T00:07:00.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:00.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:00.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:00.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: osdmap e65: 11 total, 11 up, 11 in 2024-05-09T00:07:00.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:07:00.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:00.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:00.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:00.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:01.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:01 smithi169 ceph-mon[34777]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T00:07:01.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:01 smithi169 ceph-mon[34777]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:07:01.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:01 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:07:01.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:01 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:07:01.262 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:01 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:01.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:01 smithi142 ceph-mon[34533]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T00:07:01.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:01 smithi142 ceph-mon[34533]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:07:01.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:01 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:07:01.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:01 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:07:01.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:01 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:01.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:01 smithi046 ceph-mon[30151]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T00:07:01.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:01 smithi046 ceph-mon[30151]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:07:01.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:01 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T00:07:01.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:01 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T00:07:01.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:01 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:02.342 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:02 smithi169 ceph-mon[34777]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:02.354 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:02 smithi142 ceph-mon[34533]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:02.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:02 smithi046 ceph-mon[30151]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:03.546 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:03 smithi142 ceph-mon[34533]: from='client.? 172.21.15.169:0/3657842949' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56b04b15-6fcd-497a-aa66-19da9624b656"}]: dispatch 2024-05-09T00:07:03.547 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:03 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56b04b15-6fcd-497a-aa66-19da9624b656"}]: dispatch 2024-05-09T00:07:03.547 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:03 smithi142 ceph-mon[34533]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56b04b15-6fcd-497a-aa66-19da9624b656"}]': finished 2024-05-09T00:07:03.547 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:03 smithi142 ceph-mon[34533]: osdmap e67: 12 total, 11 up, 12 in 2024-05-09T00:07:03.547 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:03 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:03.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:03 smithi046 ceph-mon[30151]: from='client.? 172.21.15.169:0/3657842949' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56b04b15-6fcd-497a-aa66-19da9624b656"}]: dispatch 2024-05-09T00:07:03.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:03 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56b04b15-6fcd-497a-aa66-19da9624b656"}]: dispatch 2024-05-09T00:07:03.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:03 smithi046 ceph-mon[30151]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56b04b15-6fcd-497a-aa66-19da9624b656"}]': finished 2024-05-09T00:07:03.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:03 smithi046 ceph-mon[30151]: osdmap e67: 12 total, 11 up, 12 in 2024-05-09T00:07:03.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:03 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:03.573 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:03 smithi169 ceph-mon[34777]: from='client.? 172.21.15.169:0/3657842949' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56b04b15-6fcd-497a-aa66-19da9624b656"}]: dispatch 2024-05-09T00:07:03.573 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:03 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56b04b15-6fcd-497a-aa66-19da9624b656"}]: dispatch 2024-05-09T00:07:03.573 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:03 smithi169 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56b04b15-6fcd-497a-aa66-19da9624b656"}]': finished 2024-05-09T00:07:03.573 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:03 smithi169 ceph-mon[34777]: osdmap e67: 12 total, 11 up, 12 in 2024-05-09T00:07:03.573 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:03 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:04.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:04 smithi046 ceph-mon[30151]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:04.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:04 smithi046 ceph-mon[30151]: from='client.? 172.21.15.169:0/1530425395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T00:07:04.600 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:04 smithi169 ceph-mon[34777]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:04.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:04 smithi169 ceph-mon[34777]: from='client.? 172.21.15.169:0/1530425395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T00:07:04.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:04 smithi142 ceph-mon[34533]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:04.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:04 smithi142 ceph-mon[34533]: from='client.? 172.21.15.169:0/1530425395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T00:07:06.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:06 smithi046 ceph-mon[30151]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-09T00:07:06.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:06 smithi169 ceph-mon[34777]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-09T00:07:06.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:06 smithi142 ceph-mon[34533]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-09T00:07:08.342 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:08 smithi169 ceph-mon[34777]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:07:08.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:08 smithi046 ceph-mon[30151]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:07:08.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:08 smithi142 ceph-mon[34533]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T00:07:10.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:10 smithi046 ceph-mon[30151]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-09T00:07:10.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:10 smithi169 ceph-mon[34777]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-09T00:07:10.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:10 smithi142 ceph-mon[34533]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-09T00:07:11.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:11 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T00:07:11.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:11 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:11.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:11 smithi046 ceph-mon[30151]: Deploying daemon osd.11 on smithi169 2024-05-09T00:07:11.588 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:11 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T00:07:11.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:11 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:11.589 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:11 smithi169 ceph-mon[34777]: Deploying daemon osd.11 on smithi169 2024-05-09T00:07:11.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:11 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T00:07:11.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:11 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:11.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:11 smithi142 ceph-mon[34533]: Deploying daemon osd.11 on smithi169 2024-05-09T00:07:12.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:12 smithi169 ceph-mon[34777]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:07:12.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:12 smithi046 ceph-mon[30151]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:07:12.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:12 smithi142 ceph-mon[34533]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:07:14.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:14 smithi169 ceph-mon[34777]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:07:14.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:14 smithi046 ceph-mon[30151]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:07:14.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:14 smithi142 ceph-mon[34533]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T00:07:16.588 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:16 smithi169 ceph-mon[34777]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T00:07:16.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:16 smithi142 ceph-mon[34533]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T00:07:16.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:16 smithi046 ceph-mon[30151]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T00:07:18.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:17 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:18.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:17 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:18.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:17 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:07:18.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:17 smithi169 ceph-mon[34777]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:18.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:17 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:18.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:17 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:18.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:17 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:07:18.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:17 smithi142 ceph-mon[34533]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:18.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:17 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:18.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:17 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:18.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:17 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:07:18.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:17 smithi046 ceph-mon[30151]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:20.496 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:20 smithi169 ceph-mon[34777]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:20.496 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:20.496 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:20.496 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:20.496 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:20.496 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:20.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:20 smithi142 ceph-mon[34533]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:20.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:20.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:20.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:20.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:20.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:20.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:20 smithi046 ceph-mon[30151]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:20.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:20.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:20.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:20.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:20.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:20.850 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.855+0000 7f79d0ff9700 1 -- 172.21.15.169:0/4130050617 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f79d4063c00 con 0x7f79c00b2880 2024-05-09T00:07:20.851 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 11 on host 'smithi169' 2024-05-09T00:07:20.853 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.858+0000 7f79d96cc700 1 -- 172.21.15.169:0/4130050617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f79c00b2880 msgr2=0x7f79c00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:20.853 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.858+0000 7f79d96cc700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f79c00b2880 0x7f79c00b4d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f79d419bb10 tx=0x7f79bc009190 comp rx=0 tx=0).stop 2024-05-09T00:07:20.853 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.858+0000 7f79d96cc700 1 -- 172.21.15.169:0/4130050617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d4104340 msgr2=0x7f79d419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:20.853 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.858+0000 7f79d96cc700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d4104340 0x7f79d419aab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f79c40057c0 tx=0x7f79c40057f0 comp rx=0 tx=0).stop 2024-05-09T00:07:20.853 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.858+0000 7f79d96cc700 1 -- 172.21.15.169:0/4130050617 shutdown_connections 2024-05-09T00:07:20.853 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.858+0000 7f79d96cc700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f79c00b2880 0x7f79c00b4d90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:20.853 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.858+0000 7f79d96cc700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79d4105e10 0x7f79d41a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:20.853 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.858+0000 7f79d96cc700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d4104340 0x7f79d419aab0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:20.854 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.858+0000 7f79d96cc700 1 --2- 172.21.15.169:0/4130050617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d4105470 0x7f79d419aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:20.854 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.858+0000 7f79d96cc700 1 -- 172.21.15.169:0/4130050617 >> 172.21.15.169:0/4130050617 conn(0x7f79d40ff9c0 msgr2=0x7f79d41086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:20.854 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.858+0000 7f79d96cc700 1 -- 172.21.15.169:0/4130050617 shutdown_connections 2024-05-09T00:07:20.854 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:20.858+0000 7f79d96cc700 1 -- 172.21.15.169:0/4130050617 wait complete. 2024-05-09T00:07:21.467 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:21 smithi169 ceph-mon[34777]: from='osd.11 [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T00:07:21.467 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:21 smithi169 ceph-mon[34777]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T00:07:21.467 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:21.467 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:21.467 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:07:21.493 DEBUG:teuthology.orchestra.run.smithi169:osd.11> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.11.service 2024-05-09T00:07:21.496 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-09T00:07:21.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-09T00:07:21.570 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:21 smithi046 ceph-mon[30151]: from='osd.11 [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T00:07:21.570 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:21 smithi046 ceph-mon[30151]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T00:07:21.570 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:21.570 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:21.570 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:07:21.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:21 smithi142 ceph-mon[34533]: from='osd.11 [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T00:07:21.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:21 smithi142 ceph-mon[34533]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T00:07:21.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:21.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:21.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:07:21.763 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:07:22.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:22 smithi046 ceph-mon[30151]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:22.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:22 smithi046 ceph-mon[30151]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T00:07:22.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:22 smithi046 ceph-mon[30151]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T00:07:22.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:22 smithi046 ceph-mon[30151]: from='osd.11 [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:07:22.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:22 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:22.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:22 smithi046 ceph-mon[30151]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:07:22.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:22 smithi169 ceph-mon[34777]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:22.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:22 smithi169 ceph-mon[34777]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T00:07:22.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:22 smithi169 ceph-mon[34777]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T00:07:22.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:22 smithi169 ceph-mon[34777]: from='osd.11 [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:07:22.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:22 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:22.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:22 smithi169 ceph-mon[34777]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:07:22.602 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:07:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:07:22.335+0000 7f86ade5d700 -1 osd.11 0 waiting for initial osdmap 2024-05-09T00:07:22.602 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:07:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:07:22.350+0000 7f86a9c12700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T00:07:22.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:22 smithi142 ceph-mon[34533]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:22.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:22 smithi142 ceph-mon[34533]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T00:07:22.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:22 smithi142 ceph-mon[34533]: osdmap e68: 12 total, 11 up, 12 in 2024-05-09T00:07:22.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:22 smithi142 ceph-mon[34533]: from='osd.11 [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:07:22.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:22 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:22.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:22 smithi142 ceph-mon[34533]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-05-09T00:07:23.586 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:23 smithi046 ceph-mon[30151]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T00:07:23.587 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:23 smithi046 ceph-mon[30151]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T00:07:23.587 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:23.587 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:23.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:23 smithi169 ceph-mon[34777]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T00:07:23.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:23 smithi169 ceph-mon[34777]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T00:07:23.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:23.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:23.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.636+0000 7f71cee4e700 1 -- 172.21.15.46:0/3381553165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71c81059b0 msgr2=0x7f71c8105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:23.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.637+0000 7f71cee4e700 1 --2- 172.21.15.46:0/3381553165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71c81059b0 0x7f71c8105dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009990 tx=0x7f71bc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:07:23.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.637+0000 7f71cee4e700 1 -- 172.21.15.46:0/3381553165 shutdown_connections 2024-05-09T00:07:23.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.637+0000 7f71cee4e700 1 --2- 172.21.15.46:0/3381553165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71c81049e0 0x7f71c810c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:23.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.637+0000 7f71cee4e700 1 --2- 172.21.15.46:0/3381553165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71c81059b0 0x7f71c8105dc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.637+0000 7f71cee4e700 1 --2- 172.21.15.46:0/3381553165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c8103f60 0x7f71c8104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.637+0000 7f71cee4e700 1 -- 172.21.15.46:0/3381553165 >> 172.21.15.46:0/3381553165 conn(0x7f71c80ff9d0 msgr2=0x7f71c8101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.637+0000 7f71cee4e700 1 -- 172.21.15.46:0/3381553165 shutdown_connections 2024-05-09T00:07:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.637+0000 7f71cee4e700 1 -- 172.21.15.46:0/3381553165 wait complete. 2024-05-09T00:07:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.637+0000 7f71cee4e700 1 Processor -- start 2024-05-09T00:07:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.637+0000 7f71cee4e700 1 -- start start 2024-05-09T00:07:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cee4e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c8103f60 0x7f71c8078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cee4e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71c81049e0 0x7f71c8076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cee4e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71c81059b0 0x7f71c8077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cee4e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71c810fed0 con 0x7f71c81059b0 2024-05-09T00:07:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cee4e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71c810fd50 con 0x7f71c8103f60 2024-05-09T00:07:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cee4e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8110050 con 0x7f71c81049e0 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cd3eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71c81059b0 0x7f71c8077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cd3eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71c81059b0 0x7f71c8077160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50638/0 (socket says 172.21.15.46:50638) 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cd3eb700 1 -- 172.21.15.46:0/436811358 learned_addr learned my addr 172.21.15.46:0/436811358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71c7fff700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71c81049e0 0x7f71c8076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71ccbea700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c8103f60 0x7f71c8078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cd3eb700 1 -- 172.21.15.46:0/436811358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71c81049e0 msgr2=0x7f71c8076c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cd3eb700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71c81049e0 0x7f71c8076c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cd3eb700 1 -- 172.21.15.46:0/436811358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c8103f60 msgr2=0x7f71c8078570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cd3eb700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c8103f60 0x7f71c8078570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cd3eb700 1 -- 172.21.15.46:0/436811358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c80778c0 con 0x7f71c81059b0 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71ccbea700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c8103f60 0x7f71c8078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cd3eb700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71c81059b0 0x7f71c8077160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f71c000ef30 tx=0x7f71c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71c5ffb700 1 -- 172.21.15.46:0/436811358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00092e0 con 0x7f71c81059b0 2024-05-09T00:07:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71c5ffb700 1 -- 172.21.15.46:0/436811358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c0004bb0 con 0x7f71c81059b0 2024-05-09T00:07:23.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71cee4e700 1 -- 172.21.15.46:0/436811358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c81a4b10 con 0x7f71c81059b0 2024-05-09T00:07:23.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.638+0000 7f71c5ffb700 1 -- 172.21.15.46:0/436811358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0004280 con 0x7f71c81059b0 2024-05-09T00:07:23.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.639+0000 7f71cee4e700 1 -- 172.21.15.46:0/436811358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c81a4f70 con 0x7f71c81059b0 2024-05-09T00:07:23.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.640+0000 7f71cee4e700 1 -- 172.21.15.46:0/436811358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c80512c0 con 0x7f71c81059b0 2024-05-09T00:07:23.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.641+0000 7f71c5ffb700 1 -- 172.21.15.46:0/436811358 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f71c001b030 con 0x7f71c81059b0 2024-05-09T00:07:23.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.642+0000 7f71c5ffb700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71b80b2880 0x7f71b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:23.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.642+0000 7f71ccbea700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71b80b2880 0x7f71b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:23.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.642+0000 7f71c5ffb700 1 -- 172.21.15.46:0/436811358 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f71c00e2360 con 0x7f71c81059b0 2024-05-09T00:07:23.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.642+0000 7f71ccbea700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71b80b2880 0x7f71b80b4d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b2e0 tx=0x7f71bc00b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:23.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.646+0000 7f71c5ffb700 1 -- 172.21.15.46:0/436811358 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71c0022020 con 0x7f71c81059b0 2024-05-09T00:07:23.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:23 smithi142 ceph-mon[34533]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-05-09T00:07:23.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:23 smithi142 ceph-mon[34533]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T00:07:23.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:23.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.861+0000 7f71cee4e700 1 -- 172.21.15.46:0/436811358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f71c802e960 con 0x7f71c81059b0 2024-05-09T00:07:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.861+0000 7f71c5ffb700 1 -- 172.21.15.46:0/436811358 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f71c0010040 con 0x7f71c81059b0 2024-05-09T00:07:23.856 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:07:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.863+0000 7f71cee4e700 1 -- 172.21.15.46:0/436811358 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71b80b2880 msgr2=0x7f71b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:23.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.863+0000 7f71cee4e700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71b80b2880 0x7f71b80b4d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b2e0 tx=0x7f71bc00b720 comp rx=0 tx=0).stop 2024-05-09T00:07:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.863+0000 7f71cee4e700 1 -- 172.21.15.46:0/436811358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71c81059b0 msgr2=0x7f71c8077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.863+0000 7f71cee4e700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71c81059b0 0x7f71c8077160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f71c000ef30 tx=0x7f71c000c3a0 comp rx=0 tx=0).stop 2024-05-09T00:07:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.864+0000 7f71cee4e700 1 -- 172.21.15.46:0/436811358 shutdown_connections 2024-05-09T00:07:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.864+0000 7f71cee4e700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71b80b2880 0x7f71b80b4d90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.864+0000 7f71cee4e700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71c81059b0 0x7f71c8077160 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.864+0000 7f71cee4e700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c8103f60 0x7f71c8078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.864+0000 7f71cee4e700 1 --2- 172.21.15.46:0/436811358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71c81049e0 0x7f71c8076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.864+0000 7f71cee4e700 1 -- 172.21.15.46:0/436811358 >> 172.21.15.46:0/436811358 conn(0x7f71c80ff9d0 msgr2=0x7f71c81086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.864+0000 7f71cee4e700 1 -- 172.21.15.46:0/436811358 shutdown_connections 2024-05-09T00:07:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:23.864+0000 7f71cee4e700 1 -- 172.21.15.46:0/436811358 wait complete. 2024-05-09T00:07:24.427 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:24 smithi169 ceph-mon[34777]: purged_snaps scrub starts 2024-05-09T00:07:24.427 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:24 smithi169 ceph-mon[34777]: purged_snaps scrub ok 2024-05-09T00:07:24.427 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:24 smithi169 ceph-mon[34777]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:24.427 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:24 smithi169 ceph-mon[34777]: osd.11 [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976] boot 2024-05-09T00:07:24.427 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:24 smithi169 ceph-mon[34777]: osdmap e70: 12 total, 12 up, 12 in 2024-05-09T00:07:24.427 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:24 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:24.427 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:24 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/436811358' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T00:07:24.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:24 smithi046 ceph-mon[30151]: purged_snaps scrub starts 2024-05-09T00:07:24.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:24 smithi046 ceph-mon[30151]: purged_snaps scrub ok 2024-05-09T00:07:24.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:24 smithi046 ceph-mon[30151]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:24.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:24 smithi046 ceph-mon[30151]: osd.11 [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976] boot 2024-05-09T00:07:24.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:24 smithi046 ceph-mon[30151]: osdmap e70: 12 total, 12 up, 12 in 2024-05-09T00:07:24.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:24 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:24.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:24 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/436811358' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T00:07:24.601 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1715213243,"num_in_osds":12,"osd_in_since":1715213223,"num_remapped_pgs":0} 2024-05-09T00:07:24.601 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:07:24.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:24 smithi142 ceph-mon[34533]: purged_snaps scrub starts 2024-05-09T00:07:24.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:24 smithi142 ceph-mon[34533]: purged_snaps scrub ok 2024-05-09T00:07:24.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:24 smithi142 ceph-mon[34533]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-09T00:07:24.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:24 smithi142 ceph-mon[34533]: osd.11 [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976] boot 2024-05-09T00:07:24.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:24 smithi142 ceph-mon[34533]: osdmap e70: 12 total, 12 up, 12 in 2024-05-09T00:07:24.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:24 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T00:07:24.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:24 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/436811358' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T00:07:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:07:25.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T00:07:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: Detected new or changed devices on smithi169 2024-05-09T00:07:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi169 to 4272M 2024-05-09T00:07:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:25.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:25.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T00:07:25.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: Detected new or changed devices on smithi169 2024-05-09T00:07:25.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:25.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:25.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:25.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:25.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:25.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:25.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi169 to 4272M 2024-05-09T00:07:25.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:25.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:25.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:25.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:25.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: osdmap e71: 12 total, 12 up, 12 in 2024-05-09T00:07:25.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: Detected new or changed devices on smithi169 2024-05-09T00:07:25.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:25.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:25.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:25.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:25.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:25.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:07:25.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi169 to 4272M 2024-05-09T00:07:25.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:25.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:25.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:25.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:26.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:26 smithi046 ceph-mon[30151]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:26.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.565+0000 7f6ebcd31700 1 -- 172.21.15.46:0/4171741905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb8102790 msgr2=0x7f6eb8102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:26.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.565+0000 7f6ebcd31700 1 --2- 172.21.15.46:0/4171741905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb8102790 0x7f6eb8102ba0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009990 tx=0x7f6eac009ca0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.566+0000 7f6ebcd31700 1 -- 172.21.15.46:0/4171741905 shutdown_connections 2024-05-09T00:07:26.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.566+0000 7f6ebcd31700 1 --2- 172.21.15.46:0/4171741905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb81030e0 0x7f6eb8111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.566+0000 7f6ebcd31700 1 --2- 172.21.15.46:0/4171741905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb8109300 0x7f6eb8109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.566+0000 7f6ebcd31700 1 --2- 172.21.15.46:0/4171741905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb8102790 0x7f6eb8102ba0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.566+0000 7f6ebcd31700 1 -- 172.21.15.46:0/4171741905 >> 172.21.15.46:0/4171741905 conn(0x7f6eb80fe520 msgr2=0x7f6eb8100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:26.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.566+0000 7f6ebcd31700 1 -- 172.21.15.46:0/4171741905 shutdown_connections 2024-05-09T00:07:26.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.566+0000 7f6ebcd31700 1 -- 172.21.15.46:0/4171741905 wait complete. 2024-05-09T00:07:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.566+0000 7f6ebcd31700 1 Processor -- start 2024-05-09T00:07:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6ebcd31700 1 -- start start 2024-05-09T00:07:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6ebcd31700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb8102790 0x7f6eb8103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6ebcd31700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb81030e0 0x7f6eb8103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6ebcd31700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb8109300 0x7f6eb81043f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6ebcd31700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb81145a0 con 0x7f6eb8109300 2024-05-09T00:07:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6ebcd31700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb8114420 con 0x7f6eb81030e0 2024-05-09T00:07:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6ebcd31700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb8114720 con 0x7f6eb8102790 2024-05-09T00:07:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6eb6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb8109300 0x7f6eb81043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6eb6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb8109300 0x7f6eb81043f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42710/0 (socket says 172.21.15.46:42710) 2024-05-09T00:07:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6eb6d9d700 1 -- 172.21.15.46:0/698207894 learned_addr learned my addr 172.21.15.46:0/698207894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6eb5d9b700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb81030e0 0x7f6eb8103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6eb659c700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb8102790 0x7f6eb8103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6eb6d9d700 1 -- 172.21.15.46:0/698207894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb8102790 msgr2=0x7f6eb8103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6eb6d9d700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb8102790 0x7f6eb8103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6eb6d9d700 1 -- 172.21.15.46:0/698207894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb81030e0 msgr2=0x7f6eb8103eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6eb6d9d700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb81030e0 0x7f6eb8103eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.567+0000 7f6eb6d9d700 1 -- 172.21.15.46:0/698207894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb81a9290 con 0x7f6eb8109300 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.568+0000 7f6eb6d9d700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb8109300 0x7f6eb81043f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0007ee0 tx=0x7f6eb000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.568+0000 7f6ea77fe700 1 -- 172.21.15.46:0/698207894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000dbd0 con 0x7f6eb8109300 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.568+0000 7f6ebcd31700 1 -- 172.21.15.46:0/698207894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb81a9580 con 0x7f6eb8109300 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.568+0000 7f6ea77fe700 1 -- 172.21.15.46:0/698207894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb000dd70 con 0x7f6eb8109300 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.568+0000 7f6ea77fe700 1 -- 172.21.15.46:0/698207894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0005990 con 0x7f6eb8109300 2024-05-09T00:07:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.568+0000 7f6ebcd31700 1 -- 172.21.15.46:0/698207894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb81a9a40 con 0x7f6eb8109300 2024-05-09T00:07:26.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.569+0000 7f6ebcd31700 1 -- 172.21.15.46:0/698207894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb80512c0 con 0x7f6eb8109300 2024-05-09T00:07:26.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.570+0000 7f6ea77fe700 1 -- 172.21.15.46:0/698207894 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6eb001a020 con 0x7f6eb8109300 2024-05-09T00:07:26.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.571+0000 7f6ea77fe700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6ea00b2880 0x7f6ea00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:26.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.571+0000 7f6eb659c700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6ea00b2880 0x7f6ea00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:26.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.571+0000 7f6ea77fe700 1 -- 172.21.15.46:0/698207894 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f6eb00e1510 con 0x7f6eb8109300 2024-05-09T00:07:26.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.572+0000 7f6eb659c700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6ea00b2880 0x7f6ea00b4d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8005950 tx=0x7f6ea80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:26.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.576+0000 7f6ea77fe700 1 -- 172.21.15.46:0/698207894 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6eb0010070 con 0x7f6eb8109300 2024-05-09T00:07:26.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.794+0000 7f6ebcd31700 1 -- 172.21.15.46:0/698207894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6eb802e5f0 con 0x7f6eb8109300 2024-05-09T00:07:26.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.795+0000 7f6ea77fe700 1 -- 172.21.15.46:0/698207894 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18452 (secure 0 0 0) 0x7f6eb00a96d0 con 0x7f6eb8109300 2024-05-09T00:07:26.790 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:07:26.790 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":71,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","created":"2024-05-09T00:00:14.356524+0000","modified":"2024-05-09T00:07:24.339514+0000","last_up_change":"2024-05-09T00:07:23.332796+0000","last_in_change":"2024-05-09T00:07:03.238539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:03:33.278910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"740d6388-5c1f-45eb-93b0-7583bfa5a225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6803","nonce":3727342125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6805","nonce":3727342125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6809","nonce":3727342125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6807","nonce":3727342125}]},"public_addr":"172.21.15.46:6803/3727342125","cluster_addr":"172.21.15.46:6805/3727342125","heartbeat_back_addr":"172.21.15.46:6809/3727342125","heartbeat_front_addr":"172.21.15.46:6807/3727342125","state":["exists","up"]},{"osd":1,"uuid":"85fbf3ae-0be8-4504-84d6-66fae0d2169d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6801","nonce":3156042469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6803","nonce":3156042469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6807","nonce":3156042469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6805","nonce":3156042469}]},"public_addr":"172.21.15.142:6801/3156042469","cluster_addr":"172.21.15.142:6803/3156042469","heartbeat_back_addr":"172.21.15.142:6807/3156042469","heartbeat_front_addr":"172.21.15.142:6805/3156042469","state":["exists","up"]},{"osd":2,"uuid":"05c070e7-83ad-4949-9834-4ce51ac0a2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6801","nonce":2680673064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6803","nonce":2680673064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6807","nonce":2680673064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6805","nonce":2680673064}]},"public_addr":"172.21.15.169:6801/2680673064","cluster_addr":"172.21.15.169:6803/2680673064","heartbeat_back_addr":"172.21.15.169:6807/2680673064","heartbeat_front_addr":"172.21.15.169:6805/2680673064","state":["exists","up"]},{"osd":3,"uuid":"09d7bbe1-58b6-4407-bef2-920f179dfb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6811","nonce":1158536261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6813","nonce":1158536261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6817","nonce":1158536261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6815","nonce":1158536261}]},"public_addr":"172.21.15.46:6811/1158536261","cluster_addr":"172.21.15.46:6813/1158536261","heartbeat_back_addr":"172.21.15.46:6817/1158536261","heartbeat_front_addr":"172.21.15.46:6815/1158536261","state":["exists","up"]},{"osd":4,"uuid":"66c9644b-6b0b-4fe7-aa63-b9d87bff7153","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6809","nonce":2299616865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6811","nonce":2299616865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6815","nonce":2299616865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6813","nonce":2299616865}]},"public_addr":"172.21.15.142:6809/2299616865","cluster_addr":"172.21.15.142:6811/2299616865","heartbeat_back_addr":"172.21.15.142:6815/2299616865","heartbeat_front_addr":"172.21.15.142:6813/2299616865","state":["exists","up"]},{"osd":5,"uuid":"96a72e01-1c5f-418e-bb9b-b790d9b3059b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6809","nonce":445044713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6811","nonce":445044713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6815","nonce":445044713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6813","nonce":445044713}]},"public_addr":"172.21.15.169:6809/445044713","cluster_addr":"172.21.15.169:6811/445044713","heartbeat_back_addr":"172.21.15.169:6815/445044713","heartbeat_front_addr":"172.21.15.169:6813/445044713","state":["exists","up"]},{"osd":6,"uuid":"6b459542-0e56-4369-b542-e36c46a18f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6819","nonce":1394035482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6821","nonce":1394035482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6825","nonce":1394035482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6823","nonce":1394035482}]},"public_addr":"172.21.15.46:6819/1394035482","cluster_addr":"172.21.15.46:6821/1394035482","heartbeat_back_addr":"172.21.15.46:6825/1394035482","heartbeat_front_addr":"172.21.15.46:6823/1394035482","state":["exists","up"]},{"osd":7,"uuid":"786b4517-4cbd-4fa3-b5a0-39cccf839893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6817","nonce":596805607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6819","nonce":596805607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6823","nonce":596805607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6821","nonce":596805607}]},"public_addr":"172.21.15.142:6817/596805607","cluster_addr":"172.21.15.142:6819/596805607","heartbeat_back_addr":"172.21.15.142:6823/596805607","heartbeat_front_addr":"172.21.15.142:6821/596805607","state":["exists","up"]},{"osd":8,"uuid":"b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6817","nonce":2650652376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6819","nonce":2650652376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6823","nonce":2650652376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6821","nonce":2650652376}]},"public_addr":"172.21.15.169:6817/2650652376","cluster_addr":"172.21.15.169:6819/2650652376","heartbeat_back_addr":"172.21.15.169:6823/2650652376","heartbeat_front_addr":"172.21.15.169:6821/2650652376","state":["exists","up"]},{"osd":9,"uuid":"32df0dab-5875-4beb-b733-60a915c94a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6827","nonce":374861248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6829","nonce":374861248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6833","nonce":374861248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6831","nonce":374861248}]},"public_addr":"172.21.15.46:6827/374861248","cluster_addr":"172.21.15.46:6829/374861248","heartbeat_back_addr":"172.21.15.46:6833/374861248","heartbeat_front_addr":"172.21.15.46:6831/374861248","state":["exists","up"]},{"osd":10,"uuid":"380a4f47-e528-4258-8836-32eac0e6e630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6825","nonce":1364891982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6827","nonce":1364891982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6831","nonce":1364891982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6829","nonce":1364891982}]},"public_addr":"172.21.15.142:6825/1364891982","cluster_addr":"172.21.15.142:6827/1364891982","heartbeat_back_addr":"172.21.15.142:6831/1364891982","heartbeat_front_addr":"172.21.15.142:6829/1364891982","state":["exists","up"]},{"osd":11,"uuid":"56b04b15-6fcd-497a-aa66-19da9624b656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6825","nonce":4262478976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6827","nonce":4262478976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6831","nonce":4262478976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6829","nonce":4262478976}]},"public_addr":"172.21.15.169:6825/4262478976","cluster_addr":"172.21.15.169:6827/4262478976","heartbeat_back_addr":"172.21.15.169:6831/4262478976","heartbeat_front_addr":"172.21.15.169:6829/4262478976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:02:42.159298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:06.053328+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:29.587137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:54.573782+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:20.276073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:45.320565+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:10.985522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:36.806137+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:02.266736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:29.788773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:55.827639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:07:21.762382+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/1072138393":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2786891364":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/7723180":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/626888490":"2024-05-10T00:01:09.191239+0000","172.21.15.46:6800/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/4236543921":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2168313126":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6800/953818590":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6801/953818590":"2024-05-10T00:00:41.833323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:07:26.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.797+0000 7f6ebcd31700 1 -- 172.21.15.46:0/698207894 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6ea00b2880 msgr2=0x7f6ea00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:26.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.797+0000 7f6ebcd31700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6ea00b2880 0x7f6ea00b4d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8005950 tx=0x7f6ea80058e0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.797+0000 7f6ebcd31700 1 -- 172.21.15.46:0/698207894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb8109300 msgr2=0x7f6eb81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:26.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.797+0000 7f6ebcd31700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb8109300 0x7f6eb81043f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0007ee0 tx=0x7f6eb000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.797+0000 7f6ebcd31700 1 -- 172.21.15.46:0/698207894 shutdown_connections 2024-05-09T00:07:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.797+0000 7f6ebcd31700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6ea00b2880 0x7f6ea00b4d90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.797+0000 7f6ebcd31700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb8109300 0x7f6eb81043f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.797+0000 7f6ebcd31700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb8102790 0x7f6eb8103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.797+0000 7f6ebcd31700 1 --2- 172.21.15.46:0/698207894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb81030e0 0x7f6eb8103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.797+0000 7f6ebcd31700 1 -- 172.21.15.46:0/698207894 >> 172.21.15.46:0/698207894 conn(0x7f6eb80fe520 msgr2=0x7f6eb8107140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.797+0000 7f6ebcd31700 1 -- 172.21.15.46:0/698207894 shutdown_connections 2024-05-09T00:07:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:26.797+0000 7f6ebcd31700 1 -- 172.21.15.46:0/698207894 wait complete. 2024-05-09T00:07:26.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:26 smithi169 ceph-mon[34777]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:26.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:26 smithi142 ceph-mon[34533]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:27.585 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T00:03:33.278910+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T00:07:27.585 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-09T00:07:27.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:27 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/698207894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:07:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:07:27.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:27 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/698207894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:07:27.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:27 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/698207894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:07:28.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:28 smithi046 ceph-mon[30151]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:28.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:28 smithi169 ceph-mon[34777]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:28.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:28 smithi142 ceph-mon[34533]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:29.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.513+0000 7f7f56d23700 1 -- 172.21.15.46:0/1864395426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50109870 msgr2=0x7f7f501111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:29.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.513+0000 7f7f56d23700 1 --2- 172.21.15.46:0/1864395426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50109870 0x7f7f501111d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800b0d0 tx=0x7f7f4800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:07:29.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.514+0000 7f7f56d23700 1 -- 172.21.15.46:0/1864395426 shutdown_connections 2024-05-09T00:07:29.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.514+0000 7f7f56d23700 1 --2- 172.21.15.46:0/1864395426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50109870 0x7f7f501111d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:29.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.514+0000 7f7f56d23700 1 --2- 172.21.15.46:0/1864395426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f501045f0 0x7f7f50104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:29.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.514+0000 7f7f56d23700 1 --2- 172.21.15.46:0/1864395426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f50108f20 0x7f7f50109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:29.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.514+0000 7f7f56d23700 1 -- 172.21.15.46:0/1864395426 >> 172.21.15.46:0/1864395426 conn(0x7f7f500fffe0 msgr2=0x7f7f50102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:29.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.514+0000 7f7f56d23700 1 -- 172.21.15.46:0/1864395426 shutdown_connections 2024-05-09T00:07:29.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.514+0000 7f7f56d23700 1 -- 172.21.15.46:0/1864395426 wait complete. 2024-05-09T00:07:29.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.514+0000 7f7f56d23700 1 Processor -- start 2024-05-09T00:07:29.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.514+0000 7f7f56d23700 1 -- start start 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f56d23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f501045f0 0x7f7f5019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f56d23700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f50108f20 0x7f7f5019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f56d23700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f50109870 0x7f7f5019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f56d23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501144a0 con 0x7f7f501045f0 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f56d23700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50114320 con 0x7f7f50108f20 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f56d23700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50114620 con 0x7f7f50109870 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f54abf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f501045f0 0x7f7f5019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f54abf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f501045f0 0x7f7f5019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42732/0 (socket says 172.21.15.46:42732) 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f54abf700 1 -- 172.21.15.46:0/1787949950 learned_addr learned my addr 172.21.15.46:0/1787949950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f50108f20 0x7f7f5019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f552c0700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f50109870 0x7f7f5019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f54abf700 1 -- 172.21.15.46:0/1787949950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f50109870 msgr2=0x7f7f5019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f54abf700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f50109870 0x7f7f5019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:29.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f54abf700 1 -- 172.21.15.46:0/1787949950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f50108f20 msgr2=0x7f7f5019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:29.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f54abf700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f50108f20 0x7f7f5019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:29.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f54abf700 1 -- 172.21.15.46:0/1787949950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5019fcc0 con 0x7f7f501045f0 2024-05-09T00:07:29.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.515+0000 7f7f54abf700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f501045f0 0x7f7f5019adf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000d950 tx=0x7f7f4000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:29.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.516+0000 7f7f4dffb700 1 -- 172.21.15.46:0/1787949950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40016070 con 0x7f7f501045f0 2024-05-09T00:07:29.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.516+0000 7f7f4dffb700 1 -- 172.21.15.46:0/1787949950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f4000fd40 con 0x7f7f501045f0 2024-05-09T00:07:29.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.516+0000 7f7f56d23700 1 -- 172.21.15.46:0/1787949950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5019ffb0 con 0x7f7f501045f0 2024-05-09T00:07:29.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.516+0000 7f7f4dffb700 1 -- 172.21.15.46:0/1787949950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4000e3b0 con 0x7f7f501045f0 2024-05-09T00:07:29.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.516+0000 7f7f56d23700 1 -- 172.21.15.46:0/1787949950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f501a78d0 con 0x7f7f501045f0 2024-05-09T00:07:29.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.517+0000 7f7f56d23700 1 -- 172.21.15.46:0/1787949950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f500512c0 con 0x7f7f501045f0 2024-05-09T00:07:29.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.518+0000 7f7f4dffb700 1 -- 172.21.15.46:0/1787949950 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7f40019070 con 0x7f7f501045f0 2024-05-09T00:07:29.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.519+0000 7f7f4dffb700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7f380b2880 0x7f7f380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:29.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.519+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7f380b2880 0x7f7f380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:29.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.519+0000 7f7f4dffb700 1 -- 172.21.15.46:0/1787949950 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7f400e1860 con 0x7f7f501045f0 2024-05-09T00:07:29.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.520+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7f380b2880 0x7f7f380b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7f50108800 tx=0x7f7f440073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:29.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.525+0000 7f7f4dffb700 1 -- 172.21.15.46:0/1787949950 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f400a9980 con 0x7f7f501045f0 2024-05-09T00:07:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.738+0000 7f7f56d23700 1 -- 172.21.15.46:0/1787949950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7f5002e5f0 con 0x7f7f501045f0 2024-05-09T00:07:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.739+0000 7f7f4dffb700 1 -- 172.21.15.46:0/1787949950 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f7f400a9280 con 0x7f7f501045f0 2024-05-09T00:07:29.734 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-09T00:07:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.741+0000 7f7f56d23700 1 -- 172.21.15.46:0/1787949950 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7f380b2880 msgr2=0x7f7f380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.741+0000 7f7f56d23700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7f380b2880 0x7f7f380b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7f50108800 tx=0x7f7f440073d0 comp rx=0 tx=0).stop 2024-05-09T00:07:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.741+0000 7f7f56d23700 1 -- 172.21.15.46:0/1787949950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f501045f0 msgr2=0x7f7f5019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.741+0000 7f7f56d23700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f501045f0 0x7f7f5019adf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000d950 tx=0x7f7f4000dc60 comp rx=0 tx=0).stop 2024-05-09T00:07:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.741+0000 7f7f56d23700 1 -- 172.21.15.46:0/1787949950 shutdown_connections 2024-05-09T00:07:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.741+0000 7f7f56d23700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7f380b2880 0x7f7f380b4d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.741+0000 7f7f56d23700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f50109870 0x7f7f5019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.741+0000 7f7f56d23700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f501045f0 0x7f7f5019adf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.741+0000 7f7f56d23700 1 --2- 172.21.15.46:0/1787949950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f50108f20 0x7f7f5019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.741+0000 7f7f56d23700 1 -- 172.21.15.46:0/1787949950 >> 172.21.15.46:0/1787949950 conn(0x7f7f500fffe0 msgr2=0x7f7f50106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.741+0000 7f7f56d23700 1 -- 172.21.15.46:0/1787949950 shutdown_connections 2024-05-09T00:07:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:29.741+0000 7f7f56d23700 1 -- 172.21.15.46:0/1787949950 wait complete. 2024-05-09T00:07:30.556 INFO:tasks.cephadm:Adding mds.a on smithi046 2024-05-09T00:07:30.556 INFO:tasks.cephadm:Adding mds.d on smithi046 2024-05-09T00:07:30.556 INFO:tasks.cephadm:Adding mds.g on smithi046 2024-05-09T00:07:30.556 INFO:tasks.cephadm:Adding mds.j on smithi046 2024-05-09T00:07:30.556 INFO:tasks.cephadm:Adding mds.b on smithi142 2024-05-09T00:07:30.557 INFO:tasks.cephadm:Adding mds.e on smithi142 2024-05-09T00:07:30.557 INFO:tasks.cephadm:Adding mds.h on smithi142 2024-05-09T00:07:30.557 INFO:tasks.cephadm:Adding mds.k on smithi142 2024-05-09T00:07:30.557 INFO:tasks.cephadm:Adding mds.c on smithi169 2024-05-09T00:07:30.557 INFO:tasks.cephadm:Adding mds.f on smithi169 2024-05-09T00:07:30.557 INFO:tasks.cephadm:Adding mds.i on smithi169 2024-05-09T00:07:30.557 INFO:tasks.cephadm:Adding mds.l on smithi169 2024-05-09T00:07:30.557 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l' 2024-05-09T00:07:30.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:30 smithi046 ceph-mon[30151]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:30.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:30 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1787949950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:07:30.810 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:07:30.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:30 smithi169 ceph-mon[34777]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:30.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:30 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1787949950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:07:30.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:30 smithi142 ceph-mon[34533]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:30.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:30 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1787949950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:07:32.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.536+0000 7ff78b632700 1 -- 172.21.15.169:0/1304440613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7841049e0 msgr2=0x7ff78410c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:32.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.536+0000 7ff78b632700 1 --2- 172.21.15.169:0/1304440613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7841049e0 0x7ff78410c6f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00b0a0 tx=0x7ff77c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:07:32.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.536+0000 7ff78b632700 1 -- 172.21.15.169:0/1304440613 shutdown_connections 2024-05-09T00:07:32.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.536+0000 7ff78b632700 1 --2- 172.21.15.169:0/1304440613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7841049e0 0x7ff78410c6f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:32.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.536+0000 7ff78b632700 1 --2- 172.21.15.169:0/1304440613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7841059b0 0x7ff784105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:32.533 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.536+0000 7ff78b632700 1 --2- 172.21.15.169:0/1304440613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff784103f60 0x7ff784104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.536+0000 7ff78b632700 1 -- 172.21.15.169:0/1304440613 >> 172.21.15.169:0/1304440613 conn(0x7ff7840ff9d0 msgr2=0x7ff784101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.536+0000 7ff78b632700 1 -- 172.21.15.169:0/1304440613 shutdown_connections 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.536+0000 7ff78b632700 1 -- 172.21.15.169:0/1304440613 wait complete. 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff78b632700 1 Processor -- start 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff78b632700 1 -- start start 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff78b632700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff784103f60 0x7ff78419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff78b632700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7841049e0 0x7ff78419d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff78b632700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7841059b0 0x7ff7841a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff78b632700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff78410fef0 con 0x7ff7841059b0 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff78b632700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff78410fd70 con 0x7ff7841049e0 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff78b632700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff784110070 con 0x7ff784103f60 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff7893ce700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff784103f60 0x7ff78419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff7893ce700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff784103f60 0x7ff78419cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42084/0 (socket says 172.21.15.169:42084) 2024-05-09T00:07:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff7893ce700 1 -- 172.21.15.169:0/3930480195 learned_addr learned my addr 172.21.15.169:0/3930480195 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-09T00:07:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff788bcd700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7841049e0 0x7ff78419d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff789bcf700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7841059b0 0x7ff7841a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff7893ce700 1 -- 172.21.15.169:0/3930480195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7841049e0 msgr2=0x7ff78419d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff7893ce700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7841049e0 0x7ff78419d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff7893ce700 1 -- 172.21.15.169:0/3930480195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7841059b0 msgr2=0x7ff7841a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff7893ce700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7841059b0 0x7ff7841a4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff7893ce700 1 -- 172.21.15.169:0/3930480195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7841a46c0 con 0x7ff784103f60 2024-05-09T00:07:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff789bcf700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7841059b0 0x7ff7841a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:07:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.537+0000 7ff7893ce700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff784103f60 0x7ff78419cb60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff78000d950 tx=0x7ff78000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.538+0000 7ff77a7fc700 1 -- 172.21.15.169:0/3930480195 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780016070 con 0x7ff784103f60 2024-05-09T00:07:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.538+0000 7ff77a7fc700 1 -- 172.21.15.169:0/3930480195 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff78000be00 con 0x7ff784103f60 2024-05-09T00:07:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.538+0000 7ff78b632700 1 -- 172.21.15.169:0/3930480195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7841a49b0 con 0x7ff784103f60 2024-05-09T00:07:32.536 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.538+0000 7ff77a7fc700 1 -- 172.21.15.169:0/3930480195 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78000f3c0 con 0x7ff784103f60 2024-05-09T00:07:32.536 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.538+0000 7ff78b632700 1 -- 172.21.15.169:0/3930480195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7841a4ef0 con 0x7ff784103f60 2024-05-09T00:07:32.536 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.539+0000 7ff78b632700 1 -- 172.21.15.169:0/3930480195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7840512c0 con 0x7ff784103f60 2024-05-09T00:07:32.536 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.541+0000 7ff77a7fc700 1 -- 172.21.15.169:0/3930480195 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff780019070 con 0x7ff784103f60 2024-05-09T00:07:32.537 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.542+0000 7ff77a7fc700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff7700b2880 0x7ff7700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:32.537 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.542+0000 7ff77a7fc700 1 -- 172.21.15.169:0/3930480195 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff7800e2150 con 0x7ff784103f60 2024-05-09T00:07:32.537 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.542+0000 7ff788bcd700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff7700b2880 0x7ff7700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:32.537 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.542+0000 7ff788bcd700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff7700b2880 0x7ff7700b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff78419dc10 tx=0x7ff774009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:32.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.545+0000 7ff77a7fc700 1 -- 172.21.15.169:0/3930480195 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff780020020 con 0x7ff784103f60 2024-05-09T00:07:32.755 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.759+0000 7ff78b632700 1 -- 172.21.15.169:0/3930480195 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}) -- 0x7ff784078320 con 0x7ff7700b2880 2024-05-09T00:07:32.763 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.768+0000 7ff77a7fc700 1 -- 172.21.15.169:0/3930480195 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ff784078320 con 0x7ff7700b2880 2024-05-09T00:07:32.763 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mds.all update... 2024-05-09T00:07:32.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.770+0000 7ff78b632700 1 -- 172.21.15.169:0/3930480195 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff7700b2880 msgr2=0x7ff7700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:32.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.770+0000 7ff78b632700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff7700b2880 0x7ff7700b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff78419dc10 tx=0x7ff774009260 comp rx=0 tx=0).stop 2024-05-09T00:07:32.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.770+0000 7ff78b632700 1 -- 172.21.15.169:0/3930480195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff784103f60 msgr2=0x7ff78419cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:32.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.770+0000 7ff78b632700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff784103f60 0x7ff78419cb60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff78000d950 tx=0x7ff78000dc60 comp rx=0 tx=0).stop 2024-05-09T00:07:32.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.770+0000 7ff78b632700 1 -- 172.21.15.169:0/3930480195 shutdown_connections 2024-05-09T00:07:32.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.770+0000 7ff78b632700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff7700b2880 0x7ff7700b4d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:32.767 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.770+0000 7ff78b632700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7841059b0 0x7ff7841a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:32.767 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.770+0000 7ff78b632700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff784103f60 0x7ff78419cb60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:32.767 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.770+0000 7ff78b632700 1 --2- 172.21.15.169:0/3930480195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7841049e0 0x7ff78419d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:32.767 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.770+0000 7ff78b632700 1 -- 172.21.15.169:0/3930480195 >> 172.21.15.169:0/3930480195 conn(0x7ff7840ff9d0 msgr2=0x7ff784076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:32.767 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.771+0000 7ff78b632700 1 -- 172.21.15.169:0/3930480195 shutdown_connections 2024-05-09T00:07:32.767 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-09T00:07:32.771+0000 7ff78b632700 1 -- 172.21.15.169:0/3930480195 wait complete. 2024-05-09T00:07:32.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:32 smithi046 ceph-mon[30151]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:32.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:32 smithi169 ceph-mon[34777]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:32.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:32 smithi142 ceph-mon[34533]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:33.425 DEBUG:teuthology.orchestra.run.smithi046:mds.a> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.a.service 2024-05-09T00:07:33.428 DEBUG:teuthology.orchestra.run.smithi046:mds.d> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.d.service 2024-05-09T00:07:33.471 DEBUG:teuthology.orchestra.run.smithi046:mds.g> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.g.service 2024-05-09T00:07:33.515 DEBUG:teuthology.orchestra.run.smithi046:mds.j> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.j.service 2024-05-09T00:07:33.558 DEBUG:teuthology.orchestra.run.smithi142:mds.b> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.b.service 2024-05-09T00:07:33.560 DEBUG:teuthology.orchestra.run.smithi142:mds.e> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.e.service 2024-05-09T00:07:33.605 DEBUG:teuthology.orchestra.run.smithi142:mds.h> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.h.service 2024-05-09T00:07:33.648 DEBUG:teuthology.orchestra.run.smithi142:mds.k> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.k.service 2024-05-09T00:07:33.691 DEBUG:teuthology.orchestra.run.smithi169:mds.c> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.c.service 2024-05-09T00:07:33.694 DEBUG:teuthology.orchestra.run.smithi169:mds.f> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.f.service 2024-05-09T00:07:33.737 DEBUG:teuthology.orchestra.run.smithi169:mds.i> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.i.service 2024-05-09T00:07:33.780 DEBUG:teuthology.orchestra.run.smithi169:mds.l> sudo journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.l.service 2024-05-09T00:07:33.824 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-09T00:07:33.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:07:33.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:33 smithi142 ceph-mon[34533]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:07:33.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:33 smithi142 ceph-mon[34533]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-05-09T00:07:33.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:33.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:07:33.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:33 smithi142 ceph-mon[34533]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:33.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:33.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:33.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:33.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:33.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:33.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:33 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:33.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:33 smithi142 ceph-mon[34533]: Deploying daemon mds.l on smithi169 2024-05-09T00:07:34.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:33 smithi046 ceph-mon[30151]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:07:34.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:33 smithi046 ceph-mon[30151]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-05-09T00:07:34.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:34.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:07:34.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:33 smithi046 ceph-mon[30151]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:34.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:34.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:34.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:34.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:34.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:34.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:33 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:34.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:33 smithi046 ceph-mon[30151]: Deploying daemon mds.l on smithi169 2024-05-09T00:07:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:07:34.086 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:33 smithi169 ceph-mon[34777]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:07:34.086 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:33 smithi169 ceph-mon[34777]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-05-09T00:07:34.086 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:34.086 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:07:34.086 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:33 smithi169 ceph-mon[34777]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:34.086 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:34.086 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:07:34.086 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:34.086 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:34.087 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:34.087 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:33 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:34.087 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:33 smithi169 ceph-mon[34777]: Deploying daemon mds.l on smithi169 2024-05-09T00:07:35.800 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:07:35 smithi169 systemd[1]: Starting Ceph mds.l for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:07:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.846+0000 7f0a03f91700 1 -- 172.21.15.46:0/152690923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc105480 msgr2=0x7f09fc105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.846+0000 7f0a03f91700 1 --2- 172.21.15.46:0/152690923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc105480 0x7f09fc105890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f09f8009990 tx=0x7f09f8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.846+0000 7f0a03f91700 1 -- 172.21.15.46:0/152690923 shutdown_connections 2024-05-09T00:07:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.846+0000 7f0a03f91700 1 --2- 172.21.15.46:0/152690923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09fc10e8f0 0x7f09fc110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.846+0000 7f0a03f91700 1 --2- 172.21.15.46:0/152690923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc105480 0x7f09fc105890 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.846+0000 7f0a03f91700 1 --2- 172.21.15.46:0/152690923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09fc105dd0 0x7f09fc10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.846+0000 7f0a03f91700 1 -- 172.21.15.46:0/152690923 >> 172.21.15.46:0/152690923 conn(0x7f09fc0fef30 msgr2=0x7f09fc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.847+0000 7f0a03f91700 1 -- 172.21.15.46:0/152690923 shutdown_connections 2024-05-09T00:07:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.847+0000 7f0a03f91700 1 -- 172.21.15.46:0/152690923 wait complete. 2024-05-09T00:07:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.847+0000 7f0a03f91700 1 Processor -- start 2024-05-09T00:07:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.847+0000 7f0a03f91700 1 -- start start 2024-05-09T00:07:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a03f91700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09fc105480 0x7f09fc19d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a03f91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc105dd0 0x7f09fc19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a03f91700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09fc10e8f0 0x7f09fc1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a03f91700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc114440 con 0x7f09fc105dd0 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a03f91700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1142c0 con 0x7f09fc10e8f0 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a03f91700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1145c0 con 0x7f09fc105480 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a0152c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc105dd0 0x7f09fc19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a0152c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc105dd0 0x7f09fc19d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42752/0 (socket says 172.21.15.46:42752) 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a0152c700 1 -- 172.21.15.46:0/3202322259 learned_addr learned my addr 172.21.15.46:0/3202322259 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a01d2d700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09fc105480 0x7f09fc19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a0252e700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09fc10e8f0 0x7f09fc1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a0152c700 1 -- 172.21.15.46:0/3202322259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09fc105480 msgr2=0x7f09fc19d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a0152c700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09fc105480 0x7f09fc19d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a0152c700 1 -- 172.21.15.46:0/3202322259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09fc10e8f0 msgr2=0x7f09fc1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a0152c700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09fc10e8f0 0x7f09fc1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a0152c700 1 -- 172.21.15.46:0/3202322259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc1a1f00 con 0x7f09fc105dd0 2024-05-09T00:07:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.848+0000 7f0a0152c700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc105dd0 0x7f09fc19d5d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00d950 tx=0x7f09ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.849+0000 7f09f2ffd700 1 -- 172.21.15.46:0/3202322259 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec016070 con 0x7f09fc105dd0 2024-05-09T00:07:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.849+0000 7f0a03f91700 1 -- 172.21.15.46:0/3202322259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09fc1a21f0 con 0x7f09fc105dd0 2024-05-09T00:07:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.849+0000 7f09f2ffd700 1 -- 172.21.15.46:0/3202322259 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09ec00fd40 con 0x7f09fc105dd0 2024-05-09T00:07:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.849+0000 7f09f2ffd700 1 -- 172.21.15.46:0/3202322259 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00e3b0 con 0x7f09fc105dd0 2024-05-09T00:07:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.849+0000 7f0a03f91700 1 -- 172.21.15.46:0/3202322259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09fc1a9b10 con 0x7f09fc105dd0 2024-05-09T00:07:35.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.850+0000 7f0a03f91700 1 -- 172.21.15.46:0/3202322259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09fc0512c0 con 0x7f09fc105dd0 2024-05-09T00:07:35.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.852+0000 7f09f2ffd700 1 -- 172.21.15.46:0/3202322259 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f09ec019070 con 0x7f09fc105dd0 2024-05-09T00:07:35.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.853+0000 7f09f2ffd700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f09e80b2880 0x7f09e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:35.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.853+0000 7f09f2ffd700 1 -- 172.21.15.46:0/3202322259 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f09ec0e17c0 con 0x7f09fc105dd0 2024-05-09T00:07:35.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.853+0000 7f0a01d2d700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f09e80b2880 0x7f09e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:35.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.853+0000 7f0a01d2d700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f09e80b2880 0x7f09e80b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f09f800b2e0 tx=0x7f09f8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:35.857+0000 7f09f2ffd700 1 -- 172.21.15.46:0/3202322259 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f09ec0a9980 con 0x7f09fc105dd0 2024-05-09T00:07:36.081 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:07:35 smithi169 podman[64333]: 2024-05-09 00:07:35.707512441 +0000 UTC m=+0.018216236 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:07:36.081 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:07:35 smithi169 podman[64333]: 2024-05-09 00:07:35.825379123 +0000 UTC m=+0.136082901 container create b0c6cb191e9f9c1dd7d77e64e840f2f26f118b59ef84b29f9f9100cbb72496e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-l, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-09T00:07:36.081 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:07:36 smithi169 podman[64333]: 2024-05-09 00:07:36.086906933 +0000 UTC m=+0.397610710 container init b0c6cb191e9f9c1dd7d77e64e840f2f26f118b59ef84b29f9f9100cbb72496e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T00:07:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.121+0000 7f0a03f91700 1 -- 172.21.15.46:0/3202322259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09fc1a9ce0 con 0x7f09fc105dd0 2024-05-09T00:07:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.122+0000 7f09f2ffd700 1 -- 172.21.15.46:0/3202322259 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f09ec0a9280 con 0x7f09fc105dd0 2024-05-09T00:07:36.117 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:07:36.117 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-09T00:07:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.124+0000 7f0a03f91700 1 -- 172.21.15.46:0/3202322259 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f09e80b2880 msgr2=0x7f09e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.124+0000 7f0a03f91700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f09e80b2880 0x7f09e80b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f09f800b2e0 tx=0x7f09f8005960 comp rx=0 tx=0).stop 2024-05-09T00:07:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.124+0000 7f0a03f91700 1 -- 172.21.15.46:0/3202322259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc105dd0 msgr2=0x7f09fc19d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.124+0000 7f0a03f91700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc105dd0 0x7f09fc19d5d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00d950 tx=0x7f09ec00dc60 comp rx=0 tx=0).stop 2024-05-09T00:07:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.124+0000 7f0a03f91700 1 -- 172.21.15.46:0/3202322259 shutdown_connections 2024-05-09T00:07:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.124+0000 7f0a03f91700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f09e80b2880 0x7f09e80b4d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.124+0000 7f0a03f91700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09fc10e8f0 0x7f09fc1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.124+0000 7f0a03f91700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09fc105480 0x7f09fc19d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.124+0000 7f0a03f91700 1 --2- 172.21.15.46:0/3202322259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09fc105dd0 0x7f09fc19d5d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:36.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.124+0000 7f0a03f91700 1 -- 172.21.15.46:0/3202322259 >> 172.21.15.46:0/3202322259 conn(0x7f09fc0fef30 msgr2=0x7f09fc1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:36.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.124+0000 7f0a03f91700 1 -- 172.21.15.46:0/3202322259 shutdown_connections 2024-05-09T00:07:36.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:36.124+0000 7f0a03f91700 1 -- 172.21.15.46:0/3202322259 wait complete. 2024-05-09T00:07:36.120 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 1 2024-05-09T00:07:36.351 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:07:36 smithi169 podman[64333]: 2024-05-09 00:07:36.089573586 +0000 UTC m=+0.400277370 container start b0c6cb191e9f9c1dd7d77e64e840f2f26f118b59ef84b29f9f9100cbb72496e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T00:07:36.351 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:07:36 smithi169 ceph-mds[64352]: -- 172.21.15.169:0/2159561455 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fca331fa40 con 0x55fca3367a80 2024-05-09T00:07:36.351 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:07:36 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-l[64348]: starting mds.l at 2024-05-09T00:07:36.351 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:07:36 smithi169 bash[64333]: b0c6cb191e9f9c1dd7d77e64e840f2f26f118b59ef84b29f9f9100cbb72496e9 2024-05-09T00:07:36.351 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:07:36 smithi169 systemd[1]: Started Ceph mds.l for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:07:36.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:36 smithi169 ceph-mon[34777]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:36.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:36 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3202322259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:36.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:36 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.169:6832/2870369857,v1:172.21.15.169:6833/2870369857] up:boot 2024-05-09T00:07:36.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:36 smithi169 ceph-mon[34777]: fsmap 1 up:standby 2024-05-09T00:07:36.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:07:36.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:36.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:36.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:36.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:36.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:36.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:36 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:36.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:36 smithi046 ceph-mon[30151]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:36.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:36 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3202322259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:36.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:36 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.169:6832/2870369857,v1:172.21.15.169:6833/2870369857] up:boot 2024-05-09T00:07:36.437 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:36 smithi046 ceph-mon[30151]: fsmap 1 up:standby 2024-05-09T00:07:36.437 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:07:36.437 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:36.437 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:36.437 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:36.437 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:36.437 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:36.437 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:36 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:36.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:36 smithi142 ceph-mon[34533]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:36.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:36 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3202322259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:36.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:36 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.169:6832/2870369857,v1:172.21.15.169:6833/2870369857] up:boot 2024-05-09T00:07:36.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:36 smithi142 ceph-mon[34533]: fsmap 1 up:standby 2024-05-09T00:07:36.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T00:07:36.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:36.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:36.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:36.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:36.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:36.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:36 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:37.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:37 smithi142 ceph-mon[34533]: Deploying daemon mds.k on smithi142 2024-05-09T00:07:37.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:37 smithi169 ceph-mon[34777]: Deploying daemon mds.k on smithi142 2024-05-09T00:07:37.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:37 smithi046 ceph-mon[30151]: Deploying daemon mds.k on smithi142 2024-05-09T00:07:38.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:38 smithi169 ceph-mon[34777]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:38.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:38 smithi142 ceph-mon[34533]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:38.647 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:07:38 smithi142 systemd[1]: Starting Ceph mds.k for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:07:38.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:38 smithi046 ceph-mon[30151]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:38.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:07:38.938 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:07:38 smithi142 podman[63965]: 2024-05-09 00:07:38.775064073 +0000 UTC m=+0.018105141 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:07:38.939 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:07:38 smithi142 podman[63965]: 2024-05-09 00:07:38.877022854 +0000 UTC m=+0.120063924 container create 2b3a54540edf4897d3c8d56f6a276ee7137d2e90c84cad3076c00c422bfb3cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-k, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-09T00:07:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:07:39.438 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:07:39 smithi142 podman[63965]: 2024-05-09 00:07:39.113005949 +0000 UTC m=+0.356047017 container init 2b3a54540edf4897d3c8d56f6a276ee7137d2e90c84cad3076c00c422bfb3cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5) 2024-05-09T00:07:39.438 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:07:39 smithi142 podman[63965]: 2024-05-09 00:07:39.115915228 +0000 UTC m=+0.358956296 container start 2b3a54540edf4897d3c8d56f6a276ee7137d2e90c84cad3076c00c422bfb3cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-k, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T00:07:39.438 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:07:39 smithi142 ceph-mds[63985]: -- 172.21.15.142:0/816475630 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b6ef9ba40 con 0x555b6efe3600 2024-05-09T00:07:39.439 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:07:39 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-k[63981]: starting mds.k at 2024-05-09T00:07:39.439 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:07:39 smithi142 bash[63965]: 2b3a54540edf4897d3c8d56f6a276ee7137d2e90c84cad3076c00c422bfb3cfc 2024-05-09T00:07:39.439 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:07:39 smithi142 systemd[1]: Started Ceph mds.k for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:07:40.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:40 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:40.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:40 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:40.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:40 smithi046 ceph-mon[30151]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:40.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:40 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:40.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:40 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:40.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:40 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:40.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:40 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:40.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:40 smithi046 ceph-mon[30151]: Deploying daemon mds.j on smithi046 2024-05-09T00:07:40.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:40 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.142:6832/512022880,v1:172.21.15.142:6833/512022880] up:boot 2024-05-09T00:07:40.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:40 smithi046 ceph-mon[30151]: fsmap 2 up:standby 2024-05-09T00:07:40.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:40 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:07:40.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:40 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:40.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:40 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:40.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:40 smithi169 ceph-mon[34777]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:40.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:40 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:40.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:40 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:40.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:40 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:40.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:40 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:40.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:40 smithi169 ceph-mon[34777]: Deploying daemon mds.j on smithi046 2024-05-09T00:07:40.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:40 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.142:6832/512022880,v1:172.21.15.142:6833/512022880] up:boot 2024-05-09T00:07:40.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:40 smithi169 ceph-mon[34777]: fsmap 2 up:standby 2024-05-09T00:07:40.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:40 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:07:40.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:40 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:40.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:40 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:40.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:40 smithi142 ceph-mon[34533]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:40.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:40 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:40.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:40 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:40.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:40 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:40.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:40 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:40.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:40 smithi142 ceph-mon[34533]: Deploying daemon mds.j on smithi046 2024-05-09T00:07:40.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:40 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.142:6832/512022880,v1:172.21.15.142:6833/512022880] up:boot 2024-05-09T00:07:40.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:40 smithi142 ceph-mon[34533]: fsmap 2 up:standby 2024-05-09T00:07:40.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:40 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T00:07:41.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.622+0000 7f629464a700 1 -- 172.21.15.46:0/3705909449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f628c103550 msgr2=0x7f628c103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.622+0000 7f629464a700 1 --2- 172.21.15.46:0/3705909449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f628c103550 0x7f628c103960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f62880099c0 tx=0x7f6288009cd0 comp rx=0 tx=0).stop 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.622+0000 7f629464a700 1 -- 172.21.15.46:0/3705909449 shutdown_connections 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.622+0000 7f629464a700 1 --2- 172.21.15.46:0/3705909449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628c108780 0x7f628c1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.622+0000 7f629464a700 1 --2- 172.21.15.46:0/3705909449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f628c103550 0x7f628c103960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.622+0000 7f629464a700 1 --2- 172.21.15.46:0/3705909449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f628c107e30 0x7f628c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.622+0000 7f629464a700 1 -- 172.21.15.46:0/3705909449 >> 172.21.15.46:0/3705909449 conn(0x7f628c078e00 msgr2=0x7f628c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.622+0000 7f629464a700 1 -- 172.21.15.46:0/3705909449 shutdown_connections 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.622+0000 7f629464a700 1 -- 172.21.15.46:0/3705909449 wait complete. 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.623+0000 7f629464a700 1 Processor -- start 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.623+0000 7f629464a700 1 -- start start 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.623+0000 7f629464a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628c103550 0x7f628c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.623+0000 7f629464a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f628c107e30 0x7f628c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f629464a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f628c108780 0x7f628c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f629464a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f628c112310 con 0x7f628c107e30 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f629464a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f628c112190 con 0x7f628c108780 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f629464a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f628c112490 con 0x7f628c103550 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f6291be5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f628c107e30 0x7f628c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f62923e6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628c103550 0x7f628c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f6291be5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f628c107e30 0x7f628c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40262/0 (socket says 172.21.15.46:40262) 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f6291be5700 1 -- 172.21.15.46:0/3610370644 learned_addr learned my addr 172.21.15.46:0/3610370644 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f6292be7700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f628c108780 0x7f628c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f6291be5700 1 -- 172.21.15.46:0/3610370644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628c103550 msgr2=0x7f628c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f6291be5700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628c103550 0x7f628c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f6291be5700 1 -- 172.21.15.46:0/3610370644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f628c108780 msgr2=0x7f628c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f6291be5700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f628c108780 0x7f628c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f6291be5700 1 -- 172.21.15.46:0/3610370644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628c199120 con 0x7f628c107e30 2024-05-09T00:07:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f6291be5700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f628c107e30 0x7f628c19ed90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f627c0099f0 tx=0x7f627c009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f62837fe700 1 -- 172.21.15.46:0/3610370644 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c004830 con 0x7f628c107e30 2024-05-09T00:07:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.624+0000 7f629464a700 1 -- 172.21.15.46:0/3610370644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628c199410 con 0x7f628c107e30 2024-05-09T00:07:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.625+0000 7f62837fe700 1 -- 172.21.15.46:0/3610370644 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f627c00c760 con 0x7f628c107e30 2024-05-09T00:07:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.625+0000 7f62837fe700 1 -- 172.21.15.46:0/3610370644 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c005260 con 0x7f628c107e30 2024-05-09T00:07:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.625+0000 7f629464a700 1 -- 172.21.15.46:0/3610370644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f628c1a56e0 con 0x7f628c107e30 2024-05-09T00:07:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.626+0000 7f629464a700 1 -- 172.21.15.46:0/3610370644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f628c0512c0 con 0x7f628c107e30 2024-05-09T00:07:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.627+0000 7f62837fe700 1 -- 172.21.15.46:0/3610370644 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f627c0049d0 con 0x7f628c107e30 2024-05-09T00:07:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.628+0000 7f62837fe700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f62780b2880 0x7f62780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.628+0000 7f62923e6700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f62780b2880 0x7f62780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.628+0000 7f62837fe700 1 -- 172.21.15.46:0/3610370644 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f627c0e1220 con 0x7f628c107e30 2024-05-09T00:07:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.629+0000 7f62923e6700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f62780b2880 0x7f62780b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6288006010 tx=0x7f6288000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.633+0000 7f62837fe700 1 -- 172.21.15.46:0/3610370644 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f627c0a93e0 con 0x7f628c107e30 2024-05-09T00:07:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.899+0000 7f629464a700 1 -- 172.21.15.46:0/3610370644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f628c199fa0 con 0x7f628c107e30 2024-05-09T00:07:41.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.899+0000 7f62837fe700 1 -- 172.21.15.46:0/3610370644 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f627c0a8ce0 con 0x7f628c107e30 2024-05-09T00:07:41.894 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:07:41.894 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T00:07:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.901+0000 7f629464a700 1 -- 172.21.15.46:0/3610370644 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f62780b2880 msgr2=0x7f62780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.901+0000 7f629464a700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f62780b2880 0x7f62780b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6288006010 tx=0x7f6288000bc0 comp rx=0 tx=0).stop 2024-05-09T00:07:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.901+0000 7f629464a700 1 -- 172.21.15.46:0/3610370644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f628c107e30 msgr2=0x7f628c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.901+0000 7f629464a700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f628c107e30 0x7f628c19ed90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f627c0099f0 tx=0x7f627c009d00 comp rx=0 tx=0).stop 2024-05-09T00:07:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.902+0000 7f629464a700 1 -- 172.21.15.46:0/3610370644 shutdown_connections 2024-05-09T00:07:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.902+0000 7f629464a700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f62780b2880 0x7f62780b4d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.902+0000 7f629464a700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f628c108780 0x7f628c198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.902+0000 7f629464a700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628c103550 0x7f628c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.902+0000 7f629464a700 1 --2- 172.21.15.46:0/3610370644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f628c107e30 0x7f628c19ed90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.902+0000 7f629464a700 1 -- 172.21.15.46:0/3610370644 >> 172.21.15.46:0/3610370644 conn(0x7f628c078e00 msgr2=0x7f628c100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.902+0000 7f629464a700 1 -- 172.21.15.46:0/3610370644 shutdown_connections 2024-05-09T00:07:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:41.902+0000 7f629464a700 1 -- 172.21.15.46:0/3610370644 wait complete. 2024-05-09T00:07:41.898 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 3 2024-05-09T00:07:42.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:42 smithi046 ceph-mon[30151]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:42.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:42 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3610370644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:42.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:42 smithi169 ceph-mon[34777]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:42.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:42 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3610370644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:42.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:42 smithi142 ceph-mon[34533]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:42.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:42 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3610370644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:42.984 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:07:42 smithi046 systemd[1]: Starting Ceph mds.j for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:07:43.299 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:07:43 smithi046 podman[68803]: 2024-05-09 00:07:43.182362621 +0000 UTC m=+0.098097588 container create 4b11d193045daa5b7e9e781152a25c7932e0b437afef9b3b3a4fe1382bf251b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-09T00:07:43.299 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:07:43 smithi046 podman[68803]: 2024-05-09 00:07:43.103577336 +0000 UTC m=+0.019312283 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:07:43.809 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:07:43 smithi046 podman[68803]: 2024-05-09 00:07:43.435198359 +0000 UTC m=+0.350933294 container init 4b11d193045daa5b7e9e781152a25c7932e0b437afef9b3b3a4fe1382bf251b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-j, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:07:43.809 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:07:43 smithi046 podman[68803]: 2024-05-09 00:07:43.438325094 +0000 UTC m=+0.354060027 container start 4b11d193045daa5b7e9e781152a25c7932e0b437afef9b3b3a4fe1382bf251b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-j, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:07:43.809 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:07:43 smithi046 ceph-mds[68822]: -- 172.21.15.46:0/807609082 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e36bb9a40 con 0x558e36c01a80 2024-05-09T00:07:43.809 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:07:43 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-j[68818]: starting mds.j at 2024-05-09T00:07:43.809 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:07:43 smithi046 bash[68803]: 4b11d193045daa5b7e9e781152a25c7932e0b437afef9b3b3a4fe1382bf251b8 2024-05-09T00:07:43.809 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:07:43 smithi046 systemd[1]: Started Ceph mds.j for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:07:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:44 smithi169 ceph-mon[34777]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:44.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:44.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:44.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:44.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:44 smithi169 ceph-mon[34777]: Deploying daemon mds.d on smithi046 2024-05-09T00:07:44.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:44 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.46:6834/1791470384,v1:172.21.15.46:6835/1791470384] up:boot 2024-05-09T00:07:44.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:44 smithi169 ceph-mon[34777]: fsmap 3 up:standby 2024-05-09T00:07:44.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T00:07:44.937 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:44 smithi046 ceph-mon[30151]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:44.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:44.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:44.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:44.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:44.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:44.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:44.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:44 smithi046 ceph-mon[30151]: Deploying daemon mds.d on smithi046 2024-05-09T00:07:44.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:44 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.46:6834/1791470384,v1:172.21.15.46:6835/1791470384] up:boot 2024-05-09T00:07:44.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:44 smithi046 ceph-mon[30151]: fsmap 3 up:standby 2024-05-09T00:07:44.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T00:07:44.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:44 smithi142 ceph-mon[34533]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:44.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:44.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:44.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:44.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:44.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:44.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:44.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:44 smithi142 ceph-mon[34533]: Deploying daemon mds.d on smithi046 2024-05-09T00:07:44.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:44 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.46:6834/1791470384,v1:172.21.15.46:6835/1791470384] up:boot 2024-05-09T00:07:44.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:44 smithi142 ceph-mon[34533]: fsmap 3 up:standby 2024-05-09T00:07:44.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T00:07:45.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:07:45.408 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:07:46.699 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:46 smithi046 ceph-mon[30151]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:46.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:46 smithi169 ceph-mon[34777]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:46.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:46 smithi142 ceph-mon[34533]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.275+0000 7fb322f54700 1 -- 172.21.15.46:0/4106909930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c107df0 msgr2=0x7fb31c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.275+0000 7fb322f54700 1 --2- 172.21.15.46:0/4106909930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c107df0 0x7fb31c108200 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb3080099c0 tx=0x7fb308009cd0 comp rx=0 tx=0).stop 2024-05-09T00:07:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.275+0000 7fb322f54700 1 -- 172.21.15.46:0/4106909930 shutdown_connections 2024-05-09T00:07:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.275+0000 7fb322f54700 1 --2- 172.21.15.46:0/4106909930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb31c108740 0x7fb31c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.275+0000 7fb322f54700 1 --2- 172.21.15.46:0/4106909930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb31c1034c0 0x7fb31c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.275+0000 7fb322f54700 1 --2- 172.21.15.46:0/4106909930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c107df0 0x7fb31c108200 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.275+0000 7fb322f54700 1 -- 172.21.15.46:0/4106909930 >> 172.21.15.46:0/4106909930 conn(0x7fb31c0fedf0 msgr2=0x7fb31c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.275+0000 7fb322f54700 1 -- 172.21.15.46:0/4106909930 shutdown_connections 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.275+0000 7fb322f54700 1 -- 172.21.15.46:0/4106909930 wait complete. 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.276+0000 7fb322f54700 1 Processor -- start 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.276+0000 7fb322f54700 1 -- start start 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.276+0000 7fb322f54700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb31c1034c0 0x7fb31c19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.276+0000 7fb322f54700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb31c107df0 0x7fb31c19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.276+0000 7fb322f54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c108740 0x7fb31c1a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.276+0000 7fb322f54700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c113350 con 0x7fb31c108740 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.276+0000 7fb322f54700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1131d0 con 0x7fb31c107df0 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.276+0000 7fb322f54700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1134d0 con 0x7fb31c1034c0 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.276+0000 7fb3214f1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c108740 0x7fb31c1a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.276+0000 7fb3214f1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c108740 0x7fb31c1a6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52220/0 (socket says 172.21.15.46:52220) 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.276+0000 7fb3214f1700 1 -- 172.21.15.46:0/1849434545 learned_addr learned my addr 172.21.15.46:0/1849434545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb31bfff700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb31c107df0 0x7fb31c19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb320cf0700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb31c1034c0 0x7fb31c19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb3214f1700 1 -- 172.21.15.46:0/1849434545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb31c1034c0 msgr2=0x7fb31c19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb3214f1700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb31c1034c0 0x7fb31c19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb3214f1700 1 -- 172.21.15.46:0/1849434545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb31c107df0 msgr2=0x7fb31c19c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb3214f1700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb31c107df0 0x7fb31c19c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb3214f1700 1 -- 172.21.15.46:0/1849434545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c19ca40 con 0x7fb31c108740 2024-05-09T00:07:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb3214f1700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c108740 0x7fb31c1a6cd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb31400ada0 tx=0x7fb31400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb319ffb700 1 -- 172.21.15.46:0/1849434545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314017020 con 0x7fb31c108740 2024-05-09T00:07:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb319ffb700 1 -- 172.21.15.46:0/1849434545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb31400ce30 con 0x7fb31c108740 2024-05-09T00:07:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb322f54700 1 -- 172.21.15.46:0/1849434545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c1a7330 con 0x7fb31c108740 2024-05-09T00:07:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb319ffb700 1 -- 172.21.15.46:0/1849434545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314005350 con 0x7fb31c108740 2024-05-09T00:07:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.277+0000 7fb322f54700 1 -- 172.21.15.46:0/1849434545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c1a77f0 con 0x7fb31c108740 2024-05-09T00:07:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.278+0000 7fb322f54700 1 -- 172.21.15.46:0/1849434545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31c0512c0 con 0x7fb31c108740 2024-05-09T00:07:47.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.281+0000 7fb319ffb700 1 -- 172.21.15.46:0/1849434545 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb31400c980 con 0x7fb31c108740 2024-05-09T00:07:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.282+0000 7fb319ffb700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb30c0b2880 0x7fb30c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.282+0000 7fb320cf0700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb30c0b2880 0x7fb30c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.282+0000 7fb319ffb700 1 -- 172.21.15.46:0/1849434545 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fb3140e1a80 con 0x7fb31c108740 2024-05-09T00:07:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.283+0000 7fb320cf0700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb30c0b2880 0x7fb30c0b4d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb310005fd0 tx=0x7fb310005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.286+0000 7fb319ffb700 1 -- 172.21.15.46:0/1849434545 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb314021020 con 0x7fb31c108740 2024-05-09T00:07:47.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.556+0000 7fb322f54700 1 -- 172.21.15.46:0/1849434545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb31c02e7b0 con 0x7fb31c108740 2024-05-09T00:07:47.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.556+0000 7fb319ffb700 1 -- 172.21.15.46:0/1849434545 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fb3140a9c30 con 0x7fb31c108740 2024-05-09T00:07:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:07:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T00:07:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.559+0000 7fb322f54700 1 -- 172.21.15.46:0/1849434545 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb30c0b2880 msgr2=0x7fb30c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.559+0000 7fb322f54700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb30c0b2880 0x7fb30c0b4d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb310005fd0 tx=0x7fb310005f60 comp rx=0 tx=0).stop 2024-05-09T00:07:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.559+0000 7fb322f54700 1 -- 172.21.15.46:0/1849434545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c108740 msgr2=0x7fb31c1a6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.559+0000 7fb322f54700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c108740 0x7fb31c1a6cd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb31400ada0 tx=0x7fb31400c330 comp rx=0 tx=0).stop 2024-05-09T00:07:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.559+0000 7fb322f54700 1 -- 172.21.15.46:0/1849434545 shutdown_connections 2024-05-09T00:07:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.559+0000 7fb322f54700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb30c0b2880 0x7fb30c0b4d90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.559+0000 7fb322f54700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c108740 0x7fb31c1a6cd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.559+0000 7fb322f54700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb31c1034c0 0x7fb31c19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.559+0000 7fb322f54700 1 --2- 172.21.15.46:0/1849434545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb31c107df0 0x7fb31c19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.559+0000 7fb322f54700 1 -- 172.21.15.46:0/1849434545 >> 172.21.15.46:0/1849434545 conn(0x7fb31c0fedf0 msgr2=0x7fb31c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.559+0000 7fb322f54700 1 -- 172.21.15.46:0/1849434545 shutdown_connections 2024-05-09T00:07:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:47.559+0000 7fb322f54700 1 -- 172.21.15.46:0/1849434545 wait complete. 2024-05-09T00:07:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 4 2024-05-09T00:07:47.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:47 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1849434545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:47.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:47 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1849434545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:47.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:47 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1849434545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:48.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:48 smithi142 ceph-mon[34533]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:48.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:48 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:48.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:48 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:48.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:48 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:48.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:48 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:48.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:48 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:48.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:48 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:48.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:48 smithi046 ceph-mon[30151]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:48.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:48 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:48.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:48 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:48.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:48 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:48.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:48 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:48.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:48 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:48.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:48 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:48.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:48 smithi169 ceph-mon[34777]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:48.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:48 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:48.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:48 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:48.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:48 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:48.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:48 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:48.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:48 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:48.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:48 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:49.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:49 smithi142 ceph-mon[34533]: Deploying daemon mds.e on smithi142 2024-05-09T00:07:49.798 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:49 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.46:6836/4102968476,v1:172.21.15.46:6837/4102968476] up:boot 2024-05-09T00:07:49.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:49 smithi142 ceph-mon[34533]: fsmap 4 up:standby 2024-05-09T00:07:49.799 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T00:07:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:49 smithi046 ceph-mon[30151]: Deploying daemon mds.e on smithi142 2024-05-09T00:07:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:49 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.46:6836/4102968476,v1:172.21.15.46:6837/4102968476] up:boot 2024-05-09T00:07:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:49 smithi046 ceph-mon[30151]: fsmap 4 up:standby 2024-05-09T00:07:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T00:07:50.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:49 smithi169 ceph-mon[34777]: Deploying daemon mds.e on smithi142 2024-05-09T00:07:50.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:49 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.46:6836/4102968476,v1:172.21.15.46:6837/4102968476] up:boot 2024-05-09T00:07:50.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:49 smithi169 ceph-mon[34777]: fsmap 4 up:standby 2024-05-09T00:07:50.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T00:07:50.439 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:07:50 smithi142 podman[64288]: 2024-05-09 00:07:50.422501962 +0000 UTC m=+0.135918473 container create 8c35d1b6fb6e103280bbbcfaa4c7dfab5b15e36db3c88f0eb359065e1bec628d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T00:07:50.455 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:07:50.723 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:07:50.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:50 smithi046 ceph-mon[30151]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:50.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:50 smithi142 ceph-mon[34533]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:50.939 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:07:50 smithi142 podman[64288]: 2024-05-09 00:07:50.675479261 +0000 UTC m=+0.388895782 container init 8c35d1b6fb6e103280bbbcfaa4c7dfab5b15e36db3c88f0eb359065e1bec628d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-09T00:07:50.939 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:07:50 smithi142 podman[64288]: 2024-05-09 00:07:50.678187786 +0000 UTC m=+0.391604304 container start 8c35d1b6fb6e103280bbbcfaa4c7dfab5b15e36db3c88f0eb359065e1bec628d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-09T00:07:50.939 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:07:50 smithi142 ceph-mds[64307]: -- 172.21.15.142:0/1122424882 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf602a5a40 con 0x55cf602eda80 2024-05-09T00:07:50.939 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:07:50 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-e[64303]: starting mds.e at 2024-05-09T00:07:50.939 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:07:50 smithi142 bash[64288]: 8c35d1b6fb6e103280bbbcfaa4c7dfab5b15e36db3c88f0eb359065e1bec628d 2024-05-09T00:07:50.939 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:07:50 smithi142 systemd[1]: Started Ceph mds.e for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:07:51.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:50 smithi169 ceph-mon[34777]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:52.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:51 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:52.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:51 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:52.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:51 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:52.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:51 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:52.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:51 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:52.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:51 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:52.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:51 smithi046 ceph-mon[30151]: Deploying daemon mds.f on smithi169 2024-05-09T00:07:52.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:51 smithi046 ceph-mon[30151]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:52.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:51 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.142:6834/2701851507,v1:172.21.15.142:6835/2701851507] up:boot 2024-05-09T00:07:52.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:51 smithi046 ceph-mon[30151]: fsmap 5 up:standby 2024-05-09T00:07:52.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:51 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T00:07:52.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:51 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:52.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:51 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:52.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:51 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:52.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:51 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:52.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:51 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:52.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:51 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:52.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:51 smithi169 ceph-mon[34777]: Deploying daemon mds.f on smithi169 2024-05-09T00:07:52.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:51 smithi169 ceph-mon[34777]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:52.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:51 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.142:6834/2701851507,v1:172.21.15.142:6835/2701851507] up:boot 2024-05-09T00:07:52.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:51 smithi169 ceph-mon[34777]: fsmap 5 up:standby 2024-05-09T00:07:52.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:51 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T00:07:52.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:51 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:52.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:51 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:52.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:51 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:52.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:51 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:52.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:51 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:52.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:51 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:52.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:51 smithi142 ceph-mon[34533]: Deploying daemon mds.f on smithi169 2024-05-09T00:07:52.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:51 smithi142 ceph-mon[34533]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:52.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:51 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.142:6834/2701851507,v1:172.21.15.142:6835/2701851507] up:boot 2024-05-09T00:07:52.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:51 smithi142 ceph-mon[34533]: fsmap 5 up:standby 2024-05-09T00:07:52.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:51 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T00:07:52.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.338+0000 7f58d0e70700 1 -- 172.21.15.46:0/2540545851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58cc1034c0 msgr2=0x7f58cc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:52.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.338+0000 7f58d0e70700 1 --2- 172.21.15.46:0/2540545851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58cc1034c0 0x7f58cc1038d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00b0d0 tx=0x7f58bc00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:07:52.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.338+0000 7f58d0e70700 1 -- 172.21.15.46:0/2540545851 shutdown_connections 2024-05-09T00:07:52.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.338+0000 7f58d0e70700 1 --2- 172.21.15.46:0/2540545851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58cc108740 0x7f58cc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:52.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.338+0000 7f58d0e70700 1 --2- 172.21.15.46:0/2540545851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58cc1034c0 0x7f58cc1038d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:52.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.338+0000 7f58d0e70700 1 --2- 172.21.15.46:0/2540545851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58cc107df0 0x7f58cc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:52.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.338+0000 7f58d0e70700 1 -- 172.21.15.46:0/2540545851 >> 172.21.15.46:0/2540545851 conn(0x7f58cc0fedf0 msgr2=0x7f58cc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:52.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.339+0000 7f58d0e70700 1 -- 172.21.15.46:0/2540545851 shutdown_connections 2024-05-09T00:07:52.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.339+0000 7f58d0e70700 1 -- 172.21.15.46:0/2540545851 wait complete. 2024-05-09T00:07:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.340+0000 7f58d0e70700 1 Processor -- start 2024-05-09T00:07:52.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.340+0000 7f58d0e70700 1 -- start start 2024-05-09T00:07:52.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.340+0000 7f58d0e70700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58cc1034c0 0x7f58cc19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:52.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.340+0000 7f58d0e70700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58cc107df0 0x7f58cc19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:52.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.340+0000 7f58d0e70700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58cc108740 0x7f58cc19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:52.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.340+0000 7f58d0e70700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc113370 con 0x7f58cc1034c0 2024-05-09T00:07:52.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.340+0000 7f58d0e70700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc1131f0 con 0x7f58cc107df0 2024-05-09T00:07:52.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.340+0000 7f58d0e70700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc1134f0 con 0x7f58cc108740 2024-05-09T00:07:52.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.340+0000 7f58ca59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58cc1034c0 0x7f58cc19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:52.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.340+0000 7f58ca59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58cc1034c0 0x7f58cc19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52250/0 (socket says 172.21.15.46:52250) 2024-05-09T00:07:52.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.340+0000 7f58ca59c700 1 -- 172.21.15.46:0/1589259570 learned_addr learned my addr 172.21.15.46:0/1589259570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:07:52.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.340+0000 7f58cad9d700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58cc108740 0x7f58cc19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.341+0000 7f58ca59c700 1 -- 172.21.15.46:0/1589259570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58cc108740 msgr2=0x7f58cc19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.341+0000 7f58ca59c700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58cc108740 0x7f58cc19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.341+0000 7f58ca59c700 1 -- 172.21.15.46:0/1589259570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58cc107df0 msgr2=0x7f58cc19b3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:07:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.341+0000 7f58ca59c700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58cc107df0 0x7f58cc19b3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.341+0000 7f58ca59c700 1 -- 172.21.15.46:0/1589259570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc19fd00 con 0x7f58cc1034c0 2024-05-09T00:07:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.341+0000 7f58ca59c700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58cc1034c0 0x7f58cc19ae30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f58bc007a30 tx=0x7f58bc007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.341+0000 7f58c37fe700 1 -- 172.21.15.46:0/1589259570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc01f070 con 0x7f58cc1034c0 2024-05-09T00:07:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.341+0000 7f58d0e70700 1 -- 172.21.15.46:0/1589259570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc19ff90 con 0x7f58cc1034c0 2024-05-09T00:07:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.341+0000 7f58d0e70700 1 -- 172.21.15.46:0/1589259570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc1a7910 con 0x7f58cc1034c0 2024-05-09T00:07:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.341+0000 7f58c37fe700 1 -- 172.21.15.46:0/1589259570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58bc004c60 con 0x7f58cc1034c0 2024-05-09T00:07:52.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.341+0000 7f58c37fe700 1 -- 172.21.15.46:0/1589259570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc0075c0 con 0x7f58cc1034c0 2024-05-09T00:07:52.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.342+0000 7f58d0e70700 1 -- 172.21.15.46:0/1589259570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ac005330 con 0x7f58cc1034c0 2024-05-09T00:07:52.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.345+0000 7f58c37fe700 1 -- 172.21.15.46:0/1589259570 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f58bc022070 con 0x7f58cc1034c0 2024-05-09T00:07:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.346+0000 7f58c37fe700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58b80b2a20 0x7f58b80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.346+0000 7f58c9d9b700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58b80b2a20 0x7f58b80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:52.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.346+0000 7f58c37fe700 1 -- 172.21.15.46:0/1589259570 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f58bc0e30d0 con 0x7f58cc1034c0 2024-05-09T00:07:52.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.347+0000 7f58c9d9b700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58b80b2a20 0x7f58b80b4f30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f58cc1020a0 tx=0x7f58b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:52.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.349+0000 7f58c37fe700 1 -- 172.21.15.46:0/1589259570 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f58bc029020 con 0x7f58cc1034c0 2024-05-09T00:07:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.613+0000 7f58d0e70700 1 -- 172.21.15.46:0/1589259570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58ac005620 con 0x7f58cc1034c0 2024-05-09T00:07:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.614+0000 7f58c37fe700 1 -- 172.21.15.46:0/1589259570 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f58bc0ab280 con 0x7f58cc1034c0 2024-05-09T00:07:52.610 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:07:52.610 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-09T00:07:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.616+0000 7f58d0e70700 1 -- 172.21.15.46:0/1589259570 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58b80b2a20 msgr2=0x7f58b80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.616+0000 7f58d0e70700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58b80b2a20 0x7f58b80b4f30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f58cc1020a0 tx=0x7f58b4009260 comp rx=0 tx=0).stop 2024-05-09T00:07:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.616+0000 7f58d0e70700 1 -- 172.21.15.46:0/1589259570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58cc1034c0 msgr2=0x7f58cc19ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.616+0000 7f58d0e70700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58cc1034c0 0x7f58cc19ae30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f58bc007a30 tx=0x7f58bc007a60 comp rx=0 tx=0).stop 2024-05-09T00:07:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.616+0000 7f58d0e70700 1 -- 172.21.15.46:0/1589259570 shutdown_connections 2024-05-09T00:07:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.616+0000 7f58d0e70700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58b80b2a20 0x7f58b80b4f30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.616+0000 7f58d0e70700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58cc108740 0x7f58cc19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.616+0000 7f58d0e70700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58cc1034c0 0x7f58cc19ae30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.616+0000 7f58d0e70700 1 --2- 172.21.15.46:0/1589259570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58cc107df0 0x7f58cc19b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.616+0000 7f58d0e70700 1 -- 172.21.15.46:0/1589259570 >> 172.21.15.46:0/1589259570 conn(0x7f58cc0fedf0 msgr2=0x7f58cc1063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.616+0000 7f58d0e70700 1 -- 172.21.15.46:0/1589259570 shutdown_connections 2024-05-09T00:07:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:52.616+0000 7f58d0e70700 1 -- 172.21.15.46:0/1589259570 wait complete. 2024-05-09T00:07:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2024-05-09T00:07:52.930 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:52 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1589259570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:53.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:52 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1589259570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:53.102 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:07:52 smithi169 systemd[1]: Starting Ceph mds.f for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:07:53.188 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:52 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1589259570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:53.430 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:07:53 smithi169 podman[64643]: 2024-05-09 00:07:53.068829098 +0000 UTC m=+0.018731290 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:07:53.431 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:07:53 smithi169 podman[64643]: 2024-05-09 00:07:53.180739306 +0000 UTC m=+0.130641474 container create c8111ddbd5979f0b4ca84ec0d385bd531a9e80ab8ac3aeed343c300b99122aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:07:53.431 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:07:53 smithi169 podman[64643]: 2024-05-09 00:07:53.400357087 +0000 UTC m=+0.350259255 container init c8111ddbd5979f0b4ca84ec0d385bd531a9e80ab8ac3aeed343c300b99122aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T00:07:53.431 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:07:53 smithi169 podman[64643]: 2024-05-09 00:07:53.402999894 +0000 UTC m=+0.352902063 container start c8111ddbd5979f0b4ca84ec0d385bd531a9e80ab8ac3aeed343c300b99122aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-09T00:07:53.851 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:07:53 smithi169 ceph-mds[64662]: -- 172.21.15.169:0/1883707485 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558bf1cfa40 con 0x5558bf217180 2024-05-09T00:07:53.851 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:07:53 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f[64658]: starting mds.f at 2024-05-09T00:07:53.851 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:07:53 smithi169 bash[64643]: c8111ddbd5979f0b4ca84ec0d385bd531a9e80ab8ac3aeed343c300b99122aad 2024-05-09T00:07:53.852 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:07:53 smithi169 systemd[1]: Started Ceph mds.f for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:07:54.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:54 smithi046 ceph-mon[30151]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:54.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:54.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:54.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:54.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:54.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:54.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:54.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:54 smithi046 ceph-mon[30151]: Deploying daemon mds.c on smithi169 2024-05-09T00:07:54.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:54 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.169:6834/1842121683,v1:172.21.15.169:6835/1842121683] up:boot 2024-05-09T00:07:54.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:54 smithi046 ceph-mon[30151]: fsmap 6 up:standby 2024-05-09T00:07:54.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T00:07:54.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:54 smithi169 ceph-mon[34777]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:54.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:54.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:54.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:54.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:54.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:54.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:54.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:54 smithi169 ceph-mon[34777]: Deploying daemon mds.c on smithi169 2024-05-09T00:07:54.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:54 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.169:6834/1842121683,v1:172.21.15.169:6835/1842121683] up:boot 2024-05-09T00:07:54.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:54 smithi169 ceph-mon[34777]: fsmap 6 up:standby 2024-05-09T00:07:54.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T00:07:54.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:54 smithi142 ceph-mon[34533]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:54.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:54.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:54.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:54.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:54.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:54.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:54.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:54 smithi142 ceph-mon[34533]: Deploying daemon mds.c on smithi169 2024-05-09T00:07:54.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:54 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.169:6834/1842121683,v1:172.21.15.169:6835/1842121683] up:boot 2024-05-09T00:07:54.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:54 smithi142 ceph-mon[34533]: fsmap 6 up:standby 2024-05-09T00:07:54.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T00:07:55.274 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:07:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:07:55.851 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:07:55 smithi169 systemd[1]: Starting Ceph mds.c for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:07:56.142 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:07:55 smithi169 podman[64955]: 2024-05-09 00:07:55.769682522 +0000 UTC m=+0.018439349 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:07:56.143 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:07:55 smithi169 podman[64955]: 2024-05-09 00:07:55.894806713 +0000 UTC m=+0.143563516 container create e161ca7048f4bc123c0ed39387229fa363c97aaa029a989a459255faf6bc13b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:07:56.143 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:07:56 smithi169 podman[64955]: 2024-05-09 00:07:56.14784699 +0000 UTC m=+0.396603793 container init e161ca7048f4bc123c0ed39387229fa363c97aaa029a989a459255faf6bc13b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T00:07:56.541 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:07:56 smithi169 podman[64955]: 2024-05-09 00:07:56.150594397 +0000 UTC m=+0.399351208 container start e161ca7048f4bc123c0ed39387229fa363c97aaa029a989a459255faf6bc13b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-09T00:07:56.541 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:07:56 smithi169 ceph-mds[64974]: -- 172.21.15.169:0/262357578 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d0d66afa40 con 0x55d0d66f7a80 2024-05-09T00:07:56.541 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:07:56 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c[64970]: starting mds.c at 2024-05-09T00:07:56.541 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:07:56 smithi169 bash[64955]: e161ca7048f4bc123c0ed39387229fa363c97aaa029a989a459255faf6bc13b3 2024-05-09T00:07:56.541 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:07:56 smithi169 systemd[1]: Started Ceph mds.c for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:07:56.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:56 smithi046 ceph-mon[30151]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:56.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:56.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:56.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:56.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:56.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:56.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:56.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:56 smithi169 ceph-mon[34777]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:56.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:56.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:56.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:56.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:56.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:56.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:56.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:56 smithi142 ceph-mon[34533]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:56.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:56.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:56.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:07:56.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:07:56.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:07:56.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:07:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.375+0000 7f32fdee0700 1 -- 172.21.15.46:0/909867883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f8108740 msgr2=0x7f32f81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.375+0000 7f32fdee0700 1 --2- 172.21.15.46:0/909867883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f8108740 0x7f32f81100a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f32e80099c0 tx=0x7f32e8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.375+0000 7f32fdee0700 1 -- 172.21.15.46:0/909867883 shutdown_connections 2024-05-09T00:07:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.375+0000 7f32fdee0700 1 --2- 172.21.15.46:0/909867883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f8108740 0x7f32f81100a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.375+0000 7f32fdee0700 1 --2- 172.21.15.46:0/909867883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f81034c0 0x7f32f81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.375+0000 7f32fdee0700 1 --2- 172.21.15.46:0/909867883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f8107df0 0x7f32f8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.375+0000 7f32fdee0700 1 -- 172.21.15.46:0/909867883 >> 172.21.15.46:0/909867883 conn(0x7f32f80fedf0 msgr2=0x7f32f8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.375+0000 7f32fdee0700 1 -- 172.21.15.46:0/909867883 shutdown_connections 2024-05-09T00:07:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.375+0000 7f32fdee0700 1 -- 172.21.15.46:0/909867883 wait complete. 2024-05-09T00:07:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.375+0000 7f32fdee0700 1 Processor -- start 2024-05-09T00:07:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32fdee0700 1 -- start start 2024-05-09T00:07:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32fdee0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f81034c0 0x7f32f8100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32fdee0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f8107df0 0x7f32f80ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32fdee0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f8108740 0x7f32f80ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32fdee0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8113370 con 0x7f32f81034c0 2024-05-09T00:07:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32fdee0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32f81131f0 con 0x7f32f8107df0 2024-05-09T00:07:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32fdee0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32f81134f0 con 0x7f32f8108740 2024-05-09T00:07:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32f77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f81034c0 0x7f32f8100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32f77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f81034c0 0x7f32f8100e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40388/0 (socket says 172.21.15.46:40388) 2024-05-09T00:07:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32f77fe700 1 -- 172.21.15.46:0/2862755858 learned_addr learned my addr 172.21.15.46:0/2862755858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:07:57.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32f6ffd700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f8107df0 0x7f32f80ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32f7fff700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f8108740 0x7f32f80ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32f77fe700 1 -- 172.21.15.46:0/2862755858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f8108740 msgr2=0x7f32f80ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32f77fe700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f8108740 0x7f32f80ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.376+0000 7f32f77fe700 1 -- 172.21.15.46:0/2862755858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f8107df0 msgr2=0x7f32f80ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.377+0000 7f32f77fe700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f8107df0 0x7f32f80ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.377+0000 7f32f77fe700 1 -- 172.21.15.46:0/2862755858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f81000e0 con 0x7f32f81034c0 2024-05-09T00:07:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.377+0000 7f32f77fe700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f81034c0 0x7f32f8100e70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f32f00060b0 tx=0x7f32f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.377+0000 7f32f4ff9700 1 -- 172.21.15.46:0/2862755858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f001a070 con 0x7f32f81034c0 2024-05-09T00:07:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.377+0000 7f32f4ff9700 1 -- 172.21.15.46:0/2862755858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f000dd20 con 0x7f32f81034c0 2024-05-09T00:07:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.377+0000 7f32fdee0700 1 -- 172.21.15.46:0/2862755858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f81ad860 con 0x7f32f81034c0 2024-05-09T00:07:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.377+0000 7f32f4ff9700 1 -- 172.21.15.46:0/2862755858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0014600 con 0x7f32f81034c0 2024-05-09T00:07:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.377+0000 7f32fdee0700 1 -- 172.21.15.46:0/2862755858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f81addc0 con 0x7f32f81034c0 2024-05-09T00:07:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.378+0000 7f32fdee0700 1 -- 172.21.15.46:0/2862755858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f8104970 con 0x7f32f81034c0 2024-05-09T00:07:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.380+0000 7f32f4ff9700 1 -- 172.21.15.46:0/2862755858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f32f001d070 con 0x7f32f81034c0 2024-05-09T00:07:57.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.380+0000 7f32f4ff9700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f32d80b2900 0x7f32d80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:07:57.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.381+0000 7f32f6ffd700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f32d80b2900 0x7f32d80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:07:57.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.381+0000 7f32f4ff9700 1 -- 172.21.15.46:0/2862755858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f32f00e19a0 con 0x7f32f81034c0 2024-05-09T00:07:57.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.381+0000 7f32f6ffd700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f32d80b2900 0x7f32d80b4e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f32f8076b40 tx=0x7f32e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:07:57.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.384+0000 7f32f4ff9700 1 -- 172.21.15.46:0/2862755858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32f0010070 con 0x7f32f81034c0 2024-05-09T00:07:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.646+0000 7f32fdee0700 1 -- 172.21.15.46:0/2862755858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32f8100800 con 0x7f32f81034c0 2024-05-09T00:07:57.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:57 smithi046 ceph-mon[30151]: Deploying daemon mds.h on smithi142 2024-05-09T00:07:57.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:57 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] up:boot 2024-05-09T00:07:57.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:57 smithi046 ceph-mon[30151]: fsmap 7 up:standby 2024-05-09T00:07:57.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:07:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.646+0000 7f32f4ff9700 1 -- 172.21.15.46:0/2862755858 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f32f00a9b60 con 0x7f32f81034c0 2024-05-09T00:07:57.643 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:07:57.643 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-09T00:07:57.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.649+0000 7f32fdee0700 1 -- 172.21.15.46:0/2862755858 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f32d80b2900 msgr2=0x7f32d80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.649+0000 7f32fdee0700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f32d80b2900 0x7f32d80b4e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f32f8076b40 tx=0x7f32e0009260 comp rx=0 tx=0).stop 2024-05-09T00:07:57.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.649+0000 7f32fdee0700 1 -- 172.21.15.46:0/2862755858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f81034c0 msgr2=0x7f32f8100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:07:57.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.649+0000 7f32fdee0700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f81034c0 0x7f32f8100e70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f32f00060b0 tx=0x7f32f000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.649+0000 7f32fdee0700 1 -- 172.21.15.46:0/2862755858 shutdown_connections 2024-05-09T00:07:57.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.649+0000 7f32fdee0700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f32d80b2900 0x7f32d80b4e10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.649+0000 7f32fdee0700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f8108740 0x7f32f80ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.649+0000 7f32fdee0700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f81034c0 0x7f32f8100e70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.649+0000 7f32fdee0700 1 --2- 172.21.15.46:0/2862755858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f8107df0 0x7f32f80ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:07:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.649+0000 7f32fdee0700 1 -- 172.21.15.46:0/2862755858 >> 172.21.15.46:0/2862755858 conn(0x7f32f80fedf0 msgr2=0x7f32f8075c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:07:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.649+0000 7f32fdee0700 1 -- 172.21.15.46:0/2862755858 shutdown_connections 2024-05-09T00:07:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:07:57.649+0000 7f32fdee0700 1 -- 172.21.15.46:0/2862755858 wait complete. 2024-05-09T00:07:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-05-09T00:07:57.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:57 smithi142 ceph-mon[34533]: Deploying daemon mds.h on smithi142 2024-05-09T00:07:57.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:57 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] up:boot 2024-05-09T00:07:57.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:57 smithi142 ceph-mon[34533]: fsmap 7 up:standby 2024-05-09T00:07:57.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:07:57.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:57 smithi169 ceph-mon[34777]: Deploying daemon mds.h on smithi142 2024-05-09T00:07:57.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:57 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] up:boot 2024-05-09T00:07:57.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:57 smithi169 ceph-mon[34777]: fsmap 7 up:standby 2024-05-09T00:07:57.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:07:58.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:58 smithi142 ceph-mon[34533]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:58.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:07:58 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2862755858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:58.689 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:07:58 smithi142 systemd[1]: Starting Ceph mds.h for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:07:58.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:58 smithi046 ceph-mon[30151]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:58.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:07:58 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2862755858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:58.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:58 smithi169 ceph-mon[34777]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:07:58.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:07:58 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2862755858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:07:59.132 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:07:58 smithi142 podman[64602]: 2024-05-09 00:07:58.747012235 +0000 UTC m=+0.018107955 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:07:59.132 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:07:58 smithi142 podman[64602]: 2024-05-09 00:07:58.868116518 +0000 UTC m=+0.139212219 container create f07c30c5b6cc2c1b648145c06bc26f6599d9666762d3cf9b0ec2b2b8741ad286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:07:59.132 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:07:59 smithi142 podman[64602]: 2024-05-09 00:07:59.137286485 +0000 UTC m=+0.408382184 container init f07c30c5b6cc2c1b648145c06bc26f6599d9666762d3cf9b0ec2b2b8741ad286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-09T00:07:59.439 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:07:59 smithi142 podman[64602]: 2024-05-09 00:07:59.140269211 +0000 UTC m=+0.411364911 container start f07c30c5b6cc2c1b648145c06bc26f6599d9666762d3cf9b0ec2b2b8741ad286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-09T00:07:59.439 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:07:59 smithi142 ceph-mds[64621]: -- 172.21.15.142:0/1376719749 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555ec6f93a40 con 0x555ec6fdb600 2024-05-09T00:07:59.439 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:07:59 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-h[64617]: starting mds.h at 2024-05-09T00:07:59.439 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:07:59 smithi142 bash[64602]: f07c30c5b6cc2c1b648145c06bc26f6599d9666762d3cf9b0ec2b2b8741ad286 2024-05-09T00:07:59.439 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:07:59 smithi142 systemd[1]: Started Ceph mds.h for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:08:00.303 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:08:00.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:00 smithi046 ceph-mon[30151]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:00.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:00.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:00.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:00.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:08:00.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:08:00.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:00.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:00 smithi046 ceph-mon[30151]: Deploying daemon mds.i on smithi169 2024-05-09T00:08:00.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:00 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.142:6836/3779152362,v1:172.21.15.142:6837/3779152362] up:boot 2024-05-09T00:08:00.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:00 smithi046 ceph-mon[30151]: fsmap 8 up:standby 2024-05-09T00:08:00.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T00:08:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:00.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:00 smithi169 ceph-mon[34777]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:00.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:00.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:00.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:00.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:08:00.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:08:00.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:00.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:00 smithi169 ceph-mon[34777]: Deploying daemon mds.i on smithi169 2024-05-09T00:08:00.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:00 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.142:6836/3779152362,v1:172.21.15.142:6837/3779152362] up:boot 2024-05-09T00:08:00.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:00 smithi169 ceph-mon[34777]: fsmap 8 up:standby 2024-05-09T00:08:00.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T00:08:00.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:00 smithi142 ceph-mon[34533]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:00.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:00.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:00.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:00.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:08:00.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:08:00.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:00.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:00 smithi142 ceph-mon[34533]: Deploying daemon mds.i on smithi169 2024-05-09T00:08:00.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:00 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.142:6836/3779152362,v1:172.21.15.142:6837/3779152362] up:boot 2024-05-09T00:08:00.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:00 smithi142 ceph-mon[34533]: fsmap 8 up:standby 2024-05-09T00:08:00.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T00:08:01.601 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:08:01 smithi169 systemd[1]: Starting Ceph mds.i for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:08:01.935 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:08:01 smithi169 podman[65263]: 2024-05-09 00:08:01.559108847 +0000 UTC m=+0.018262240 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:08:01.935 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:08:01 smithi169 podman[65263]: 2024-05-09 00:08:01.667846993 +0000 UTC m=+0.127000361 container create 83bac6934c60fb5fae0b83b8264586d348aca69738fab93dc1cc1725b0473481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T00:08:01.935 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:08:01 smithi169 podman[65263]: 2024-05-09 00:08:01.903792502 +0000 UTC m=+0.362945873 container init 83bac6934c60fb5fae0b83b8264586d348aca69738fab93dc1cc1725b0473481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-09T00:08:01.935 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:08:01 smithi169 podman[65263]: 2024-05-09 00:08:01.90649959 +0000 UTC m=+0.365652965 container start 83bac6934c60fb5fae0b83b8264586d348aca69738fab93dc1cc1725b0473481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T00:08:02.299 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:08:01 smithi169 ceph-mds[65282]: -- 172.21.15.169:0/408097511 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562971e1fa40 con 0x562971e67a80 2024-05-09T00:08:02.299 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:08:01 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i[65278]: starting mds.i at 2024-05-09T00:08:02.300 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:08:02 smithi169 bash[65263]: 83bac6934c60fb5fae0b83b8264586d348aca69738fab93dc1cc1725b0473481 2024-05-09T00:08:02.300 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:08:02 smithi169 systemd[1]: Started Ceph mds.i for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:08:02.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.340+0000 7f2a1223b700 1 -- 172.21.15.46:0/1904997126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c108f20 msgr2=0x7f2a0c109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.340+0000 7f2a1223b700 1 --2- 172.21.15.46:0/1904997126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c108f20 0x7f2a0c109330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2a000099c0 tx=0x7f2a00009cd0 comp rx=0 tx=0).stop 2024-05-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.340+0000 7f2a1223b700 1 -- 172.21.15.46:0/1904997126 shutdown_connections 2024-05-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.340+0000 7f2a1223b700 1 --2- 172.21.15.46:0/1904997126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0c109870 0x7f2a0c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.340+0000 7f2a1223b700 1 --2- 172.21.15.46:0/1904997126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a0c1045f0 0x7f2a0c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.340+0000 7f2a1223b700 1 --2- 172.21.15.46:0/1904997126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c108f20 0x7f2a0c109330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.340+0000 7f2a1223b700 1 -- 172.21.15.46:0/1904997126 >> 172.21.15.46:0/1904997126 conn(0x7f2a0c100040 msgr2=0x7f2a0c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.340+0000 7f2a1223b700 1 -- 172.21.15.46:0/1904997126 shutdown_connections 2024-05-09T00:08:02.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.340+0000 7f2a1223b700 1 -- 172.21.15.46:0/1904997126 wait complete. 2024-05-09T00:08:02.339 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:02 smithi046 ceph-mon[30151]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:02.339 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:02.339 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:02.339 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:02.339 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:08:02.339 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:08:02.339 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:02.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.341+0000 7f2a1223b700 1 Processor -- start 2024-05-09T00:08:02.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.341+0000 7f2a1223b700 1 -- start start 2024-05-09T00:08:02.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a1223b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0c1045f0 0x7f2a0c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:02.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a1223b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c108f20 0x7f2a0c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:02.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a1223b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a0c109870 0x7f2a0c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:02.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a1223b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c114480 con 0x7f2a0c108f20 2024-05-09T00:08:02.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a1223b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c114300 con 0x7f2a0c1045f0 2024-05-09T00:08:02.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a1223b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c114600 con 0x7f2a0c109870 2024-05-09T00:08:02.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a0affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c108f20 0x7f2a0c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:02.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a0affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c108f20 0x7f2a0c19b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40410/0 (socket says 172.21.15.46:40410) 2024-05-09T00:08:02.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a0affd700 1 -- 172.21.15.46:0/4042863574 learned_addr learned my addr 172.21.15.46:0/4042863574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:02.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a0b7fe700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0c1045f0 0x7f2a0c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:02.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a0bfff700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a0c109870 0x7f2a0c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:02.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a0affd700 1 -- 172.21.15.46:0/4042863574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a0c109870 msgr2=0x7f2a0c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:02.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a0affd700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a0c109870 0x7f2a0c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:02.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a0affd700 1 -- 172.21.15.46:0/4042863574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0c1045f0 msgr2=0x7f2a0c19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:02.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a0affd700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0c1045f0 0x7f2a0c19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a0affd700 1 -- 172.21.15.46:0/4042863574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c19fd80 con 0x7f2a0c108f20 2024-05-09T00:08:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.342+0000 7f2a0affd700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c108f20 0x7f2a0c19b450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a00005f50 tx=0x7f2a00004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.343+0000 7f2a08ff9700 1 -- 172.21.15.46:0/4042863574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00026070 con 0x7f2a0c108f20 2024-05-09T00:08:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.343+0000 7f2a08ff9700 1 -- 172.21.15.46:0/4042863574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a00003c70 con 0x7f2a0c108f20 2024-05-09T00:08:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.343+0000 7f2a1223b700 1 -- 172.21.15.46:0/4042863574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c1a0010 con 0x7f2a0c108f20 2024-05-09T00:08:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.343+0000 7f2a08ff9700 1 -- 172.21.15.46:0/4042863574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000f9c0 con 0x7f2a0c108f20 2024-05-09T00:08:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.344+0000 7f2a1223b700 1 -- 172.21.15.46:0/4042863574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c1a04f0 con 0x7f2a0c108f20 2024-05-09T00:08:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.345+0000 7f2a1223b700 1 -- 172.21.15.46:0/4042863574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0c0512c0 con 0x7f2a0c108f20 2024-05-09T00:08:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.346+0000 7f2a08ff9700 1 -- 172.21.15.46:0/4042863574 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2a00029070 con 0x7f2a0c108f20 2024-05-09T00:08:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.347+0000 7f2a08ff9700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f29f40b2880 0x7f29f40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.347+0000 7f2a0b7fe700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f29f40b2880 0x7f29f40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.347+0000 7f2a08ff9700 1 -- 172.21.15.46:0/4042863574 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f2a000e2390 con 0x7f2a0c108f20 2024-05-09T00:08:02.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.347+0000 7f2a0b7fe700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f29f40b2880 0x7f29f40b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009c60 tx=0x7f29fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:02.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.352+0000 7f2a08ff9700 1 -- 172.21.15.46:0/4042863574 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a00013070 con 0x7f2a0c108f20 2024-05-09T00:08:02.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:02 smithi169 ceph-mon[34777]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:02.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:02.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:02.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:02.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:08:02.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:08:02.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.623+0000 7f2a1223b700 1 -- 172.21.15.46:0/4042863574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a0c02e5f0 con 0x7f2a0c108f20 2024-05-09T00:08:02.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.623+0000 7f2a08ff9700 1 -- 172.21.15.46:0/4042863574 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8013 (secure 0 0 0) 0x7f2a000aa550 con 0x7f2a0c108f20 2024-05-09T00:08:02.620 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:08:02.620 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T00:08:02.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.626+0000 7f2a1223b700 1 -- 172.21.15.46:0/4042863574 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f29f40b2880 msgr2=0x7f29f40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:02.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.626+0000 7f2a1223b700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f29f40b2880 0x7f29f40b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009c60 tx=0x7f29fc009260 comp rx=0 tx=0).stop 2024-05-09T00:08:02.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.626+0000 7f2a1223b700 1 -- 172.21.15.46:0/4042863574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c108f20 msgr2=0x7f2a0c19b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:02.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.626+0000 7f2a1223b700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c108f20 0x7f2a0c19b450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a00005f50 tx=0x7f2a00004270 comp rx=0 tx=0).stop 2024-05-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.626+0000 7f2a1223b700 1 -- 172.21.15.46:0/4042863574 shutdown_connections 2024-05-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.626+0000 7f2a1223b700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f29f40b2880 0x7f29f40b4d90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.626+0000 7f2a1223b700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a0c109870 0x7f2a0c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.626+0000 7f2a1223b700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0c1045f0 0x7f2a0c19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.626+0000 7f2a1223b700 1 --2- 172.21.15.46:0/4042863574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c108f20 0x7f2a0c19b450 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.626+0000 7f2a1223b700 1 -- 172.21.15.46:0/4042863574 >> 172.21.15.46:0/4042863574 conn(0x7f2a0c100040 msgr2=0x7f2a0c10a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.626+0000 7f2a1223b700 1 -- 172.21.15.46:0/4042863574 shutdown_connections 2024-05-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:02.627+0000 7f2a1223b700 1 -- 172.21.15.46:0/4042863574 wait complete. 2024-05-09T00:08:02.624 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 10 2024-05-09T00:08:02.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:02 smithi142 ceph-mon[34533]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:02.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:02.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:02.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:02.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:08:02.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:08:02.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:03.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:03 smithi046 ceph-mon[30151]: Deploying daemon mds.g on smithi046 2024-05-09T00:08:03.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:03 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] up:boot 2024-05-09T00:08:03.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:03 smithi046 ceph-mon[30151]: fsmap 9 up:standby 2024-05-09T00:08:03.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:03 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T00:08:03.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:03 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/4042863574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:08:03.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:03 smithi169 ceph-mon[34777]: Deploying daemon mds.g on smithi046 2024-05-09T00:08:03.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:03 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] up:boot 2024-05-09T00:08:03.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:03 smithi169 ceph-mon[34777]: fsmap 9 up:standby 2024-05-09T00:08:03.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:03 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T00:08:03.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:03 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/4042863574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:08:03.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:03 smithi142 ceph-mon[34533]: Deploying daemon mds.g on smithi046 2024-05-09T00:08:03.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:03 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] up:boot 2024-05-09T00:08:03.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:03 smithi142 ceph-mon[34533]: fsmap 9 up:standby 2024-05-09T00:08:03.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:03 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T00:08:03.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:03 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/4042863574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:08:04.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:04 smithi046 ceph-mon[30151]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:04.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:04 smithi169 ceph-mon[34777]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:04.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:04 smithi142 ceph-mon[34533]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:05.059 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:08:04 smithi046 systemd[1]: Starting Ceph mds.g for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:08:05.474 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:08:05 smithi046 podman[70075]: 2024-05-09 00:08:05.118493278 +0000 UTC m=+0.018227375 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:08:05.474 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:08:05 smithi046 podman[70075]: 2024-05-09 00:08:05.234879914 +0000 UTC m=+0.134613997 container create 552d4e2443f6f68ba0e9aa23a3dd69123367cdbc10fbcae1c6ca1a6fedbc8a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-g, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T00:08:05.475 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:08:05 smithi046 podman[70075]: 2024-05-09 00:08:05.47958103 +0000 UTC m=+0.379315121 container init 552d4e2443f6f68ba0e9aa23a3dd69123367cdbc10fbcae1c6ca1a6fedbc8a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T00:08:05.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:08:05.808 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:08:05 smithi046 podman[70075]: 2024-05-09 00:08:05.482699157 +0000 UTC m=+0.382433242 container start 552d4e2443f6f68ba0e9aa23a3dd69123367cdbc10fbcae1c6ca1a6fedbc8a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:08:05.808 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:08:05 smithi046 ceph-mds[70094]: -- 172.21.15.46:0/2850183682 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a6916ba40 con 0x557a691b3600 2024-05-09T00:08:05.808 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:08:05 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-g[70090]: starting mds.g at 2024-05-09T00:08:05.808 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:08:05 smithi046 bash[70075]: 552d4e2443f6f68ba0e9aa23a3dd69123367cdbc10fbcae1c6ca1a6fedbc8a14 2024-05-09T00:08:05.808 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:08:05 smithi046 systemd[1]: Started Ceph mds.g for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:08:05.848 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:06.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:06 smithi142 ceph-mon[34533]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:06.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:06 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:06.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:06 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:06.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:06 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:06.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:06 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:08:06.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:06 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:08:06.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:06 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:06.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:06 smithi142 ceph-mon[34533]: Deploying daemon mds.a on smithi046 2024-05-09T00:08:06.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:06 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.46:6838/612832273,v1:172.21.15.46:6839/612832273] up:boot 2024-05-09T00:08:06.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:06 smithi142 ceph-mon[34533]: fsmap 10 up:standby 2024-05-09T00:08:06.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:06 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T00:08:07.016 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:06 smithi046 ceph-mon[30151]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:07.016 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:06 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:07.016 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:06 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:07.016 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:06 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:07.016 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:06 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:08:07.017 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:06 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:08:07.017 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:06 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:07.017 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:06 smithi046 ceph-mon[30151]: Deploying daemon mds.a on smithi046 2024-05-09T00:08:07.017 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:06 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.46:6838/612832273,v1:172.21.15.46:6839/612832273] up:boot 2024-05-09T00:08:07.017 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:06 smithi046 ceph-mon[30151]: fsmap 10 up:standby 2024-05-09T00:08:07.017 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:06 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T00:08:07.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:06 smithi169 ceph-mon[34777]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:07.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:06 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:07.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:06 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:07.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:06 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:07.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:06 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:08:07.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:06 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:08:07.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:06 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:07.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:06 smithi169 ceph-mon[34777]: Deploying daemon mds.a on smithi046 2024-05-09T00:08:07.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:06 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.46:6838/612832273,v1:172.21.15.46:6839/612832273] up:boot 2024-05-09T00:08:07.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:06 smithi169 ceph-mon[34777]: fsmap 10 up:standby 2024-05-09T00:08:07.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:06 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T00:08:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.303+0000 7f546f45b700 1 -- 172.21.15.46:0/3352684586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5468109870 msgr2=0x7f54681111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.303+0000 7f546f45b700 1 --2- 172.21.15.46:0/3352684586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5468109870 0x7f54681111d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f546000b0d0 tx=0x7f546000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:08:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.303+0000 7f546f45b700 1 -- 172.21.15.46:0/3352684586 shutdown_connections 2024-05-09T00:08:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.303+0000 7f546f45b700 1 --2- 172.21.15.46:0/3352684586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5468109870 0x7f54681111d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.303+0000 7f546f45b700 1 --2- 172.21.15.46:0/3352684586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54681045f0 0x7f5468104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.303+0000 7f546f45b700 1 --2- 172.21.15.46:0/3352684586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5468108f20 0x7f5468109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.303+0000 7f546f45b700 1 -- 172.21.15.46:0/3352684586 >> 172.21.15.46:0/3352684586 conn(0x7f5468100040 msgr2=0x7f5468102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.303+0000 7f546f45b700 1 -- 172.21.15.46:0/3352684586 shutdown_connections 2024-05-09T00:08:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.303+0000 7f546f45b700 1 -- 172.21.15.46:0/3352684586 wait complete. 2024-05-09T00:08:08.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.304+0000 7f546f45b700 1 Processor -- start 2024-05-09T00:08:08.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.304+0000 7f546f45b700 1 -- start start 2024-05-09T00:08:08.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546f45b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54681045f0 0x7f546819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546f45b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5468108f20 0x7f546819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546f45b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5468109870 0x7f546819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546f45b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f54681144a0 con 0x7f5468108f20 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546f45b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5468114320 con 0x7f54681045f0 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546f45b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5468114620 con 0x7f5468109870 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546c9f6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5468108f20 0x7f546819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546c9f6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5468108f20 0x7f546819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47556/0 (socket says 172.21.15.46:47556) 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546c9f6700 1 -- 172.21.15.46:0/1049629354 learned_addr learned my addr 172.21.15.46:0/1049629354 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546d1f7700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54681045f0 0x7f546819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546d9f8700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5468109870 0x7f546819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546c9f6700 1 -- 172.21.15.46:0/1049629354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5468109870 msgr2=0x7f546819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546c9f6700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5468109870 0x7f546819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546c9f6700 1 -- 172.21.15.46:0/1049629354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54681045f0 msgr2=0x7f546819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:08.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546c9f6700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54681045f0 0x7f546819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:08.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546c9f6700 1 -- 172.21.15.46:0/1049629354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546819fc70 con 0x7f5468108f20 2024-05-09T00:08:08.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546d9f8700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5468109870 0x7f546819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:08:08.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546c9f6700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5468108f20 0x7f546819b3f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f545800c9e0 tx=0x7f545800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:08.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f545e7fc700 1 -- 172.21.15.46:0/1049629354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458007aa0 con 0x7f5468108f20 2024-05-09T00:08:08.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f546f45b700 1 -- 172.21.15.46:0/1049629354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546819ff60 con 0x7f5468108f20 2024-05-09T00:08:08.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f545e7fc700 1 -- 172.21.15.46:0/1049629354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5458007c40 con 0x7f5468108f20 2024-05-09T00:08:08.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.305+0000 7f545e7fc700 1 -- 172.21.15.46:0/1049629354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458018660 con 0x7f5468108f20 2024-05-09T00:08:08.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.306+0000 7f546f45b700 1 -- 172.21.15.46:0/1049629354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54681a7930 con 0x7f5468108f20 2024-05-09T00:08:08.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.307+0000 7f546f45b700 1 -- 172.21.15.46:0/1049629354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54680512c0 con 0x7f5468108f20 2024-05-09T00:08:08.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.308+0000 7f545e7fc700 1 -- 172.21.15.46:0/1049629354 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5458018800 con 0x7f5468108f20 2024-05-09T00:08:08.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.309+0000 7f545e7fc700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f54540b2880 0x7f54540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:08.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.309+0000 7f546d1f7700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f54540b2880 0x7f54540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:08.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.309+0000 7f545e7fc700 1 -- 172.21.15.46:0/1049629354 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f54580e1d50 con 0x7f5468108f20 2024-05-09T00:08:08.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.309+0000 7f546d1f7700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f54540b2880 0x7f54540b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5464005950 tx=0x7f546400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:08.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.314+0000 7f545e7fc700 1 -- 172.21.15.46:0/1049629354 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5458014030 con 0x7f5468108f20 2024-05-09T00:08:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.578+0000 7f546f45b700 1 -- 172.21.15.46:0/1049629354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f546819c1b0 con 0x7f5468108f20 2024-05-09T00:08:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.579+0000 7f545e7fc700 1 -- 172.21.15.46:0/1049629354 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f54580a9e70 con 0x7f5468108f20 2024-05-09T00:08:08.575 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:08:08.576 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T00:08:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.581+0000 7f546f45b700 1 -- 172.21.15.46:0/1049629354 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f54540b2880 msgr2=0x7f54540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.581+0000 7f546f45b700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f54540b2880 0x7f54540b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5464005950 tx=0x7f546400b410 comp rx=0 tx=0).stop 2024-05-09T00:08:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.581+0000 7f546f45b700 1 -- 172.21.15.46:0/1049629354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5468108f20 msgr2=0x7f546819b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.581+0000 7f546f45b700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5468108f20 0x7f546819b3f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f545800c9e0 tx=0x7f545800ccf0 comp rx=0 tx=0).stop 2024-05-09T00:08:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.582+0000 7f546f45b700 1 -- 172.21.15.46:0/1049629354 shutdown_connections 2024-05-09T00:08:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.582+0000 7f546f45b700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f54540b2880 0x7f54540b4d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:08.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.582+0000 7f546f45b700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5468109870 0x7f546819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:08.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.582+0000 7f546f45b700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54681045f0 0x7f546819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:08.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.582+0000 7f546f45b700 1 --2- 172.21.15.46:0/1049629354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5468108f20 0x7f546819b3f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:08.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.582+0000 7f546f45b700 1 -- 172.21.15.46:0/1049629354 >> 172.21.15.46:0/1049629354 conn(0x7f5468100040 msgr2=0x7f5468108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:08.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.582+0000 7f546f45b700 1 -- 172.21.15.46:0/1049629354 shutdown_connections 2024-05-09T00:08:08.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:08.582+0000 7f546f45b700 1 -- 172.21.15.46:0/1049629354 wait complete. 2024-05-09T00:08:08.579 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 11 2024-05-09T00:08:08.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:08 smithi046 ceph-mon[30151]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:08.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:08 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1049629354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:08:08.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:08 smithi169 ceph-mon[34777]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:08.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:08 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1049629354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:08:08.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:08 smithi142 ceph-mon[34533]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:08.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:08 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1049629354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:08:09.545 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:08:09 smithi046 systemd[1]: Starting Ceph mds.a for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:08:09.796 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:08:09 smithi046 podman[70538]: 2024-05-09 00:08:09.687356907 +0000 UTC m=+0.018134299 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:08:09.796 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:08:09 smithi046 podman[70538]: 2024-05-09 00:08:09.801229863 +0000 UTC m=+0.132007235 container create f835197cfca6fa27458448d2775723a88ac7b5b3012a4a918cb734f529827a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-a, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:08:10.058 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:08:10 smithi046 podman[70538]: 2024-05-09 00:08:10.047121779 +0000 UTC m=+0.377899170 container init f835197cfca6fa27458448d2775723a88ac7b5b3012a4a918cb734f529827a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-09T00:08:10.058 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:08:10 smithi046 podman[70538]: 2024-05-09 00:08:10.050647543 +0000 UTC m=+0.381424925 container start f835197cfca6fa27458448d2775723a88ac7b5b3012a4a918cb734f529827a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-a, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T00:08:10.558 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:08:10 smithi046 ceph-mds[70557]: -- 172.21.15.46:0/3573736760 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5571a2e17a40 con 0x5571a2e5fa80 2024-05-09T00:08:10.558 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:08:10 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-a[70553]: starting mds.a at 2024-05-09T00:08:10.558 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:08:10 smithi046 bash[70538]: f835197cfca6fa27458448d2775723a88ac7b5b3012a4a918cb734f529827a7b 2024-05-09T00:08:10.559 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:08:10 smithi046 systemd[1]: Started Ceph mds.a for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:08:10.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:10 smithi142 ceph-mon[34533]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:10.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:10.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:10.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:10.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:08:10.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:08:10.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:10 smithi046 ceph-mon[30151]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:08:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:08:11.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:11.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:10 smithi169 ceph-mon[34777]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:11.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:11.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:11.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:11.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T00:08:11.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T00:08:11.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:11.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:08:11.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:11 smithi046 ceph-mon[30151]: Deploying daemon mds.b on smithi142 2024-05-09T00:08:11.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:11 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.46:6840/1387176303,v1:172.21.15.46:6841/1387176303] up:boot 2024-05-09T00:08:11.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:11 smithi046 ceph-mon[30151]: fsmap 11 up:standby 2024-05-09T00:08:11.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:11 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T00:08:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:11.938 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:11 smithi142 ceph-mon[34533]: Deploying daemon mds.b on smithi142 2024-05-09T00:08:11.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:11 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.46:6840/1387176303,v1:172.21.15.46:6841/1387176303] up:boot 2024-05-09T00:08:11.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:11 smithi142 ceph-mon[34533]: fsmap 11 up:standby 2024-05-09T00:08:11.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:11 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T00:08:12.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:11 smithi169 ceph-mon[34777]: Deploying daemon mds.b on smithi142 2024-05-09T00:08:12.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:11 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.46:6840/1387176303,v1:172.21.15.46:6841/1387176303] up:boot 2024-05-09T00:08:12.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:11 smithi169 ceph-mon[34777]: fsmap 11 up:standby 2024-05-09T00:08:12.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:11 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T00:08:12.634 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:08:12 smithi142 systemd[1]: Starting Ceph mds.b for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:08:12.912 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:12 smithi142 ceph-mon[34533]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:12.912 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:08:12 smithi142 podman[64921]: 2024-05-09 00:08:12.540361276 +0000 UTC m=+0.017928503 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:08:12.912 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:08:12 smithi142 podman[64921]: 2024-05-09 00:08:12.647478011 +0000 UTC m=+0.125045218 container create 09a12a4d824b60e257bb0fb639ee8191c8f059f37bb37402734ac01dafb0dfcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-b, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-05-09T00:08:12.912 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:08:12 smithi142 podman[64921]: 2024-05-09 00:08:12.916985594 +0000 UTC m=+0.394552801 container init 09a12a4d824b60e257bb0fb639ee8191c8f059f37bb37402734ac01dafb0dfcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-b, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-09T00:08:13.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:12 smithi046 ceph-mon[30151]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:13.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:12 smithi169 ceph-mon[34777]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:13.189 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:08:12 smithi142 podman[64921]: 2024-05-09 00:08:12.919872264 +0000 UTC m=+0.397439474 container start 09a12a4d824b60e257bb0fb639ee8191c8f059f37bb37402734ac01dafb0dfcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T00:08:13.189 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:08:12 smithi142 ceph-mds[64940]: -- 172.21.15.142:0/410474620 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598d83efa40 con 0x5598d8437a80 2024-05-09T00:08:13.189 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:08:12 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-b[64936]: starting mds.b at 2024-05-09T00:08:13.189 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:08:13 smithi142 bash[64921]: 09a12a4d824b60e257bb0fb639ee8191c8f059f37bb37402734ac01dafb0dfcb 2024-05-09T00:08:13.189 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:08:13 smithi142 systemd[1]: Started Ceph mds.b for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:08:13.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.661+0000 7f8047444700 1 -- 172.21.15.46:0/3292169963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8040109860 msgr2=0x7f80401111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.661+0000 7f8047444700 1 --2- 172.21.15.46:0/3292169963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8040109860 0x7f80401111c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f80380099c0 tx=0x7f8038009cd0 comp rx=0 tx=0).stop 2024-05-09T00:08:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.661+0000 7f8047444700 1 -- 172.21.15.46:0/3292169963 shutdown_connections 2024-05-09T00:08:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.661+0000 7f8047444700 1 --2- 172.21.15.46:0/3292169963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8040109860 0x7f80401111c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.661+0000 7f8047444700 1 --2- 172.21.15.46:0/3292169963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80401045e0 0x7f80401049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.661+0000 7f8047444700 1 --2- 172.21.15.46:0/3292169963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8040108f10 0x7f8040109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.661+0000 7f8047444700 1 -- 172.21.15.46:0/3292169963 >> 172.21.15.46:0/3292169963 conn(0x7f80400ffe70 msgr2=0x7f80401022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.661+0000 7f8047444700 1 -- 172.21.15.46:0/3292169963 shutdown_connections 2024-05-09T00:08:13.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.661+0000 7f8047444700 1 -- 172.21.15.46:0/3292169963 wait complete. 2024-05-09T00:08:13.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.661+0000 7f8047444700 1 Processor -- start 2024-05-09T00:08:13.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f8047444700 1 -- start start 2024-05-09T00:08:13.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f8047444700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80401045e0 0x7f804019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:13.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f8047444700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8040108f10 0x7f804019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:13.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f8047444700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8040109860 0x7f804019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:13.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f8047444700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8040114490 con 0x7f8040108f10 2024-05-09T00:08:13.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f8047444700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8040114310 con 0x7f8040109860 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f8047444700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8040114610 con 0x7f80401045e0 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f80449df700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8040108f10 0x7f804019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f80449df700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8040108f10 0x7f804019b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47608/0 (socket says 172.21.15.46:47608) 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f80449df700 1 -- 172.21.15.46:0/2366988634 learned_addr learned my addr 172.21.15.46:0/2366988634 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f80451e0700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80401045e0 0x7f804019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f80459e1700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8040109860 0x7f804019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f80449df700 1 -- 172.21.15.46:0/2366988634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80401045e0 msgr2=0x7f804019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f80449df700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80401045e0 0x7f804019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f80449df700 1 -- 172.21.15.46:0/2366988634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8040109860 msgr2=0x7f804019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f80449df700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8040109860 0x7f804019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f80449df700 1 -- 172.21.15.46:0/2366988634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804019fcf0 con 0x7f8040108f10 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.662+0000 7f80451e0700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80401045e0 0x7f804019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:08:13.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.663+0000 7f80449df700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8040108f10 0x7f804019b3c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f803000e970 tx=0x7f803000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:13.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.663+0000 7f80367fc700 1 -- 172.21.15.46:0/2366988634 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030004a30 con 0x7f8040108f10 2024-05-09T00:08:13.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.663+0000 7f80367fc700 1 -- 172.21.15.46:0/2366988634 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f803000cc90 con 0x7f8040108f10 2024-05-09T00:08:13.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.663+0000 7f8047444700 1 -- 172.21.15.46:0/2366988634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804019ffe0 con 0x7f8040108f10 2024-05-09T00:08:13.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.663+0000 7f80367fc700 1 -- 172.21.15.46:0/2366988634 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030005610 con 0x7f8040108f10 2024-05-09T00:08:13.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.663+0000 7f8047444700 1 -- 172.21.15.46:0/2366988634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80401a7900 con 0x7f8040108f10 2024-05-09T00:08:13.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.664+0000 7f8047444700 1 -- 172.21.15.46:0/2366988634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80400512c0 con 0x7f8040108f10 2024-05-09T00:08:13.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.667+0000 7f80367fc700 1 -- 172.21.15.46:0/2366988634 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f803000c7e0 con 0x7f8040108f10 2024-05-09T00:08:13.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.668+0000 7f80367fc700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f802c0b2900 0x7f802c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:13.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.668+0000 7f80451e0700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f802c0b2900 0x7f802c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:13.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.668+0000 7f80367fc700 1 -- 172.21.15.46:0/2366988634 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f80300e1fe0 con 0x7f8040108f10 2024-05-09T00:08:13.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.668+0000 7f80451e0700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f802c0b2900 0x7f802c0b4e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f803c009480 tx=0x7f803c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.671+0000 7f80367fc700 1 -- 172.21.15.46:0/2366988634 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8030054df0 con 0x7f8040108f10 2024-05-09T00:08:13.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.949+0000 7f8047444700 1 -- 172.21.15.46:0/2366988634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f804019c180 con 0x7f8040108f10 2024-05-09T00:08:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.950+0000 7f80367fc700 1 -- 172.21.15.46:0/2366988634 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7f80300aa1a0 con 0x7f8040108f10 2024-05-09T00:08:13.946 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:08:13.946 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T00:08:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.952+0000 7f8047444700 1 -- 172.21.15.46:0/2366988634 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f802c0b2900 msgr2=0x7f802c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.952+0000 7f8047444700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f802c0b2900 0x7f802c0b4e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f803c009480 tx=0x7f803c00b040 comp rx=0 tx=0).stop 2024-05-09T00:08:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.952+0000 7f8047444700 1 -- 172.21.15.46:0/2366988634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8040108f10 msgr2=0x7f804019b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.952+0000 7f8047444700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8040108f10 0x7f804019b3c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f803000e970 tx=0x7f803000ec80 comp rx=0 tx=0).stop 2024-05-09T00:08:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.952+0000 7f8047444700 1 -- 172.21.15.46:0/2366988634 shutdown_connections 2024-05-09T00:08:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.952+0000 7f8047444700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f802c0b2900 0x7f802c0b4e10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.952+0000 7f8047444700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8040109860 0x7f804019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.952+0000 7f8047444700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80401045e0 0x7f804019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.952+0000 7f8047444700 1 --2- 172.21.15.46:0/2366988634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8040108f10 0x7f804019b3c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.952+0000 7f8047444700 1 -- 172.21.15.46:0/2366988634 >> 172.21.15.46:0/2366988634 conn(0x7f80400ffe70 msgr2=0x7f8040106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.952+0000 7f8047444700 1 -- 172.21.15.46:0/2366988634 shutdown_connections 2024-05-09T00:08:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:13.952+0000 7f8047444700 1 -- 172.21.15.46:0/2366988634 wait complete. 2024-05-09T00:08:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2024-05-09T00:08:14.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:08:14.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:14 smithi046 ceph-mon[30151]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:14.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:14 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] up:boot 2024-05-09T00:08:14.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:14 smithi046 ceph-mon[30151]: fsmap 12 up:standby 2024-05-09T00:08:14.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:14 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T00:08:14.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:14 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2366988634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:08:14.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:08:14.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:14 smithi169 ceph-mon[34777]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:14.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:14 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] up:boot 2024-05-09T00:08:14.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:14 smithi169 ceph-mon[34777]: fsmap 12 up:standby 2024-05-09T00:08:14.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:14 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T00:08:14.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:14 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2366988634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:08:14.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:14.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:08:14.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:14 smithi142 ceph-mon[34533]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:14.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:14 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] up:boot 2024-05-09T00:08:14.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:14 smithi142 ceph-mon[34533]: fsmap 12 up:standby 2024-05-09T00:08:14.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:14 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T00:08:14.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:14 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2366988634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:08:15.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-09T00:08:15.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:15 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.510 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:15.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:15 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:15.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:15 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:16.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:16 smithi046 ceph-mon[30151]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:16.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:16 smithi169 ceph-mon[34777]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:16.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:16 smithi142 ceph-mon[34533]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.248+0000 7f5aa6db7700 1 -- 172.21.15.46:0/3626544281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa0109860 msgr2=0x7f5aa01111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.248+0000 7f5aa6db7700 1 --2- 172.21.15.46:0/3626544281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa0109860 0x7f5aa01111c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800b0d0 tx=0x7f5a9800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.248+0000 7f5aa6db7700 1 -- 172.21.15.46:0/3626544281 shutdown_connections 2024-05-09T00:08:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.248+0000 7f5aa6db7700 1 --2- 172.21.15.46:0/3626544281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa0109860 0x7f5aa01111c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.248+0000 7f5aa6db7700 1 --2- 172.21.15.46:0/3626544281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa01045e0 0x7f5aa01049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.248+0000 7f5aa6db7700 1 --2- 172.21.15.46:0/3626544281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa0108f10 0x7f5aa0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.248+0000 7f5aa6db7700 1 -- 172.21.15.46:0/3626544281 >> 172.21.15.46:0/3626544281 conn(0x7f5aa00ffe70 msgr2=0x7f5aa01022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.248+0000 7f5aa6db7700 1 -- 172.21.15.46:0/3626544281 shutdown_connections 2024-05-09T00:08:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.248+0000 7f5aa6db7700 1 -- 172.21.15.46:0/3626544281 wait complete. 2024-05-09T00:08:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.248+0000 7f5aa6db7700 1 Processor -- start 2024-05-09T00:08:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa6db7700 1 -- start start 2024-05-09T00:08:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa6db7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa01045e0 0x7f5aa019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa6db7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa0108f10 0x7f5aa019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa6db7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa0109860 0x7f5aa01a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa6db7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0114490 con 0x7f5aa01045e0 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa6db7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0114310 con 0x7f5aa0109860 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa6db7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0114610 con 0x7f5aa0108f10 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa4b53700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa01045e0 0x7f5aa019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa4b53700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa01045e0 0x7f5aa019cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35064/0 (socket says 172.21.15.46:35064) 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa4b53700 1 -- 172.21.15.46:0/1262509019 learned_addr learned my addr 172.21.15.46:0/1262509019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa5354700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa0109860 0x7f5aa01a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5a9ffff700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa0108f10 0x7f5aa019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa4b53700 1 -- 172.21.15.46:0/1262509019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa0108f10 msgr2=0x7f5aa019d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa4b53700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa0108f10 0x7f5aa019d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa4b53700 1 -- 172.21.15.46:0/1262509019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa0109860 msgr2=0x7f5aa01a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa4b53700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa0109860 0x7f5aa01a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa4b53700 1 -- 172.21.15.46:0/1262509019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa01a1e00 con 0x7f5aa01045e0 2024-05-09T00:08:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.249+0000 7f5aa5354700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa0109860 0x7f5aa01a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:08:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.250+0000 7f5aa4b53700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa01045e0 0x7f5aa019cfe0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400c9b0 tx=0x7f5a9400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.250+0000 7f5a9dffb700 1 -- 172.21.15.46:0/1262509019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94007b00 con 0x7f5aa01045e0 2024-05-09T00:08:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.250+0000 7f5a9dffb700 1 -- 172.21.15.46:0/1262509019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a94007ca0 con 0x7f5aa01045e0 2024-05-09T00:08:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.250+0000 7f5aa6db7700 1 -- 172.21.15.46:0/1262509019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa01a20f0 con 0x7f5aa01045e0 2024-05-09T00:08:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.250+0000 7f5a9dffb700 1 -- 172.21.15.46:0/1262509019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94018730 con 0x7f5aa01045e0 2024-05-09T00:08:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.250+0000 7f5aa6db7700 1 -- 172.21.15.46:0/1262509019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa01a2630 con 0x7f5aa01045e0 2024-05-09T00:08:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.251+0000 7f5aa6db7700 1 -- 172.21.15.46:0/1262509019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa00512c0 con 0x7f5aa01045e0 2024-05-09T00:08:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.253+0000 7f5a9dffb700 1 -- 172.21.15.46:0/1262509019 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5a9400f450 con 0x7f5aa01045e0 2024-05-09T00:08:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.254+0000 7f5a9dffb700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5a900b2880 0x7f5a900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.254+0000 7f5a9ffff700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5a900b2880 0x7f5a900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.254+0000 7f5a9dffb700 1 -- 172.21.15.46:0/1262509019 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f5a940e2100 con 0x7f5aa01045e0 2024-05-09T00:08:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.254+0000 7f5a9ffff700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5a900b2880 0x7f5a900b4d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c005950 tx=0x7f5a8c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:17.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.258+0000 7f5a9dffb700 1 -- 172.21.15.46:0/1262509019 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a94014030 con 0x7f5aa01045e0 2024-05-09T00:08:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.476+0000 7f5aa6db7700 1 -- 172.21.15.46:0/1262509019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5aa01a9bb0 con 0x7f5aa01045e0 2024-05-09T00:08:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.477+0000 7f5a9dffb700 1 -- 172.21.15.46:0/1262509019 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f5a940aa350 con 0x7f5aa01045e0 2024-05-09T00:08:17.473 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:08:17.473 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-05-09T00:08:17.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:08:17.474 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-05-09T00:08:17.474 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-05-09T00:08:17.474 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:08:17.474 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-05-09T00:08:17.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.479+0000 7f5aa6db7700 1 -- 172.21.15.46:0/1262509019 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5a900b2880 msgr2=0x7f5a900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.479+0000 7f5aa6db7700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5a900b2880 0x7f5a900b4d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c005950 tx=0x7f5a8c0058e0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.479+0000 7f5aa6db7700 1 -- 172.21.15.46:0/1262509019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa01045e0 msgr2=0x7f5aa019cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.479+0000 7f5aa6db7700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa01045e0 0x7f5aa019cfe0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400c9b0 tx=0x7f5a9400cd70 comp rx=0 tx=0).stop 2024-05-09T00:08:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.479+0000 7f5aa6db7700 1 -- 172.21.15.46:0/1262509019 shutdown_connections 2024-05-09T00:08:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.479+0000 7f5aa6db7700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5a900b2880 0x7f5a900b4d90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.479+0000 7f5aa6db7700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa0109860 0x7f5aa01a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.479+0000 7f5aa6db7700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa01045e0 0x7f5aa019cfe0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.479+0000 7f5aa6db7700 1 --2- 172.21.15.46:0/1262509019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa0108f10 0x7f5aa019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.479+0000 7f5aa6db7700 1 -- 172.21.15.46:0/1262509019 >> 172.21.15.46:0/1262509019 conn(0x7f5aa00ffe70 msgr2=0x7f5aa010fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.479+0000 7f5aa6db7700 1 -- 172.21.15.46:0/1262509019 shutdown_connections 2024-05-09T00:08:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:17.479+0000 7f5aa6db7700 1 -- 172.21.15.46:0/1262509019 wait complete. 2024-05-09T00:08:18.101 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:17 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:18.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:17 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:18.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:17 smithi169 ceph-mon[34777]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:18.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:17 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:18.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:17 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:08:18.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:17 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:18.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:17 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1262509019' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T00:08:18.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:17 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:08:18.124 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:17 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:18.124 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:17 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:18.124 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:17 smithi046 ceph-mon[30151]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:18.124 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:17 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:18.124 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:17 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:08:18.124 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:17 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:18.124 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:17 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1262509019' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T00:08:18.124 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:17 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:08:18.154 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T00:08:18.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T00:08:18.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:17 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:18.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:17 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:18.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:17 smithi142 ceph-mon[34533]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:18.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:17 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:18.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:17 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:08:18.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:17 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:18.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:17 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1262509019' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T00:08:18.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:17 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:08:18.987 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:20.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:20 smithi046 ceph-mon[30151]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:20.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:20.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:20.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:20.509 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:20 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:20.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:20 smithi169 ceph-mon[34777]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:20.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:20.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:20.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:20.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:20 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:20.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:20 smithi142 ceph-mon[34533]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:20.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:20.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:20.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:20.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:20 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.760+0000 7fe921220700 1 -- 172.21.15.46:0/818777215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe91c105e40 msgr2=0x7fe91c10e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.760+0000 7fe921220700 1 --2- 172.21.15.46:0/818777215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe91c105e40 0x7fe91c10e390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe904009990 tx=0x7fe904009ca0 comp rx=0 tx=0).stop 2024-05-09T00:08:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.760+0000 7fe921220700 1 -- 172.21.15.46:0/818777215 shutdown_connections 2024-05-09T00:08:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.760+0000 7fe921220700 1 --2- 172.21.15.46:0/818777215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe91c10e920 0x7fe91c110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.760+0000 7fe921220700 1 --2- 172.21.15.46:0/818777215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe91c1054f0 0x7fe91c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.760+0000 7fe921220700 1 --2- 172.21.15.46:0/818777215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe91c105e40 0x7fe91c10e390 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.760+0000 7fe921220700 1 -- 172.21.15.46:0/818777215 >> 172.21.15.46:0/818777215 conn(0x7fe91c078e00 msgr2=0x7fe91c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.760+0000 7fe921220700 1 -- 172.21.15.46:0/818777215 shutdown_connections 2024-05-09T00:08:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.760+0000 7fe921220700 1 -- 172.21.15.46:0/818777215 wait complete. 2024-05-09T00:08:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.761+0000 7fe921220700 1 Processor -- start 2024-05-09T00:08:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.761+0000 7fe921220700 1 -- start start 2024-05-09T00:08:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.762+0000 7fe921220700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe91c1054f0 0x7fe91c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.762+0000 7fe921220700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe91c105e40 0x7fe91c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.762+0000 7fe921220700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe91c10e920 0x7fe91c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.762+0000 7fe921220700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c114480 con 0x7fe91c10e920 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.762+0000 7fe921220700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c114300 con 0x7fe91c1054f0 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.762+0000 7fe921220700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c114600 con 0x7fe91c105e40 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.762+0000 7fe91b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe91c10e920 0x7fe91c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.762+0000 7fe91a59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe91c105e40 0x7fe91c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.762+0000 7fe91b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe91c10e920 0x7fe91c19f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35082/0 (socket says 172.21.15.46:35082) 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.762+0000 7fe91a59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe91c105e40 0x7fe91c19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:43402/0 (socket says 172.21.15.46:43402) 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.762+0000 7fe91b59e700 1 -- 172.21.15.46:0/1648306404 learned_addr learned my addr 172.21.15.46:0/1648306404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.762+0000 7fe91ad9d700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe91c1054f0 0x7fe91c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.763+0000 7fe91b59e700 1 -- 172.21.15.46:0/1648306404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe91c105e40 msgr2=0x7fe91c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.763+0000 7fe91b59e700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe91c105e40 0x7fe91c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.763+0000 7fe91b59e700 1 -- 172.21.15.46:0/1648306404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe91c1054f0 msgr2=0x7fe91c19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.763+0000 7fe91b59e700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe91c1054f0 0x7fe91c19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.763+0000 7fe91b59e700 1 -- 172.21.15.46:0/1648306404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c19fc10 con 0x7fe91c10e920 2024-05-09T00:08:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.763+0000 7fe91b59e700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe91c10e920 0x7fe91c19f620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe914007ee0 tx=0x7fe91400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.763+0000 7fe913fff700 1 -- 172.21.15.46:0/1648306404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914004bb0 con 0x7fe91c10e920 2024-05-09T00:08:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.763+0000 7fe921220700 1 -- 172.21.15.46:0/1648306404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91c19ff00 con 0x7fe91c10e920 2024-05-09T00:08:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.763+0000 7fe921220700 1 -- 172.21.15.46:0/1648306404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91c1a0410 con 0x7fe91c10e920 2024-05-09T00:08:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.763+0000 7fe913fff700 1 -- 172.21.15.46:0/1648306404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe914004d50 con 0x7fe91c10e920 2024-05-09T00:08:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.763+0000 7fe913fff700 1 -- 172.21.15.46:0/1648306404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914005920 con 0x7fe91c10e920 2024-05-09T00:08:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.764+0000 7fe921220700 1 -- 172.21.15.46:0/1648306404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc005330 con 0x7fe91c10e920 2024-05-09T00:08:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.766+0000 7fe913fff700 1 -- 172.21.15.46:0/1648306404 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe91401a020 con 0x7fe91c10e920 2024-05-09T00:08:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.767+0000 7fe913fff700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe9080b2900 0x7fe9080b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.767+0000 7fe913fff700 1 -- 172.21.15.46:0/1648306404 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fe9140e1840 con 0x7fe91c10e920 2024-05-09T00:08:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.767+0000 7fe91ad9d700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe9080b2900 0x7fe9080b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.768+0000 7fe91ad9d700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe9080b2900 0x7fe9080b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00ac50 tx=0x7fe90c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:20.775+0000 7fe913fff700 1 -- 172.21.15.46:0/1648306404 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe914010070 con 0x7fe91c10e920 2024-05-09T00:08:21.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:21.008+0000 7fe921220700 1 -- 172.21.15.46:0/1648306404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe8fc005620 con 0x7fe91c10e920 2024-05-09T00:08:21.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.001+0000 7fe913fff700 1 -- 172.21.15.46:0/1648306404 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7fe9140a9a00 con 0x7fe91c10e920 2024-05-09T00:08:22.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.004+0000 7fe921220700 1 -- 172.21.15.46:0/1648306404 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe9080b2900 msgr2=0x7fe9080b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:22.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.004+0000 7fe921220700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe9080b2900 0x7fe9080b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00ac50 tx=0x7fe90c00a380 comp rx=0 tx=0).stop 2024-05-09T00:08:22.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.004+0000 7fe921220700 1 -- 172.21.15.46:0/1648306404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe91c10e920 msgr2=0x7fe91c19f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:22.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.004+0000 7fe921220700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe91c10e920 0x7fe91c19f620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe914007ee0 tx=0x7fe91400d3b0 comp rx=0 tx=0).stop 2024-05-09T00:08:22.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.005+0000 7fe921220700 1 -- 172.21.15.46:0/1648306404 shutdown_connections 2024-05-09T00:08:22.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.005+0000 7fe921220700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe9080b2900 0x7fe9080b4e10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:22.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.005+0000 7fe921220700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe91c10e920 0x7fe91c19f620 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:22.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.005+0000 7fe921220700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe91c1054f0 0x7fe91c19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:22.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.005+0000 7fe921220700 1 --2- 172.21.15.46:0/1648306404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe91c105e40 0x7fe91c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:22.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.005+0000 7fe921220700 1 -- 172.21.15.46:0/1648306404 >> 172.21.15.46:0/1648306404 conn(0x7fe91c078e00 msgr2=0x7fe91c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:22.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.005+0000 7fe921220700 1 -- 172.21.15.46:0/1648306404 shutdown_connections 2024-05-09T00:08:22.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:22.005+0000 7fe921220700 1 -- 172.21.15.46:0/1648306404 wait complete. 2024-05-09T00:08:22.003 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-05-09T00:08:22.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:22.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:22.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:22.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:08:22.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:22.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:21 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1648306404' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T00:08:22.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:21 smithi046 ceph-mon[30151]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:22.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:22.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:22.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:22.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:08:22.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:22.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:21 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1648306404' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T00:08:22.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:21 smithi169 ceph-mon[34777]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:22.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:22.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:22.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:08:22.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:08:22.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:08:22.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:21 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1648306404' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T00:08:22.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:21 smithi142 ceph-mon[34533]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:22.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T00:08:22.931 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:23.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:23 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1648306404' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T00:08:23.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:23 smithi046 ceph-mon[30151]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T00:08:23.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:23 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1648306404' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T00:08:23.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:23 smithi169 ceph-mon[34777]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T00:08:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:23 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1648306404' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T00:08:23.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:23 smithi142 ceph-mon[34533]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T00:08:24.130 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:24 smithi046 ceph-mon[30151]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T00:08:24.130 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:24 smithi046 ceph-mon[30151]: pgmap v280: 33 pgs: 6 creating+peering, 4 active+clean, 23 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:24.130 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:24 smithi046 ceph-mon[30151]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T00:08:24.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:24 smithi169 ceph-mon[34777]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T00:08:24.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:24 smithi169 ceph-mon[34777]: pgmap v280: 33 pgs: 6 creating+peering, 4 active+clean, 23 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:24.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:24 smithi169 ceph-mon[34777]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T00:08:24.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:24 smithi142 ceph-mon[34533]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T00:08:24.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:24 smithi142 ceph-mon[34533]: pgmap v280: 33 pgs: 6 creating+peering, 4 active+clean, 23 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:24.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:24 smithi142 ceph-mon[34533]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T00:08:24.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.694+0000 7f363f59e700 1 -- 172.21.15.46:0/815279564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3640107c10 msgr2=0x7f3640108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.694+0000 7f363f59e700 1 --2- 172.21.15.46:0/815279564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3640107c10 0x7f3640108020 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f36280099c0 tx=0x7f3628009cd0 comp rx=0 tx=0).stop 2024-05-09T00:08:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.695+0000 7f363f59e700 1 -- 172.21.15.46:0/815279564 shutdown_connections 2024-05-09T00:08:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.695+0000 7f363f59e700 1 --2- 172.21.15.46:0/815279564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3640108560 0x7f364010fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.695+0000 7f363f59e700 1 --2- 172.21.15.46:0/815279564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3640103330 0x7f3640103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.695+0000 7f363f59e700 1 --2- 172.21.15.46:0/815279564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3640107c10 0x7f3640108020 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.695+0000 7f363f59e700 1 -- 172.21.15.46:0/815279564 >> 172.21.15.46:0/815279564 conn(0x7f36400fed20 msgr2=0x7f3640101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.695+0000 7f363f59e700 1 -- 172.21.15.46:0/815279564 shutdown_connections 2024-05-09T00:08:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.695+0000 7f363f59e700 1 -- 172.21.15.46:0/815279564 wait complete. 2024-05-09T00:08:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.695+0000 7f363f59e700 1 Processor -- start 2024-05-09T00:08:24.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.696+0000 7f363f59e700 1 -- start start 2024-05-09T00:08:24.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.696+0000 7f363f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3640103330 0x7f36401a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:24.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.696+0000 7f363f59e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3640107c10 0x7f36401a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:24.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.696+0000 7f363f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3640108560 0x7f364019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.696+0000 7f363f59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3640114330 con 0x7f3640103330 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.696+0000 7f363f59e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f36401141b0 con 0x7f3640107c10 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.696+0000 7f363f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f36401144b0 con 0x7f3640108560 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.696+0000 7f363e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3640103330 0x7f36401a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.696+0000 7f363e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3640103330 0x7f36401a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35098/0 (socket says 172.21.15.46:35098) 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.696+0000 7f363e59c700 1 -- 172.21.15.46:0/3892050036 learned_addr learned my addr 172.21.15.46:0/3892050036 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.696+0000 7f3635d9b700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3640107c10 0x7f36401a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.696+0000 7f363ed9d700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3640108560 0x7f364019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.697+0000 7f363e59c700 1 -- 172.21.15.46:0/3892050036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3640108560 msgr2=0x7f364019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.697+0000 7f363e59c700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3640108560 0x7f364019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.697+0000 7f363e59c700 1 -- 172.21.15.46:0/3892050036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3640107c10 msgr2=0x7f36401a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.697+0000 7f363e59c700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3640107c10 0x7f36401a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.697+0000 7f363e59c700 1 -- 172.21.15.46:0/3892050036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364019b1b0 con 0x7f3640103330 2024-05-09T00:08:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.697+0000 7f363e59c700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3640103330 0x7f36401a0890 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f363000b530 tx=0x7f363000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.697+0000 7f3637fff700 1 -- 172.21.15.46:0/3892050036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630015020 con 0x7f3640103330 2024-05-09T00:08:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.697+0000 7f3637fff700 1 -- 172.21.15.46:0/3892050036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3630011ce0 con 0x7f3640103330 2024-05-09T00:08:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.697+0000 7f3637fff700 1 -- 172.21.15.46:0/3892050036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630010dd0 con 0x7f3640103330 2024-05-09T00:08:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.697+0000 7f363f59e700 1 -- 172.21.15.46:0/3892050036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364019b4a0 con 0x7f3640103330 2024-05-09T00:08:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.697+0000 7f363f59e700 1 -- 172.21.15.46:0/3892050036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36401a7740 con 0x7f3640103330 2024-05-09T00:08:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.698+0000 7f363f59e700 1 -- 172.21.15.46:0/3892050036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f364006a620 con 0x7f3640103330 2024-05-09T00:08:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.700+0000 7f3637fff700 1 -- 172.21.15.46:0/3892050036 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3630018020 con 0x7f3640103330 2024-05-09T00:08:24.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.701+0000 7f3637fff700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f362c0b2900 0x7f362c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:24.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.701+0000 7f3635d9b700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f362c0b2900 0x7f362c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:24.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.701+0000 7f3637fff700 1 -- 172.21.15.46:0/3892050036 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7f36300e1a70 con 0x7f3640103330 2024-05-09T00:08:24.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.702+0000 7f3635d9b700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f362c0b2900 0x7f362c0b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3628006010 tx=0x7f362800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:24.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.705+0000 7f3637fff700 1 -- 172.21.15.46:0/3892050036 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f363001f020 con 0x7f3640103330 2024-05-09T00:08:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:24.917+0000 7f363f59e700 1 -- 172.21.15.46:0/3892050036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f364002e5f0 con 0x7f3640103330 2024-05-09T00:08:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.026+0000 7f3637fff700 1 -- 172.21.15.46:0/3892050036 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7f36300a9ac0 con 0x7f3640103330 2024-05-09T00:08:25.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.029+0000 7f363f59e700 1 -- 172.21.15.46:0/3892050036 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f362c0b2900 msgr2=0x7f362c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.029+0000 7f363f59e700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f362c0b2900 0x7f362c0b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3628006010 tx=0x7f362800b540 comp rx=0 tx=0).stop 2024-05-09T00:08:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.029+0000 7f363f59e700 1 -- 172.21.15.46:0/3892050036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3640103330 msgr2=0x7f36401a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.029+0000 7f363f59e700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3640103330 0x7f36401a0890 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f363000b530 tx=0x7f363000b840 comp rx=0 tx=0).stop 2024-05-09T00:08:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.029+0000 7f363f59e700 1 -- 172.21.15.46:0/3892050036 shutdown_connections 2024-05-09T00:08:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.029+0000 7f363f59e700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f362c0b2900 0x7f362c0b4e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.029+0000 7f363f59e700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3640108560 0x7f364019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.029+0000 7f363f59e700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3640103330 0x7f36401a0890 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.029+0000 7f363f59e700 1 --2- 172.21.15.46:0/3892050036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3640107c10 0x7f36401a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.029+0000 7f363f59e700 1 -- 172.21.15.46:0/3892050036 >> 172.21.15.46:0/3892050036 conn(0x7f36400fed20 msgr2=0x7f36401094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:25.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.029+0000 7f363f59e700 1 -- 172.21.15.46:0/3892050036 shutdown_connections 2024-05-09T00:08:25.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:25.029+0000 7f363f59e700 1 -- 172.21.15.46:0/3892050036 wait complete. 2024-05-09T00:08:25.027 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-05-09T00:08:25.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:25 smithi046 ceph-mon[30151]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T00:08:25.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:25 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3892050036' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T00:08:25.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:25 smithi169 ceph-mon[34777]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T00:08:25.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:25 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3892050036' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T00:08:25.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:25 smithi142 ceph-mon[34533]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T00:08:25.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:25 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3892050036' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T00:08:25.666 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T00:08:25.917 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:26.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:26 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3892050036' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T00:08:26.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:26 smithi046 ceph-mon[30151]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T00:08:26.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:26 smithi046 ceph-mon[30151]: pgmap v283: 97 pgs: 6 creating+peering, 20 active+clean, 71 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:26.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:26 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3892050036' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T00:08:26.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:26 smithi169 ceph-mon[34777]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T00:08:26.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:26 smithi169 ceph-mon[34777]: pgmap v283: 97 pgs: 6 creating+peering, 20 active+clean, 71 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:26.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:26 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3892050036' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T00:08:26.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:26 smithi142 ceph-mon[34533]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T00:08:26.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:26 smithi142 ceph-mon[34533]: pgmap v283: 97 pgs: 6 creating+peering, 20 active+clean, 71 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:27.155 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:27 smithi046 ceph-mon[30151]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T00:08:27.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:27 smithi169 ceph-mon[34777]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T00:08:27.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:27 smithi142 ceph-mon[34533]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T00:08:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.735+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2296931982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a4108740 msgr2=0x7fc8a41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.735+0000 7fc8a9f04700 1 --2- 172.21.15.46:0/2296931982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a4108740 0x7fc8a41100a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00b0d0 tx=0x7fc89c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:08:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.735+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2296931982 shutdown_connections 2024-05-09T00:08:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.735+0000 7fc8a9f04700 1 --2- 172.21.15.46:0/2296931982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a4108740 0x7fc8a41100a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.735+0000 7fc8a9f04700 1 --2- 172.21.15.46:0/2296931982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8a41034c0 0x7fc8a41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.735+0000 7fc8a9f04700 1 --2- 172.21.15.46:0/2296931982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a4107df0 0x7fc8a4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.735+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2296931982 >> 172.21.15.46:0/2296931982 conn(0x7fc8a40fedf0 msgr2=0x7fc8a4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.735+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2296931982 shutdown_connections 2024-05-09T00:08:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.735+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2296931982 wait complete. 2024-05-09T00:08:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.735+0000 7fc8a9f04700 1 Processor -- start 2024-05-09T00:08:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a9f04700 1 -- start start 2024-05-09T00:08:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a9f04700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a41034c0 0x7fc8a419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a9f04700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8a4107df0 0x7fc8a419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a9f04700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a4108740 0x7fc8a4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a9f04700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4113370 con 0x7fc8a41034c0 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a9f04700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41131f0 con 0x7fc8a4108740 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a9f04700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41134f0 con 0x7fc8a4107df0 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a41034c0 0x7fc8a419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a41034c0 0x7fc8a419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51410/0 (socket says 172.21.15.46:51410) 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a37fe700 1 -- 172.21.15.46:0/2211798658 learned_addr learned my addr 172.21.15.46:0/2211798658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a3fff700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a4108740 0x7fc8a4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a2ffd700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8a4107df0 0x7fc8a419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a37fe700 1 -- 172.21.15.46:0/2211798658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8a4107df0 msgr2=0x7fc8a419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a37fe700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8a4107df0 0x7fc8a419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a37fe700 1 -- 172.21.15.46:0/2211798658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a4108740 msgr2=0x7fc8a4199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a37fe700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a4108740 0x7fc8a4199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.736+0000 7fc8a37fe700 1 -- 172.21.15.46:0/2211798658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a419a160 con 0x7fc8a41034c0 2024-05-09T00:08:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.737+0000 7fc8a37fe700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a41034c0 0x7fc8a419f890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc894007910 tx=0x7fc894007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.737+0000 7fc8a0ff9700 1 -- 172.21.15.46:0/2211798658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894007e60 con 0x7fc8a41034c0 2024-05-09T00:08:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.737+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2211798658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a419a450 con 0x7fc8a41034c0 2024-05-09T00:08:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.737+0000 7fc8a0ff9700 1 -- 172.21.15.46:0/2211798658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8940044d0 con 0x7fc8a41034c0 2024-05-09T00:08:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.737+0000 7fc8a0ff9700 1 -- 172.21.15.46:0/2211798658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894005350 con 0x7fc8a41034c0 2024-05-09T00:08:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.737+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2211798658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a40759f0 con 0x7fc8a41034c0 2024-05-09T00:08:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.738+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2211798658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a40512c0 con 0x7fc8a41034c0 2024-05-09T00:08:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.740+0000 7fc8a0ff9700 1 -- 172.21.15.46:0/2211798658 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc89401f020 con 0x7fc8a41034c0 2024-05-09T00:08:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.740+0000 7fc8a0ff9700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc88c0b2900 0x7fc88c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.741+0000 7fc8a2ffd700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc88c0b2900 0x7fc88c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.741+0000 7fc8a0ff9700 1 -- 172.21.15.46:0/2211798658 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7fc8940e3a80 con 0x7fc8a41034c0 2024-05-09T00:08:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.741+0000 7fc8a2ffd700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc88c0b2900 0x7fc88c0b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc898005950 tx=0x7fc8980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:27.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:27.744+0000 7fc8a0ff9700 1 -- 172.21.15.46:0/2211798658 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc894026030 con 0x7fc8a41034c0 2024-05-09T00:08:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.007+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2211798658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc8a4076470 con 0x7fc8a41034c0 2024-05-09T00:08:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.046+0000 7fc8a0ff9700 1 -- 172.21.15.46:0/2211798658 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc894004750 con 0x7fc8a41034c0 2024-05-09T00:08:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.049+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2211798658 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc88c0b2900 msgr2=0x7fc88c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.049+0000 7fc8a9f04700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc88c0b2900 0x7fc88c0b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc898005950 tx=0x7fc8980058e0 comp rx=0 tx=0).stop 2024-05-09T00:08:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.049+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2211798658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a41034c0 msgr2=0x7fc8a419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.049+0000 7fc8a9f04700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a41034c0 0x7fc8a419f890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc894007910 tx=0x7fc894007c20 comp rx=0 tx=0).stop 2024-05-09T00:08:28.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.049+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2211798658 shutdown_connections 2024-05-09T00:08:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.049+0000 7fc8a9f04700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc88c0b2900 0x7fc88c0b4e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.049+0000 7fc8a9f04700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a4108740 0x7fc8a4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.049+0000 7fc8a9f04700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a41034c0 0x7fc8a419f890 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.049+0000 7fc8a9f04700 1 --2- 172.21.15.46:0/2211798658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8a4107df0 0x7fc8a419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.049+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2211798658 >> 172.21.15.46:0/2211798658 conn(0x7fc8a40fedf0 msgr2=0x7fc8a4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.049+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2211798658 shutdown_connections 2024-05-09T00:08:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:28.049+0000 7fc8a9f04700 1 -- 172.21.15.46:0/2211798658 wait complete. 2024-05-09T00:08:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T00:08:28.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:28 smithi046 ceph-mon[30151]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T00:08:28.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:28 smithi046 ceph-mon[30151]: pgmap v286: 97 pgs: 6 creating+peering, 40 active+clean, 51 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:28.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:28 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2211798658' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T00:08:28.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:28 smithi046 ceph-mon[30151]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T00:08:28.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:28 smithi169 ceph-mon[34777]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T00:08:28.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:28 smithi169 ceph-mon[34777]: pgmap v286: 97 pgs: 6 creating+peering, 40 active+clean, 51 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:28.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:28 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2211798658' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T00:08:28.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:28 smithi169 ceph-mon[34777]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T00:08:28.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:28 smithi142 ceph-mon[34533]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T00:08:28.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:28 smithi142 ceph-mon[34533]: pgmap v286: 97 pgs: 6 creating+peering, 40 active+clean, 51 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:08:28.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:28 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2211798658' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T00:08:28.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:28 smithi142 ceph-mon[34533]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-09T00:08:28.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:08:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:29.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:29 smithi046 ceph-mon[30151]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T00:08:29.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:29 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2211798658' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T00:08:29.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:29 smithi046 ceph-mon[30151]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T00:08:29.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:29 smithi046 ceph-mon[30151]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T00:08:29.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:29 smithi169 ceph-mon[34777]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T00:08:29.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:29 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2211798658' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T00:08:29.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:29 smithi169 ceph-mon[34777]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T00:08:29.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:29 smithi169 ceph-mon[34777]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T00:08:29.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:29 smithi142 ceph-mon[34533]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T00:08:29.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:29 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2211798658' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T00:08:29.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:29 smithi142 ceph-mon[34533]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-09T00:08:29.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:29 smithi142 ceph-mon[34533]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-09T00:08:30.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:30 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] up:active 2024-05-09T00:08:30.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:30 smithi046 ceph-mon[30151]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:08:30.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:30 smithi046 ceph-mon[30151]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:08:30.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:30 smithi046 ceph-mon[30151]: pgmap v288: 97 pgs: 82 active+clean, 15 unknown; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 963 B/s wr, 7 op/s 2024-05-09T00:08:30.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:30 smithi046 ceph-mon[30151]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T00:08:30.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:30 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] up:active 2024-05-09T00:08:30.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:30 smithi169 ceph-mon[34777]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:08:30.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:30 smithi169 ceph-mon[34777]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:08:30.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:30 smithi169 ceph-mon[34777]: pgmap v288: 97 pgs: 82 active+clean, 15 unknown; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 963 B/s wr, 7 op/s 2024-05-09T00:08:30.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:30 smithi169 ceph-mon[34777]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T00:08:30.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:30 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] up:active 2024-05-09T00:08:30.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:30 smithi142 ceph-mon[34533]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:08:30.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:30 smithi142 ceph-mon[34533]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:08:30.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:30 smithi142 ceph-mon[34533]: pgmap v288: 97 pgs: 82 active+clean, 15 unknown; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 963 B/s wr, 7 op/s 2024-05-09T00:08:30.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:30 smithi142 ceph-mon[34533]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T00:08:30.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.968+0000 7f34394d8700 1 -- 172.21.15.46:0/145721779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3434107df0 msgr2=0x7f3434108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:30.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.968+0000 7f34394d8700 1 --2- 172.21.15.46:0/145721779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3434107df0 0x7f3434108200 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f34280099c0 tx=0x7f3428009cd0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.968+0000 7f34394d8700 1 -- 172.21.15.46:0/145721779 shutdown_connections 2024-05-09T00:08:30.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.968+0000 7f34394d8700 1 --2- 172.21.15.46:0/145721779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3434108740 0x7f34341100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.968+0000 7f34394d8700 1 --2- 172.21.15.46:0/145721779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34341034c0 0x7f34341038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.968+0000 7f34394d8700 1 --2- 172.21.15.46:0/145721779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3434107df0 0x7f3434108200 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.968+0000 7f34394d8700 1 -- 172.21.15.46:0/145721779 >> 172.21.15.46:0/145721779 conn(0x7f34340fedf0 msgr2=0x7f3434101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:30.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.968+0000 7f34394d8700 1 -- 172.21.15.46:0/145721779 shutdown_connections 2024-05-09T00:08:30.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.968+0000 7f34394d8700 1 -- 172.21.15.46:0/145721779 wait complete. 2024-05-09T00:08:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.969+0000 7f34394d8700 1 Processor -- start 2024-05-09T00:08:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.969+0000 7f34394d8700 1 -- start start 2024-05-09T00:08:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.969+0000 7f34394d8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34341034c0 0x7f34340ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.969+0000 7f34394d8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3434107df0 0x7f34340ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.969+0000 7f34394d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3434108740 0x7f34340ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.969+0000 7f34394d8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34341133a0 con 0x7f3434108740 2024-05-09T00:08:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.969+0000 7f34394d8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3434113220 con 0x7f34341034c0 2024-05-09T00:08:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.969+0000 7f34394d8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3434113520 con 0x7f3434107df0 2024-05-09T00:08:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.969+0000 7f34337fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3434108740 0x7f34340ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.969+0000 7f34337fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3434108740 0x7f34340ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51432/0 (socket says 172.21.15.46:51432) 2024-05-09T00:08:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.969+0000 7f34337fe700 1 -- 172.21.15.46:0/512167331 learned_addr learned my addr 172.21.15.46:0/512167331 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.969+0000 7f3432ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34341034c0 0x7f34340ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.970+0000 7f34327fc700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3434107df0 0x7f34340ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.970+0000 7f34337fe700 1 -- 172.21.15.46:0/512167331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3434107df0 msgr2=0x7f34340ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.970+0000 7f34337fe700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3434107df0 0x7f34340ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.970+0000 7f34337fe700 1 -- 172.21.15.46:0/512167331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34341034c0 msgr2=0x7f34340ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.970+0000 7f34337fe700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34341034c0 0x7f34340ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.970+0000 7f34337fe700 1 -- 172.21.15.46:0/512167331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34341a9200 con 0x7f3434108740 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.970+0000 7f34337fe700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3434108740 0x7f34340ffd50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f342c007ee0 tx=0x7f342c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.970+0000 7f341bfff700 1 -- 172.21.15.46:0/512167331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c00dbd0 con 0x7f3434108740 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.970+0000 7f34394d8700 1 -- 172.21.15.46:0/512167331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34341a94f0 con 0x7f3434108740 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.970+0000 7f341bfff700 1 -- 172.21.15.46:0/512167331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f342c00dd70 con 0x7f3434108740 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.970+0000 7f341bfff700 1 -- 172.21.15.46:0/512167331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c005990 con 0x7f3434108740 2024-05-09T00:08:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.970+0000 7f34394d8700 1 -- 172.21.15.46:0/512167331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34341a9a00 con 0x7f3434108740 2024-05-09T00:08:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.971+0000 7f34394d8700 1 -- 172.21.15.46:0/512167331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3434104970 con 0x7f3434108740 2024-05-09T00:08:30.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.974+0000 7f341bfff700 1 -- 172.21.15.46:0/512167331 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f342c01a020 con 0x7f3434108740 2024-05-09T00:08:30.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.974+0000 7f341bfff700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f341c0b2880 0x7f341c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.975+0000 7f3432ffd700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f341c0b2880 0x7f341c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.975+0000 7f341bfff700 1 -- 172.21.15.46:0/512167331 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f342c0e1eb0 con 0x7f3434108740 2024-05-09T00:08:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.975+0000 7f3432ffd700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f341c0b2880 0x7f341c0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3424005950 tx=0x7f342400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:30.978+0000 7f341bfff700 1 -- 172.21.15.46:0/512167331 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f342c010070 con 0x7f3434108740 2024-05-09T00:08:31.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.196+0000 7f34394d8700 1 -- 172.21.15.46:0/512167331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f343402e5f0 con 0x7f3434108740 2024-05-09T00:08:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.197+0000 7f341bfff700 1 -- 172.21.15.46:0/512167331 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22215 (secure 0 0 0) 0x7f342c0a9d20 con 0x7f3434108740 2024-05-09T00:08:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:08:31.194 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","created":"2024-05-09T00:00:14.356524+0000","modified":"2024-05-09T00:08:28.009855+0000","last_up_change":"2024-05-09T00:07:23.332796+0000","last_in_change":"2024-05-09T00:07:03.238539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:03:33.278910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:08:21.014416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:08:24.922968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"740d6388-5c1f-45eb-93b0-7583bfa5a225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6803","nonce":3727342125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6805","nonce":3727342125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6809","nonce":3727342125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6807","nonce":3727342125}]},"public_addr":"172.21.15.46:6803/3727342125","cluster_addr":"172.21.15.46:6805/3727342125","heartbeat_back_addr":"172.21.15.46:6809/3727342125","heartbeat_front_addr":"172.21.15.46:6807/3727342125","state":["exists","up"]},{"osd":1,"uuid":"85fbf3ae-0be8-4504-84d6-66fae0d2169d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6801","nonce":3156042469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6803","nonce":3156042469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6807","nonce":3156042469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6805","nonce":3156042469}]},"public_addr":"172.21.15.142:6801/3156042469","cluster_addr":"172.21.15.142:6803/3156042469","heartbeat_back_addr":"172.21.15.142:6807/3156042469","heartbeat_front_addr":"172.21.15.142:6805/3156042469","state":["exists","up"]},{"osd":2,"uuid":"05c070e7-83ad-4949-9834-4ce51ac0a2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6801","nonce":2680673064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6803","nonce":2680673064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6807","nonce":2680673064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6805","nonce":2680673064}]},"public_addr":"172.21.15.169:6801/2680673064","cluster_addr":"172.21.15.169:6803/2680673064","heartbeat_back_addr":"172.21.15.169:6807/2680673064","heartbeat_front_addr":"172.21.15.169:6805/2680673064","state":["exists","up"]},{"osd":3,"uuid":"09d7bbe1-58b6-4407-bef2-920f179dfb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6811","nonce":1158536261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6813","nonce":1158536261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6817","nonce":1158536261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6815","nonce":1158536261}]},"public_addr":"172.21.15.46:6811/1158536261","cluster_addr":"172.21.15.46:6813/1158536261","heartbeat_back_addr":"172.21.15.46:6817/1158536261","heartbeat_front_addr":"172.21.15.46:6815/1158536261","state":["exists","up"]},{"osd":4,"uuid":"66c9644b-6b0b-4fe7-aa63-b9d87bff7153","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6809","nonce":2299616865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6811","nonce":2299616865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6815","nonce":2299616865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6813","nonce":2299616865}]},"public_addr":"172.21.15.142:6809/2299616865","cluster_addr":"172.21.15.142:6811/2299616865","heartbeat_back_addr":"172.21.15.142:6815/2299616865","heartbeat_front_addr":"172.21.15.142:6813/2299616865","state":["exists","up"]},{"osd":5,"uuid":"96a72e01-1c5f-418e-bb9b-b790d9b3059b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6809","nonce":445044713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6811","nonce":445044713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6815","nonce":445044713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6813","nonce":445044713}]},"public_addr":"172.21.15.169:6809/445044713","cluster_addr":"172.21.15.169:6811/445044713","heartbeat_back_addr":"172.21.15.169:6815/445044713","heartbeat_front_addr":"172.21.15.169:6813/445044713","state":["exists","up"]},{"osd":6,"uuid":"6b459542-0e56-4369-b542-e36c46a18f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6819","nonce":1394035482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6821","nonce":1394035482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6825","nonce":1394035482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6823","nonce":1394035482}]},"public_addr":"172.21.15.46:6819/1394035482","cluster_addr":"172.21.15.46:6821/1394035482","heartbeat_back_addr":"172.21.15.46:6825/1394035482","heartbeat_front_addr":"172.21.15.46:6823/1394035482","state":["exists","up"]},{"osd":7,"uuid":"786b4517-4cbd-4fa3-b5a0-39cccf839893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6817","nonce":596805607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6819","nonce":596805607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6823","nonce":596805607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6821","nonce":596805607}]},"public_addr":"172.21.15.142:6817/596805607","cluster_addr":"172.21.15.142:6819/596805607","heartbeat_back_addr":"172.21.15.142:6823/596805607","heartbeat_front_addr":"172.21.15.142:6821/596805607","state":["exists","up"]},{"osd":8,"uuid":"b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6817","nonce":2650652376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6819","nonce":2650652376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6823","nonce":2650652376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6821","nonce":2650652376}]},"public_addr":"172.21.15.169:6817/2650652376","cluster_addr":"172.21.15.169:6819/2650652376","heartbeat_back_addr":"172.21.15.169:6823/2650652376","heartbeat_front_addr":"172.21.15.169:6821/2650652376","state":["exists","up"]},{"osd":9,"uuid":"32df0dab-5875-4beb-b733-60a915c94a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6827","nonce":374861248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6829","nonce":374861248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6833","nonce":374861248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6831","nonce":374861248}]},"public_addr":"172.21.15.46:6827/374861248","cluster_addr":"172.21.15.46:6829/374861248","heartbeat_back_addr":"172.21.15.46:6833/374861248","heartbeat_front_addr":"172.21.15.46:6831/374861248","state":["exists","up"]},{"osd":10,"uuid":"380a4f47-e528-4258-8836-32eac0e6e630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6825","nonce":1364891982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6827","nonce":1364891982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6831","nonce":1364891982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6829","nonce":1364891982}]},"public_addr":"172.21.15.142:6825/1364891982","cluster_addr":"172.21.15.142:6827/1364891982","heartbeat_back_addr":"172.21.15.142:6831/1364891982","heartbeat_front_addr":"172.21.15.142:6829/1364891982","state":["exists","up"]},{"osd":11,"uuid":"56b04b15-6fcd-497a-aa66-19da9624b656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6825","nonce":4262478976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6827","nonce":4262478976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6831","nonce":4262478976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6829","nonce":4262478976}]},"public_addr":"172.21.15.169:6825/4262478976","cluster_addr":"172.21.15.169:6827/4262478976","heartbeat_back_addr":"172.21.15.169:6831/4262478976","heartbeat_front_addr":"172.21.15.169:6829/4262478976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:02:42.159298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:06.053328+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:29.587137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:54.573782+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:20.276073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:45.320565+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:10.985522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:36.806137+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:02.266736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:29.788773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:55.827639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:07:21.762382+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/1072138393":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2786891364":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/7723180":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/626888490":"2024-05-10T00:01:09.191239+0000","172.21.15.46:6800/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/4236543921":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2168313126":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6800/953818590":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6801/953818590":"2024-05-10T00:00:41.833323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:08:31.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.200+0000 7f34394d8700 1 -- 172.21.15.46:0/512167331 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f341c0b2880 msgr2=0x7f341c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.200+0000 7f34394d8700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f341c0b2880 0x7f341c0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3424005950 tx=0x7f342400b410 comp rx=0 tx=0).stop 2024-05-09T00:08:31.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.200+0000 7f34394d8700 1 -- 172.21.15.46:0/512167331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3434108740 msgr2=0x7f34340ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.200+0000 7f34394d8700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3434108740 0x7f34340ffd50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f342c007ee0 tx=0x7f342c00d3b0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.200+0000 7f34394d8700 1 -- 172.21.15.46:0/512167331 shutdown_connections 2024-05-09T00:08:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.200+0000 7f34394d8700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f341c0b2880 0x7f341c0b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.200+0000 7f34394d8700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3434108740 0x7f34340ffd50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.200+0000 7f34394d8700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34341034c0 0x7f34340ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.200+0000 7f34394d8700 1 --2- 172.21.15.46:0/512167331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3434107df0 0x7f34340ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.200+0000 7f34394d8700 1 -- 172.21.15.46:0/512167331 >> 172.21.15.46:0/512167331 conn(0x7f34340fedf0 msgr2=0x7f3434075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:31.197 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:31 smithi046 ceph-mon[30151]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T00:08:31.197 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:31 smithi046 ceph-mon[30151]: Cluster is now healthy 2024-05-09T00:08:31.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.200+0000 7f34394d8700 1 -- 172.21.15.46:0/512167331 shutdown_connections 2024-05-09T00:08:31.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:31.200+0000 7f34394d8700 1 -- 172.21.15.46:0/512167331 wait complete. 2024-05-09T00:08:31.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:31 smithi169 ceph-mon[34777]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T00:08:31.351 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:31 smithi169 ceph-mon[34777]: Cluster is now healthy 2024-05-09T00:08:31.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:31 smithi142 ceph-mon[34533]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T00:08:31.439 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:31 smithi142 ceph-mon[34533]: Cluster is now healthy 2024-05-09T00:08:31.898 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:08:32.154 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:32 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/512167331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:08:32.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:32 smithi046 ceph-mon[30151]: pgmap v289: 97 pgs: 97 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2024-05-09T00:08:32.340 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:32 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/512167331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:08:32.340 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:32 smithi142 ceph-mon[34533]: pgmap v289: 97 pgs: 97 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2024-05-09T00:08:32.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:32 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/512167331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:08:32.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:32 smithi169 ceph-mon[34777]: pgmap v289: 97 pgs: 97 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2024-05-09T00:08:32.352 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:08:32 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i[65278]: 2024-05-09T00:08:32.305+0000 7f127432b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T00:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.025+0000 7fd8096fc700 1 -- 172.21.15.46:0/2114823457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd804108760 msgr2=0x7fd8041100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.025+0000 7fd8096fc700 1 --2- 172.21.15.46:0/2114823457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd804108760 0x7fd8041100c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00b0d0 tx=0x7fd7fc00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.025+0000 7fd8096fc700 1 -- 172.21.15.46:0/2114823457 shutdown_connections 2024-05-09T00:08:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.025+0000 7fd8096fc700 1 --2- 172.21.15.46:0/2114823457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd804108760 0x7fd8041100c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.025+0000 7fd8096fc700 1 --2- 172.21.15.46:0/2114823457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8041034e0 0x7fd8041038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.025+0000 7fd8096fc700 1 --2- 172.21.15.46:0/2114823457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804107e10 0x7fd804108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.025+0000 7fd8096fc700 1 -- 172.21.15.46:0/2114823457 >> 172.21.15.46:0/2114823457 conn(0x7fd8040fef30 msgr2=0x7fd804101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.025+0000 7fd8096fc700 1 -- 172.21.15.46:0/2114823457 shutdown_connections 2024-05-09T00:08:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.025+0000 7fd8096fc700 1 -- 172.21.15.46:0/2114823457 wait complete. 2024-05-09T00:08:34.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.026+0000 7fd8096fc700 1 Processor -- start 2024-05-09T00:08:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.026+0000 7fd8096fc700 1 -- start start 2024-05-09T00:08:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8096fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8041034e0 0x7fd804199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8096fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804107e10 0x7fd80419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8096fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd804108760 0x7fd80419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8096fc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd804113390 con 0x7fd804108760 2024-05-09T00:08:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8096fc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd804113210 con 0x7fd8041034e0 2024-05-09T00:08:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8096fc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd804113510 con 0x7fd804107e10 2024-05-09T00:08:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8037fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd804108760 0x7fd80419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8037fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd804108760 0x7fd80419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51448/0 (socket says 172.21.15.46:51448) 2024-05-09T00:08:34.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8037fe700 1 -- 172.21.15.46:0/741489673 learned_addr learned my addr 172.21.15.46:0/741489673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8027fc700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804107e10 0x7fd80419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd802ffd700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8041034e0 0x7fd804199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8037fe700 1 -- 172.21.15.46:0/741489673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804107e10 msgr2=0x7fd80419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8037fe700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804107e10 0x7fd80419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8037fe700 1 -- 172.21.15.46:0/741489673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8041034e0 msgr2=0x7fd804199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8037fe700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8041034e0 0x7fd804199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8037fe700 1 -- 172.21.15.46:0/741489673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80419ec50 con 0x7fd804108760 2024-05-09T00:08:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd8037fe700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd804108760 0x7fd80419e5b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0095c0 tx=0x7fd7fc007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.027+0000 7fd7ebfff700 1 -- 172.21.15.46:0/741489673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc01f070 con 0x7fd804108760 2024-05-09T00:08:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.028+0000 7fd7ebfff700 1 -- 172.21.15.46:0/741489673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7fc003c00 con 0x7fd804108760 2024-05-09T00:08:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.028+0000 7fd8096fc700 1 -- 172.21.15.46:0/741489673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80419eee0 con 0x7fd804108760 2024-05-09T00:08:34.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.028+0000 7fd7ebfff700 1 -- 172.21.15.46:0/741489673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc01b6a0 con 0x7fd804108760 2024-05-09T00:08:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.028+0000 7fd8096fc700 1 -- 172.21.15.46:0/741489673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80419f3c0 con 0x7fd804108760 2024-05-09T00:08:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.029+0000 7fd8096fc700 1 -- 172.21.15.46:0/741489673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8040512c0 con 0x7fd804108760 2024-05-09T00:08:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.030+0000 7fd7ebfff700 1 -- 172.21.15.46:0/741489673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd7fc022070 con 0x7fd804108760 2024-05-09T00:08:34.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.031+0000 7fd7ebfff700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7ec0b2880 0x7fd7ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.031+0000 7fd802ffd700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7ec0b2880 0x7fd7ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.031+0000 7fd7ebfff700 1 -- 172.21.15.46:0/741489673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fd7fc00f020 con 0x7fd804108760 2024-05-09T00:08:34.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.031+0000 7fd802ffd700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7ec0b2880 0x7fd7ec0b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4005fd0 tx=0x7fd7f400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:34.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.036+0000 7fd7ebfff700 1 -- 172.21.15.46:0/741489673 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd7fc029020 con 0x7fd804108760 2024-05-09T00:08:34.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.255+0000 7fd8096fc700 1 -- 172.21.15.46:0/741489673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd80402e5f0 con 0x7fd804108760 2024-05-09T00:08:34.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.257+0000 7fd7ebfff700 1 -- 172.21.15.46:0/741489673 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22215 (secure 0 0 0) 0x7fd7fc0ab260 con 0x7fd804108760 2024-05-09T00:08:34.252 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:08:34.252 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","created":"2024-05-09T00:00:14.356524+0000","modified":"2024-05-09T00:08:28.009855+0000","last_up_change":"2024-05-09T00:07:23.332796+0000","last_in_change":"2024-05-09T00:07:03.238539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:03:33.278910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:08:21.014416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:08:24.922968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"740d6388-5c1f-45eb-93b0-7583bfa5a225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6803","nonce":3727342125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6805","nonce":3727342125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6809","nonce":3727342125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6807","nonce":3727342125}]},"public_addr":"172.21.15.46:6803/3727342125","cluster_addr":"172.21.15.46:6805/3727342125","heartbeat_back_addr":"172.21.15.46:6809/3727342125","heartbeat_front_addr":"172.21.15.46:6807/3727342125","state":["exists","up"]},{"osd":1,"uuid":"85fbf3ae-0be8-4504-84d6-66fae0d2169d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6801","nonce":3156042469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6803","nonce":3156042469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6807","nonce":3156042469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6805","nonce":3156042469}]},"public_addr":"172.21.15.142:6801/3156042469","cluster_addr":"172.21.15.142:6803/3156042469","heartbeat_back_addr":"172.21.15.142:6807/3156042469","heartbeat_front_addr":"172.21.15.142:6805/3156042469","state":["exists","up"]},{"osd":2,"uuid":"05c070e7-83ad-4949-9834-4ce51ac0a2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6801","nonce":2680673064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6803","nonce":2680673064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6807","nonce":2680673064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6805","nonce":2680673064}]},"public_addr":"172.21.15.169:6801/2680673064","cluster_addr":"172.21.15.169:6803/2680673064","heartbeat_back_addr":"172.21.15.169:6807/2680673064","heartbeat_front_addr":"172.21.15.169:6805/2680673064","state":["exists","up"]},{"osd":3,"uuid":"09d7bbe1-58b6-4407-bef2-920f179dfb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6811","nonce":1158536261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6813","nonce":1158536261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6817","nonce":1158536261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6815","nonce":1158536261}]},"public_addr":"172.21.15.46:6811/1158536261","cluster_addr":"172.21.15.46:6813/1158536261","heartbeat_back_addr":"172.21.15.46:6817/1158536261","heartbeat_front_addr":"172.21.15.46:6815/1158536261","state":["exists","up"]},{"osd":4,"uuid":"66c9644b-6b0b-4fe7-aa63-b9d87bff7153","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6809","nonce":2299616865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6811","nonce":2299616865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6815","nonce":2299616865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6813","nonce":2299616865}]},"public_addr":"172.21.15.142:6809/2299616865","cluster_addr":"172.21.15.142:6811/2299616865","heartbeat_back_addr":"172.21.15.142:6815/2299616865","heartbeat_front_addr":"172.21.15.142:6813/2299616865","state":["exists","up"]},{"osd":5,"uuid":"96a72e01-1c5f-418e-bb9b-b790d9b3059b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6809","nonce":445044713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6811","nonce":445044713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6815","nonce":445044713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6813","nonce":445044713}]},"public_addr":"172.21.15.169:6809/445044713","cluster_addr":"172.21.15.169:6811/445044713","heartbeat_back_addr":"172.21.15.169:6815/445044713","heartbeat_front_addr":"172.21.15.169:6813/445044713","state":["exists","up"]},{"osd":6,"uuid":"6b459542-0e56-4369-b542-e36c46a18f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6819","nonce":1394035482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6821","nonce":1394035482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6825","nonce":1394035482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6823","nonce":1394035482}]},"public_addr":"172.21.15.46:6819/1394035482","cluster_addr":"172.21.15.46:6821/1394035482","heartbeat_back_addr":"172.21.15.46:6825/1394035482","heartbeat_front_addr":"172.21.15.46:6823/1394035482","state":["exists","up"]},{"osd":7,"uuid":"786b4517-4cbd-4fa3-b5a0-39cccf839893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6817","nonce":596805607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6819","nonce":596805607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6823","nonce":596805607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6821","nonce":596805607}]},"public_addr":"172.21.15.142:6817/596805607","cluster_addr":"172.21.15.142:6819/596805607","heartbeat_back_addr":"172.21.15.142:6823/596805607","heartbeat_front_addr":"172.21.15.142:6821/596805607","state":["exists","up"]},{"osd":8,"uuid":"b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6817","nonce":2650652376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6819","nonce":2650652376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6823","nonce":2650652376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6821","nonce":2650652376}]},"public_addr":"172.21.15.169:6817/2650652376","cluster_addr":"172.21.15.169:6819/2650652376","heartbeat_back_addr":"172.21.15.169:6823/2650652376","heartbeat_front_addr":"172.21.15.169:6821/2650652376","state":["exists","up"]},{"osd":9,"uuid":"32df0dab-5875-4beb-b733-60a915c94a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6827","nonce":374861248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6829","nonce":374861248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6833","nonce":374861248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6831","nonce":374861248}]},"public_addr":"172.21.15.46:6827/374861248","cluster_addr":"172.21.15.46:6829/374861248","heartbeat_back_addr":"172.21.15.46:6833/374861248","heartbeat_front_addr":"172.21.15.46:6831/374861248","state":["exists","up"]},{"osd":10,"uuid":"380a4f47-e528-4258-8836-32eac0e6e630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6825","nonce":1364891982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6827","nonce":1364891982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6831","nonce":1364891982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6829","nonce":1364891982}]},"public_addr":"172.21.15.142:6825/1364891982","cluster_addr":"172.21.15.142:6827/1364891982","heartbeat_back_addr":"172.21.15.142:6831/1364891982","heartbeat_front_addr":"172.21.15.142:6829/1364891982","state":["exists","up"]},{"osd":11,"uuid":"56b04b15-6fcd-497a-aa66-19da9624b656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6825","nonce":4262478976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6827","nonce":4262478976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6831","nonce":4262478976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6829","nonce":4262478976}]},"public_addr":"172.21.15.169:6825/4262478976","cluster_addr":"172.21.15.169:6827/4262478976","heartbeat_back_addr":"172.21.15.169:6831/4262478976","heartbeat_front_addr":"172.21.15.169:6829/4262478976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:02:42.159298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:06.053328+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:29.587137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:54.573782+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:20.276073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:45.320565+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:10.985522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:36.806137+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:02.266736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:29.788773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:55.827639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:07:21.762382+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/1072138393":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2786891364":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/7723180":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/626888490":"2024-05-10T00:01:09.191239+0000","172.21.15.46:6800/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/4236543921":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2168313126":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6800/953818590":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6801/953818590":"2024-05-10T00:00:41.833323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:08:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.260+0000 7fd8096fc700 1 -- 172.21.15.46:0/741489673 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7ec0b2880 msgr2=0x7fd7ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.260+0000 7fd8096fc700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7ec0b2880 0x7fd7ec0b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4005fd0 tx=0x7fd7f400a560 comp rx=0 tx=0).stop 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.260+0000 7fd8096fc700 1 -- 172.21.15.46:0/741489673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd804108760 msgr2=0x7fd80419e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.260+0000 7fd8096fc700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd804108760 0x7fd80419e5b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0095c0 tx=0x7fd7fc007a20 comp rx=0 tx=0).stop 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.260+0000 7fd8096fc700 1 -- 172.21.15.46:0/741489673 shutdown_connections 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.260+0000 7fd8096fc700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7ec0b2880 0x7fd7ec0b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.260+0000 7fd8096fc700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd804108760 0x7fd80419e5b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.260+0000 7fd8096fc700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8041034e0 0x7fd804199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.260+0000 7fd8096fc700 1 --2- 172.21.15.46:0/741489673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804107e10 0x7fd80419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.260+0000 7fd8096fc700 1 -- 172.21.15.46:0/741489673 >> 172.21.15.46:0/741489673 conn(0x7fd8040fef30 msgr2=0x7fd804107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.260+0000 7fd8096fc700 1 -- 172.21.15.46:0/741489673 shutdown_connections 2024-05-09T00:08:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:34.260+0000 7fd8096fc700 1 -- 172.21.15.46:0/741489673 wait complete. 2024-05-09T00:08:34.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:34 smithi046 ceph-mon[30151]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-09T00:08:34.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:34 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/741489673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:08:34.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:34 smithi169 ceph-mon[34777]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-09T00:08:34.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:34 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/741489673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:08:34.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:34 smithi142 ceph-mon[34533]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-09T00:08:34.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:34 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/741489673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:08:35.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-09T00:08:35.299 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:36.435 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:36 smithi046 ceph-mon[30151]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T00:08:36.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:36 smithi169 ceph-mon[34777]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T00:08:36.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:36 smithi142 ceph-mon[34533]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T00:08:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.037+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2351272618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a141045e0 msgr2=0x7f3a141049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.037+0000 7f3a1c11d700 1 --2- 172.21.15.46:0/2351272618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a141045e0 0x7f3a141049f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3a100099c0 tx=0x7f3a10009cd0 comp rx=0 tx=0).stop 2024-05-09T00:08:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.037+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2351272618 shutdown_connections 2024-05-09T00:08:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.037+0000 7f3a1c11d700 1 --2- 172.21.15.46:0/2351272618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a14109860 0x7f3a141111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.037+0000 7f3a1c11d700 1 --2- 172.21.15.46:0/2351272618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a141045e0 0x7f3a141049f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.037+0000 7f3a1c11d700 1 --2- 172.21.15.46:0/2351272618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14108f10 0x7f3a14109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.037+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2351272618 >> 172.21.15.46:0/2351272618 conn(0x7f3a140ffed0 msgr2=0x7f3a14102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.037+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2351272618 shutdown_connections 2024-05-09T00:08:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.037+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2351272618 wait complete. 2024-05-09T00:08:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a1c11d700 1 Processor -- start 2024-05-09T00:08:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a1c11d700 1 -- start start 2024-05-09T00:08:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a1c11d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a141045e0 0x7f3a14076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a1c11d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14108f10 0x7f3a14077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a1c11d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a14109860 0x7f3a14077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a1c11d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14114490 con 0x7f3a14109860 2024-05-09T00:08:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a1c11d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14114310 con 0x7f3a14108f10 2024-05-09T00:08:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a1c11d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14114610 con 0x7f3a141045e0 2024-05-09T00:08:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a1a6ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a14109860 0x7f3a14077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a1a6ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a14109860 0x7f3a14077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50988/0 (socket says 172.21.15.46:50988) 2024-05-09T00:08:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a1a6ba700 1 -- 172.21.15.46:0/2369168617 learned_addr learned my addr 172.21.15.46:0/2369168617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:37.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a19eb9700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a141045e0 0x7f3a14076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.038+0000 7f3a196b8700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14108f10 0x7f3a14077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.039+0000 7f3a1a6ba700 1 -- 172.21.15.46:0/2369168617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a141045e0 msgr2=0x7f3a14076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.039+0000 7f3a1a6ba700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a141045e0 0x7f3a14076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.039+0000 7f3a1a6ba700 1 -- 172.21.15.46:0/2369168617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14108f10 msgr2=0x7f3a14077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.039+0000 7f3a1a6ba700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14108f10 0x7f3a14077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.039+0000 7f3a1a6ba700 1 -- 172.21.15.46:0/2369168617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a141a9150 con 0x7f3a14109860 2024-05-09T00:08:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.039+0000 7f3a1a6ba700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a14109860 0x7f3a14077650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c007d70 tx=0x7f3a0c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.039+0000 7f3a0affd700 1 -- 172.21.15.46:0/2369168617 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c016020 con 0x7f3a14109860 2024-05-09T00:08:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.039+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2369168617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a141a9440 con 0x7f3a14109860 2024-05-09T00:08:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.039+0000 7f3a0affd700 1 -- 172.21.15.46:0/2369168617 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0c00ad40 con 0x7f3a14109860 2024-05-09T00:08:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.039+0000 7f3a0affd700 1 -- 172.21.15.46:0/2369168617 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0115e0 con 0x7f3a14109860 2024-05-09T00:08:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.039+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2369168617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a141a9980 con 0x7f3a14109860 2024-05-09T00:08:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.040+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2369168617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a140512c0 con 0x7f3a14109860 2024-05-09T00:08:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.042+0000 7f3a0affd700 1 -- 172.21.15.46:0/2369168617 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3a0c019020 con 0x7f3a14109860 2024-05-09T00:08:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.042+0000 7f3a0affd700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3a000b2880 0x7f3a000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.043+0000 7f3a19eb9700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3a000b2880 0x7f3a000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.043+0000 7f3a0affd700 1 -- 172.21.15.46:0/2369168617 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f3a0c0e2030 con 0x7f3a14109860 2024-05-09T00:08:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.043+0000 7f3a19eb9700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3a000b2880 0x7f3a000b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009990 tx=0x7f3a10000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:37.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.048+0000 7f3a0affd700 1 -- 172.21.15.46:0/2369168617 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3a0c020020 con 0x7f3a14109860 2024-05-09T00:08:37.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.311+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2369168617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3a1402e5f0 con 0x7f3a14109860 2024-05-09T00:08:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.343+0000 7f3a0affd700 1 -- 172.21.15.46:0/2369168617 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f3a0c0a9ea0 con 0x7f3a14109860 2024-05-09T00:08:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.346+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2369168617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3a000b2880 msgr2=0x7f3a000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.346+0000 7f3a1c11d700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3a000b2880 0x7f3a000b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009990 tx=0x7f3a10000bc0 comp rx=0 tx=0).stop 2024-05-09T00:08:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.346+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2369168617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a14109860 msgr2=0x7f3a14077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.346+0000 7f3a1c11d700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a14109860 0x7f3a14077650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c007d70 tx=0x7f3a0c00a340 comp rx=0 tx=0).stop 2024-05-09T00:08:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.346+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2369168617 shutdown_connections 2024-05-09T00:08:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.346+0000 7f3a1c11d700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3a000b2880 0x7f3a000b4d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.346+0000 7f3a1c11d700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a14109860 0x7f3a14077650 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.346+0000 7f3a1c11d700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a141045e0 0x7f3a14076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.346+0000 7f3a1c11d700 1 --2- 172.21.15.46:0/2369168617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a14108f10 0x7f3a14077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.346+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2369168617 >> 172.21.15.46:0/2369168617 conn(0x7f3a140ffed0 msgr2=0x7f3a141022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.346+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2369168617 shutdown_connections 2024-05-09T00:08:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:37.346+0000 7f3a1c11d700 1 -- 172.21.15.46:0/2369168617 wait complete. 2024-05-09T00:08:37.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:37 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2369168617' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T00:08:37.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:37 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2369168617' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T00:08:37.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:37 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2369168617' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T00:08:37.999 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-09T00:08:38.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-09T00:08:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:38.591 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:38 smithi046 ceph-mon[30151]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T00:08:38.592 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:38 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2369168617' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T00:08:38.592 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:38 smithi046 ceph-mon[30151]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:08:38.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:38 smithi169 ceph-mon[34777]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T00:08:38.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:38 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2369168617' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T00:08:38.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:38 smithi169 ceph-mon[34777]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:08:38.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:38 smithi142 ceph-mon[34533]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T00:08:38.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:38 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2369168617' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T00:08:38.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:38 smithi142 ceph-mon[34533]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:08:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.004+0000 7fc1e85f1700 1 -- 172.21.15.46:0/2944933566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e0108f00 msgr2=0x7fc1e0109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.004+0000 7fc1e85f1700 1 --2- 172.21.15.46:0/2944933566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e0108f00 0x7fc1e0109310 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00099c0 tx=0x7fc1d0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:08:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.004+0000 7fc1e85f1700 1 -- 172.21.15.46:0/2944933566 shutdown_connections 2024-05-09T00:08:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.004+0000 7fc1e85f1700 1 --2- 172.21.15.46:0/2944933566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1e0109850 0x7fc1e01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.004+0000 7fc1e85f1700 1 --2- 172.21.15.46:0/2944933566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1e0102ae0 0x7fc1e0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.004+0000 7fc1e85f1700 1 --2- 172.21.15.46:0/2944933566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e0108f00 0x7fc1e0109310 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.004+0000 7fc1e85f1700 1 -- 172.21.15.46:0/2944933566 >> 172.21.15.46:0/2944933566 conn(0x7fc1e00fe540 msgr2=0x7fc1e0100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.004+0000 7fc1e85f1700 1 -- 172.21.15.46:0/2944933566 shutdown_connections 2024-05-09T00:08:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.004+0000 7fc1e85f1700 1 -- 172.21.15.46:0/2944933566 wait complete. 2024-05-09T00:08:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.005+0000 7fc1e85f1700 1 Processor -- start 2024-05-09T00:08:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.005+0000 7fc1e85f1700 1 -- start start 2024-05-09T00:08:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.005+0000 7fc1e85f1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1e0102ae0 0x7fc1e0103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.005+0000 7fc1e85f1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1e0108f00 0x7fc1e0103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.005+0000 7fc1e85f1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e0109850 0x7fc1e01044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.005+0000 7fc1e85f1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e0114460 con 0x7fc1e0109850 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.005+0000 7fc1e85f1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e01142e0 con 0x7fc1e0108f00 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.005+0000 7fc1e85f1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e01145e0 con 0x7fc1e0102ae0 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e6b8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e0109850 0x7fc1e01044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e6b8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e0109850 0x7fc1e01044b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51016/0 (socket says 172.21.15.46:51016) 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e6b8e700 1 -- 172.21.15.46:0/3106906794 learned_addr learned my addr 172.21.15.46:0/3106906794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e638d700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1e0102ae0 0x7fc1e0103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e5b8c700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1e0108f00 0x7fc1e0103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e6b8e700 1 -- 172.21.15.46:0/3106906794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1e0102ae0 msgr2=0x7fc1e0103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e6b8e700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1e0102ae0 0x7fc1e0103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e6b8e700 1 -- 172.21.15.46:0/3106906794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1e0108f00 msgr2=0x7fc1e0103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e6b8e700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1e0108f00 0x7fc1e0103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e6b8e700 1 -- 172.21.15.46:0/3106906794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e01a9210 con 0x7fc1e0109850 2024-05-09T00:08:40.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e6b8e700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e0109850 0x7fc1e01044b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8007ee0 tx=0x7fc1d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:40.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1d77fe700 1 -- 172.21.15.46:0/3106906794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d800dd20 con 0x7fc1e0109850 2024-05-09T00:08:40.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1d77fe700 1 -- 172.21.15.46:0/3106906794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d8004020 con 0x7fc1e0109850 2024-05-09T00:08:40.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e85f1700 1 -- 172.21.15.46:0/3106906794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e01a9500 con 0x7fc1e0109850 2024-05-09T00:08:40.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1d77fe700 1 -- 172.21.15.46:0/3106906794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8015660 con 0x7fc1e0109850 2024-05-09T00:08:40.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.006+0000 7fc1e85f1700 1 -- 172.21.15.46:0/3106906794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e01a9a10 con 0x7fc1e0109850 2024-05-09T00:08:40.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.008+0000 7fc1e85f1700 1 -- 172.21.15.46:0/3106906794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e0105b70 con 0x7fc1e0109850 2024-05-09T00:08:40.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.010+0000 7fc1d77fe700 1 -- 172.21.15.46:0/3106906794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc1d80041d0 con 0x7fc1e0109850 2024-05-09T00:08:40.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.011+0000 7fc1d77fe700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc1cc0b2900 0x7fc1cc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:40.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.011+0000 7fc1e638d700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc1cc0b2900 0x7fc1cc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:40.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.011+0000 7fc1d77fe700 1 -- 172.21.15.46:0/3106906794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fc1d80e2180 con 0x7fc1e0109850 2024-05-09T00:08:40.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.012+0000 7fc1e638d700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc1cc0b2900 0x7fc1cc0b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc007be0 tx=0x7fc1dc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:40.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.015+0000 7fc1d77fe700 1 -- 172.21.15.46:0/3106906794 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1d8010070 con 0x7fc1e0109850 2024-05-09T00:08:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.277+0000 7fc1e85f1700 1 -- 172.21.15.46:0/3106906794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fc1e002e5f0 con 0x7fc1e0109850 2024-05-09T00:08:40.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.347+0000 7fc1d77fe700 1 -- 172.21.15.46:0/3106906794 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fc1d80a9ff0 con 0x7fc1e0109850 2024-05-09T00:08:40.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.350+0000 7fc1e85f1700 1 -- 172.21.15.46:0/3106906794 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc1cc0b2900 msgr2=0x7fc1cc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:40.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.350+0000 7fc1e85f1700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc1cc0b2900 0x7fc1cc0b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc007be0 tx=0x7fc1dc0073d0 comp rx=0 tx=0).stop 2024-05-09T00:08:40.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.350+0000 7fc1e85f1700 1 -- 172.21.15.46:0/3106906794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e0109850 msgr2=0x7fc1e01044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:40.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.350+0000 7fc1e85f1700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e0109850 0x7fc1e01044b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8007ee0 tx=0x7fc1d800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:08:40.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.350+0000 7fc1e85f1700 1 -- 172.21.15.46:0/3106906794 shutdown_connections 2024-05-09T00:08:40.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.350+0000 7fc1e85f1700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc1cc0b2900 0x7fc1cc0b4e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:40.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.350+0000 7fc1e85f1700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e0109850 0x7fc1e01044b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:40.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.350+0000 7fc1e85f1700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1e0102ae0 0x7fc1e0103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:40.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.350+0000 7fc1e85f1700 1 --2- 172.21.15.46:0/3106906794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1e0108f00 0x7fc1e0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:40.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.350+0000 7fc1e85f1700 1 -- 172.21.15.46:0/3106906794 >> 172.21.15.46:0/3106906794 conn(0x7fc1e00fe540 msgr2=0x7fc1e0107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:40.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.350+0000 7fc1e85f1700 1 -- 172.21.15.46:0/3106906794 shutdown_connections 2024-05-09T00:08:40.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:40.350+0000 7fc1e85f1700 1 -- 172.21.15.46:0/3106906794 wait complete. 2024-05-09T00:08:40.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:40 smithi046 ceph-mon[30151]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-09T00:08:40.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:40 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3106906794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-09T00:08:40.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:40 smithi169 ceph-mon[34777]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-09T00:08:40.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:40 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3106906794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-09T00:08:40.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:40 smithi142 ceph-mon[34533]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-09T00:08:40.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:40 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3106906794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-09T00:08:41.181 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-09T00:08:41.453 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:41.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:41 smithi046 ceph-mon[30151]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T00:08:41.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:41 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3106906794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-09T00:08:41.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:41 smithi046 ceph-mon[30151]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T00:08:41.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:41 smithi046 ceph-mon[30151]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:08:41.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:41 smithi046 ceph-mon[30151]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-05-09T00:08:41.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:41 smithi046 ceph-mon[30151]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-09T00:08:41.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:41 smithi142 ceph-mon[34533]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T00:08:41.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:41 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3106906794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-09T00:08:41.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:41 smithi142 ceph-mon[34533]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T00:08:41.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:41 smithi142 ceph-mon[34533]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:08:41.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:41 smithi142 ceph-mon[34533]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-05-09T00:08:41.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:41 smithi142 ceph-mon[34533]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-09T00:08:41.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:41 smithi169 ceph-mon[34777]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T00:08:41.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:41 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3106906794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-09T00:08:41.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:41 smithi169 ceph-mon[34777]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T00:08:41.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:41 smithi169 ceph-mon[34777]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-09T00:08:41.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:41 smithi169 ceph-mon[34777]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-05-09T00:08:41.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:41 smithi169 ceph-mon[34777]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-05-09T00:08:42.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:42 smithi046 ceph-mon[30151]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 1 op/s 2024-05-09T00:08:42.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:42 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] up:active 2024-05-09T00:08:42.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:42 smithi046 ceph-mon[30151]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T00:08:42.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:42 smithi046 ceph-mon[30151]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T00:08:42.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:42 smithi046 ceph-mon[30151]: Cluster is now healthy 2024-05-09T00:08:42.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:42 smithi046 ceph-mon[30151]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-05-09T00:08:42.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:42 smithi046 ceph-mon[30151]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-05-09T00:08:42.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:42 smithi046 ceph-mon[30151]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-09T00:08:42.602 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:08:42 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i[65278]: 2024-05-09T00:08:42.306+0000 7f127432b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-09T00:08:42.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:42 smithi169 ceph-mon[34777]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 1 op/s 2024-05-09T00:08:42.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:42 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] up:active 2024-05-09T00:08:42.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:42 smithi169 ceph-mon[34777]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T00:08:42.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:42 smithi169 ceph-mon[34777]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T00:08:42.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:42 smithi169 ceph-mon[34777]: Cluster is now healthy 2024-05-09T00:08:42.602 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:42 smithi169 ceph-mon[34777]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-05-09T00:08:42.603 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:42 smithi169 ceph-mon[34777]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-05-09T00:08:42.603 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:42 smithi169 ceph-mon[34777]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-09T00:08:42.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:42 smithi142 ceph-mon[34533]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 1 op/s 2024-05-09T00:08:42.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:42 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] up:active 2024-05-09T00:08:42.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:42 smithi142 ceph-mon[34533]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T00:08:42.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:42 smithi142 ceph-mon[34533]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T00:08:42.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:42 smithi142 ceph-mon[34533]: Cluster is now healthy 2024-05-09T00:08:42.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:42 smithi142 ceph-mon[34533]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-05-09T00:08:42.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:42 smithi142 ceph-mon[34533]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-05-09T00:08:42.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:42 smithi142 ceph-mon[34533]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-09T00:08:43.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.276+0000 7f60630f0700 1 -- 172.21.15.46:0/307387948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605c103c10 msgr2=0x7f605c104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:43.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.276+0000 7f60630f0700 1 --2- 172.21.15.46:0/307387948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605c103c10 0x7f605c104020 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f604400b0d0 tx=0x7f604400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:08:43.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.277+0000 7f60630f0700 1 -- 172.21.15.46:0/307387948 shutdown_connections 2024-05-09T00:08:43.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.277+0000 7f60630f0700 1 --2- 172.21.15.46:0/307387948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f605c108970 0x7f605c1102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:43.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.277+0000 7f60630f0700 1 --2- 172.21.15.46:0/307387948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605c103c10 0x7f605c104020 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:43.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.277+0000 7f60630f0700 1 --2- 172.21.15.46:0/307387948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f605c108020 0x7f605c108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:43.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.277+0000 7f60630f0700 1 -- 172.21.15.46:0/307387948 >> 172.21.15.46:0/307387948 conn(0x7f605c0fe540 msgr2=0x7f605c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:43.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.277+0000 7f60630f0700 1 -- 172.21.15.46:0/307387948 shutdown_connections 2024-05-09T00:08:43.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.277+0000 7f60630f0700 1 -- 172.21.15.46:0/307387948 wait complete. 2024-05-09T00:08:43.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.277+0000 7f60630f0700 1 Processor -- start 2024-05-09T00:08:43.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f60630f0700 1 -- start start 2024-05-09T00:08:43.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f60630f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605c103c10 0x7f605c0fea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:43.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f60630f0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f605c108020 0x7f605c0fef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:43.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f60630f0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f605c108970 0x7f605c0ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:43.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f60630f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f605c1135a0 con 0x7f605c103c10 2024-05-09T00:08:43.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f60630f0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f605c113420 con 0x7f605c108020 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f60630f0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f605c113720 con 0x7f605c108970 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f6060e8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605c103c10 0x7f605c0fea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f6060e8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605c103c10 0x7f605c0fea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51030/0 (socket says 172.21.15.46:51030) 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f6060e8c700 1 -- 172.21.15.46:0/1951924366 learned_addr learned my addr 172.21.15.46:0/1951924366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f605bfff700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f605c108020 0x7f605c0fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f606168d700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f605c108970 0x7f605c0ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f6060e8c700 1 -- 172.21.15.46:0/1951924366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f605c108970 msgr2=0x7f605c0ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f6060e8c700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f605c108970 0x7f605c0ff4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f6060e8c700 1 -- 172.21.15.46:0/1951924366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f605c108020 msgr2=0x7f605c0fef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f6060e8c700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f605c108020 0x7f605c0fef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f6060e8c700 1 -- 172.21.15.46:0/1951924366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605c1a93d0 con 0x7f605c103c10 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.278+0000 7f6060e8c700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605c103c10 0x7f605c0fea20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6044007a20 tx=0x7f6044007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.279+0000 7f6059ffb700 1 -- 172.21.15.46:0/1951924366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604401f070 con 0x7f605c103c10 2024-05-09T00:08:43.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.279+0000 7f60630f0700 1 -- 172.21.15.46:0/1951924366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605c1a9660 con 0x7f605c103c10 2024-05-09T00:08:43.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.279+0000 7f6059ffb700 1 -- 172.21.15.46:0/1951924366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6044004c50 con 0x7f605c103c10 2024-05-09T00:08:43.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.279+0000 7f6059ffb700 1 -- 172.21.15.46:0/1951924366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604401b690 con 0x7f605c103c10 2024-05-09T00:08:43.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.279+0000 7f60630f0700 1 -- 172.21.15.46:0/1951924366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605c1a9b10 con 0x7f605c103c10 2024-05-09T00:08:43.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.280+0000 7f60630f0700 1 -- 172.21.15.46:0/1951924366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605c0512c0 con 0x7f605c103c10 2024-05-09T00:08:43.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.282+0000 7f6059ffb700 1 -- 172.21.15.46:0/1951924366 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6044022070 con 0x7f605c103c10 2024-05-09T00:08:43.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.282+0000 7f6059ffb700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f60480b2880 0x7f60480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:43.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.282+0000 7f6059ffb700 1 -- 172.21.15.46:0/1951924366 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f604400f020 con 0x7f605c103c10 2024-05-09T00:08:43.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.282+0000 7f605bfff700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f60480b2880 0x7f60480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:43.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.283+0000 7f605bfff700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f60480b2880 0x7f60480b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f605c101bd0 tx=0x7f604c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:43.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.286+0000 7f6059ffb700 1 -- 172.21.15.46:0/1951924366 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6044029020 con 0x7f605c103c10 2024-05-09T00:08:43.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:43.553+0000 7f60630f0700 1 -- 172.21.15.46:0/1951924366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f605c06c160 con 0x7f605c103c10 2024-05-09T00:08:43.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:43 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] up:active 2024-05-09T00:08:43.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:43 smithi046 ceph-mon[30151]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-09T00:08:43.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:43 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] up:active 2024-05-09T00:08:43.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:43 smithi142 ceph-mon[34533]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-09T00:08:43.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:43 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] up:active 2024-05-09T00:08:43.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:43 smithi169 ceph-mon[34777]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-09T00:08:44.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.381+0000 7f6059ffb700 1 -- 172.21.15.46:0/1951924366 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f60440ab4c0 con 0x7f605c103c10 2024-05-09T00:08:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.385+0000 7f60630f0700 1 -- 172.21.15.46:0/1951924366 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f60480b2880 msgr2=0x7f60480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.385+0000 7f60630f0700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f60480b2880 0x7f60480b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f605c101bd0 tx=0x7f604c009260 comp rx=0 tx=0).stop 2024-05-09T00:08:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.385+0000 7f60630f0700 1 -- 172.21.15.46:0/1951924366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605c103c10 msgr2=0x7f605c0fea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.385+0000 7f60630f0700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605c103c10 0x7f605c0fea20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6044007a20 tx=0x7f6044007a50 comp rx=0 tx=0).stop 2024-05-09T00:08:44.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.385+0000 7f60630f0700 1 -- 172.21.15.46:0/1951924366 shutdown_connections 2024-05-09T00:08:44.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.385+0000 7f60630f0700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f60480b2880 0x7f60480b4d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.385+0000 7f60630f0700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f605c108970 0x7f605c0ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.385+0000 7f60630f0700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605c103c10 0x7f605c0fea20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.385+0000 7f60630f0700 1 --2- 172.21.15.46:0/1951924366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f605c108020 0x7f605c0fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:44.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.385+0000 7f60630f0700 1 -- 172.21.15.46:0/1951924366 >> 172.21.15.46:0/1951924366 conn(0x7f605c0fe540 msgr2=0x7f605c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:44.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.385+0000 7f60630f0700 1 -- 172.21.15.46:0/1951924366 shutdown_connections 2024-05-09T00:08:44.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:44.386+0000 7f60630f0700 1 -- 172.21.15.46:0/1951924366 wait complete. 2024-05-09T00:08:44.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:44 smithi142 ceph-mon[34533]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 4 op/s 2024-05-09T00:08:44.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:44 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1951924366' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T00:08:44.706 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:44 smithi046 ceph-mon[30151]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 4 op/s 2024-05-09T00:08:44.706 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:44 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1951924366' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T00:08:44.851 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:44 smithi169 ceph-mon[34777]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 4 op/s 2024-05-09T00:08:44.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:44 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1951924366' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T00:08:45.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-09T00:08:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:45.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:45 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1951924366' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T00:08:45.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:45 smithi046 ceph-mon[30151]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-09T00:08:45.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:45 smithi046 ceph-mon[30151]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:08:45.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:45 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1951924366' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T00:08:45.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:45 smithi142 ceph-mon[34533]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-09T00:08:45.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:45 smithi142 ceph-mon[34533]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:08:45.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:45 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1951924366' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T00:08:45.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:45 smithi169 ceph-mon[34777]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-05-09T00:08:45.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:45 smithi169 ceph-mon[34777]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:08:46.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:46 smithi046 ceph-mon[30151]: pgmap v296: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-09T00:08:46.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:46 smithi142 ceph-mon[34533]: pgmap v296: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-09T00:08:46.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:46 smithi169 ceph-mon[34777]: pgmap v296: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-09T00:08:47.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.145+0000 7f81c409d700 1 -- 172.21.15.46:0/1763021471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc108740 msgr2=0x7f81bc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:47.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.145+0000 7f81c409d700 1 --2- 172.21.15.46:0/1763021471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc108740 0x7f81bc1100a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f81b400b0d0 tx=0x7f81b400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:08:47.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.145+0000 7f81c409d700 1 -- 172.21.15.46:0/1763021471 shutdown_connections 2024-05-09T00:08:47.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.145+0000 7f81c409d700 1 --2- 172.21.15.46:0/1763021471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc108740 0x7f81bc1100a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:47.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.145+0000 7f81c409d700 1 --2- 172.21.15.46:0/1763021471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81bc1034c0 0x7f81bc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:47.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.145+0000 7f81c409d700 1 --2- 172.21.15.46:0/1763021471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc107df0 0x7f81bc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:47.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.145+0000 7f81c409d700 1 -- 172.21.15.46:0/1763021471 >> 172.21.15.46:0/1763021471 conn(0x7f81bc0fedc0 msgr2=0x7f81bc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:47.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.145+0000 7f81c409d700 1 -- 172.21.15.46:0/1763021471 shutdown_connections 2024-05-09T00:08:47.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.145+0000 7f81c409d700 1 -- 172.21.15.46:0/1763021471 wait complete. 2024-05-09T00:08:47.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.146+0000 7f81c409d700 1 Processor -- start 2024-05-09T00:08:47.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.146+0000 7f81c409d700 1 -- start start 2024-05-09T00:08:47.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.146+0000 7f81c409d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc1034c0 0x7f81bc19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:47.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.146+0000 7f81c409d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81bc107df0 0x7f81bc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.146+0000 7f81c409d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc108740 0x7f81bc19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.146+0000 7f81c409d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc113370 con 0x7f81bc108740 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.146+0000 7f81c409d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1131f0 con 0x7f81bc107df0 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.146+0000 7f81c409d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1134f0 con 0x7f81bc1034c0 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.146+0000 7f81c263a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc108740 0x7f81bc19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81c1e39700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc1034c0 0x7f81bc19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81c263a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc108740 0x7f81bc19f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52540/0 (socket says 172.21.15.46:52540) 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81c263a700 1 -- 172.21.15.46:0/2786653381 learned_addr learned my addr 172.21.15.46:0/2786653381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81c1638700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81bc107df0 0x7f81bc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81c263a700 1 -- 172.21.15.46:0/2786653381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc1034c0 msgr2=0x7f81bc19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81c263a700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc1034c0 0x7f81bc19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81c263a700 1 -- 172.21.15.46:0/2786653381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81bc107df0 msgr2=0x7f81bc19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81c263a700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81bc107df0 0x7f81bc19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:47.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81c263a700 1 -- 172.21.15.46:0/2786653381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81bc19fbd0 con 0x7f81bc108740 2024-05-09T00:08:47.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81c263a700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc108740 0x7f81bc19f5e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f81b40095c0 tx=0x7f81b4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:47.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81aeffd700 1 -- 172.21.15.46:0/2786653381 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b401f070 con 0x7f81bc108740 2024-05-09T00:08:47.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81aeffd700 1 -- 172.21.15.46:0/2786653381 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b4007e20 con 0x7f81bc108740 2024-05-09T00:08:47.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81c409d700 1 -- 172.21.15.46:0/2786653381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81bc19fe60 con 0x7f81bc108740 2024-05-09T00:08:47.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81aeffd700 1 -- 172.21.15.46:0/2786653381 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b401b4a0 con 0x7f81bc108740 2024-05-09T00:08:47.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.147+0000 7f81c409d700 1 -- 172.21.15.46:0/2786653381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81bc1a0390 con 0x7f81bc108740 2024-05-09T00:08:47.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.149+0000 7f81c409d700 1 -- 172.21.15.46:0/2786653381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81bc0512c0 con 0x7f81bc108740 2024-05-09T00:08:47.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.150+0000 7f81aeffd700 1 -- 172.21.15.46:0/2786653381 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f81b4022070 con 0x7f81bc108740 2024-05-09T00:08:47.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.151+0000 7f81aeffd700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f81a80b2900 0x7f81a80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:47.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.151+0000 7f81c1e39700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f81a80b2900 0x7f81a80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:47.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.151+0000 7f81aeffd700 1 -- 172.21.15.46:0/2786653381 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f81b400f020 con 0x7f81bc108740 2024-05-09T00:08:47.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.152+0000 7f81c1e39700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f81a80b2900 0x7f81a80b4e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f81b800ac50 tx=0x7f81b800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:47.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.155+0000 7f81aeffd700 1 -- 172.21.15.46:0/2786653381 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f81b4029020 con 0x7f81bc108740 2024-05-09T00:08:47.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:47.422+0000 7f81c409d700 1 -- 172.21.15.46:0/2786653381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f81bc1a7890 con 0x7f81bc108740 2024-05-09T00:08:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.398+0000 7f81aeffd700 1 -- 172.21.15.46:0/2786653381 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f81b40ab4f0 con 0x7f81bc108740 2024-05-09T00:08:48.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.402+0000 7f81c409d700 1 -- 172.21.15.46:0/2786653381 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f81a80b2900 msgr2=0x7f81a80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:48.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.402+0000 7f81c409d700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f81a80b2900 0x7f81a80b4e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f81b800ac50 tx=0x7f81b800a380 comp rx=0 tx=0).stop 2024-05-09T00:08:48.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.402+0000 7f81c409d700 1 -- 172.21.15.46:0/2786653381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc108740 msgr2=0x7f81bc19f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.402+0000 7f81c409d700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc108740 0x7f81bc19f5e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f81b40095c0 tx=0x7f81b4007a20 comp rx=0 tx=0).stop 2024-05-09T00:08:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.402+0000 7f81c409d700 1 -- 172.21.15.46:0/2786653381 shutdown_connections 2024-05-09T00:08:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.402+0000 7f81c409d700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f81a80b2900 0x7f81a80b4e10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.402+0000 7f81c409d700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81bc108740 0x7f81bc19f5e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.402+0000 7f81c409d700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc1034c0 0x7f81bc19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.402+0000 7f81c409d700 1 --2- 172.21.15.46:0/2786653381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81bc107df0 0x7f81bc19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.402+0000 7f81c409d700 1 -- 172.21.15.46:0/2786653381 >> 172.21.15.46:0/2786653381 conn(0x7f81bc0fedc0 msgr2=0x7f81bc107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.403+0000 7f81c409d700 1 -- 172.21.15.46:0/2786653381 shutdown_connections 2024-05-09T00:08:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:48.403+0000 7f81c409d700 1 -- 172.21.15.46:0/2786653381 wait complete. 2024-05-09T00:08:48.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:48 smithi142 ceph-mon[34533]: pgmap v297: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-09T00:08:48.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:48 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2786653381' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T00:08:48.730 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:48 smithi046 ceph-mon[30151]: pgmap v297: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-09T00:08:48.730 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:48 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2786653381' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T00:08:48.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:48 smithi169 ceph-mon[34777]: pgmap v297: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-09T00:08:48.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:48 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2786653381' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T00:08:49.176 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-05-09T00:08:49.176 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-09T00:08:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:49.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:49 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2786653381' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T00:08:49.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:49 smithi142 ceph-mon[34533]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:08:49.712 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:49 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2786653381' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T00:08:49.712 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:49 smithi046 ceph-mon[30151]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:08:49.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:49 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2786653381' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T00:08:49.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:49 smithi169 ceph-mon[34777]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:08:50.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:50 smithi046 ceph-mon[30151]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-09T00:08:50.601 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:50 smithi169 ceph-mon[34777]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-09T00:08:50.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:50 smithi142 ceph-mon[34533]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-09T00:08:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.161+0000 7f718deeb700 1 -- 172.21.15.46:0/4059204260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7188108f20 msgr2=0x7f7188109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.161+0000 7f718deeb700 1 --2- 172.21.15.46:0/4059204260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7188108f20 0x7f7188109330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f717c0099c0 tx=0x7f717c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:08:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.161+0000 7f718deeb700 1 -- 172.21.15.46:0/4059204260 shutdown_connections 2024-05-09T00:08:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.161+0000 7f718deeb700 1 --2- 172.21.15.46:0/4059204260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7188109870 0x7f71881111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.161+0000 7f718deeb700 1 --2- 172.21.15.46:0/4059204260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71881045f0 0x7f7188104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.162+0000 7f718deeb700 1 --2- 172.21.15.46:0/4059204260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7188108f20 0x7f7188109330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.162+0000 7f718deeb700 1 -- 172.21.15.46:0/4059204260 >> 172.21.15.46:0/4059204260 conn(0x7f7188100040 msgr2=0x7f7188102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.162+0000 7f718deeb700 1 -- 172.21.15.46:0/4059204260 shutdown_connections 2024-05-09T00:08:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.162+0000 7f718deeb700 1 -- 172.21.15.46:0/4059204260 wait complete. 2024-05-09T00:08:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.162+0000 7f718deeb700 1 Processor -- start 2024-05-09T00:08:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f718deeb700 1 -- start start 2024-05-09T00:08:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f718deeb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71881045f0 0x7f718819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f718deeb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7188108f20 0x7f718819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f718deeb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7188109870 0x7f718819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f718deeb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7188114570 con 0x7f71881045f0 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f718deeb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71881143f0 con 0x7f7188108f20 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f718deeb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71881146f0 con 0x7f7188109870 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f71877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71881045f0 0x7f718819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f71877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71881045f0 0x7f718819af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52570/0 (socket says 172.21.15.46:52570) 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f71877fe700 1 -- 172.21.15.46:0/3528457961 learned_addr learned my addr 172.21.15.46:0/3528457961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f7186ffd700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7188108f20 0x7f718819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f7187fff700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7188109870 0x7f718819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f71877fe700 1 -- 172.21.15.46:0/3528457961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7188109870 msgr2=0x7f718819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f71877fe700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7188109870 0x7f718819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f71877fe700 1 -- 172.21.15.46:0/3528457961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7188108f20 msgr2=0x7f718819b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f71877fe700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7188108f20 0x7f718819b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f71877fe700 1 -- 172.21.15.46:0/3528457961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718819fdf0 con 0x7f71881045f0 2024-05-09T00:08:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.163+0000 7f7187fff700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7188109870 0x7f718819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:08:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.164+0000 7f71877fe700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71881045f0 0x7f718819af20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f717800b530 tx=0x7f717800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.164+0000 7f7184ff9700 1 -- 172.21.15.46:0/3528457961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178015020 con 0x7f71881045f0 2024-05-09T00:08:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.164+0000 7f718deeb700 1 -- 172.21.15.46:0/3528457961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71881a00e0 con 0x7f71881045f0 2024-05-09T00:08:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.164+0000 7f7184ff9700 1 -- 172.21.15.46:0/3528457961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7178011ce0 con 0x7f71881045f0 2024-05-09T00:08:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.164+0000 7f7184ff9700 1 -- 172.21.15.46:0/3528457961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178010e40 con 0x7f71881045f0 2024-05-09T00:08:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.164+0000 7f718deeb700 1 -- 172.21.15.46:0/3528457961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71881a7a00 con 0x7f71881045f0 2024-05-09T00:08:51.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.165+0000 7f718deeb700 1 -- 172.21.15.46:0/3528457961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71880512c0 con 0x7f71881045f0 2024-05-09T00:08:51.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.167+0000 7f7184ff9700 1 -- 172.21.15.46:0/3528457961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7178018020 con 0x7f71881045f0 2024-05-09T00:08:51.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.167+0000 7f7184ff9700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71700b2880 0x7f71700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:51.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.167+0000 7f7186ffd700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71700b2880 0x7f71700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:51.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.168+0000 7f7184ff9700 1 -- 172.21.15.46:0/3528457961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f71780e1eb0 con 0x7f71881045f0 2024-05-09T00:08:51.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.168+0000 7f7186ffd700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71700b2880 0x7f71700b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f717c006010 tx=0x7f717c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.171+0000 7f7184ff9700 1 -- 172.21.15.46:0/3528457961 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f717801f020 con 0x7f71881045f0 2024-05-09T00:08:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.419+0000 7f718deeb700 1 -- 172.21.15.46:0/3528457961 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f718819c280 con 0x7f71700b2880 2024-05-09T00:08:51.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.447+0000 7f7184ff9700 1 -- 172.21.15.46:0/3528457961 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f718819c280 con 0x7f71700b2880 2024-05-09T00:08:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.449+0000 7f718deeb700 1 -- 172.21.15.46:0/3528457961 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71700b2880 msgr2=0x7f71700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.449+0000 7f718deeb700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71700b2880 0x7f71700b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f717c006010 tx=0x7f717c000bc0 comp rx=0 tx=0).stop 2024-05-09T00:08:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.449+0000 7f718deeb700 1 -- 172.21.15.46:0/3528457961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71881045f0 msgr2=0x7f718819af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.449+0000 7f718deeb700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71881045f0 0x7f718819af20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f717800b530 tx=0x7f717800b840 comp rx=0 tx=0).stop 2024-05-09T00:08:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.449+0000 7f718deeb700 1 -- 172.21.15.46:0/3528457961 shutdown_connections 2024-05-09T00:08:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.449+0000 7f718deeb700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71700b2880 0x7f71700b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.449+0000 7f718deeb700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7188109870 0x7f718819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.449+0000 7f718deeb700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71881045f0 0x7f718819af20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.449+0000 7f718deeb700 1 --2- 172.21.15.46:0/3528457961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7188108f20 0x7f718819b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.449+0000 7f718deeb700 1 -- 172.21.15.46:0/3528457961 >> 172.21.15.46:0/3528457961 conn(0x7f7188100040 msgr2=0x7f7188108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.449+0000 7f718deeb700 1 -- 172.21.15.46:0/3528457961 shutdown_connections 2024-05-09T00:08:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:51.449+0000 7f718deeb700 1 -- 172.21.15.46:0/3528457961 wait complete. 2024-05-09T00:08:51.734 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:08:51 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:08:51.424+0000 7f170f849700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T00:08:51.735 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:08:51 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:08:51.424+0000 7f170f849700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T00:08:51.735 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:08:51 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:08:51.424+0000 7f170f849700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T00:08:51.735 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:08:51 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:08:51.424+0000 7f170f849700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T00:08:51.735 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:08:51 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:08:51.424+0000 7f170f849700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T00:08:52.153 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-09T00:08:52.415 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:52.671 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:52 smithi046 ceph-mon[30151]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-09T00:08:52.672 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:52 smithi046 ceph-mon[30151]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:08:52.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:52 smithi142 ceph-mon[34533]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-09T00:08:52.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:52 smithi142 ceph-mon[34533]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:08:52.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:52 smithi169 ceph-mon[34777]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-09T00:08:52.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:52 smithi169 ceph-mon[34777]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:08:54.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.224+0000 7f5b41ea8700 1 -- 172.21.15.46:0/3896961210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b3c107df0 msgr2=0x7f5b3c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:54.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.224+0000 7f5b41ea8700 1 --2- 172.21.15.46:0/3896961210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b3c107df0 0x7f5b3c108200 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b240099c0 tx=0x7f5b24009cd0 comp rx=0 tx=0).stop 2024-05-09T00:08:54.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.224+0000 7f5b41ea8700 1 -- 172.21.15.46:0/3896961210 shutdown_connections 2024-05-09T00:08:54.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.224+0000 7f5b41ea8700 1 --2- 172.21.15.46:0/3896961210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b3c108740 0x7f5b3c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:54.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.224+0000 7f5b41ea8700 1 --2- 172.21.15.46:0/3896961210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b3c1034c0 0x7f5b3c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:54.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.224+0000 7f5b41ea8700 1 --2- 172.21.15.46:0/3896961210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b3c107df0 0x7f5b3c108200 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:54.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.224+0000 7f5b41ea8700 1 -- 172.21.15.46:0/3896961210 >> 172.21.15.46:0/3896961210 conn(0x7f5b3c0fedf0 msgr2=0x7f5b3c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:54.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.225+0000 7f5b41ea8700 1 -- 172.21.15.46:0/3896961210 shutdown_connections 2024-05-09T00:08:54.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.225+0000 7f5b41ea8700 1 -- 172.21.15.46:0/3896961210 wait complete. 2024-05-09T00:08:54.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.225+0000 7f5b41ea8700 1 Processor -- start 2024-05-09T00:08:54.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b41ea8700 1 -- start start 2024-05-09T00:08:54.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b41ea8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b3c1034c0 0x7f5b3c0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b41ea8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b3c107df0 0x7f5b3c0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b41ea8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b3c108740 0x7f5b3c0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b41ea8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c113350 con 0x7f5b3c1034c0 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b41ea8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1131d0 con 0x7f5b3c108740 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b41ea8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1134d0 con 0x7f5b3c107df0 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b3c1034c0 0x7f5b3c0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b3c1034c0 0x7f5b3c0ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52602/0 (socket says 172.21.15.46:52602) 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b3b7fe700 1 -- 172.21.15.46:0/2855115396 learned_addr learned my addr 172.21.15.46:0/2855115396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b3affd700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b3c107df0 0x7f5b3c0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b3bfff700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b3c108740 0x7f5b3c0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b3b7fe700 1 -- 172.21.15.46:0/2855115396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b3c107df0 msgr2=0x7f5b3c0ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b3b7fe700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b3c107df0 0x7f5b3c0ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b3b7fe700 1 -- 172.21.15.46:0/2855115396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b3c108740 msgr2=0x7f5b3c0ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b3b7fe700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b3c108740 0x7f5b3c0ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.226+0000 7f5b3b7fe700 1 -- 172.21.15.46:0/2855115396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c1a91f0 con 0x7f5b3c1034c0 2024-05-09T00:08:54.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.227+0000 7f5b3b7fe700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b3c1034c0 0x7f5b3c0ff2d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00ada0 tx=0x7f5b2c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:54.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.227+0000 7f5b38ff9700 1 -- 172.21.15.46:0/2855115396 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c017020 con 0x7f5b3c1034c0 2024-05-09T00:08:54.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.227+0000 7f5b38ff9700 1 -- 172.21.15.46:0/2855115396 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b2c00ce30 con 0x7f5b3c1034c0 2024-05-09T00:08:54.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.227+0000 7f5b41ea8700 1 -- 172.21.15.46:0/2855115396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c1a94e0 con 0x7f5b3c1034c0 2024-05-09T00:08:54.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.227+0000 7f5b38ff9700 1 -- 172.21.15.46:0/2855115396 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0055a0 con 0x7f5b3c1034c0 2024-05-09T00:08:54.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.227+0000 7f5b41ea8700 1 -- 172.21.15.46:0/2855115396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c1a9a40 con 0x7f5b3c1034c0 2024-05-09T00:08:54.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.228+0000 7f5b41ea8700 1 -- 172.21.15.46:0/2855115396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b3c0512c0 con 0x7f5b3c1034c0 2024-05-09T00:08:54.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.230+0000 7f5b38ff9700 1 -- 172.21.15.46:0/2855115396 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5b2c004c40 con 0x7f5b3c1034c0 2024-05-09T00:08:54.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.230+0000 7f5b38ff9700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5b280b2990 0x7f5b280b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:54.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.231+0000 7f5b3affd700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5b280b2990 0x7f5b280b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:54.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.231+0000 7f5b38ff9700 1 -- 172.21.15.46:0/2855115396 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f5b2c0e1fa0 con 0x7f5b3c1034c0 2024-05-09T00:08:54.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.231+0000 7f5b3affd700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5b280b2990 0x7f5b280b4ea0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b24000c00 tx=0x7f5b2401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.234+0000 7f5b38ff9700 1 -- 172.21.15.46:0/2855115396 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b2c021020 con 0x7f5b3c1034c0 2024-05-09T00:08:54.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.483+0000 7f5b41ea8700 1 -- 172.21.15.46:0/2855115396 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5b3c063c00 con 0x7f5b280b2990 2024-05-09T00:08:54.479 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:54 smithi046 ceph-mon[30151]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-09T00:08:54.479 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:54 smithi046 ceph-mon[30151]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T00:08:54.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.512+0000 7f5b38ff9700 1 -- 172.21.15.46:0/2855115396 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5b3c063c00 con 0x7f5b280b2990 2024-05-09T00:08:54.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.515+0000 7f5b41ea8700 1 -- 172.21.15.46:0/2855115396 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5b280b2990 msgr2=0x7f5b280b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:54.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.515+0000 7f5b41ea8700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5b280b2990 0x7f5b280b4ea0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b24000c00 tx=0x7f5b2401a040 comp rx=0 tx=0).stop 2024-05-09T00:08:54.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.515+0000 7f5b41ea8700 1 -- 172.21.15.46:0/2855115396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b3c1034c0 msgr2=0x7f5b3c0ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:54.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.515+0000 7f5b41ea8700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b3c1034c0 0x7f5b3c0ff2d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00ada0 tx=0x7f5b2c00c330 comp rx=0 tx=0).stop 2024-05-09T00:08:54.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.515+0000 7f5b41ea8700 1 -- 172.21.15.46:0/2855115396 shutdown_connections 2024-05-09T00:08:54.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.515+0000 7f5b41ea8700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5b280b2990 0x7f5b280b4ea0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:54.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.515+0000 7f5b41ea8700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b3c108740 0x7f5b3c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:54.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.515+0000 7f5b41ea8700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b3c1034c0 0x7f5b3c0ff2d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:54.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.515+0000 7f5b41ea8700 1 --2- 172.21.15.46:0/2855115396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b3c107df0 0x7f5b3c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:54.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.515+0000 7f5b41ea8700 1 -- 172.21.15.46:0/2855115396 >> 172.21.15.46:0/2855115396 conn(0x7f5b3c0fedf0 msgr2=0x7f5b3c075dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:54.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.515+0000 7f5b41ea8700 1 -- 172.21.15.46:0/2855115396 shutdown_connections 2024-05-09T00:08:54.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:54.515+0000 7f5b41ea8700 1 -- 172.21.15.46:0/2855115396 wait complete. 2024-05-09T00:08:54.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:54 smithi142 ceph-mon[34533]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-09T00:08:54.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:54 smithi142 ceph-mon[34533]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T00:08:54.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:54 smithi169 ceph-mon[34777]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-09T00:08:54.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:54 smithi169 ceph-mon[34777]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T00:08:55.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-09T00:08:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:55.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:55 smithi046 ceph-mon[30151]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:08:55.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:55 smithi142 ceph-mon[34533]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:08:55.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:55 smithi169 ceph-mon[34777]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:08:56.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:56 smithi142 ceph-mon[34533]: pgmap v301: 97 pgs: 97 active+clean; 588 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-05-09T00:08:56.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:56 smithi046 ceph-mon[30151]: pgmap v301: 97 pgs: 97 active+clean; 588 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-05-09T00:08:56.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:56 smithi169 ceph-mon[34777]: pgmap v301: 97 pgs: 97 active+clean; 588 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-05-09T00:08:57.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.406+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/2432690388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4109870 msgr2=0x7f2eb41111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:57.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.406+0000 7f2eb9fb9700 1 --2- 172.21.15.46:0/2432690388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4109870 0x7f2eb41111d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00b0d0 tx=0x7f2eac00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:08:57.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.406+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/2432690388 shutdown_connections 2024-05-09T00:08:57.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.406+0000 7f2eb9fb9700 1 --2- 172.21.15.46:0/2432690388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4109870 0x7f2eb41111d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:57.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.406+0000 7f2eb9fb9700 1 --2- 172.21.15.46:0/2432690388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2eb41045f0 0x7f2eb4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:57.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.406+0000 7f2eb9fb9700 1 --2- 172.21.15.46:0/2432690388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eb4108f20 0x7f2eb4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.406+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/2432690388 >> 172.21.15.46:0/2432690388 conn(0x7f2eb4100040 msgr2=0x7f2eb4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.406+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/2432690388 shutdown_connections 2024-05-09T00:08:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.406+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/2432690388 wait complete. 2024-05-09T00:08:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb9fb9700 1 Processor -- start 2024-05-09T00:08:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb9fb9700 1 -- start start 2024-05-09T00:08:57.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb9fb9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eb41045f0 0x7f2eb419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:57.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb9fb9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2eb4108f20 0x7f2eb419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:57.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb9fb9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4109870 0x7f2eb419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:57.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb9fb9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb41144a0 con 0x7f2eb4109870 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb9fb9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4114320 con 0x7f2eb4108f20 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb9fb9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4114620 con 0x7f2eb41045f0 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4109870 0x7f2eb419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4109870 0x7f2eb419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53728/0 (socket says 172.21.15.46:53728) 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb3fff700 1 -- 172.21.15.46:0/1818974245 learned_addr learned my addr 172.21.15.46:0/1818974245 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb2ffd700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2eb4108f20 0x7f2eb419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.407+0000 7f2eb37fe700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eb41045f0 0x7f2eb419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.408+0000 7f2eb3fff700 1 -- 172.21.15.46:0/1818974245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eb41045f0 msgr2=0x7f2eb419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.408+0000 7f2eb3fff700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eb41045f0 0x7f2eb419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.408+0000 7f2eb3fff700 1 -- 172.21.15.46:0/1818974245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2eb4108f20 msgr2=0x7f2eb419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.408+0000 7f2eb3fff700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2eb4108f20 0x7f2eb419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.408+0000 7f2eb3fff700 1 -- 172.21.15.46:0/1818974245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb419fcc0 con 0x7f2eb4109870 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.408+0000 7f2eb3fff700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4109870 0x7f2eb419f620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0095c0 tx=0x7f2eac007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.408+0000 7f2eb0ff9700 1 -- 172.21.15.46:0/1818974245 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac01f070 con 0x7f2eb4109870 2024-05-09T00:08:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.408+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/1818974245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb419ff50 con 0x7f2eb4109870 2024-05-09T00:08:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.408+0000 7f2eb0ff9700 1 -- 172.21.15.46:0/1818974245 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2eac007e20 con 0x7f2eb4109870 2024-05-09T00:08:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.408+0000 7f2eb0ff9700 1 -- 172.21.15.46:0/1818974245 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac01b4a0 con 0x7f2eb4109870 2024-05-09T00:08:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.408+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/1818974245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb41a78d0 con 0x7f2eb4109870 2024-05-09T00:08:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.409+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/1818974245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb40512c0 con 0x7f2eb4109870 2024-05-09T00:08:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.411+0000 7f2eb0ff9700 1 -- 172.21.15.46:0/1818974245 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2eac022070 con 0x7f2eb4109870 2024-05-09T00:08:57.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.412+0000 7f2eb0ff9700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2e9c0b2910 0x7f2e9c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:08:57.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.412+0000 7f2eb37fe700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2e9c0b2910 0x7f2e9c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:08:57.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.412+0000 7f2eb0ff9700 1 -- 172.21.15.46:0/1818974245 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f2eac00f020 con 0x7f2eb4109870 2024-05-09T00:08:57.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.412+0000 7f2eb37fe700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2e9c0b2910 0x7f2e9c0b4e20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4005950 tx=0x7f2ea400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:08:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.416+0000 7f2eb0ff9700 1 -- 172.21.15.46:0/1818974245 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2eac029020 con 0x7f2eb4109870 2024-05-09T00:08:57.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.668+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/1818974245 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2eb419c150 con 0x7f2e9c0b2910 2024-05-09T00:08:57.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.673+0000 7f2eb0ff9700 1 -- 172.21.15.46:0/1818974245 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2eb419c150 con 0x7f2e9c0b2910 2024-05-09T00:08:57.669 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_0/2d881707-8971-4a5e-9fb4-bf361ecbc2a8 2024-05-09T00:08:57.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.675+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/1818974245 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2e9c0b2910 msgr2=0x7f2e9c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:57.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.675+0000 7f2eb9fb9700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2e9c0b2910 0x7f2e9c0b4e20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4005950 tx=0x7f2ea400b410 comp rx=0 tx=0).stop 2024-05-09T00:08:57.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.675+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/1818974245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4109870 msgr2=0x7f2eb419f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:08:57.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.675+0000 7f2eb9fb9700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4109870 0x7f2eb419f620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0095c0 tx=0x7f2eac007a20 comp rx=0 tx=0).stop 2024-05-09T00:08:57.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.676+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/1818974245 shutdown_connections 2024-05-09T00:08:57.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.676+0000 7f2eb9fb9700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2e9c0b2910 0x7f2e9c0b4e20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:57.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.676+0000 7f2eb9fb9700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4109870 0x7f2eb419f620 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:57.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.676+0000 7f2eb9fb9700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eb41045f0 0x7f2eb419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:57.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.676+0000 7f2eb9fb9700 1 --2- 172.21.15.46:0/1818974245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2eb4108f20 0x7f2eb419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:08:57.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.676+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/1818974245 >> 172.21.15.46:0/1818974245 conn(0x7f2eb4100040 msgr2=0x7f2eb4108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:08:57.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.676+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/1818974245 shutdown_connections 2024-05-09T00:08:57.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:08:57.676+0000 7f2eb9fb9700 1 -- 172.21.15.46:0/1818974245 wait complete. 2024-05-09T00:08:58.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-09T00:08:58.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:58 smithi046 ceph-mon[30151]: pgmap v302: 97 pgs: 97 active+clean; 588 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-09T00:08:58.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:08:58 smithi046 ceph-mon[30151]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:08:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:08:58.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:58 smithi142 ceph-mon[34533]: pgmap v302: 97 pgs: 97 active+clean; 588 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-09T00:08:58.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:08:58 smithi142 ceph-mon[34533]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:08:58.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:58 smithi169 ceph-mon[34777]: pgmap v302: 97 pgs: 97 active+clean; 588 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-09T00:08:58.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:08:58 smithi169 ceph-mon[34777]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:09:00.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.308+0000 7fd00031d700 1 -- 172.21.15.46:0/92137435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcff81034c0 msgr2=0x7fcff81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:00.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.308+0000 7fd00031d700 1 --2- 172.21.15.46:0/92137435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcff81034c0 0x7fcff81038d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcff400b0d0 tx=0x7fcff400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:09:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.308+0000 7fd00031d700 1 -- 172.21.15.46:0/92137435 shutdown_connections 2024-05-09T00:09:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.308+0000 7fd00031d700 1 --2- 172.21.15.46:0/92137435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff8108740 0x7fcff81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.308+0000 7fd00031d700 1 --2- 172.21.15.46:0/92137435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcff81034c0 0x7fcff81038d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.308+0000 7fd00031d700 1 --2- 172.21.15.46:0/92137435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcff8107df0 0x7fcff8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.308+0000 7fd00031d700 1 -- 172.21.15.46:0/92137435 >> 172.21.15.46:0/92137435 conn(0x7fcff80fedc0 msgr2=0x7fcff81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.308+0000 7fd00031d700 1 -- 172.21.15.46:0/92137435 shutdown_connections 2024-05-09T00:09:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.308+0000 7fd00031d700 1 -- 172.21.15.46:0/92137435 wait complete. 2024-05-09T00:09:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fd00031d700 1 Processor -- start 2024-05-09T00:09:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fd00031d700 1 -- start start 2024-05-09T00:09:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fd00031d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcff81034c0 0x7fcff819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fd00031d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcff8107df0 0x7fcff819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fd00031d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff8108740 0x7fcff819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fd00031d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81133f0 con 0x7fcff81034c0 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fd00031d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8113270 con 0x7fcff8108740 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fd00031d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8113570 con 0x7fcff8107df0 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fcffe0b9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcff81034c0 0x7fcff819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fcffe0b9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcff81034c0 0x7fcff819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53752/0 (socket says 172.21.15.46:53752) 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fcffe0b9700 1 -- 172.21.15.46:0/374577147 learned_addr learned my addr 172.21.15.46:0/374577147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fcffd8b8700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcff8107df0 0x7fcff819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fcffe0b9700 1 -- 172.21.15.46:0/374577147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcff8107df0 msgr2=0x7fcff819b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fcffe0b9700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcff8107df0 0x7fcff819b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fcffe0b9700 1 -- 172.21.15.46:0/374577147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff8108740 msgr2=0x7fcff819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fcffe8ba700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff8108740 0x7fcff819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fcffe0b9700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff8108740 0x7fcff819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.309+0000 7fcffe0b9700 1 -- 172.21.15.46:0/374577147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff819fcf0 con 0x7fcff81034c0 2024-05-09T00:09:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.310+0000 7fcffe8ba700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff8108740 0x7fcff819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:09:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.310+0000 7fcffe0b9700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcff81034c0 0x7fcff819aed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcff4007a20 tx=0x7fcff4007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.310+0000 7fcfef7fe700 1 -- 172.21.15.46:0/374577147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff401f070 con 0x7fcff81034c0 2024-05-09T00:09:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.310+0000 7fcfef7fe700 1 -- 172.21.15.46:0/374577147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff4003c00 con 0x7fcff81034c0 2024-05-09T00:09:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.310+0000 7fd00031d700 1 -- 172.21.15.46:0/374577147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff819ff80 con 0x7fcff81034c0 2024-05-09T00:09:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.310+0000 7fcfef7fe700 1 -- 172.21.15.46:0/374577147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff401b730 con 0x7fcff81034c0 2024-05-09T00:09:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.310+0000 7fd00031d700 1 -- 172.21.15.46:0/374577147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff81a0460 con 0x7fcff81034c0 2024-05-09T00:09:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.311+0000 7fd00031d700 1 -- 172.21.15.46:0/374577147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff80512c0 con 0x7fcff81034c0 2024-05-09T00:09:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.313+0000 7fcfef7fe700 1 -- 172.21.15.46:0/374577147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcff4022070 con 0x7fcff81034c0 2024-05-09T00:09:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.313+0000 7fcfef7fe700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcfe40b2910 0x7fcfe40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.314+0000 7fcffd8b8700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcfe40b2910 0x7fcfe40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.314+0000 7fcfef7fe700 1 -- 172.21.15.46:0/374577147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fcff400f020 con 0x7fcff81034c0 2024-05-09T00:09:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.314+0000 7fcffd8b8700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcfe40b2910 0x7fcfe40b4e20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8005ea0 tx=0x7fcfe8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:00.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.317+0000 7fcfef7fe700 1 -- 172.21.15.46:0/374577147 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcff4029020 con 0x7fcff81034c0 2024-05-09T00:09:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.567+0000 7fd00031d700 1 -- 172.21.15.46:0/374577147 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcff819c230 con 0x7fcfe40b2910 2024-05-09T00:09:00.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.595+0000 7fcfef7fe700 1 -- 172.21.15.46:0/374577147 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcff819c230 con 0x7fcfe40b2910 2024-05-09T00:09:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.597+0000 7fd00031d700 1 -- 172.21.15.46:0/374577147 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcfe40b2910 msgr2=0x7fcfe40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.597+0000 7fd00031d700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcfe40b2910 0x7fcfe40b4e20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8005ea0 tx=0x7fcfe8005e30 comp rx=0 tx=0).stop 2024-05-09T00:09:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.597+0000 7fd00031d700 1 -- 172.21.15.46:0/374577147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcff81034c0 msgr2=0x7fcff819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.597+0000 7fd00031d700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcff81034c0 0x7fcff819aed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcff4007a20 tx=0x7fcff4007b00 comp rx=0 tx=0).stop 2024-05-09T00:09:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.597+0000 7fd00031d700 1 -- 172.21.15.46:0/374577147 shutdown_connections 2024-05-09T00:09:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.597+0000 7fd00031d700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcfe40b2910 0x7fcfe40b4e20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.597+0000 7fd00031d700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcff8108740 0x7fcff819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.597+0000 7fd00031d700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcff81034c0 0x7fcff819aed0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.597+0000 7fd00031d700 1 --2- 172.21.15.46:0/374577147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcff8107df0 0x7fcff819b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.597+0000 7fd00031d700 1 -- 172.21.15.46:0/374577147 >> 172.21.15.46:0/374577147 conn(0x7fcff80fedc0 msgr2=0x7fcff806bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.598+0000 7fd00031d700 1 -- 172.21.15.46:0/374577147 shutdown_connections 2024-05-09T00:09:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:00.598+0000 7fd00031d700 1 -- 172.21.15.46:0/374577147 wait complete. 2024-05-09T00:09:00.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:00 smithi142 ceph-mon[34533]: pgmap v303: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-05-09T00:09:00.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:00 smithi046 ceph-mon[30151]: pgmap v303: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-05-09T00:09:00.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:00 smithi169 ceph-mon[34777]: pgmap v303: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-05-09T00:09:01.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-09T00:09:01.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:01 smithi046 ceph-mon[30151]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:09:01.565 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:01.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:01 smithi142 ceph-mon[34533]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:09:01.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:01 smithi169 ceph-mon[34777]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:09:02.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:02 smithi142 ceph-mon[34533]: pgmap v304: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-05-09T00:09:02.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:02 smithi046 ceph-mon[30151]: pgmap v304: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-05-09T00:09:02.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:02 smithi169 ceph-mon[34777]: pgmap v304: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-05-09T00:09:03.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.255+0000 7fb0b1710700 1 -- 172.21.15.46:0/3633021110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ac109360 msgr2=0x7fb0ac109770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:03.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.255+0000 7fb0b1710700 1 --2- 172.21.15.46:0/3633021110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ac109360 0x7fb0ac109770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00b0a0 tx=0x7fb09c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:09:03.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.256+0000 7fb0b1710700 1 -- 172.21.15.46:0/3633021110 shutdown_connections 2024-05-09T00:09:03.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.256+0000 7fb0b1710700 1 --2- 172.21.15.46:0/3633021110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0ac104290 0x7fb0ac110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:03.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.256+0000 7fb0b1710700 1 --2- 172.21.15.46:0/3633021110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ac109360 0x7fb0ac109770 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:03.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.256+0000 7fb0b1710700 1 --2- 172.21.15.46:0/3633021110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0ac103940 0x7fb0ac103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:03.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.256+0000 7fb0b1710700 1 -- 172.21.15.46:0/3633021110 >> 172.21.15.46:0/3633021110 conn(0x7fb0ac0fe520 msgr2=0x7fb0ac100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:03.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.256+0000 7fb0b1710700 1 -- 172.21.15.46:0/3633021110 shutdown_connections 2024-05-09T00:09:03.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.256+0000 7fb0b1710700 1 -- 172.21.15.46:0/3633021110 wait complete. 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.256+0000 7fb0b1710700 1 Processor -- start 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.257+0000 7fb0b1710700 1 -- start start 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.257+0000 7fb0b1710700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ac103940 0x7fb0ac19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.257+0000 7fb0b1710700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0ac104290 0x7fb0ac19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.257+0000 7fb0b1710700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0ac109360 0x7fb0ac19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.257+0000 7fb0b1710700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1145c0 con 0x7fb0ac103940 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.257+0000 7fb0b1710700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac114440 con 0x7fb0ac104290 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.257+0000 7fb0b1710700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac114740 con 0x7fb0ac109360 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.257+0000 7fb0aaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ac103940 0x7fb0ac19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.257+0000 7fb0aaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ac103940 0x7fb0ac19af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53770/0 (socket says 172.21.15.46:53770) 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.257+0000 7fb0aaffd700 1 -- 172.21.15.46:0/4095509403 learned_addr learned my addr 172.21.15.46:0/4095509403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.257+0000 7fb0ab7fe700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0ac109360 0x7fb0ac19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:03.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.257+0000 7fb0aa7fc700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0ac104290 0x7fb0ac19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:03.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.258+0000 7fb0aaffd700 1 -- 172.21.15.46:0/4095509403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0ac109360 msgr2=0x7fb0ac19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:03.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.258+0000 7fb0aaffd700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0ac109360 0x7fb0ac19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:03.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.258+0000 7fb0aaffd700 1 -- 172.21.15.46:0/4095509403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0ac104290 msgr2=0x7fb0ac19b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:03.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.258+0000 7fb0aaffd700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0ac104290 0x7fb0ac19b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:03.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.258+0000 7fb0aaffd700 1 -- 172.21.15.46:0/4095509403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac19fe40 con 0x7fb0ac103940 2024-05-09T00:09:03.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.258+0000 7fb0aaffd700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ac103940 0x7fb0ac19af70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0062a0 tx=0x7fb09c004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:03.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.258+0000 7fb093fff700 1 -- 172.21.15.46:0/4095509403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c021070 con 0x7fb0ac103940 2024-05-09T00:09:03.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.258+0000 7fb093fff700 1 -- 172.21.15.46:0/4095509403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb09c007c40 con 0x7fb0ac103940 2024-05-09T00:09:03.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.258+0000 7fb0b1710700 1 -- 172.21.15.46:0/4095509403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac1a00d0 con 0x7fb0ac103940 2024-05-09T00:09:03.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.258+0000 7fb093fff700 1 -- 172.21.15.46:0/4095509403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0052a0 con 0x7fb0ac103940 2024-05-09T00:09:03.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.258+0000 7fb0b1710700 1 -- 172.21.15.46:0/4095509403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac1a7a50 con 0x7fb0ac103940 2024-05-09T00:09:03.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.259+0000 7fb0b1710700 1 -- 172.21.15.46:0/4095509403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0ac104b60 con 0x7fb0ac103940 2024-05-09T00:09:03.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.261+0000 7fb093fff700 1 -- 172.21.15.46:0/4095509403 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb09c024070 con 0x7fb0ac103940 2024-05-09T00:09:03.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.262+0000 7fb093fff700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb0940b2910 0x7fb0940b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:03.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.262+0000 7fb0aa7fc700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb0940b2910 0x7fb0940b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:03.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.262+0000 7fb093fff700 1 -- 172.21.15.46:0/4095509403 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fb09c00f020 con 0x7fb0ac103940 2024-05-09T00:09:03.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.262+0000 7fb0aa7fc700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb0940b2910 0x7fb0940b4e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac108830 tx=0x7fb0a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:03.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.265+0000 7fb093fff700 1 -- 172.21.15.46:0/4095509403 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb09c02b020 con 0x7fb0ac103940 2024-05-09T00:09:03.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.520+0000 7fb0b1710700 1 -- 172.21.15.46:0/4095509403 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb0ac19c2d0 con 0x7fb0940b2910 2024-05-09T00:09:03.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.525+0000 7fb093fff700 1 -- 172.21.15.46:0/4095509403 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb0ac19c2d0 con 0x7fb0940b2910 2024-05-09T00:09:03.521 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_1/d5361cba-819b-4ad7-a343-3956db6678cb 2024-05-09T00:09:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.528+0000 7fb0b1710700 1 -- 172.21.15.46:0/4095509403 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb0940b2910 msgr2=0x7fb0940b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.528+0000 7fb0b1710700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb0940b2910 0x7fb0940b4e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac108830 tx=0x7fb0a000b410 comp rx=0 tx=0).stop 2024-05-09T00:09:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.528+0000 7fb0b1710700 1 -- 172.21.15.46:0/4095509403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ac103940 msgr2=0x7fb0ac19af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.528+0000 7fb0b1710700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ac103940 0x7fb0ac19af70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0062a0 tx=0x7fb09c004d50 comp rx=0 tx=0).stop 2024-05-09T00:09:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.528+0000 7fb0b1710700 1 -- 172.21.15.46:0/4095509403 shutdown_connections 2024-05-09T00:09:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.528+0000 7fb0b1710700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb0940b2910 0x7fb0940b4e20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.528+0000 7fb0b1710700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0ac109360 0x7fb0ac19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.528+0000 7fb0b1710700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ac103940 0x7fb0ac19af70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.528+0000 7fb0b1710700 1 --2- 172.21.15.46:0/4095509403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0ac104290 0x7fb0ac19b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.528+0000 7fb0b1710700 1 -- 172.21.15.46:0/4095509403 >> 172.21.15.46:0/4095509403 conn(0x7fb0ac0fe520 msgr2=0x7fb0ac100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.528+0000 7fb0b1710700 1 -- 172.21.15.46:0/4095509403 shutdown_connections 2024-05-09T00:09:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:03.528+0000 7fb0b1710700 1 -- 172.21.15.46:0/4095509403 wait complete. 2024-05-09T00:09:04.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:09:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:04.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:04 smithi046 ceph-mon[30151]: pgmap v305: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2024-05-09T00:09:04.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:04 smithi046 ceph-mon[30151]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:09:04.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:04 smithi169 ceph-mon[34777]: pgmap v305: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2024-05-09T00:09:04.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:04 smithi169 ceph-mon[34777]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:09:04.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:04 smithi142 ceph-mon[34533]: pgmap v305: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2024-05-09T00:09:04.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:04 smithi142 ceph-mon[34533]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:09:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.077+0000 7f9d562ff700 1 -- 172.21.15.46:0/2959282030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d50108740 msgr2=0x7f9d501100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.077+0000 7f9d562ff700 1 --2- 172.21.15.46:0/2959282030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d50108740 0x7f9d501100a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800b0d0 tx=0x7f9d4800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.077+0000 7f9d562ff700 1 -- 172.21.15.46:0/2959282030 shutdown_connections 2024-05-09T00:09:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.077+0000 7f9d562ff700 1 --2- 172.21.15.46:0/2959282030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d50108740 0x7f9d501100a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.077+0000 7f9d562ff700 1 --2- 172.21.15.46:0/2959282030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d501034c0 0x7f9d501038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.077+0000 7f9d562ff700 1 --2- 172.21.15.46:0/2959282030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d50107df0 0x7f9d50108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.077+0000 7f9d562ff700 1 -- 172.21.15.46:0/2959282030 >> 172.21.15.46:0/2959282030 conn(0x7f9d500fedc0 msgr2=0x7f9d501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.077+0000 7f9d562ff700 1 -- 172.21.15.46:0/2959282030 shutdown_connections 2024-05-09T00:09:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.077+0000 7f9d562ff700 1 -- 172.21.15.46:0/2959282030 wait complete. 2024-05-09T00:09:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.077+0000 7f9d562ff700 1 Processor -- start 2024-05-09T00:09:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.077+0000 7f9d562ff700 1 -- start start 2024-05-09T00:09:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d562ff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d501034c0 0x7f9d5019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d562ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d50107df0 0x7f9d5019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d562ff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d50108740 0x7f9d5019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d562ff700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9d501144e0 con 0x7f9d50108740 2024-05-09T00:09:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d562ff700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9d50114360 con 0x7f9d501034c0 2024-05-09T00:09:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d562ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9d50114660 con 0x7f9d50107df0 2024-05-09T00:09:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d5489c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d50108740 0x7f9d5019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d5489c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d50108740 0x7f9d5019f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53792/0 (socket says 172.21.15.46:53792) 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d5489c700 1 -- 172.21.15.46:0/1536539867 learned_addr learned my addr 172.21.15.46:0/1536539867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d47fff700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d50107df0 0x7f9d5019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d4ffff700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d501034c0 0x7f9d5019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d5489c700 1 -- 172.21.15.46:0/1536539867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d50107df0 msgr2=0x7f9d5019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d5489c700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d50107df0 0x7f9d5019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d5489c700 1 -- 172.21.15.46:0/1536539867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d501034c0 msgr2=0x7f9d5019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d5489c700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d501034c0 0x7f9d5019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d5489c700 1 -- 172.21.15.46:0/1536539867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5019fce0 con 0x7f9d50108740 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.078+0000 7f9d5489c700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d50108740 0x7f9d5019f640 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9d480095c0 tx=0x7f9d48007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.079+0000 7f9d4dffb700 1 -- 172.21.15.46:0/1536539867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4801f070 con 0x7f9d50108740 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.079+0000 7f9d4dffb700 1 -- 172.21.15.46:0/1536539867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d48003c00 con 0x7f9d50108740 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.079+0000 7f9d562ff700 1 -- 172.21.15.46:0/1536539867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5019ff70 con 0x7f9d50108740 2024-05-09T00:09:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.079+0000 7f9d4dffb700 1 -- 172.21.15.46:0/1536539867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4801b6a0 con 0x7f9d50108740 2024-05-09T00:09:06.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.079+0000 7f9d562ff700 1 -- 172.21.15.46:0/1536539867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d501a78f0 con 0x7f9d50108740 2024-05-09T00:09:06.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.080+0000 7f9d562ff700 1 -- 172.21.15.46:0/1536539867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d50104970 con 0x7f9d50108740 2024-05-09T00:09:06.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.081+0000 7f9d4dffb700 1 -- 172.21.15.46:0/1536539867 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9d48022070 con 0x7f9d50108740 2024-05-09T00:09:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.082+0000 7f9d4dffb700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9d3c0b2990 0x7f9d3c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.082+0000 7f9d4ffff700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9d3c0b2990 0x7f9d3c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.082+0000 7f9d4dffb700 1 -- 172.21.15.46:0/1536539867 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f9d4800f020 con 0x7f9d50108740 2024-05-09T00:09:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.083+0000 7f9d4ffff700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9d3c0b2990 0x7f9d3c0b4ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9d38005950 tx=0x7f9d380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:06.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.087+0000 7f9d4dffb700 1 -- 172.21.15.46:0/1536539867 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9d48029020 con 0x7f9d50108740 2024-05-09T00:09:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.348+0000 7f9d562ff700 1 -- 172.21.15.46:0/1536539867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d5019c170 con 0x7f9d50108740 2024-05-09T00:09:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.349+0000 7f9d4dffb700 1 -- 172.21.15.46:0/1536539867 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11879 (secure 0 0 0) 0x7f9d480ab200 con 0x7f9d50108740 2024-05-09T00:09:06.345 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:09:06.345 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:08:47.423159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24497,24499,24487]},"id":1}]} 2024-05-09T00:09:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.351+0000 7f9d562ff700 1 -- 172.21.15.46:0/1536539867 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9d3c0b2990 msgr2=0x7f9d3c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.351+0000 7f9d562ff700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9d3c0b2990 0x7f9d3c0b4ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9d38005950 tx=0x7f9d380058e0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.351+0000 7f9d562ff700 1 -- 172.21.15.46:0/1536539867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d50108740 msgr2=0x7f9d5019f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.351+0000 7f9d562ff700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d50108740 0x7f9d5019f640 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9d480095c0 tx=0x7f9d48007a20 comp rx=0 tx=0).stop 2024-05-09T00:09:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.351+0000 7f9d562ff700 1 -- 172.21.15.46:0/1536539867 shutdown_connections 2024-05-09T00:09:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.351+0000 7f9d562ff700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9d3c0b2990 0x7f9d3c0b4ea0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.351+0000 7f9d562ff700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d50108740 0x7f9d5019f640 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.351+0000 7f9d562ff700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d501034c0 0x7f9d5019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.352+0000 7f9d562ff700 1 --2- 172.21.15.46:0/1536539867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d50107df0 0x7f9d5019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.352+0000 7f9d562ff700 1 -- 172.21.15.46:0/1536539867 >> 172.21.15.46:0/1536539867 conn(0x7f9d500fedc0 msgr2=0x7f9d50107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.352+0000 7f9d562ff700 1 -- 172.21.15.46:0/1536539867 shutdown_connections 2024-05-09T00:09:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:06.352+0000 7f9d562ff700 1 -- 172.21.15.46:0/1536539867 wait complete. 2024-05-09T00:09:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T00:09:06.717 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:06 smithi046 ceph-mon[30151]: pgmap v306: 97 pgs: 97 active+clean; 630 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-05-09T00:09:06.717 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:06 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1536539867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:09:06.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:06 smithi169 ceph-mon[34777]: pgmap v306: 97 pgs: 97 active+clean; 630 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-05-09T00:09:06.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:06 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1536539867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:09:06.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:06 smithi142 ceph-mon[34533]: pgmap v306: 97 pgs: 97 active+clean; 630 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-05-09T00:09:06.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:06 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1536539867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:09:07.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:09:07.319 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:08.805 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:08 smithi046 ceph-mon[30151]: pgmap v307: 97 pgs: 97 active+clean; 630 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-09T00:09:08.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:08 smithi169 ceph-mon[34777]: pgmap v307: 97 pgs: 97 active+clean; 630 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-09T00:09:08.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:08 smithi142 ceph-mon[34533]: pgmap v307: 97 pgs: 97 active+clean; 630 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-09T00:09:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.094+0000 7f2a52bd1700 1 -- 172.21.15.46:0/4046084929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a4c108760 msgr2=0x7f2a4c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.094+0000 7f2a52bd1700 1 --2- 172.21.15.46:0/4046084929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a4c108760 0x7f2a4c1100c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400b0d0 tx=0x7f2a4400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:09:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.094+0000 7f2a52bd1700 1 -- 172.21.15.46:0/4046084929 shutdown_connections 2024-05-09T00:09:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.094+0000 7f2a52bd1700 1 --2- 172.21.15.46:0/4046084929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a4c108760 0x7f2a4c1100c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.094+0000 7f2a52bd1700 1 --2- 172.21.15.46:0/4046084929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a4c1034e0 0x7f2a4c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.094+0000 7f2a52bd1700 1 --2- 172.21.15.46:0/4046084929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a4c107e10 0x7f2a4c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.094+0000 7f2a52bd1700 1 -- 172.21.15.46:0/4046084929 >> 172.21.15.46:0/4046084929 conn(0x7f2a4c0fef50 msgr2=0x7f2a4c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.095+0000 7f2a52bd1700 1 -- 172.21.15.46:0/4046084929 shutdown_connections 2024-05-09T00:09:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.095+0000 7f2a52bd1700 1 -- 172.21.15.46:0/4046084929 wait complete. 2024-05-09T00:09:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.095+0000 7f2a52bd1700 1 Processor -- start 2024-05-09T00:09:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.095+0000 7f2a52bd1700 1 -- start start 2024-05-09T00:09:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a52bd1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a4c1034e0 0x7f2a4c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a52bd1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a4c107e10 0x7f2a4c19c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a52bd1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a4c108760 0x7f2a4c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a52bd1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c113390 con 0x7f2a4c108760 2024-05-09T00:09:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a52bd1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c113210 con 0x7f2a4c107e10 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a52bd1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c113510 con 0x7f2a4c1034e0 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a5116e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a4c108760 0x7f2a4c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a5116e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a4c108760 0x7f2a4c19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55560/0 (socket says 172.21.15.46:55560) 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a5116e700 1 -- 172.21.15.46:0/774534135 learned_addr learned my addr 172.21.15.46:0/774534135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a4bfff700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a4c107e10 0x7f2a4c19c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a5096d700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a4c1034e0 0x7f2a4c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a5116e700 1 -- 172.21.15.46:0/774534135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a4c1034e0 msgr2=0x7f2a4c19bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a5116e700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a4c1034e0 0x7f2a4c19bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a5116e700 1 -- 172.21.15.46:0/774534135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a4c107e10 msgr2=0x7f2a4c19c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a5116e700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a4c107e10 0x7f2a4c19c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a5116e700 1 -- 172.21.15.46:0/774534135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4c1a6f40 con 0x7f2a4c108760 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a5116e700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a4c108760 0x7f2a4c19c690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2a440095c0 tx=0x7f2a44007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.096+0000 7f2a49ffb700 1 -- 172.21.15.46:0/774534135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4401f070 con 0x7f2a4c108760 2024-05-09T00:09:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.097+0000 7f2a49ffb700 1 -- 172.21.15.46:0/774534135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a44003c00 con 0x7f2a4c108760 2024-05-09T00:09:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.097+0000 7f2a52bd1700 1 -- 172.21.15.46:0/774534135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4c1a71d0 con 0x7f2a4c108760 2024-05-09T00:09:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.097+0000 7f2a49ffb700 1 -- 172.21.15.46:0/774534135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4401b6a0 con 0x7f2a4c108760 2024-05-09T00:09:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.097+0000 7f2a52bd1700 1 -- 172.21.15.46:0/774534135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4c1a7620 con 0x7f2a4c108760 2024-05-09T00:09:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.098+0000 7f2a52bd1700 1 -- 172.21.15.46:0/774534135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a4c0512c0 con 0x7f2a4c108760 2024-05-09T00:09:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.100+0000 7f2a49ffb700 1 -- 172.21.15.46:0/774534135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2a44022070 con 0x7f2a4c108760 2024-05-09T00:09:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.100+0000 7f2a49ffb700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2a340b2910 0x7f2a340b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.100+0000 7f2a5096d700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2a340b2910 0x7f2a340b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.101+0000 7f2a49ffb700 1 -- 172.21.15.46:0/774534135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f2a4400f020 con 0x7f2a4c108760 2024-05-09T00:09:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.101+0000 7f2a5096d700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2a340b2910 0x7f2a340b4e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c005950 tx=0x7f2a3c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:09.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.104+0000 7f2a49ffb700 1 -- 172.21.15.46:0/774534135 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a44029020 con 0x7f2a4c108760 2024-05-09T00:09:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.328+0000 7f2a52bd1700 1 -- 172.21.15.46:0/774534135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a4c02e7b0 con 0x7f2a4c108760 2024-05-09T00:09:09.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.330+0000 7f2a49ffb700 1 -- 172.21.15.46:0/774534135 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22215 (secure 0 0 0) 0x7f2a440ab7b0 con 0x7f2a4c108760 2024-05-09T00:09:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:09:09.326 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","created":"2024-05-09T00:00:14.356524+0000","modified":"2024-05-09T00:08:28.009855+0000","last_up_change":"2024-05-09T00:07:23.332796+0000","last_in_change":"2024-05-09T00:07:03.238539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:03:33.278910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:08:21.014416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:08:24.922968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"740d6388-5c1f-45eb-93b0-7583bfa5a225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6803","nonce":3727342125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6805","nonce":3727342125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6809","nonce":3727342125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6807","nonce":3727342125}]},"public_addr":"172.21.15.46:6803/3727342125","cluster_addr":"172.21.15.46:6805/3727342125","heartbeat_back_addr":"172.21.15.46:6809/3727342125","heartbeat_front_addr":"172.21.15.46:6807/3727342125","state":["exists","up"]},{"osd":1,"uuid":"85fbf3ae-0be8-4504-84d6-66fae0d2169d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6801","nonce":3156042469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6803","nonce":3156042469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6807","nonce":3156042469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6805","nonce":3156042469}]},"public_addr":"172.21.15.142:6801/3156042469","cluster_addr":"172.21.15.142:6803/3156042469","heartbeat_back_addr":"172.21.15.142:6807/3156042469","heartbeat_front_addr":"172.21.15.142:6805/3156042469","state":["exists","up"]},{"osd":2,"uuid":"05c070e7-83ad-4949-9834-4ce51ac0a2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6801","nonce":2680673064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6803","nonce":2680673064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6807","nonce":2680673064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6805","nonce":2680673064}]},"public_addr":"172.21.15.169:6801/2680673064","cluster_addr":"172.21.15.169:6803/2680673064","heartbeat_back_addr":"172.21.15.169:6807/2680673064","heartbeat_front_addr":"172.21.15.169:6805/2680673064","state":["exists","up"]},{"osd":3,"uuid":"09d7bbe1-58b6-4407-bef2-920f179dfb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6811","nonce":1158536261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6813","nonce":1158536261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6817","nonce":1158536261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6815","nonce":1158536261}]},"public_addr":"172.21.15.46:6811/1158536261","cluster_addr":"172.21.15.46:6813/1158536261","heartbeat_back_addr":"172.21.15.46:6817/1158536261","heartbeat_front_addr":"172.21.15.46:6815/1158536261","state":["exists","up"]},{"osd":4,"uuid":"66c9644b-6b0b-4fe7-aa63-b9d87bff7153","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6809","nonce":2299616865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6811","nonce":2299616865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6815","nonce":2299616865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6813","nonce":2299616865}]},"public_addr":"172.21.15.142:6809/2299616865","cluster_addr":"172.21.15.142:6811/2299616865","heartbeat_back_addr":"172.21.15.142:6815/2299616865","heartbeat_front_addr":"172.21.15.142:6813/2299616865","state":["exists","up"]},{"osd":5,"uuid":"96a72e01-1c5f-418e-bb9b-b790d9b3059b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6809","nonce":445044713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6811","nonce":445044713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6815","nonce":445044713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6813","nonce":445044713}]},"public_addr":"172.21.15.169:6809/445044713","cluster_addr":"172.21.15.169:6811/445044713","heartbeat_back_addr":"172.21.15.169:6815/445044713","heartbeat_front_addr":"172.21.15.169:6813/445044713","state":["exists","up"]},{"osd":6,"uuid":"6b459542-0e56-4369-b542-e36c46a18f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6819","nonce":1394035482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6821","nonce":1394035482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6825","nonce":1394035482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6823","nonce":1394035482}]},"public_addr":"172.21.15.46:6819/1394035482","cluster_addr":"172.21.15.46:6821/1394035482","heartbeat_back_addr":"172.21.15.46:6825/1394035482","heartbeat_front_addr":"172.21.15.46:6823/1394035482","state":["exists","up"]},{"osd":7,"uuid":"786b4517-4cbd-4fa3-b5a0-39cccf839893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6817","nonce":596805607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6819","nonce":596805607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6823","nonce":596805607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6821","nonce":596805607}]},"public_addr":"172.21.15.142:6817/596805607","cluster_addr":"172.21.15.142:6819/596805607","heartbeat_back_addr":"172.21.15.142:6823/596805607","heartbeat_front_addr":"172.21.15.142:6821/596805607","state":["exists","up"]},{"osd":8,"uuid":"b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6817","nonce":2650652376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6819","nonce":2650652376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6823","nonce":2650652376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6821","nonce":2650652376}]},"public_addr":"172.21.15.169:6817/2650652376","cluster_addr":"172.21.15.169:6819/2650652376","heartbeat_back_addr":"172.21.15.169:6823/2650652376","heartbeat_front_addr":"172.21.15.169:6821/2650652376","state":["exists","up"]},{"osd":9,"uuid":"32df0dab-5875-4beb-b733-60a915c94a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6827","nonce":374861248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6829","nonce":374861248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6833","nonce":374861248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6831","nonce":374861248}]},"public_addr":"172.21.15.46:6827/374861248","cluster_addr":"172.21.15.46:6829/374861248","heartbeat_back_addr":"172.21.15.46:6833/374861248","heartbeat_front_addr":"172.21.15.46:6831/374861248","state":["exists","up"]},{"osd":10,"uuid":"380a4f47-e528-4258-8836-32eac0e6e630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6825","nonce":1364891982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6827","nonce":1364891982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6831","nonce":1364891982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6829","nonce":1364891982}]},"public_addr":"172.21.15.142:6825/1364891982","cluster_addr":"172.21.15.142:6827/1364891982","heartbeat_back_addr":"172.21.15.142:6831/1364891982","heartbeat_front_addr":"172.21.15.142:6829/1364891982","state":["exists","up"]},{"osd":11,"uuid":"56b04b15-6fcd-497a-aa66-19da9624b656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6825","nonce":4262478976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6827","nonce":4262478976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6831","nonce":4262478976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6829","nonce":4262478976}]},"public_addr":"172.21.15.169:6825/4262478976","cluster_addr":"172.21.15.169:6827/4262478976","heartbeat_back_addr":"172.21.15.169:6831/4262478976","heartbeat_front_addr":"172.21.15.169:6829/4262478976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:02:42.159298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:06.053328+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:29.587137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:54.573782+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:20.276073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:45.320565+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:10.985522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:36.806137+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:02.266736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:29.788773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:55.827639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:07:21.762382+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/1072138393":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2786891364":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/7723180":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/626888490":"2024-05-10T00:01:09.191239+0000","172.21.15.46:6800/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/4236543921":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2168313126":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6800/953818590":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6801/953818590":"2024-05-10T00:00:41.833323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:09:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.332+0000 7f2a52bd1700 1 -- 172.21.15.46:0/774534135 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2a340b2910 msgr2=0x7f2a340b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.332+0000 7f2a52bd1700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2a340b2910 0x7f2a340b4e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c005950 tx=0x7f2a3c0058e0 comp rx=0 tx=0).stop 2024-05-09T00:09:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.332+0000 7f2a52bd1700 1 -- 172.21.15.46:0/774534135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a4c108760 msgr2=0x7f2a4c19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.332+0000 7f2a52bd1700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a4c108760 0x7f2a4c19c690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2a440095c0 tx=0x7f2a44007ad0 comp rx=0 tx=0).stop 2024-05-09T00:09:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.332+0000 7f2a52bd1700 1 -- 172.21.15.46:0/774534135 shutdown_connections 2024-05-09T00:09:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.332+0000 7f2a52bd1700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2a340b2910 0x7f2a340b4e20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.332+0000 7f2a52bd1700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a4c108760 0x7f2a4c19c690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.332+0000 7f2a52bd1700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a4c1034e0 0x7f2a4c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.332+0000 7f2a52bd1700 1 --2- 172.21.15.46:0/774534135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a4c107e10 0x7f2a4c19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.332+0000 7f2a52bd1700 1 -- 172.21.15.46:0/774534135 >> 172.21.15.46:0/774534135 conn(0x7f2a4c0fef50 msgr2=0x7f2a4c1074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.333+0000 7f2a52bd1700 1 -- 172.21.15.46:0/774534135 shutdown_connections 2024-05-09T00:09:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:09.333+0000 7f2a52bd1700 1 -- 172.21.15.46:0/774534135 wait complete. 2024-05-09T00:09:09.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-09T00:09:09.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T00:09:09.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-05-09T00:09:09.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-09T00:09:09.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-05-09T00:09:09.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-05-09T00:09:09.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-09T00:09:09.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-05-09T00:09:09.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:09 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-05-09T00:09:09.709 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:09 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/774534135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:09:09.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-09T00:09:09.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T00:09:09.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-05-09T00:09:09.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-09T00:09:09.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-05-09T00:09:09.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-05-09T00:09:09.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-09T00:09:09.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-05-09T00:09:09.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:09 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-05-09T00:09:09.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:09 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/774534135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:09:09.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-09T00:09:09.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T00:09:09.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-05-09T00:09:09.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-09T00:09:09.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-05-09T00:09:09.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-05-09T00:09:09.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-09T00:09:09.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-05-09T00:09:09.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:09 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-05-09T00:09:09.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:09 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/774534135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:09:10.031 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T00:09:10.032 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T00:09:10.285 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:10.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:10 smithi046 ceph-mon[30151]: pgmap v308: 97 pgs: 97 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-05-09T00:09:10.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-09T00:09:10.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T00:09:10.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-05-09T00:09:10.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-09T00:09:10.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-05-09T00:09:10.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-05-09T00:09:10.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-09T00:09:10.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-05-09T00:09:10.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:10 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-05-09T00:09:10.566 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:10 smithi046 ceph-mon[30151]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T00:09:10.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:10 smithi169 ceph-mon[34777]: pgmap v308: 97 pgs: 97 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-05-09T00:09:10.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-09T00:09:10.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T00:09:10.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-05-09T00:09:10.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-09T00:09:10.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-05-09T00:09:10.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-05-09T00:09:10.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-09T00:09:10.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-05-09T00:09:10.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:10 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-05-09T00:09:10.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:10 smithi169 ceph-mon[34777]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T00:09:10.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:10 smithi142 ceph-mon[34533]: pgmap v308: 97 pgs: 97 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-05-09T00:09:10.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-09T00:09:10.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T00:09:10.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-05-09T00:09:10.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-09T00:09:10.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-05-09T00:09:10.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-05-09T00:09:10.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-09T00:09:10.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-05-09T00:09:10.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:10 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-05-09T00:09:10.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:10 smithi142 ceph-mon[34533]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T00:09:11.710 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:11 smithi046 ceph-mon[30151]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T00:09:11.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:11 smithi169 ceph-mon[34777]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T00:09:11.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:11 smithi142 ceph-mon[34533]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T00:09:12.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.995+0000 7f2753ad2700 1 -- 172.21.15.46:0/2425331213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f274c107e10 msgr2=0x7f274c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:12.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.995+0000 7f2753ad2700 1 --2- 172.21.15.46:0/2425331213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f274c107e10 0x7f274c108220 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f273c0099c0 tx=0x7f273c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:12.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.995+0000 7f2753ad2700 1 -- 172.21.15.46:0/2425331213 shutdown_connections 2024-05-09T00:09:12.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.995+0000 7f2753ad2700 1 --2- 172.21.15.46:0/2425331213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f274c108760 0x7f274c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:12.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.995+0000 7f2753ad2700 1 --2- 172.21.15.46:0/2425331213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f274c1034e0 0x7f274c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.995+0000 7f2753ad2700 1 --2- 172.21.15.46:0/2425331213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f274c107e10 0x7f274c108220 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.995+0000 7f2753ad2700 1 -- 172.21.15.46:0/2425331213 >> 172.21.15.46:0/2425331213 conn(0x7f274c0fef30 msgr2=0x7f274c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.995+0000 7f2753ad2700 1 -- 172.21.15.46:0/2425331213 shutdown_connections 2024-05-09T00:09:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.995+0000 7f2753ad2700 1 -- 172.21.15.46:0/2425331213 wait complete. 2024-05-09T00:09:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.996+0000 7f2753ad2700 1 Processor -- start 2024-05-09T00:09:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.996+0000 7f2753ad2700 1 -- start start 2024-05-09T00:09:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.996+0000 7f2753ad2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f274c1034e0 0x7f274c19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f2753ad2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f274c107e10 0x7f274c19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f2753ad2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f274c108760 0x7f274c1a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f2753ad2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f274c1133c0 con 0x7f274c107e10 2024-05-09T00:09:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f2753ad2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f274c113240 con 0x7f274c1034e0 2024-05-09T00:09:12.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f2753ad2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f274c113540 con 0x7f274c108760 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f275106d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f274c107e10 0x7f274c19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f275106d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f274c107e10 0x7f274c19c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55572/0 (socket says 172.21.15.46:55572) 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f275106d700 1 -- 172.21.15.46:0/642260278 learned_addr learned my addr 172.21.15.46:0/642260278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f275206f700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f274c108760 0x7f274c1a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f275186e700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f274c1034e0 0x7f274c19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f275106d700 1 -- 172.21.15.46:0/642260278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f274c108760 msgr2=0x7f274c1a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f275106d700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f274c108760 0x7f274c1a6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f275106d700 1 -- 172.21.15.46:0/642260278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f274c1034e0 msgr2=0x7f274c19bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f275106d700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f274c1034e0 0x7f274c19bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f275106d700 1 -- 172.21.15.46:0/642260278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c19ca90 con 0x7f274c107e10 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.997+0000 7f275106d700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f274c107e10 0x7f274c19c1e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f273c006010 tx=0x7f273c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.998+0000 7f2742ffd700 1 -- 172.21.15.46:0/642260278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c027070 con 0x7f274c107e10 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.998+0000 7f2753ad2700 1 -- 172.21.15.46:0/642260278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c1a7320 con 0x7f274c107e10 2024-05-09T00:09:12.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.998+0000 7f2742ffd700 1 -- 172.21.15.46:0/642260278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f273c003f20 con 0x7f274c107e10 2024-05-09T00:09:12.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.998+0000 7f2742ffd700 1 -- 172.21.15.46:0/642260278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c022c60 con 0x7f274c107e10 2024-05-09T00:09:12.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:11.998+0000 7f2753ad2700 1 -- 172.21.15.46:0/642260278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c1a7770 con 0x7f274c107e10 2024-05-09T00:09:12.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.000+0000 7f2753ad2700 1 -- 172.21.15.46:0/642260278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c0512c0 con 0x7f274c107e10 2024-05-09T00:09:12.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.001+0000 7f2742ffd700 1 -- 172.21.15.46:0/642260278 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f273c02a070 con 0x7f274c107e10 2024-05-09T00:09:12.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.002+0000 7f2742ffd700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f27380b2910 0x7f27380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:12.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.003+0000 7f275186e700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f27380b2910 0x7f27380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:12.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.003+0000 7f2742ffd700 1 -- 172.21.15.46:0/642260278 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f273c022e00 con 0x7f274c107e10 2024-05-09T00:09:12.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.003+0000 7f275186e700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f27380b2910 0x7f27380b4e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2748005950 tx=0x7f274800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:12.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.007+0000 7f2742ffd700 1 -- 172.21.15.46:0/642260278 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f273c031080 con 0x7f274c107e10 2024-05-09T00:09:12.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.220+0000 7f2753ad2700 1 -- 172.21.15.46:0/642260278 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f274c063c00 con 0x7f27380b2910 2024-05-09T00:09:12.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.236+0000 7f2742ffd700 1 -- 172.21.15.46:0/642260278 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258843 (secure 0 0 0) 0x7f274c063c00 con 0x7f27380b2910 2024-05-09T00:09:12.236 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:09:12.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.240+0000 7f2753ad2700 1 -- 172.21.15.46:0/642260278 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f27380b2910 msgr2=0x7f27380b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:12.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.240+0000 7f2753ad2700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f27380b2910 0x7f27380b4e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2748005950 tx=0x7f274800b410 comp rx=0 tx=0).stop 2024-05-09T00:09:12.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.240+0000 7f2753ad2700 1 -- 172.21.15.46:0/642260278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f274c107e10 msgr2=0x7f274c19c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:12.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.240+0000 7f2753ad2700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f274c107e10 0x7f274c19c1e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f273c006010 tx=0x7f273c0047b0 comp rx=0 tx=0).stop 2024-05-09T00:09:12.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.240+0000 7f2753ad2700 1 -- 172.21.15.46:0/642260278 shutdown_connections 2024-05-09T00:09:12.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.240+0000 7f2753ad2700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f27380b2910 0x7f27380b4e20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:12.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.240+0000 7f2753ad2700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f274c108760 0x7f274c1a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:12.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.240+0000 7f2753ad2700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f274c1034e0 0x7f274c19bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:12.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.240+0000 7f2753ad2700 1 --2- 172.21.15.46:0/642260278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f274c107e10 0x7f274c19c1e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:12.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.240+0000 7f2753ad2700 1 -- 172.21.15.46:0/642260278 >> 172.21.15.46:0/642260278 conn(0x7f274c0fef30 msgr2=0x7f274c1096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:12.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.240+0000 7f2753ad2700 1 -- 172.21.15.46:0/642260278 shutdown_connections 2024-05-09T00:09:12.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:12.240+0000 7f2753ad2700 1 -- 172.21.15.46:0/642260278 wait complete. 2024-05-09T00:09:12.239 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T00:09:12.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:12 smithi046 ceph-mon[30151]: pgmap v311: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-05-09T00:09:12.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:12 smithi169 ceph-mon[34777]: pgmap v311: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-05-09T00:09:12.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:12 smithi142 ceph-mon[34533]: pgmap v311: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-05-09T00:09:12.954 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-05-09T00:09:11.292308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650184,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":250,"num_read_kb":215,"num_write":286,"num_write_kb":2257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371984,"kb_used_data":10352,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569840,"statfs":{"total":1151940427776,"available":1151559516160,"internally_reserved":0,"allocated":10600448,"data_stored":6834509,"data_compressed":108789,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27615,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":53,"num_read_kb":58,"num_write":15,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002386"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063574+0000","last_change":"2024-05-09T00:08:26.060455+0000","last_active":"2024-05-09T00:08:28.063574+0000","last_peered":"2024-05-09T00:08:28.063574+0000","last_clean":"2024-05-09T00:08:28.063574+0000","last_became_active":"2024-05-09T00:08:26.060068+0000","last_became_peered":"2024-05-09T00:08:26.060068+0000","last_unstale":"2024-05-09T00:08:28.063574+0000","last_undegraded":"2024-05-09T00:08:28.063574+0000","last_fullsized":"2024-05-09T00:08:28.063574+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:48.354188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.467081+0000","last_change":"2024-05-09T00:08:26.055737+0000","last_active":"2024-05-09T00:09:09.467081+0000","last_peered":"2024-05-09T00:09:09.467081+0000","last_clean":"2024-05-09T00:09:09.467081+0000","last_became_active":"2024-05-09T00:08:26.055354+0000","last_became_peered":"2024-05-09T00:08:26.055354+0000","last_unstale":"2024-05-09T00:09:09.467081+0000","last_undegraded":"2024-05-09T00:09:09.467081+0000","last_fullsized":"2024-05-09T00:09:09.467081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:54.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.466896+0000","last_change":"2024-05-09T00:08:26.050776+0000","last_active":"2024-05-09T00:09:09.466896+0000","last_peered":"2024-05-09T00:09:09.466896+0000","last_clean":"2024-05-09T00:09:09.466896+0000","last_became_active":"2024-05-09T00:08:26.050393+0000","last_became_peered":"2024-05-09T00:08:26.050393+0000","last_unstale":"2024-05-09T00:09:09.466896+0000","last_undegraded":"2024-05-09T00:09:09.466896+0000","last_fullsized":"2024-05-09T00:09:09.466896+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:30.819219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.045303+0000","last_change":"2024-05-09T00:08:26.065946+0000","last_active":"2024-05-09T00:08:28.045303+0000","last_peered":"2024-05-09T00:08:28.045303+0000","last_clean":"2024-05-09T00:08:28.045303+0000","last_became_active":"2024-05-09T00:08:26.065498+0000","last_became_peered":"2024-05-09T00:08:26.065498+0000","last_unstale":"2024-05-09T00:08:28.045303+0000","last_undegraded":"2024-05-09T00:08:28.045303+0000","last_fullsized":"2024-05-09T00:08:28.045303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:04.444860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.059164+0000","last_change":"2024-05-09T00:08:26.051961+0000","last_active":"2024-05-09T00:08:28.059164+0000","last_peered":"2024-05-09T00:08:28.059164+0000","last_clean":"2024-05-09T00:08:28.059164+0000","last_became_active":"2024-05-09T00:08:26.051548+0000","last_became_peered":"2024-05-09T00:08:26.051548+0000","last_unstale":"2024-05-09T00:08:28.059164+0000","last_undegraded":"2024-05-09T00:08:28.059164+0000","last_fullsized":"2024-05-09T00:08:28.059164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:07.539117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.058865+0000","last_change":"2024-05-09T00:08:26.052012+0000","last_active":"2024-05-09T00:08:28.058865+0000","last_peered":"2024-05-09T00:08:28.058865+0000","last_clean":"2024-05-09T00:08:28.058865+0000","last_became_active":"2024-05-09T00:08:26.050886+0000","last_became_peered":"2024-05-09T00:08:26.050886+0000","last_unstale":"2024-05-09T00:08:28.058865+0000","last_undegraded":"2024-05-09T00:08:28.058865+0000","last_fullsized":"2024-05-09T00:08:28.058865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:14.651888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.064287+0000","last_change":"2024-05-09T00:08:26.058418+0000","last_active":"2024-05-09T00:08:28.064287+0000","last_peered":"2024-05-09T00:08:28.064287+0000","last_clean":"2024-05-09T00:08:28.064287+0000","last_became_active":"2024-05-09T00:08:26.058039+0000","last_became_peered":"2024-05-09T00:08:26.058039+0000","last_unstale":"2024-05-09T00:08:28.064287+0000","last_undegraded":"2024-05-09T00:08:28.064287+0000","last_fullsized":"2024-05-09T00:08:28.064287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:13.678123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.060807+0000","last_change":"2024-05-09T00:08:26.057602+0000","last_active":"2024-05-09T00:08:28.060807+0000","last_peered":"2024-05-09T00:08:28.060807+0000","last_clean":"2024-05-09T00:08:28.060807+0000","last_became_active":"2024-05-09T00:08:26.057169+0000","last_became_peered":"2024-05-09T00:08:26.057169+0000","last_unstale":"2024-05-09T00:08:28.060807+0000","last_undegraded":"2024-05-09T00:08:28.060807+0000","last_fullsized":"2024-05-09T00:08:28.060807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:11.452083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063522+0000","last_change":"2024-05-09T00:08:26.067639+0000","last_active":"2024-05-09T00:08:28.063522+0000","last_peered":"2024-05-09T00:08:28.063522+0000","last_clean":"2024-05-09T00:08:28.063522+0000","last_became_active":"2024-05-09T00:08:26.067226+0000","last_became_peered":"2024-05-09T00:08:26.067226+0000","last_unstale":"2024-05-09T00:08:28.063522+0000","last_undegraded":"2024-05-09T00:08:28.063522+0000","last_fullsized":"2024-05-09T00:08:28.063522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:12.366500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.059491+0000","last_change":"2024-05-09T00:08:26.055234+0000","last_active":"2024-05-09T00:08:28.059491+0000","last_peered":"2024-05-09T00:08:28.059491+0000","last_clean":"2024-05-09T00:08:28.059491+0000","last_became_active":"2024-05-09T00:08:26.054851+0000","last_became_peered":"2024-05-09T00:08:26.054851+0000","last_unstale":"2024-05-09T00:08:28.059491+0000","last_undegraded":"2024-05-09T00:08:28.059491+0000","last_fullsized":"2024-05-09T00:08:28.059491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:54:52.121329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063075+0000","last_change":"2024-05-09T00:08:26.063375+0000","last_active":"2024-05-09T00:08:28.063075+0000","last_peered":"2024-05-09T00:08:28.063075+0000","last_clean":"2024-05-09T00:08:28.063075+0000","last_became_active":"2024-05-09T00:08:26.062962+0000","last_became_peered":"2024-05-09T00:08:26.062962+0000","last_unstale":"2024-05-09T00:08:28.063075+0000","last_undegraded":"2024-05-09T00:08:28.063075+0000","last_fullsized":"2024-05-09T00:08:28.063075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:05.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.045358+0000","last_change":"2024-05-09T00:08:26.062034+0000","last_active":"2024-05-09T00:08:28.045358+0000","last_peered":"2024-05-09T00:08:28.045358+0000","last_clean":"2024-05-09T00:08:28.045358+0000","last_became_active":"2024-05-09T00:08:26.060376+0000","last_became_peered":"2024-05-09T00:08:26.060376+0000","last_unstale":"2024-05-09T00:08:28.045358+0000","last_undegraded":"2024-05-09T00:08:28.045358+0000","last_fullsized":"2024-05-09T00:08:28.045358+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:15.830024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.043955+0000","last_change":"2024-05-09T00:08:26.053227+0000","last_active":"2024-05-09T00:08:28.043955+0000","last_peered":"2024-05-09T00:08:28.043955+0000","last_clean":"2024-05-09T00:08:28.043955+0000","last_became_active":"2024-05-09T00:08:26.052878+0000","last_became_peered":"2024-05-09T00:08:26.052878+0000","last_unstale":"2024-05-09T00:08:28.043955+0000","last_undegraded":"2024-05-09T00:08:28.043955+0000","last_fullsized":"2024-05-09T00:08:28.043955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:24.385026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.466484+0000","last_change":"2024-05-09T00:08:26.053041+0000","last_active":"2024-05-09T00:09:09.466484+0000","last_peered":"2024-05-09T00:09:09.466484+0000","last_clean":"2024-05-09T00:09:09.466484+0000","last_became_active":"2024-05-09T00:08:26.052677+0000","last_became_peered":"2024-05-09T00:08:26.052677+0000","last_unstale":"2024-05-09T00:09:09.466484+0000","last_undegraded":"2024-05-09T00:09:09.466484+0000","last_fullsized":"2024-05-09T00:09:09.466484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:24.478270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.465389+0000","last_change":"2024-05-09T00:08:26.063391+0000","last_active":"2024-05-09T00:09:09.465389+0000","last_peered":"2024-05-09T00:09:09.465389+0000","last_clean":"2024-05-09T00:09:09.465389+0000","last_became_active":"2024-05-09T00:08:26.063033+0000","last_became_peered":"2024-05-09T00:08:26.063033+0000","last_unstale":"2024-05-09T00:09:09.465389+0000","last_undegraded":"2024-05-09T00:09:09.465389+0000","last_fullsized":"2024-05-09T00:09:09.465389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:41.900453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.062036+0000","last_change":"2024-05-09T00:08:26.065018+0000","last_active":"2024-05-09T00:08:28.062036+0000","last_peered":"2024-05-09T00:08:28.062036+0000","last_clean":"2024-05-09T00:08:28.062036+0000","last_became_active":"2024-05-09T00:08:26.064642+0000","last_became_peered":"2024-05-09T00:08:26.064642+0000","last_unstale":"2024-05-09T00:08:28.062036+0000","last_undegraded":"2024-05-09T00:08:28.062036+0000","last_fullsized":"2024-05-09T00:08:28.062036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:02.403704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063982+0000","last_change":"2024-05-09T00:08:26.049770+0000","last_active":"2024-05-09T00:08:28.063982+0000","last_peered":"2024-05-09T00:08:28.063982+0000","last_clean":"2024-05-09T00:08:28.063982+0000","last_became_active":"2024-05-09T00:08:26.048555+0000","last_became_peered":"2024-05-09T00:08:26.048555+0000","last_unstale":"2024-05-09T00:08:28.063982+0000","last_undegraded":"2024-05-09T00:08:28.063982+0000","last_fullsized":"2024-05-09T00:08:28.063982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:16.009200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.058920+0000","last_change":"2024-05-09T00:08:26.048587+0000","last_active":"2024-05-09T00:08:28.058920+0000","last_peered":"2024-05-09T00:08:28.058920+0000","last_clean":"2024-05-09T00:08:28.058920+0000","last_became_active":"2024-05-09T00:08:26.048148+0000","last_became_peered":"2024-05-09T00:08:26.048148+0000","last_unstale":"2024-05-09T00:08:28.058920+0000","last_undegraded":"2024-05-09T00:08:28.058920+0000","last_fullsized":"2024-05-09T00:08:28.058920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:42.113829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.061795+0000","last_change":"2024-05-09T00:08:26.053607+0000","last_active":"2024-05-09T00:08:28.061795+0000","last_peered":"2024-05-09T00:08:28.061795+0000","last_clean":"2024-05-09T00:08:28.061795+0000","last_became_active":"2024-05-09T00:08:26.053212+0000","last_became_peered":"2024-05-09T00:08:26.053212+0000","last_unstale":"2024-05-09T00:08:28.061795+0000","last_undegraded":"2024-05-09T00:08:28.061795+0000","last_fullsized":"2024-05-09T00:08:28.061795+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:32.937049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.380448+0000","last_change":"2024-05-09T00:08:23.024481+0000","last_active":"2024-05-09T00:08:41.380448+0000","last_peered":"2024-05-09T00:08:41.380448+0000","last_clean":"2024-05-09T00:08:41.380448+0000","last_became_active":"2024-05-09T00:08:23.024132+0000","last_became_peered":"2024-05-09T00:08:23.024132+0000","last_unstale":"2024-05-09T00:08:41.380448+0000","last_undegraded":"2024-05-09T00:08:41.380448+0000","last_fullsized":"2024-05-09T00:08:41.380448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:11.188505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.061479+0000","last_change":"2024-05-09T00:08:26.054772+0000","last_active":"2024-05-09T00:08:28.061479+0000","last_peered":"2024-05-09T00:08:28.061479+0000","last_clean":"2024-05-09T00:08:28.061479+0000","last_became_active":"2024-05-09T00:08:26.054433+0000","last_became_peered":"2024-05-09T00:08:26.054433+0000","last_unstale":"2024-05-09T00:08:28.061479+0000","last_undegraded":"2024-05-09T00:08:28.061479+0000","last_fullsized":"2024-05-09T00:08:28.061479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:44.283240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.064849+0000","last_change":"2024-05-09T00:08:23.118778+0000","last_active":"2024-05-09T00:08:28.064849+0000","last_peered":"2024-05-09T00:08:28.064849+0000","last_clean":"2024-05-09T00:08:28.064849+0000","last_became_active":"2024-05-09T00:08:23.118377+0000","last_became_peered":"2024-05-09T00:08:23.118377+0000","last_unstale":"2024-05-09T00:08:28.064849+0000","last_undegraded":"2024-05-09T00:08:28.064849+0000","last_fullsized":"2024-05-09T00:08:28.064849+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:51.352442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.466418+0000","last_change":"2024-05-09T00:08:26.059953+0000","last_active":"2024-05-09T00:09:09.466418+0000","last_peered":"2024-05-09T00:09:09.466418+0000","last_clean":"2024-05-09T00:09:09.466418+0000","last_became_active":"2024-05-09T00:08:26.059571+0000","last_became_peered":"2024-05-09T00:08:26.059571+0000","last_unstale":"2024-05-09T00:09:09.466418+0000","last_undegraded":"2024-05-09T00:09:09.466418+0000","last_fullsized":"2024-05-09T00:09:09.466418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:48.056798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.378455+0000","last_change":"2024-05-09T00:08:23.021244+0000","last_active":"2024-05-09T00:08:41.378455+0000","last_peered":"2024-05-09T00:08:41.378455+0000","last_clean":"2024-05-09T00:08:41.378455+0000","last_became_active":"2024-05-09T00:08:23.020597+0000","last_became_peered":"2024-05-09T00:08:23.020597+0000","last_unstale":"2024-05-09T00:08:41.378455+0000","last_undegraded":"2024-05-09T00:08:41.378455+0000","last_fullsized":"2024-05-09T00:08:41.378455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.654261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.062710+0000","last_change":"2024-05-09T00:08:26.059834+0000","last_active":"2024-05-09T00:08:28.062710+0000","last_peered":"2024-05-09T00:08:28.062710+0000","last_clean":"2024-05-09T00:08:28.062710+0000","last_became_active":"2024-05-09T00:08:26.059349+0000","last_became_peered":"2024-05-09T00:08:26.059349+0000","last_unstale":"2024-05-09T00:08:28.062710+0000","last_undegraded":"2024-05-09T00:08:28.062710+0000","last_fullsized":"2024-05-09T00:08:28.062710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:42.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.066576+0000","last_change":"2024-05-09T00:08:23.019568+0000","last_active":"2024-05-09T00:08:28.066576+0000","last_peered":"2024-05-09T00:08:28.066576+0000","last_clean":"2024-05-09T00:08:28.066576+0000","last_became_active":"2024-05-09T00:08:23.019149+0000","last_became_peered":"2024-05-09T00:08:23.019149+0000","last_unstale":"2024-05-09T00:08:28.066576+0000","last_undegraded":"2024-05-09T00:08:28.066576+0000","last_fullsized":"2024-05-09T00:08:28.066576+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:44.569911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.044825+0000","last_change":"2024-05-09T00:08:26.061308+0000","last_active":"2024-05-09T00:08:28.044825+0000","last_peered":"2024-05-09T00:08:28.044825+0000","last_clean":"2024-05-09T00:08:28.044825+0000","last_became_active":"2024-05-09T00:08:26.060973+0000","last_became_peered":"2024-05-09T00:08:26.060973+0000","last_unstale":"2024-05-09T00:08:28.044825+0000","last_undegraded":"2024-05-09T00:08:28.044825+0000","last_fullsized":"2024-05-09T00:08:28.044825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:16.845074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:09:05.427412+0000","last_change":"2024-05-09T00:08:23.020886+0000","last_active":"2024-05-09T00:09:05.427412+0000","last_peered":"2024-05-09T00:09:05.427412+0000","last_clean":"2024-05-09T00:09:05.427412+0000","last_became_active":"2024-05-09T00:08:23.020530+0000","last_became_peered":"2024-05-09T00:08:23.020530+0000","last_unstale":"2024-05-09T00:09:05.427412+0000","last_undegraded":"2024-05-09T00:09:05.427412+0000","last_fullsized":"2024-05-09T00:09:05.427412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:34.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.059315+0000","last_change":"2024-05-09T00:08:26.051018+0000","last_active":"2024-05-09T00:08:28.059315+0000","last_peered":"2024-05-09T00:08:28.059315+0000","last_clean":"2024-05-09T00:08:28.059315+0000","last_became_active":"2024-05-09T00:08:26.050495+0000","last_became_peered":"2024-05-09T00:08:26.050495+0000","last_unstale":"2024-05-09T00:08:28.059315+0000","last_undegraded":"2024-05-09T00:08:28.059315+0000","last_fullsized":"2024-05-09T00:08:28.059315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:07.862339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.378509+0000","last_change":"2024-05-09T00:08:23.021187+0000","last_active":"2024-05-09T00:08:41.378509+0000","last_peered":"2024-05-09T00:08:41.378509+0000","last_clean":"2024-05-09T00:08:41.378509+0000","last_became_active":"2024-05-09T00:08:23.020770+0000","last_became_peered":"2024-05-09T00:08:23.020770+0000","last_unstale":"2024-05-09T00:08:41.378509+0000","last_undegraded":"2024-05-09T00:08:41.378509+0000","last_fullsized":"2024-05-09T00:08:41.378509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:08.992770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.044577+0000","last_change":"2024-05-09T00:08:26.066691+0000","last_active":"2024-05-09T00:08:28.044577+0000","last_peered":"2024-05-09T00:08:28.044577+0000","last_clean":"2024-05-09T00:08:28.044577+0000","last_became_active":"2024-05-09T00:08:26.066303+0000","last_became_peered":"2024-05-09T00:08:26.066303+0000","last_unstale":"2024-05-09T00:08:28.044577+0000","last_undegraded":"2024-05-09T00:08:28.044577+0000","last_fullsized":"2024-05-09T00:08:28.044577+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:37:18.516116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.377392+0000","last_change":"2024-05-09T00:08:23.012226+0000","last_active":"2024-05-09T00:08:41.377392+0000","last_peered":"2024-05-09T00:08:41.377392+0000","last_clean":"2024-05-09T00:08:41.377392+0000","last_became_active":"2024-05-09T00:08:23.011781+0000","last_became_peered":"2024-05-09T00:08:23.011781+0000","last_unstale":"2024-05-09T00:08:41.377392+0000","last_undegraded":"2024-05-09T00:08:41.377392+0000","last_fullsized":"2024-05-09T00:08:41.377392+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:44.452206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.060614+0000","last_change":"2024-05-09T00:08:26.043035+0000","last_active":"2024-05-09T00:08:28.060614+0000","last_peered":"2024-05-09T00:08:28.060614+0000","last_clean":"2024-05-09T00:08:28.060614+0000","last_became_active":"2024-05-09T00:08:26.042686+0000","last_became_peered":"2024-05-09T00:08:26.042686+0000","last_unstale":"2024-05-09T00:08:28.060614+0000","last_undegraded":"2024-05-09T00:08:28.060614+0000","last_fullsized":"2024-05-09T00:08:28.060614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:24.511309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.044339+0000","last_change":"2024-05-09T00:08:23.022017+0000","last_active":"2024-05-09T00:08:28.044339+0000","last_peered":"2024-05-09T00:08:28.044339+0000","last_clean":"2024-05-09T00:08:28.044339+0000","last_became_active":"2024-05-09T00:08:23.021540+0000","last_became_peered":"2024-05-09T00:08:23.021540+0000","last_unstale":"2024-05-09T00:08:28.044339+0000","last_undegraded":"2024-05-09T00:08:28.044339+0000","last_fullsized":"2024-05-09T00:08:28.044339+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:50.773211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.058460+0000","last_change":"2024-05-09T00:08:26.048039+0000","last_active":"2024-05-09T00:08:28.058460+0000","last_peered":"2024-05-09T00:08:28.058460+0000","last_clean":"2024-05-09T00:08:28.058460+0000","last_became_active":"2024-05-09T00:08:26.047695+0000","last_became_peered":"2024-05-09T00:08:26.047695+0000","last_unstale":"2024-05-09T00:08:28.058460+0000","last_undegraded":"2024-05-09T00:08:28.058460+0000","last_fullsized":"2024-05-09T00:08:28.058460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:27.363847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:09:07.430191+0000","last_change":"2024-05-09T00:08:23.019565+0000","last_active":"2024-05-09T00:09:07.430191+0000","last_peered":"2024-05-09T00:09:07.430191+0000","last_clean":"2024-05-09T00:09:07.430191+0000","last_became_active":"2024-05-09T00:08:23.019113+0000","last_became_peered":"2024-05-09T00:08:23.019113+0000","last_unstale":"2024-05-09T00:09:07.430191+0000","last_undegraded":"2024-05-09T00:09:07.430191+0000","last_fullsized":"2024-05-09T00:09:07.430191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:45.578934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063276+0000","last_change":"2024-05-09T00:08:26.058869+0000","last_active":"2024-05-09T00:08:28.063276+0000","last_peered":"2024-05-09T00:08:28.063276+0000","last_clean":"2024-05-09T00:08:28.063276+0000","last_became_active":"2024-05-09T00:08:26.058482+0000","last_became_peered":"2024-05-09T00:08:26.058482+0000","last_unstale":"2024-05-09T00:08:28.063276+0000","last_undegraded":"2024-05-09T00:08:28.063276+0000","last_fullsized":"2024-05-09T00:08:28.063276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:33.252208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:40.365445+0000","last_change":"2024-05-09T00:08:23.024425+0000","last_active":"2024-05-09T00:08:40.365445+0000","last_peered":"2024-05-09T00:08:40.365445+0000","last_clean":"2024-05-09T00:08:40.365445+0000","last_became_active":"2024-05-09T00:08:23.023991+0000","last_became_peered":"2024-05-09T00:08:23.023991+0000","last_unstale":"2024-05-09T00:08:40.365445+0000","last_undegraded":"2024-05-09T00:08:40.365445+0000","last_fullsized":"2024-05-09T00:08:40.365445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:47.907851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"peering","last_fresh":"2024-05-09T00:08:29.623924+0000","last_change":"2024-05-09T00:09:09.467043+0000","last_active":"2024-05-09T00:09:09.467043+0000","last_peered":"2024-05-09T00:08:29.623924+0000","last_clean":"2024-05-09T00:08:29.623924+0000","last_became_active":"2024-05-09T00:08:26.063306+0000","last_became_peered":"2024-05-09T00:08:26.063306+0000","last_unstale":"2024-05-09T00:08:29.623924+0000","last_undegraded":"2024-05-09T00:08:29.623924+0000","last_fullsized":"2024-05-09T00:08:29.623924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:40.366496+0000","last_change":"2024-05-09T00:08:23.118516+0000","last_active":"2024-05-09T00:08:40.366496+0000","last_peered":"2024-05-09T00:08:40.366496+0000","last_clean":"2024-05-09T00:08:40.366496+0000","last_became_active":"2024-05-09T00:08:23.118147+0000","last_became_peered":"2024-05-09T00:08:23.118147+0000","last_unstale":"2024-05-09T00:08:40.366496+0000","last_undegraded":"2024-05-09T00:08:40.366496+0000","last_fullsized":"2024-05-09T00:08:40.366496+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:24.205818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.062923+0000","last_change":"2024-05-09T00:08:26.067063+0000","last_active":"2024-05-09T00:08:28.062923+0000","last_peered":"2024-05-09T00:08:28.062923+0000","last_clean":"2024-05-09T00:08:28.062923+0000","last_became_active":"2024-05-09T00:08:26.066440+0000","last_became_peered":"2024-05-09T00:08:26.066440+0000","last_unstale":"2024-05-09T00:08:28.062923+0000","last_undegraded":"2024-05-09T00:08:28.062923+0000","last_fullsized":"2024-05-09T00:08:28.062923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:48.648485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'28","reported_seq":108,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:09:08.433999+0000","last_change":"2024-05-09T00:08:23.020757+0000","last_active":"2024-05-09T00:09:08.433999+0000","last_peered":"2024-05-09T00:09:08.433999+0000","last_clean":"2024-05-09T00:09:08.433999+0000","last_became_active":"2024-05-09T00:08:23.020355+0000","last_became_peered":"2024-05-09T00:08:23.020355+0000","last_unstale":"2024-05-09T00:09:08.433999+0000","last_undegraded":"2024-05-09T00:09:08.433999+0000","last_fullsized":"2024-05-09T00:09:08.433999+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:00.605691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55581,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":58,"num_read_kb":82,"num_write":29,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.059488+0000","last_change":"2024-05-09T00:08:26.057402+0000","last_active":"2024-05-09T00:08:28.059488+0000","last_peered":"2024-05-09T00:08:28.059488+0000","last_clean":"2024-05-09T00:08:28.059488+0000","last_became_active":"2024-05-09T00:08:26.056974+0000","last_became_peered":"2024-05-09T00:08:26.056974+0000","last_unstale":"2024-05-09T00:08:28.059488+0000","last_undegraded":"2024-05-09T00:08:28.059488+0000","last_fullsized":"2024-05-09T00:08:28.059488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:47.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.380024+0000","last_change":"2024-05-09T00:08:23.020208+0000","last_active":"2024-05-09T00:08:41.380024+0000","last_peered":"2024-05-09T00:08:41.380024+0000","last_clean":"2024-05-09T00:08:41.380024+0000","last_became_active":"2024-05-09T00:08:23.019809+0000","last_became_peered":"2024-05-09T00:08:23.019809+0000","last_unstale":"2024-05-09T00:08:41.380024+0000","last_undegraded":"2024-05-09T00:08:41.380024+0000","last_fullsized":"2024-05-09T00:08:41.380024+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:47.900731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.043928+0000","last_change":"2024-05-09T00:08:26.054273+0000","last_active":"2024-05-09T00:08:28.043928+0000","last_peered":"2024-05-09T00:08:28.043928+0000","last_clean":"2024-05-09T00:08:28.043928+0000","last_became_active":"2024-05-09T00:08:26.053840+0000","last_became_peered":"2024-05-09T00:08:26.053840+0000","last_unstale":"2024-05-09T00:08:28.043928+0000","last_undegraded":"2024-05-09T00:08:28.043928+0000","last_fullsized":"2024-05-09T00:08:28.043928+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:27.128582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:40.364962+0000","last_change":"2024-05-09T00:08:23.021594+0000","last_active":"2024-05-09T00:08:40.364962+0000","last_peered":"2024-05-09T00:08:40.364962+0000","last_clean":"2024-05-09T00:08:40.364962+0000","last_became_active":"2024-05-09T00:08:23.021152+0000","last_became_peered":"2024-05-09T00:08:23.021152+0000","last_unstale":"2024-05-09T00:08:40.364962+0000","last_undegraded":"2024-05-09T00:08:40.364962+0000","last_fullsized":"2024-05-09T00:08:40.364962+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:58.505781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063461+0000","last_change":"2024-05-09T00:08:26.064464+0000","last_active":"2024-05-09T00:08:28.063461+0000","last_peered":"2024-05-09T00:08:28.063461+0000","last_clean":"2024-05-09T00:08:28.063461+0000","last_became_active":"2024-05-09T00:08:26.064000+0000","last_became_peered":"2024-05-09T00:08:26.064000+0000","last_unstale":"2024-05-09T00:08:28.063461+0000","last_undegraded":"2024-05-09T00:08:28.063461+0000","last_fullsized":"2024-05-09T00:08:28.063461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:01.455979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'4","reported_seq":53,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:09:06.428517+0000","last_change":"2024-05-09T00:08:23.020823+0000","last_active":"2024-05-09T00:09:06.428517+0000","last_peered":"2024-05-09T00:09:06.428517+0000","last_clean":"2024-05-09T00:09:06.428517+0000","last_became_active":"2024-05-09T00:08:23.020412+0000","last_became_peered":"2024-05-09T00:08:23.020412+0000","last_unstale":"2024-05-09T00:09:06.428517+0000","last_undegraded":"2024-05-09T00:09:06.428517+0000","last_fullsized":"2024-05-09T00:09:06.428517+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:05.457452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063005+0000","last_change":"2024-05-09T00:08:26.057092+0000","last_active":"2024-05-09T00:08:28.063005+0000","last_peered":"2024-05-09T00:08:28.063005+0000","last_clean":"2024-05-09T00:08:28.063005+0000","last_became_active":"2024-05-09T00:08:26.056723+0000","last_became_peered":"2024-05-09T00:08:26.056723+0000","last_unstale":"2024-05-09T00:08:28.063005+0000","last_undegraded":"2024-05-09T00:08:28.063005+0000","last_fullsized":"2024-05-09T00:08:28.063005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:15.929284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.380913+0000","last_change":"2024-05-09T00:08:23.118857+0000","last_active":"2024-05-09T00:08:41.380913+0000","last_peered":"2024-05-09T00:08:41.380913+0000","last_clean":"2024-05-09T00:08:41.380913+0000","last_became_active":"2024-05-09T00:08:23.118433+0000","last_became_peered":"2024-05-09T00:08:23.118433+0000","last_unstale":"2024-05-09T00:08:41.380913+0000","last_undegraded":"2024-05-09T00:08:41.380913+0000","last_fullsized":"2024-05-09T00:08:41.380913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:10.617987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.044439+0000","last_change":"2024-05-09T00:08:26.066911+0000","last_active":"2024-05-09T00:08:28.044439+0000","last_peered":"2024-05-09T00:08:28.044439+0000","last_clean":"2024-05-09T00:08:28.044439+0000","last_became_active":"2024-05-09T00:08:26.066548+0000","last_became_peered":"2024-05-09T00:08:26.066548+0000","last_unstale":"2024-05-09T00:08:28.044439+0000","last_undegraded":"2024-05-09T00:08:28.044439+0000","last_fullsized":"2024-05-09T00:08:28.044439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:32.391390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:42.373469+0000","last_change":"2024-05-09T00:08:23.019829+0000","last_active":"2024-05-09T00:08:42.373469+0000","last_peered":"2024-05-09T00:08:42.373469+0000","last_clean":"2024-05-09T00:08:42.373469+0000","last_became_active":"2024-05-09T00:08:23.019336+0000","last_became_peered":"2024-05-09T00:08:23.019336+0000","last_unstale":"2024-05-09T00:08:42.373469+0000","last_undegraded":"2024-05-09T00:08:42.373469+0000","last_fullsized":"2024-05-09T00:08:42.373469+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:33.802738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063102+0000","last_change":"2024-05-09T00:08:26.055975+0000","last_active":"2024-05-09T00:08:28.063102+0000","last_peered":"2024-05-09T00:08:28.063102+0000","last_clean":"2024-05-09T00:08:28.063102+0000","last_became_active":"2024-05-09T00:08:26.055595+0000","last_became_peered":"2024-05-09T00:08:26.055595+0000","last_unstale":"2024-05-09T00:08:28.063102+0000","last_undegraded":"2024-05-09T00:08:28.063102+0000","last_fullsized":"2024-05-09T00:08:28.063102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:20.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.058677+0000","last_change":"2024-05-09T00:08:23.120353+0000","last_active":"2024-05-09T00:08:28.058677+0000","last_peered":"2024-05-09T00:08:28.058677+0000","last_clean":"2024-05-09T00:08:28.058677+0000","last_became_active":"2024-05-09T00:08:23.120004+0000","last_became_peered":"2024-05-09T00:08:23.120004+0000","last_unstale":"2024-05-09T00:08:28.058677+0000","last_undegraded":"2024-05-09T00:08:28.058677+0000","last_fullsized":"2024-05-09T00:08:28.058677+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:12.168671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.663335+0000","last_change":"2024-05-09T00:08:26.046554+0000","last_active":"2024-05-09T00:08:28.663335+0000","last_peered":"2024-05-09T00:08:28.663335+0000","last_clean":"2024-05-09T00:08:28.663335+0000","last_became_active":"2024-05-09T00:08:26.045924+0000","last_became_peered":"2024-05-09T00:08:26.045924+0000","last_unstale":"2024-05-09T00:08:28.663335+0000","last_undegraded":"2024-05-09T00:08:28.663335+0000","last_fullsized":"2024-05-09T00:08:28.663335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:01.208980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.064602+0000","last_change":"2024-05-09T00:08:23.019778+0000","last_active":"2024-05-09T00:08:28.064602+0000","last_peered":"2024-05-09T00:08:28.064602+0000","last_clean":"2024-05-09T00:08:28.064602+0000","last_became_active":"2024-05-09T00:08:23.019400+0000","last_became_peered":"2024-05-09T00:08:23.019400+0000","last_unstale":"2024-05-09T00:08:28.064602+0000","last_undegraded":"2024-05-09T00:08:28.064602+0000","last_fullsized":"2024-05-09T00:08:28.064602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:06.883167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.058852+0000","last_change":"2024-05-09T00:08:26.055200+0000","last_active":"2024-05-09T00:08:28.058852+0000","last_peered":"2024-05-09T00:08:28.058852+0000","last_clean":"2024-05-09T00:08:28.058852+0000","last_became_active":"2024-05-09T00:08:26.054839+0000","last_became_peered":"2024-05-09T00:08:26.054839+0000","last_unstale":"2024-05-09T00:08:28.058852+0000","last_undegraded":"2024-05-09T00:08:28.058852+0000","last_fullsized":"2024-05-09T00:08:28.058852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:31:40.473061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.044625+0000","last_change":"2024-05-09T00:08:26.066522+0000","last_active":"2024-05-09T00:08:28.044625+0000","last_peered":"2024-05-09T00:08:28.044625+0000","last_clean":"2024-05-09T00:08:28.044625+0000","last_became_active":"2024-05-09T00:08:26.066114+0000","last_became_peered":"2024-05-09T00:08:26.066114+0000","last_unstale":"2024-05-09T00:08:28.044625+0000","last_undegraded":"2024-05-09T00:08:28.044625+0000","last_fullsized":"2024-05-09T00:08:28.044625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:59:11.419535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:09:07.429459+0000","last_change":"2024-05-09T00:08:23.118322+0000","last_active":"2024-05-09T00:09:07.429459+0000","last_peered":"2024-05-09T00:09:07.429459+0000","last_clean":"2024-05-09T00:09:07.429459+0000","last_became_active":"2024-05-09T00:08:23.117943+0000","last_became_peered":"2024-05-09T00:08:23.117943+0000","last_unstale":"2024-05-09T00:09:07.429459+0000","last_undegraded":"2024-05-09T00:09:07.429459+0000","last_fullsized":"2024-05-09T00:09:07.429459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:07.890437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.061156+0000","last_change":"2024-05-09T00:08:26.052294+0000","last_active":"2024-05-09T00:08:28.061156+0000","last_peered":"2024-05-09T00:08:28.061156+0000","last_clean":"2024-05-09T00:08:28.061156+0000","last_became_active":"2024-05-09T00:08:26.051926+0000","last_became_peered":"2024-05-09T00:08:26.051926+0000","last_unstale":"2024-05-09T00:08:28.061156+0000","last_undegraded":"2024-05-09T00:08:28.061156+0000","last_fullsized":"2024-05-09T00:08:28.061156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:53.651401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:44.393781+0000","last_change":"2024-05-09T00:08:23.015007+0000","last_active":"2024-05-09T00:08:44.393781+0000","last_peered":"2024-05-09T00:08:44.393781+0000","last_clean":"2024-05-09T00:08:44.393781+0000","last_became_active":"2024-05-09T00:08:23.014545+0000","last_became_peered":"2024-05-09T00:08:23.014545+0000","last_unstale":"2024-05-09T00:08:44.393781+0000","last_undegraded":"2024-05-09T00:08:44.393781+0000","last_fullsized":"2024-05-09T00:08:44.393781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:19.850200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063181+0000","last_change":"2024-05-09T00:07:00.733233+0000","last_active":"2024-05-09T00:08:28.063181+0000","last_peered":"2024-05-09T00:08:28.063181+0000","last_clean":"2024-05-09T00:08:28.063181+0000","last_became_active":"2024-05-09T00:07:00.727365+0000","last_became_peered":"2024-05-09T00:07:00.727365+0000","last_unstale":"2024-05-09T00:08:28.063181+0000","last_undegraded":"2024-05-09T00:08:28.063181+0000","last_fullsized":"2024-05-09T00:08:28.063181+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_clean_scrub_stamp":"2024-05-09T00:03:33.285220+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:08.597290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.043602+0000","last_change":"2024-05-09T00:08:26.064000+0000","last_active":"2024-05-09T00:08:28.043602+0000","last_peered":"2024-05-09T00:08:28.043602+0000","last_clean":"2024-05-09T00:08:28.043602+0000","last_became_active":"2024-05-09T00:08:26.063636+0000","last_became_peered":"2024-05-09T00:08:26.063636+0000","last_unstale":"2024-05-09T00:08:28.043602+0000","last_undegraded":"2024-05-09T00:08:28.043602+0000","last_fullsized":"2024-05-09T00:08:28.043602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:59.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.380397+0000","last_change":"2024-05-09T00:08:23.023459+0000","last_active":"2024-05-09T00:08:41.380397+0000","last_peered":"2024-05-09T00:08:41.380397+0000","last_clean":"2024-05-09T00:08:41.380397+0000","last_became_active":"2024-05-09T00:08:23.023089+0000","last_became_peered":"2024-05-09T00:08:23.023089+0000","last_unstale":"2024-05-09T00:08:41.380397+0000","last_undegraded":"2024-05-09T00:08:41.380397+0000","last_fullsized":"2024-05-09T00:08:41.380397+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:41.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:40.364231+0000","last_change":"2024-05-09T00:08:23.120566+0000","last_active":"2024-05-09T00:08:40.364231+0000","last_peered":"2024-05-09T00:08:40.364231+0000","last_clean":"2024-05-09T00:08:40.364231+0000","last_became_active":"2024-05-09T00:08:23.119861+0000","last_became_peered":"2024-05-09T00:08:23.119861+0000","last_unstale":"2024-05-09T00:08:40.364231+0000","last_undegraded":"2024-05-09T00:08:40.364231+0000","last_fullsized":"2024-05-09T00:08:40.364231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:52.282934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.043882+0000","last_change":"2024-05-09T00:08:26.050685+0000","last_active":"2024-05-09T00:08:28.043882+0000","last_peered":"2024-05-09T00:08:28.043882+0000","last_clean":"2024-05-09T00:08:28.043882+0000","last_became_active":"2024-05-09T00:08:26.050312+0000","last_became_peered":"2024-05-09T00:08:26.050312+0000","last_unstale":"2024-05-09T00:08:28.043882+0000","last_undegraded":"2024-05-09T00:08:28.043882+0000","last_fullsized":"2024-05-09T00:08:28.043882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:31.623182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:09:04.060236+0000","last_change":"2024-05-09T00:08:23.019663+0000","last_active":"2024-05-09T00:09:04.060236+0000","last_peered":"2024-05-09T00:09:04.060236+0000","last_clean":"2024-05-09T00:09:04.060236+0000","last_became_active":"2024-05-09T00:08:23.019301+0000","last_became_peered":"2024-05-09T00:08:23.019301+0000","last_unstale":"2024-05-09T00:09:04.060236+0000","last_undegraded":"2024-05-09T00:09:04.060236+0000","last_fullsized":"2024-05-09T00:09:04.060236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:08.291227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.060737+0000","last_change":"2024-05-09T00:08:26.058825+0000","last_active":"2024-05-09T00:08:28.060737+0000","last_peered":"2024-05-09T00:08:28.060737+0000","last_clean":"2024-05-09T00:08:28.060737+0000","last_became_active":"2024-05-09T00:08:26.058469+0000","last_became_peered":"2024-05-09T00:08:26.058469+0000","last_unstale":"2024-05-09T00:08:28.060737+0000","last_undegraded":"2024-05-09T00:08:28.060737+0000","last_fullsized":"2024-05-09T00:08:28.060737+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:08.118819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:40.366715+0000","last_change":"2024-05-09T00:08:23.054366+0000","last_active":"2024-05-09T00:08:40.366715+0000","last_peered":"2024-05-09T00:08:40.366715+0000","last_clean":"2024-05-09T00:08:40.366715+0000","last_became_active":"2024-05-09T00:08:23.053977+0000","last_became_peered":"2024-05-09T00:08:23.053977+0000","last_unstale":"2024-05-09T00:08:40.366715+0000","last_undegraded":"2024-05-09T00:08:40.366715+0000","last_fullsized":"2024-05-09T00:08:40.366715+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:37.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.467190+0000","last_change":"2024-05-09T00:08:26.057864+0000","last_active":"2024-05-09T00:09:09.467190+0000","last_peered":"2024-05-09T00:09:09.467190+0000","last_clean":"2024-05-09T00:09:09.467190+0000","last_became_active":"2024-05-09T00:08:26.057436+0000","last_became_peered":"2024-05-09T00:08:26.057436+0000","last_unstale":"2024-05-09T00:09:09.467190+0000","last_undegraded":"2024-05-09T00:09:09.467190+0000","last_fullsized":"2024-05-09T00:09:09.467190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:19.976265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:40.365037+0000","last_change":"2024-05-09T00:08:23.118803+0000","last_active":"2024-05-09T00:08:40.365037+0000","last_peered":"2024-05-09T00:08:40.365037+0000","last_clean":"2024-05-09T00:08:40.365037+0000","last_became_active":"2024-05-09T00:08:23.118432+0000","last_became_peered":"2024-05-09T00:08:23.118432+0000","last_unstale":"2024-05-09T00:08:40.365037+0000","last_undegraded":"2024-05-09T00:08:40.365037+0000","last_fullsized":"2024-05-09T00:08:40.365037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:50.253699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.059058+0000","last_change":"2024-05-09T00:08:26.054281+0000","last_active":"2024-05-09T00:08:28.059058+0000","last_peered":"2024-05-09T00:08:28.059058+0000","last_clean":"2024-05-09T00:08:28.059058+0000","last_became_active":"2024-05-09T00:08:26.053874+0000","last_became_peered":"2024-05-09T00:08:26.053874+0000","last_unstale":"2024-05-09T00:08:28.059058+0000","last_undegraded":"2024-05-09T00:08:28.059058+0000","last_fullsized":"2024-05-09T00:08:28.059058+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:11.566337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:40.365157+0000","last_change":"2024-05-09T00:08:23.017905+0000","last_active":"2024-05-09T00:08:40.365157+0000","last_peered":"2024-05-09T00:08:40.365157+0000","last_clean":"2024-05-09T00:08:40.365157+0000","last_became_active":"2024-05-09T00:08:23.017481+0000","last_became_peered":"2024-05-09T00:08:23.017481+0000","last_unstale":"2024-05-09T00:08:40.365157+0000","last_undegraded":"2024-05-09T00:08:40.365157+0000","last_fullsized":"2024-05-09T00:08:40.365157+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:40.096415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063090+0000","last_change":"2024-05-09T00:08:26.053392+0000","last_active":"2024-05-09T00:08:28.063090+0000","last_peered":"2024-05-09T00:08:28.063090+0000","last_clean":"2024-05-09T00:08:28.063090+0000","last_became_active":"2024-05-09T00:08:26.052628+0000","last_became_peered":"2024-05-09T00:08:26.052628+0000","last_unstale":"2024-05-09T00:08:28.063090+0000","last_undegraded":"2024-05-09T00:08:28.063090+0000","last_fullsized":"2024-05-09T00:08:28.063090+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:28.013955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.060294+0000","last_change":"2024-05-09T00:08:23.020786+0000","last_active":"2024-05-09T00:08:28.060294+0000","last_peered":"2024-05-09T00:08:28.060294+0000","last_clean":"2024-05-09T00:08:28.060294+0000","last_became_active":"2024-05-09T00:08:23.019949+0000","last_became_peered":"2024-05-09T00:08:23.019949+0000","last_unstale":"2024-05-09T00:08:28.060294+0000","last_undegraded":"2024-05-09T00:08:28.060294+0000","last_fullsized":"2024-05-09T00:08:28.060294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:18.366813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.061217+0000","last_change":"2024-05-09T00:08:26.047812+0000","last_active":"2024-05-09T00:08:28.061217+0000","last_peered":"2024-05-09T00:08:28.061217+0000","last_clean":"2024-05-09T00:08:28.061217+0000","last_became_active":"2024-05-09T00:08:26.047459+0000","last_became_peered":"2024-05-09T00:08:26.047459+0000","last_unstale":"2024-05-09T00:08:28.061217+0000","last_undegraded":"2024-05-09T00:08:28.061217+0000","last_fullsized":"2024-05-09T00:08:28.061217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.257831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:40.365280+0000","last_change":"2024-05-09T00:08:23.023800+0000","last_active":"2024-05-09T00:08:40.365280+0000","last_peered":"2024-05-09T00:08:40.365280+0000","last_clean":"2024-05-09T00:08:40.365280+0000","last_became_active":"2024-05-09T00:08:23.023387+0000","last_became_peered":"2024-05-09T00:08:23.023387+0000","last_unstale":"2024-05-09T00:08:40.365280+0000","last_undegraded":"2024-05-09T00:08:40.365280+0000","last_fullsized":"2024-05-09T00:08:40.365280+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:33.714683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.064621+0000","last_change":"2024-05-09T00:08:26.056981+0000","last_active":"2024-05-09T00:08:28.064621+0000","last_peered":"2024-05-09T00:08:28.064621+0000","last_clean":"2024-05-09T00:08:28.064621+0000","last_became_active":"2024-05-09T00:08:26.056605+0000","last_became_peered":"2024-05-09T00:08:26.056605+0000","last_unstale":"2024-05-09T00:08:28.064621+0000","last_undegraded":"2024-05-09T00:08:28.064621+0000","last_fullsized":"2024-05-09T00:08:28.064621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:32.897929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.379752+0000","last_change":"2024-05-09T00:08:23.120612+0000","last_active":"2024-05-09T00:08:41.379752+0000","last_peered":"2024-05-09T00:08:41.379752+0000","last_clean":"2024-05-09T00:08:41.379752+0000","last_became_active":"2024-05-09T00:08:23.120157+0000","last_became_peered":"2024-05-09T00:08:23.120157+0000","last_unstale":"2024-05-09T00:08:41.379752+0000","last_undegraded":"2024-05-09T00:08:41.379752+0000","last_fullsized":"2024-05-09T00:08:41.379752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:08.027214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063419+0000","last_change":"2024-05-09T00:08:26.061594+0000","last_active":"2024-05-09T00:08:28.063419+0000","last_peered":"2024-05-09T00:08:28.063419+0000","last_clean":"2024-05-09T00:08:28.063419+0000","last_became_active":"2024-05-09T00:08:26.061201+0000","last_became_peered":"2024-05-09T00:08:26.061201+0000","last_unstale":"2024-05-09T00:08:28.063419+0000","last_undegraded":"2024-05-09T00:08:28.063419+0000","last_fullsized":"2024-05-09T00:08:28.063419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:50.183648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.065439+0000","last_change":"2024-05-09T00:08:23.017693+0000","last_active":"2024-05-09T00:08:28.065439+0000","last_peered":"2024-05-09T00:08:28.065439+0000","last_clean":"2024-05-09T00:08:28.065439+0000","last_became_active":"2024-05-09T00:08:23.017211+0000","last_became_peered":"2024-05-09T00:08:23.017211+0000","last_unstale":"2024-05-09T00:08:28.065439+0000","last_undegraded":"2024-05-09T00:08:28.065439+0000","last_fullsized":"2024-05-09T00:08:28.065439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:53.569932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.880483+0000","last_change":"2024-05-09T00:08:26.063922+0000","last_active":"2024-05-09T00:08:28.880483+0000","last_peered":"2024-05-09T00:08:28.880483+0000","last_clean":"2024-05-09T00:08:28.880483+0000","last_became_active":"2024-05-09T00:08:26.063554+0000","last_became_peered":"2024-05-09T00:08:26.063554+0000","last_unstale":"2024-05-09T00:08:28.880483+0000","last_undegraded":"2024-05-09T00:08:28.880483+0000","last_fullsized":"2024-05-09T00:08:28.880483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:01.754688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.064971+0000","last_change":"2024-05-09T00:08:23.020367+0000","last_active":"2024-05-09T00:08:28.064971+0000","last_peered":"2024-05-09T00:08:28.064971+0000","last_clean":"2024-05-09T00:08:28.064971+0000","last_became_active":"2024-05-09T00:08:23.019801+0000","last_became_peered":"2024-05-09T00:08:23.019801+0000","last_unstale":"2024-05-09T00:08:28.064971+0000","last_undegraded":"2024-05-09T00:08:28.064971+0000","last_fullsized":"2024-05-09T00:08:28.064971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:14.936021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.065405+0000","last_change":"2024-05-09T00:08:26.063863+0000","last_active":"2024-05-09T00:08:28.065405+0000","last_peered":"2024-05-09T00:08:28.065405+0000","last_clean":"2024-05-09T00:08:28.065405+0000","last_became_active":"2024-05-09T00:08:26.061938+0000","last_became_peered":"2024-05-09T00:08:26.061938+0000","last_unstale":"2024-05-09T00:08:28.065405+0000","last_undegraded":"2024-05-09T00:08:28.065405+0000","last_fullsized":"2024-05-09T00:08:28.065405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:04.597991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063148+0000","last_change":"2024-05-09T00:08:26.050837+0000","last_active":"2024-05-09T00:08:28.063148+0000","last_peered":"2024-05-09T00:08:28.063148+0000","last_clean":"2024-05-09T00:08:28.063148+0000","last_became_active":"2024-05-09T00:08:26.050424+0000","last_became_peered":"2024-05-09T00:08:26.050424+0000","last_unstale":"2024-05-09T00:08:28.063148+0000","last_undegraded":"2024-05-09T00:08:28.063148+0000","last_fullsized":"2024-05-09T00:08:28.063148+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:45.134811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.060078+0000","last_change":"2024-05-09T00:08:26.058471+0000","last_active":"2024-05-09T00:08:28.060078+0000","last_peered":"2024-05-09T00:08:28.060078+0000","last_clean":"2024-05-09T00:08:28.060078+0000","last_became_active":"2024-05-09T00:08:26.057527+0000","last_became_peered":"2024-05-09T00:08:26.057527+0000","last_unstale":"2024-05-09T00:08:28.060078+0000","last_undegraded":"2024-05-09T00:08:28.060078+0000","last_fullsized":"2024-05-09T00:08:28.060078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:17.272199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.059669+0000","last_change":"2024-05-09T00:08:26.058012+0000","last_active":"2024-05-09T00:08:28.059669+0000","last_peered":"2024-05-09T00:08:28.059669+0000","last_clean":"2024-05-09T00:08:28.059669+0000","last_became_active":"2024-05-09T00:08:26.057282+0000","last_became_peered":"2024-05-09T00:08:26.057282+0000","last_unstale":"2024-05-09T00:08:28.059669+0000","last_undegraded":"2024-05-09T00:08:28.059669+0000","last_fullsized":"2024-05-09T00:08:28.059669+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:53.520593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.465725+0000","last_change":"2024-05-09T00:08:26.063331+0000","last_active":"2024-05-09T00:09:09.465725+0000","last_peered":"2024-05-09T00:09:09.465725+0000","last_clean":"2024-05-09T00:09:09.465725+0000","last_became_active":"2024-05-09T00:08:26.062866+0000","last_became_peered":"2024-05-09T00:08:26.062866+0000","last_unstale":"2024-05-09T00:09:09.465725+0000","last_undegraded":"2024-05-09T00:09:09.465725+0000","last_fullsized":"2024-05-09T00:09:09.465725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:10.387852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.044038+0000","last_change":"2024-05-09T00:08:26.063735+0000","last_active":"2024-05-09T00:08:28.044038+0000","last_peered":"2024-05-09T00:08:28.044038+0000","last_clean":"2024-05-09T00:08:28.044038+0000","last_became_active":"2024-05-09T00:08:26.063358+0000","last_became_peered":"2024-05-09T00:08:26.063358+0000","last_unstale":"2024-05-09T00:08:28.044038+0000","last_undegraded":"2024-05-09T00:08:28.044038+0000","last_fullsized":"2024-05-09T00:08:28.044038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:06.832071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063227+0000","last_change":"2024-05-09T00:08:26.052746+0000","last_active":"2024-05-09T00:08:28.063227+0000","last_peered":"2024-05-09T00:08:28.063227+0000","last_clean":"2024-05-09T00:08:28.063227+0000","last_became_active":"2024-05-09T00:08:26.051544+0000","last_became_peered":"2024-05-09T00:08:26.051544+0000","last_unstale":"2024-05-09T00:08:28.063227+0000","last_undegraded":"2024-05-09T00:08:28.063227+0000","last_fullsized":"2024-05-09T00:08:28.063227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:31.963268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.061398+0000","last_change":"2024-05-09T00:08:26.058075+0000","last_active":"2024-05-09T00:08:28.061398+0000","last_peered":"2024-05-09T00:08:28.061398+0000","last_clean":"2024-05-09T00:08:28.061398+0000","last_became_active":"2024-05-09T00:08:26.057349+0000","last_became_peered":"2024-05-09T00:08:26.057349+0000","last_unstale":"2024-05-09T00:08:28.061398+0000","last_undegraded":"2024-05-09T00:08:28.061398+0000","last_fullsized":"2024-05-09T00:08:28.061398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:59.322903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.060786+0000","last_change":"2024-05-09T00:08:26.056429+0000","last_active":"2024-05-09T00:08:28.060786+0000","last_peered":"2024-05-09T00:08:28.060786+0000","last_clean":"2024-05-09T00:08:28.060786+0000","last_became_active":"2024-05-09T00:08:26.054852+0000","last_became_peered":"2024-05-09T00:08:26.054852+0000","last_unstale":"2024-05-09T00:08:28.060786+0000","last_undegraded":"2024-05-09T00:08:28.060786+0000","last_fullsized":"2024-05-09T00:08:28.060786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:05.337636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.044323+0000","last_change":"2024-05-09T00:08:26.059135+0000","last_active":"2024-05-09T00:08:28.044323+0000","last_peered":"2024-05-09T00:08:28.044323+0000","last_clean":"2024-05-09T00:08:28.044323+0000","last_became_active":"2024-05-09T00:08:26.058770+0000","last_became_peered":"2024-05-09T00:08:26.058770+0000","last_unstale":"2024-05-09T00:08:28.044323+0000","last_undegraded":"2024-05-09T00:08:28.044323+0000","last_fullsized":"2024-05-09T00:08:28.044323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:52:40.959531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.061258+0000","last_change":"2024-05-09T00:08:26.054579+0000","last_active":"2024-05-09T00:08:28.061258+0000","last_peered":"2024-05-09T00:08:28.061258+0000","last_clean":"2024-05-09T00:08:28.061258+0000","last_became_active":"2024-05-09T00:08:26.053186+0000","last_became_peered":"2024-05-09T00:08:26.053186+0000","last_unstale":"2024-05-09T00:08:28.061258+0000","last_undegraded":"2024-05-09T00:08:28.061258+0000","last_fullsized":"2024-05-09T00:08:28.061258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:37.561691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.062036+0000","last_change":"2024-05-09T00:08:26.063608+0000","last_active":"2024-05-09T00:08:28.062036+0000","last_peered":"2024-05-09T00:08:28.062036+0000","last_clean":"2024-05-09T00:08:28.062036+0000","last_became_active":"2024-05-09T00:08:26.063078+0000","last_became_peered":"2024-05-09T00:08:26.063078+0000","last_unstale":"2024-05-09T00:08:28.062036+0000","last_undegraded":"2024-05-09T00:08:28.062036+0000","last_fullsized":"2024-05-09T00:08:28.062036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:06.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.043917+0000","last_change":"2024-05-09T00:08:26.055058+0000","last_active":"2024-05-09T00:08:28.043917+0000","last_peered":"2024-05-09T00:08:28.043917+0000","last_clean":"2024-05-09T00:08:28.043917+0000","last_became_active":"2024-05-09T00:08:26.054695+0000","last_became_peered":"2024-05-09T00:08:26.054695+0000","last_unstale":"2024-05-09T00:08:28.043917+0000","last_undegraded":"2024-05-09T00:08:28.043917+0000","last_fullsized":"2024-05-09T00:08:28.043917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:58.925990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.062669+0000","last_change":"2024-05-09T00:08:26.055366+0000","last_active":"2024-05-09T00:08:28.062669+0000","last_peered":"2024-05-09T00:08:28.062669+0000","last_clean":"2024-05-09T00:08:28.062669+0000","last_became_active":"2024-05-09T00:08:26.054904+0000","last_became_peered":"2024-05-09T00:08:26.054904+0000","last_unstale":"2024-05-09T00:08:28.062669+0000","last_undegraded":"2024-05-09T00:08:28.062669+0000","last_fullsized":"2024-05-09T00:08:28.062669+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:40.754878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285241,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59582,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":174,"num_read_kb":151,"num_write":161,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":178746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710743,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":408028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":1055481,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070497,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":856064,"data_stored":408264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266727,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":1000114,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":843776,"data_stored":407653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":463027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":864256,"data_stored":408924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":921600,"data_stored":458982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":407964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":847872,"data_stored":408481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":999407,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":55581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:09:12.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T00:09:13.213 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:13.596 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:13 smithi046 ceph-mon[30151]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:13.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:13 smithi169 ceph-mon[34777]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:13.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:13 smithi142 ceph-mon[34533]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:14.723 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:14 smithi046 ceph-mon[30151]: pgmap v312: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-09T00:09:14.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:14 smithi169 ceph-mon[34777]: pgmap v312: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-09T00:09:14.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:14 smithi142 ceph-mon[34533]: pgmap v312: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-09T00:09:14.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.991+0000 7f4eca0cc700 1 -- 172.21.15.46:0/3979917304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4107df0 msgr2=0x7f4ec4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:14.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.991+0000 7f4eca0cc700 1 --2- 172.21.15.46:0/3979917304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4107df0 0x7f4ec4108200 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4eac0099c0 tx=0x7f4eac009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:14.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.991+0000 7f4eca0cc700 1 -- 172.21.15.46:0/3979917304 shutdown_connections 2024-05-09T00:09:14.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.991+0000 7f4eca0cc700 1 --2- 172.21.15.46:0/3979917304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec4108740 0x7f4ec41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:14.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.991+0000 7f4eca0cc700 1 --2- 172.21.15.46:0/3979917304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec41034c0 0x7f4ec41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:14.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.991+0000 7f4eca0cc700 1 --2- 172.21.15.46:0/3979917304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4107df0 0x7f4ec4108200 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:14.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.991+0000 7f4eca0cc700 1 -- 172.21.15.46:0/3979917304 >> 172.21.15.46:0/3979917304 conn(0x7f4ec40fedc0 msgr2=0x7f4ec41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:14.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.992+0000 7f4eca0cc700 1 -- 172.21.15.46:0/3979917304 shutdown_connections 2024-05-09T00:09:14.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.992+0000 7f4eca0cc700 1 -- 172.21.15.46:0/3979917304 wait complete. 2024-05-09T00:09:14.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.992+0000 7f4eca0cc700 1 Processor -- start 2024-05-09T00:09:14.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4eca0cc700 1 -- start start 2024-05-09T00:09:14.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4eca0cc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec41034c0 0x7f4ec40ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:14.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4eca0cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4107df0 0x7f4ec40ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4eca0cc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec4108740 0x7f4ec40ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4eca0cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41133c0 con 0x7f4ec4107df0 2024-05-09T00:09:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4eca0cc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec4113240 con 0x7f4ec41034c0 2024-05-09T00:09:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4eca0cc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec4113540 con 0x7f4ec4108740 2024-05-09T00:09:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4ec2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4107df0 0x7f4ec40ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4ec2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4107df0 0x7f4ec40ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55578/0 (socket says 172.21.15.46:55578) 2024-05-09T00:09:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4ec2ffd700 1 -- 172.21.15.46:0/997351911 learned_addr learned my addr 172.21.15.46:0/997351911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4ec3fff700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec4108740 0x7f4ec40ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4ec37fe700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec41034c0 0x7f4ec40ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4ec2ffd700 1 -- 172.21.15.46:0/997351911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec4108740 msgr2=0x7f4ec40ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:14.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4ec2ffd700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec4108740 0x7f4ec40ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4ec2ffd700 1 -- 172.21.15.46:0/997351911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec41034c0 msgr2=0x7f4ec40ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.993+0000 7f4ec2ffd700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec41034c0 0x7f4ec40ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.994+0000 7f4ec2ffd700 1 -- 172.21.15.46:0/997351911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec41a91c0 con 0x7f4ec4107df0 2024-05-09T00:09:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.994+0000 7f4ec2ffd700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4107df0 0x7f4ec40ff7e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4eac006010 tx=0x7f4eac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.994+0000 7f4ec0ff9700 1 -- 172.21.15.46:0/997351911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac027070 con 0x7f4ec4107df0 2024-05-09T00:09:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.994+0000 7f4ec0ff9700 1 -- 172.21.15.46:0/997351911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eac003f20 con 0x7f4ec4107df0 2024-05-09T00:09:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.994+0000 7f4ec0ff9700 1 -- 172.21.15.46:0/997351911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac022cd0 con 0x7f4ec4107df0 2024-05-09T00:09:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.994+0000 7f4eca0cc700 1 -- 172.21.15.46:0/997351911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec41a9450 con 0x7f4ec4107df0 2024-05-09T00:09:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.994+0000 7f4eca0cc700 1 -- 172.21.15.46:0/997351911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec41a98a0 con 0x7f4ec4107df0 2024-05-09T00:09:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.995+0000 7f4eca0cc700 1 -- 172.21.15.46:0/997351911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec40512c0 con 0x7f4ec4107df0 2024-05-09T00:09:14.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.998+0000 7f4ec0ff9700 1 -- 172.21.15.46:0/997351911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4eac02a070 con 0x7f4ec4107df0 2024-05-09T00:09:14.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.998+0000 7f4ec0ff9700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4eb00b2910 0x7f4eb00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:14.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.999+0000 7f4ec37fe700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4eb00b2910 0x7f4eb00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:14.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.999+0000 7f4ec0ff9700 1 -- 172.21.15.46:0/997351911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4eac0e3650 con 0x7f4ec4107df0 2024-05-09T00:09:14.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:14.999+0000 7f4ec37fe700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4eb00b2910 0x7f4eb00b4e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009f20 tx=0x7f4eb40094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:14.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.002+0000 7f4ec0ff9700 1 -- 172.21.15.46:0/997351911 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4eac031080 con 0x7f4ec4107df0 2024-05-09T00:09:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.212+0000 7f4eca0cc700 1 -- 172.21.15.46:0/997351911 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ec4063c00 con 0x7f4eb00b2910 2024-05-09T00:09:15.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.219+0000 7f4ec0ff9700 1 -- 172.21.15.46:0/997351911 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258881 (secure 0 0 0) 0x7f4ec4063c00 con 0x7f4eb00b2910 2024-05-09T00:09:15.218 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:09:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.222+0000 7f4eca0cc700 1 -- 172.21.15.46:0/997351911 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4eb00b2910 msgr2=0x7f4eb00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.222+0000 7f4eca0cc700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4eb00b2910 0x7f4eb00b4e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009f20 tx=0x7f4eb40094f0 comp rx=0 tx=0).stop 2024-05-09T00:09:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.222+0000 7f4eca0cc700 1 -- 172.21.15.46:0/997351911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4107df0 msgr2=0x7f4ec40ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.222+0000 7f4eca0cc700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4107df0 0x7f4ec40ff7e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4eac006010 tx=0x7f4eac0047b0 comp rx=0 tx=0).stop 2024-05-09T00:09:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.222+0000 7f4eca0cc700 1 -- 172.21.15.46:0/997351911 shutdown_connections 2024-05-09T00:09:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.222+0000 7f4eca0cc700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4eb00b2910 0x7f4eb00b4e20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.222+0000 7f4eca0cc700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec4108740 0x7f4ec40ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.222+0000 7f4eca0cc700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec41034c0 0x7f4ec40ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.222+0000 7f4eca0cc700 1 --2- 172.21.15.46:0/997351911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ec4107df0 0x7f4ec40ff7e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.222+0000 7f4eca0cc700 1 -- 172.21.15.46:0/997351911 >> 172.21.15.46:0/997351911 conn(0x7f4ec40fedc0 msgr2=0x7f4ec406b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.223+0000 7f4eca0cc700 1 -- 172.21.15.46:0/997351911 shutdown_connections 2024-05-09T00:09:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:15.223+0000 7f4eca0cc700 1 -- 172.21.15.46:0/997351911 wait complete. 2024-05-09T00:09:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T00:09:15.917 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-05-09T00:09:13.292740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650184,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":260,"num_read_kb":215,"num_write":288,"num_write_kb":2257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":287,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372144,"kb_used_data":10476,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569680,"statfs":{"total":1151940427776,"available":1151559352320,"internally_reserved":0,"allocated":10727424,"data_stored":6925889,"data_compressed":137584,"data_compressed_allocated":176128,"data_compressed_original":1851392,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5395,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":21,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002101"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063574+0000","last_change":"2024-05-09T00:08:26.060455+0000","last_active":"2024-05-09T00:08:28.063574+0000","last_peered":"2024-05-09T00:08:28.063574+0000","last_clean":"2024-05-09T00:08:28.063574+0000","last_became_active":"2024-05-09T00:08:26.060068+0000","last_became_peered":"2024-05-09T00:08:26.060068+0000","last_unstale":"2024-05-09T00:08:28.063574+0000","last_undegraded":"2024-05-09T00:08:28.063574+0000","last_fullsized":"2024-05-09T00:08:28.063574+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:48.354188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.467081+0000","last_change":"2024-05-09T00:08:26.055737+0000","last_active":"2024-05-09T00:09:09.467081+0000","last_peered":"2024-05-09T00:09:09.467081+0000","last_clean":"2024-05-09T00:09:09.467081+0000","last_became_active":"2024-05-09T00:08:26.055354+0000","last_became_peered":"2024-05-09T00:08:26.055354+0000","last_unstale":"2024-05-09T00:09:09.467081+0000","last_undegraded":"2024-05-09T00:09:09.467081+0000","last_fullsized":"2024-05-09T00:09:09.467081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:54.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.466896+0000","last_change":"2024-05-09T00:08:26.050776+0000","last_active":"2024-05-09T00:09:09.466896+0000","last_peered":"2024-05-09T00:09:09.466896+0000","last_clean":"2024-05-09T00:09:09.466896+0000","last_became_active":"2024-05-09T00:08:26.050393+0000","last_became_peered":"2024-05-09T00:08:26.050393+0000","last_unstale":"2024-05-09T00:09:09.466896+0000","last_undegraded":"2024-05-09T00:09:09.466896+0000","last_fullsized":"2024-05-09T00:09:09.466896+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:30.819219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796411+0000","last_change":"2024-05-09T00:08:26.065946+0000","last_active":"2024-05-09T00:09:10.796411+0000","last_peered":"2024-05-09T00:09:10.796411+0000","last_clean":"2024-05-09T00:09:10.796411+0000","last_became_active":"2024-05-09T00:08:26.065498+0000","last_became_peered":"2024-05-09T00:08:26.065498+0000","last_unstale":"2024-05-09T00:09:10.796411+0000","last_undegraded":"2024-05-09T00:09:10.796411+0000","last_fullsized":"2024-05-09T00:09:10.796411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:04.444860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.059164+0000","last_change":"2024-05-09T00:08:26.051961+0000","last_active":"2024-05-09T00:08:28.059164+0000","last_peered":"2024-05-09T00:08:28.059164+0000","last_clean":"2024-05-09T00:08:28.059164+0000","last_became_active":"2024-05-09T00:08:26.051548+0000","last_became_peered":"2024-05-09T00:08:26.051548+0000","last_unstale":"2024-05-09T00:08:28.059164+0000","last_undegraded":"2024-05-09T00:08:28.059164+0000","last_fullsized":"2024-05-09T00:08:28.059164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:07.539117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.058865+0000","last_change":"2024-05-09T00:08:26.052012+0000","last_active":"2024-05-09T00:08:28.058865+0000","last_peered":"2024-05-09T00:08:28.058865+0000","last_clean":"2024-05-09T00:08:28.058865+0000","last_became_active":"2024-05-09T00:08:26.050886+0000","last_became_peered":"2024-05-09T00:08:26.050886+0000","last_unstale":"2024-05-09T00:08:28.058865+0000","last_undegraded":"2024-05-09T00:08:28.058865+0000","last_fullsized":"2024-05-09T00:08:28.058865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:14.651888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.064287+0000","last_change":"2024-05-09T00:08:26.058418+0000","last_active":"2024-05-09T00:08:28.064287+0000","last_peered":"2024-05-09T00:08:28.064287+0000","last_clean":"2024-05-09T00:08:28.064287+0000","last_became_active":"2024-05-09T00:08:26.058039+0000","last_became_peered":"2024-05-09T00:08:26.058039+0000","last_unstale":"2024-05-09T00:08:28.064287+0000","last_undegraded":"2024-05-09T00:08:28.064287+0000","last_fullsized":"2024-05-09T00:08:28.064287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:13.678123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467742+0000","last_change":"2024-05-09T00:08:26.057602+0000","last_active":"2024-05-09T00:09:10.467742+0000","last_peered":"2024-05-09T00:09:10.467742+0000","last_clean":"2024-05-09T00:09:10.467742+0000","last_became_active":"2024-05-09T00:08:26.057169+0000","last_became_peered":"2024-05-09T00:08:26.057169+0000","last_unstale":"2024-05-09T00:09:10.467742+0000","last_undegraded":"2024-05-09T00:09:10.467742+0000","last_fullsized":"2024-05-09T00:09:10.467742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:11.452083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063522+0000","last_change":"2024-05-09T00:08:26.067639+0000","last_active":"2024-05-09T00:08:28.063522+0000","last_peered":"2024-05-09T00:08:28.063522+0000","last_clean":"2024-05-09T00:08:28.063522+0000","last_became_active":"2024-05-09T00:08:26.067226+0000","last_became_peered":"2024-05-09T00:08:26.067226+0000","last_unstale":"2024-05-09T00:08:28.063522+0000","last_undegraded":"2024-05-09T00:08:28.063522+0000","last_fullsized":"2024-05-09T00:08:28.063522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:12.366500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.059491+0000","last_change":"2024-05-09T00:08:26.055234+0000","last_active":"2024-05-09T00:08:28.059491+0000","last_peered":"2024-05-09T00:08:28.059491+0000","last_clean":"2024-05-09T00:08:28.059491+0000","last_became_active":"2024-05-09T00:08:26.054851+0000","last_became_peered":"2024-05-09T00:08:26.054851+0000","last_unstale":"2024-05-09T00:08:28.059491+0000","last_undegraded":"2024-05-09T00:08:28.059491+0000","last_fullsized":"2024-05-09T00:08:28.059491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:54:52.121329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063075+0000","last_change":"2024-05-09T00:08:26.063375+0000","last_active":"2024-05-09T00:08:28.063075+0000","last_peered":"2024-05-09T00:08:28.063075+0000","last_clean":"2024-05-09T00:08:28.063075+0000","last_became_active":"2024-05-09T00:08:26.062962+0000","last_became_peered":"2024-05-09T00:08:26.062962+0000","last_unstale":"2024-05-09T00:08:28.063075+0000","last_undegraded":"2024-05-09T00:08:28.063075+0000","last_fullsized":"2024-05-09T00:08:28.063075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:05.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796257+0000","last_change":"2024-05-09T00:08:26.062034+0000","last_active":"2024-05-09T00:09:10.796257+0000","last_peered":"2024-05-09T00:09:10.796257+0000","last_clean":"2024-05-09T00:09:10.796257+0000","last_became_active":"2024-05-09T00:08:26.060376+0000","last_became_peered":"2024-05-09T00:08:26.060376+0000","last_unstale":"2024-05-09T00:09:10.796257+0000","last_undegraded":"2024-05-09T00:09:10.796257+0000","last_fullsized":"2024-05-09T00:09:10.796257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:15.830024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.043955+0000","last_change":"2024-05-09T00:08:26.053227+0000","last_active":"2024-05-09T00:08:28.043955+0000","last_peered":"2024-05-09T00:08:28.043955+0000","last_clean":"2024-05-09T00:08:28.043955+0000","last_became_active":"2024-05-09T00:08:26.052878+0000","last_became_peered":"2024-05-09T00:08:26.052878+0000","last_unstale":"2024-05-09T00:08:28.043955+0000","last_undegraded":"2024-05-09T00:08:28.043955+0000","last_fullsized":"2024-05-09T00:08:28.043955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:24.385026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.466484+0000","last_change":"2024-05-09T00:08:26.053041+0000","last_active":"2024-05-09T00:09:09.466484+0000","last_peered":"2024-05-09T00:09:09.466484+0000","last_clean":"2024-05-09T00:09:09.466484+0000","last_became_active":"2024-05-09T00:08:26.052677+0000","last_became_peered":"2024-05-09T00:08:26.052677+0000","last_unstale":"2024-05-09T00:09:09.466484+0000","last_undegraded":"2024-05-09T00:09:09.466484+0000","last_fullsized":"2024-05-09T00:09:09.466484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:24.478270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.465389+0000","last_change":"2024-05-09T00:08:26.063391+0000","last_active":"2024-05-09T00:09:09.465389+0000","last_peered":"2024-05-09T00:09:09.465389+0000","last_clean":"2024-05-09T00:09:09.465389+0000","last_became_active":"2024-05-09T00:08:26.063033+0000","last_became_peered":"2024-05-09T00:08:26.063033+0000","last_unstale":"2024-05-09T00:09:09.465389+0000","last_undegraded":"2024-05-09T00:09:09.465389+0000","last_fullsized":"2024-05-09T00:09:09.465389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:41.900453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.062036+0000","last_change":"2024-05-09T00:08:26.065018+0000","last_active":"2024-05-09T00:08:28.062036+0000","last_peered":"2024-05-09T00:08:28.062036+0000","last_clean":"2024-05-09T00:08:28.062036+0000","last_became_active":"2024-05-09T00:08:26.064642+0000","last_became_peered":"2024-05-09T00:08:26.064642+0000","last_unstale":"2024-05-09T00:08:28.062036+0000","last_undegraded":"2024-05-09T00:08:28.062036+0000","last_fullsized":"2024-05-09T00:08:28.062036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:02.403704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063982+0000","last_change":"2024-05-09T00:08:26.049770+0000","last_active":"2024-05-09T00:08:28.063982+0000","last_peered":"2024-05-09T00:08:28.063982+0000","last_clean":"2024-05-09T00:08:28.063982+0000","last_became_active":"2024-05-09T00:08:26.048555+0000","last_became_peered":"2024-05-09T00:08:26.048555+0000","last_unstale":"2024-05-09T00:08:28.063982+0000","last_undegraded":"2024-05-09T00:08:28.063982+0000","last_fullsized":"2024-05-09T00:08:28.063982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:16.009200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469538+0000","last_change":"2024-05-09T00:08:26.048587+0000","last_active":"2024-05-09T00:09:10.469538+0000","last_peered":"2024-05-09T00:09:10.469538+0000","last_clean":"2024-05-09T00:09:10.469538+0000","last_became_active":"2024-05-09T00:08:26.048148+0000","last_became_peered":"2024-05-09T00:08:26.048148+0000","last_unstale":"2024-05-09T00:09:10.469538+0000","last_undegraded":"2024-05-09T00:09:10.469538+0000","last_fullsized":"2024-05-09T00:09:10.469538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:42.113829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473434+0000","last_change":"2024-05-09T00:08:26.053607+0000","last_active":"2024-05-09T00:09:10.473434+0000","last_peered":"2024-05-09T00:09:10.473434+0000","last_clean":"2024-05-09T00:09:10.473434+0000","last_became_active":"2024-05-09T00:08:26.053212+0000","last_became_peered":"2024-05-09T00:08:26.053212+0000","last_unstale":"2024-05-09T00:09:10.473434+0000","last_undegraded":"2024-05-09T00:09:10.473434+0000","last_fullsized":"2024-05-09T00:09:10.473434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:32.937049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469272+0000","last_change":"2024-05-09T00:08:23.024481+0000","last_active":"2024-05-09T00:09:10.469272+0000","last_peered":"2024-05-09T00:09:10.469272+0000","last_clean":"2024-05-09T00:09:10.469272+0000","last_became_active":"2024-05-09T00:08:23.024132+0000","last_became_peered":"2024-05-09T00:08:23.024132+0000","last_unstale":"2024-05-09T00:09:10.469272+0000","last_undegraded":"2024-05-09T00:09:10.469272+0000","last_fullsized":"2024-05-09T00:09:10.469272+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:11.188505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468045+0000","last_change":"2024-05-09T00:08:26.054772+0000","last_active":"2024-05-09T00:09:10.468045+0000","last_peered":"2024-05-09T00:09:10.468045+0000","last_clean":"2024-05-09T00:09:10.468045+0000","last_became_active":"2024-05-09T00:08:26.054433+0000","last_became_peered":"2024-05-09T00:08:26.054433+0000","last_unstale":"2024-05-09T00:09:10.468045+0000","last_undegraded":"2024-05-09T00:09:10.468045+0000","last_fullsized":"2024-05-09T00:09:10.468045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:44.283240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468325+0000","last_change":"2024-05-09T00:08:23.118778+0000","last_active":"2024-05-09T00:09:10.468325+0000","last_peered":"2024-05-09T00:09:10.468325+0000","last_clean":"2024-05-09T00:09:10.468325+0000","last_became_active":"2024-05-09T00:08:23.118377+0000","last_became_peered":"2024-05-09T00:08:23.118377+0000","last_unstale":"2024-05-09T00:09:10.468325+0000","last_undegraded":"2024-05-09T00:09:10.468325+0000","last_fullsized":"2024-05-09T00:09:10.468325+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:51.352442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.466418+0000","last_change":"2024-05-09T00:08:26.059953+0000","last_active":"2024-05-09T00:09:09.466418+0000","last_peered":"2024-05-09T00:09:09.466418+0000","last_clean":"2024-05-09T00:09:09.466418+0000","last_became_active":"2024-05-09T00:08:26.059571+0000","last_became_peered":"2024-05-09T00:08:26.059571+0000","last_unstale":"2024-05-09T00:09:09.466418+0000","last_undegraded":"2024-05-09T00:09:09.466418+0000","last_fullsized":"2024-05-09T00:09:09.466418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:48.056798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.378455+0000","last_change":"2024-05-09T00:08:23.021244+0000","last_active":"2024-05-09T00:08:41.378455+0000","last_peered":"2024-05-09T00:08:41.378455+0000","last_clean":"2024-05-09T00:08:41.378455+0000","last_became_active":"2024-05-09T00:08:23.020597+0000","last_became_peered":"2024-05-09T00:08:23.020597+0000","last_unstale":"2024-05-09T00:08:41.378455+0000","last_undegraded":"2024-05-09T00:08:41.378455+0000","last_fullsized":"2024-05-09T00:08:41.378455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.654261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.062710+0000","last_change":"2024-05-09T00:08:26.059834+0000","last_active":"2024-05-09T00:08:28.062710+0000","last_peered":"2024-05-09T00:08:28.062710+0000","last_clean":"2024-05-09T00:08:28.062710+0000","last_became_active":"2024-05-09T00:08:26.059349+0000","last_became_peered":"2024-05-09T00:08:26.059349+0000","last_unstale":"2024-05-09T00:08:28.062710+0000","last_undegraded":"2024-05-09T00:08:28.062710+0000","last_fullsized":"2024-05-09T00:08:28.062710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:42.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.066576+0000","last_change":"2024-05-09T00:08:23.019568+0000","last_active":"2024-05-09T00:08:28.066576+0000","last_peered":"2024-05-09T00:08:28.066576+0000","last_clean":"2024-05-09T00:08:28.066576+0000","last_became_active":"2024-05-09T00:08:23.019149+0000","last_became_peered":"2024-05-09T00:08:23.019149+0000","last_unstale":"2024-05-09T00:08:28.066576+0000","last_undegraded":"2024-05-09T00:08:28.066576+0000","last_fullsized":"2024-05-09T00:08:28.066576+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:44.569911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469966+0000","last_change":"2024-05-09T00:08:26.061308+0000","last_active":"2024-05-09T00:09:10.469966+0000","last_peered":"2024-05-09T00:09:10.469966+0000","last_clean":"2024-05-09T00:09:10.469966+0000","last_became_active":"2024-05-09T00:08:26.060973+0000","last_became_peered":"2024-05-09T00:08:26.060973+0000","last_unstale":"2024-05-09T00:09:10.469966+0000","last_undegraded":"2024-05-09T00:09:10.469966+0000","last_fullsized":"2024-05-09T00:09:10.469966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:16.845074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":60,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469466+0000","last_change":"2024-05-09T00:08:23.020886+0000","last_active":"2024-05-09T00:09:10.469466+0000","last_peered":"2024-05-09T00:09:10.469466+0000","last_clean":"2024-05-09T00:09:10.469466+0000","last_became_active":"2024-05-09T00:08:23.020530+0000","last_became_peered":"2024-05-09T00:08:23.020530+0000","last_unstale":"2024-05-09T00:09:10.469466+0000","last_undegraded":"2024-05-09T00:09:10.469466+0000","last_fullsized":"2024-05-09T00:09:10.469466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:34.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.059315+0000","last_change":"2024-05-09T00:08:26.051018+0000","last_active":"2024-05-09T00:08:28.059315+0000","last_peered":"2024-05-09T00:08:28.059315+0000","last_clean":"2024-05-09T00:08:28.059315+0000","last_became_active":"2024-05-09T00:08:26.050495+0000","last_became_peered":"2024-05-09T00:08:26.050495+0000","last_unstale":"2024-05-09T00:08:28.059315+0000","last_undegraded":"2024-05-09T00:08:28.059315+0000","last_fullsized":"2024-05-09T00:08:28.059315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:07.862339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.378509+0000","last_change":"2024-05-09T00:08:23.021187+0000","last_active":"2024-05-09T00:08:41.378509+0000","last_peered":"2024-05-09T00:08:41.378509+0000","last_clean":"2024-05-09T00:08:41.378509+0000","last_became_active":"2024-05-09T00:08:23.020770+0000","last_became_peered":"2024-05-09T00:08:23.020770+0000","last_unstale":"2024-05-09T00:08:41.378509+0000","last_undegraded":"2024-05-09T00:08:41.378509+0000","last_fullsized":"2024-05-09T00:08:41.378509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:08.992770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.797838+0000","last_change":"2024-05-09T00:09:10.797392+0000","last_active":"2024-05-09T00:09:10.797838+0000","last_peered":"2024-05-09T00:09:10.797838+0000","last_clean":"2024-05-09T00:09:10.797838+0000","last_became_active":"2024-05-09T00:09:10.796969+0000","last_became_peered":"2024-05-09T00:09:10.796969+0000","last_unstale":"2024-05-09T00:09:10.797838+0000","last_undegraded":"2024-05-09T00:09:10.797838+0000","last_fullsized":"2024-05-09T00:09:10.797838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:18.852678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.377392+0000","last_change":"2024-05-09T00:08:23.012226+0000","last_active":"2024-05-09T00:08:41.377392+0000","last_peered":"2024-05-09T00:08:41.377392+0000","last_clean":"2024-05-09T00:08:41.377392+0000","last_became_active":"2024-05-09T00:08:23.011781+0000","last_became_peered":"2024-05-09T00:08:23.011781+0000","last_unstale":"2024-05-09T00:08:41.377392+0000","last_undegraded":"2024-05-09T00:08:41.377392+0000","last_fullsized":"2024-05-09T00:08:41.377392+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:44.452206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467563+0000","last_change":"2024-05-09T00:08:26.043035+0000","last_active":"2024-05-09T00:09:10.467563+0000","last_peered":"2024-05-09T00:09:10.467563+0000","last_clean":"2024-05-09T00:09:10.467563+0000","last_became_active":"2024-05-09T00:08:26.042686+0000","last_became_peered":"2024-05-09T00:08:26.042686+0000","last_unstale":"2024-05-09T00:09:10.467563+0000","last_undegraded":"2024-05-09T00:09:10.467563+0000","last_fullsized":"2024-05-09T00:09:10.467563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:24.511309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469042+0000","last_change":"2024-05-09T00:08:23.022017+0000","last_active":"2024-05-09T00:09:10.469042+0000","last_peered":"2024-05-09T00:09:10.469042+0000","last_clean":"2024-05-09T00:09:10.469042+0000","last_became_active":"2024-05-09T00:08:23.021540+0000","last_became_peered":"2024-05-09T00:08:23.021540+0000","last_unstale":"2024-05-09T00:09:10.469042+0000","last_undegraded":"2024-05-09T00:09:10.469042+0000","last_fullsized":"2024-05-09T00:09:10.469042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:50.773211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469091+0000","last_change":"2024-05-09T00:08:26.048039+0000","last_active":"2024-05-09T00:09:10.469091+0000","last_peered":"2024-05-09T00:09:10.469091+0000","last_clean":"2024-05-09T00:09:10.469091+0000","last_became_active":"2024-05-09T00:08:26.047695+0000","last_became_peered":"2024-05-09T00:08:26.047695+0000","last_unstale":"2024-05-09T00:09:10.469091+0000","last_undegraded":"2024-05-09T00:09:10.469091+0000","last_fullsized":"2024-05-09T00:09:10.469091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:27.363847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:09:07.430191+0000","last_change":"2024-05-09T00:08:23.019565+0000","last_active":"2024-05-09T00:09:07.430191+0000","last_peered":"2024-05-09T00:09:07.430191+0000","last_clean":"2024-05-09T00:09:07.430191+0000","last_became_active":"2024-05-09T00:08:23.019113+0000","last_became_peered":"2024-05-09T00:08:23.019113+0000","last_unstale":"2024-05-09T00:09:07.430191+0000","last_undegraded":"2024-05-09T00:09:07.430191+0000","last_fullsized":"2024-05-09T00:09:07.430191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:45.578934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063276+0000","last_change":"2024-05-09T00:08:26.058869+0000","last_active":"2024-05-09T00:08:28.063276+0000","last_peered":"2024-05-09T00:08:28.063276+0000","last_clean":"2024-05-09T00:08:28.063276+0000","last_became_active":"2024-05-09T00:08:26.058482+0000","last_became_peered":"2024-05-09T00:08:26.058482+0000","last_unstale":"2024-05-09T00:08:28.063276+0000","last_undegraded":"2024-05-09T00:08:28.063276+0000","last_fullsized":"2024-05-09T00:08:28.063276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:33.252208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:40.365445+0000","last_change":"2024-05-09T00:08:23.024425+0000","last_active":"2024-05-09T00:08:40.365445+0000","last_peered":"2024-05-09T00:08:40.365445+0000","last_clean":"2024-05-09T00:08:40.365445+0000","last_became_active":"2024-05-09T00:08:23.023991+0000","last_became_peered":"2024-05-09T00:08:23.023991+0000","last_unstale":"2024-05-09T00:08:40.365445+0000","last_undegraded":"2024-05-09T00:08:40.365445+0000","last_fullsized":"2024-05-09T00:08:40.365445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:47.907851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"peering","last_fresh":"2024-05-09T00:08:29.623924+0000","last_change":"2024-05-09T00:09:09.467043+0000","last_active":"2024-05-09T00:09:09.467043+0000","last_peered":"2024-05-09T00:08:29.623924+0000","last_clean":"2024-05-09T00:08:29.623924+0000","last_became_active":"2024-05-09T00:08:26.063306+0000","last_became_peered":"2024-05-09T00:08:26.063306+0000","last_unstale":"2024-05-09T00:08:29.623924+0000","last_undegraded":"2024-05-09T00:08:29.623924+0000","last_fullsized":"2024-05-09T00:08:29.623924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470120+0000","last_change":"2024-05-09T00:08:23.118516+0000","last_active":"2024-05-09T00:09:10.470120+0000","last_peered":"2024-05-09T00:09:10.470120+0000","last_clean":"2024-05-09T00:09:10.470120+0000","last_became_active":"2024-05-09T00:08:23.118147+0000","last_became_peered":"2024-05-09T00:08:23.118147+0000","last_unstale":"2024-05-09T00:09:10.470120+0000","last_undegraded":"2024-05-09T00:09:10.470120+0000","last_fullsized":"2024-05-09T00:09:10.470120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:24.205818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.062923+0000","last_change":"2024-05-09T00:08:26.067063+0000","last_active":"2024-05-09T00:08:28.062923+0000","last_peered":"2024-05-09T00:08:28.062923+0000","last_clean":"2024-05-09T00:08:28.062923+0000","last_became_active":"2024-05-09T00:08:26.066440+0000","last_became_peered":"2024-05-09T00:08:26.066440+0000","last_unstale":"2024-05-09T00:08:28.062923+0000","last_undegraded":"2024-05-09T00:08:28.062923+0000","last_fullsized":"2024-05-09T00:08:28.062923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:48.648485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'28","reported_seq":108,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:09:08.433999+0000","last_change":"2024-05-09T00:08:23.020757+0000","last_active":"2024-05-09T00:09:08.433999+0000","last_peered":"2024-05-09T00:09:08.433999+0000","last_clean":"2024-05-09T00:09:08.433999+0000","last_became_active":"2024-05-09T00:08:23.020355+0000","last_became_peered":"2024-05-09T00:08:23.020355+0000","last_unstale":"2024-05-09T00:09:08.433999+0000","last_undegraded":"2024-05-09T00:09:08.433999+0000","last_fullsized":"2024-05-09T00:09:08.433999+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:00.605691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55581,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":58,"num_read_kb":82,"num_write":29,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.059488+0000","last_change":"2024-05-09T00:08:26.057402+0000","last_active":"2024-05-09T00:08:28.059488+0000","last_peered":"2024-05-09T00:08:28.059488+0000","last_clean":"2024-05-09T00:08:28.059488+0000","last_became_active":"2024-05-09T00:08:26.056974+0000","last_became_peered":"2024-05-09T00:08:26.056974+0000","last_unstale":"2024-05-09T00:08:28.059488+0000","last_undegraded":"2024-05-09T00:08:28.059488+0000","last_fullsized":"2024-05-09T00:08:28.059488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:47.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473890+0000","last_change":"2024-05-09T00:08:23.020208+0000","last_active":"2024-05-09T00:09:10.473890+0000","last_peered":"2024-05-09T00:09:10.473890+0000","last_clean":"2024-05-09T00:09:10.473890+0000","last_became_active":"2024-05-09T00:08:23.019809+0000","last_became_peered":"2024-05-09T00:08:23.019809+0000","last_unstale":"2024-05-09T00:09:10.473890+0000","last_undegraded":"2024-05-09T00:09:10.473890+0000","last_fullsized":"2024-05-09T00:09:10.473890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:47.900731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.043928+0000","last_change":"2024-05-09T00:08:26.054273+0000","last_active":"2024-05-09T00:08:28.043928+0000","last_peered":"2024-05-09T00:08:28.043928+0000","last_clean":"2024-05-09T00:08:28.043928+0000","last_became_active":"2024-05-09T00:08:26.053840+0000","last_became_peered":"2024-05-09T00:08:26.053840+0000","last_unstale":"2024-05-09T00:08:28.043928+0000","last_undegraded":"2024-05-09T00:08:28.043928+0000","last_fullsized":"2024-05-09T00:08:28.043928+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:27.128582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469240+0000","last_change":"2024-05-09T00:08:23.021594+0000","last_active":"2024-05-09T00:09:10.469240+0000","last_peered":"2024-05-09T00:09:10.469240+0000","last_clean":"2024-05-09T00:09:10.469240+0000","last_became_active":"2024-05-09T00:08:23.021152+0000","last_became_peered":"2024-05-09T00:08:23.021152+0000","last_unstale":"2024-05-09T00:09:10.469240+0000","last_undegraded":"2024-05-09T00:09:10.469240+0000","last_fullsized":"2024-05-09T00:09:10.469240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:58.505781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063461+0000","last_change":"2024-05-09T00:08:26.064464+0000","last_active":"2024-05-09T00:08:28.063461+0000","last_peered":"2024-05-09T00:08:28.063461+0000","last_clean":"2024-05-09T00:08:28.063461+0000","last_became_active":"2024-05-09T00:08:26.064000+0000","last_became_peered":"2024-05-09T00:08:26.064000+0000","last_unstale":"2024-05-09T00:08:28.063461+0000","last_undegraded":"2024-05-09T00:08:28.063461+0000","last_fullsized":"2024-05-09T00:08:28.063461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:01.455979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'4","reported_seq":62,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:11.436334+0000","last_change":"2024-05-09T00:08:23.020823+0000","last_active":"2024-05-09T00:09:11.436334+0000","last_peered":"2024-05-09T00:09:11.436334+0000","last_clean":"2024-05-09T00:09:11.436334+0000","last_became_active":"2024-05-09T00:08:23.020412+0000","last_became_peered":"2024-05-09T00:08:23.020412+0000","last_unstale":"2024-05-09T00:09:11.436334+0000","last_undegraded":"2024-05-09T00:09:11.436334+0000","last_fullsized":"2024-05-09T00:09:11.436334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:05.457452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063005+0000","last_change":"2024-05-09T00:08:26.057092+0000","last_active":"2024-05-09T00:08:28.063005+0000","last_peered":"2024-05-09T00:08:28.063005+0000","last_clean":"2024-05-09T00:08:28.063005+0000","last_became_active":"2024-05-09T00:08:26.056723+0000","last_became_peered":"2024-05-09T00:08:26.056723+0000","last_unstale":"2024-05-09T00:08:28.063005+0000","last_undegraded":"2024-05-09T00:08:28.063005+0000","last_fullsized":"2024-05-09T00:08:28.063005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:15.929284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.380913+0000","last_change":"2024-05-09T00:08:23.118857+0000","last_active":"2024-05-09T00:08:41.380913+0000","last_peered":"2024-05-09T00:08:41.380913+0000","last_clean":"2024-05-09T00:08:41.380913+0000","last_became_active":"2024-05-09T00:08:23.118433+0000","last_became_peered":"2024-05-09T00:08:23.118433+0000","last_unstale":"2024-05-09T00:08:41.380913+0000","last_undegraded":"2024-05-09T00:08:41.380913+0000","last_fullsized":"2024-05-09T00:08:41.380913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:10.617987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469212+0000","last_change":"2024-05-09T00:08:26.066911+0000","last_active":"2024-05-09T00:09:10.469212+0000","last_peered":"2024-05-09T00:09:10.469212+0000","last_clean":"2024-05-09T00:09:10.469212+0000","last_became_active":"2024-05-09T00:08:26.066548+0000","last_became_peered":"2024-05-09T00:08:26.066548+0000","last_unstale":"2024-05-09T00:09:10.469212+0000","last_undegraded":"2024-05-09T00:09:10.469212+0000","last_fullsized":"2024-05-09T00:09:10.469212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:32.391390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:42.373469+0000","last_change":"2024-05-09T00:08:23.019829+0000","last_active":"2024-05-09T00:08:42.373469+0000","last_peered":"2024-05-09T00:08:42.373469+0000","last_clean":"2024-05-09T00:08:42.373469+0000","last_became_active":"2024-05-09T00:08:23.019336+0000","last_became_peered":"2024-05-09T00:08:23.019336+0000","last_unstale":"2024-05-09T00:08:42.373469+0000","last_undegraded":"2024-05-09T00:08:42.373469+0000","last_fullsized":"2024-05-09T00:08:42.373469+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:33.802738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063102+0000","last_change":"2024-05-09T00:08:26.055975+0000","last_active":"2024-05-09T00:08:28.063102+0000","last_peered":"2024-05-09T00:08:28.063102+0000","last_clean":"2024-05-09T00:08:28.063102+0000","last_became_active":"2024-05-09T00:08:26.055595+0000","last_became_peered":"2024-05-09T00:08:26.055595+0000","last_unstale":"2024-05-09T00:08:28.063102+0000","last_undegraded":"2024-05-09T00:08:28.063102+0000","last_fullsized":"2024-05-09T00:08:28.063102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:20.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469371+0000","last_change":"2024-05-09T00:08:23.120353+0000","last_active":"2024-05-09T00:09:10.469371+0000","last_peered":"2024-05-09T00:09:10.469371+0000","last_clean":"2024-05-09T00:09:10.469371+0000","last_became_active":"2024-05-09T00:08:23.120004+0000","last_became_peered":"2024-05-09T00:08:23.120004+0000","last_unstale":"2024-05-09T00:09:10.469371+0000","last_undegraded":"2024-05-09T00:09:10.469371+0000","last_fullsized":"2024-05-09T00:09:10.469371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:12.168671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.663335+0000","last_change":"2024-05-09T00:08:26.046554+0000","last_active":"2024-05-09T00:08:28.663335+0000","last_peered":"2024-05-09T00:08:28.663335+0000","last_clean":"2024-05-09T00:08:28.663335+0000","last_became_active":"2024-05-09T00:08:26.045924+0000","last_became_peered":"2024-05-09T00:08:26.045924+0000","last_unstale":"2024-05-09T00:08:28.663335+0000","last_undegraded":"2024-05-09T00:08:28.663335+0000","last_fullsized":"2024-05-09T00:08:28.663335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:01.208980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.064602+0000","last_change":"2024-05-09T00:08:23.019778+0000","last_active":"2024-05-09T00:08:28.064602+0000","last_peered":"2024-05-09T00:08:28.064602+0000","last_clean":"2024-05-09T00:08:28.064602+0000","last_became_active":"2024-05-09T00:08:23.019400+0000","last_became_peered":"2024-05-09T00:08:23.019400+0000","last_unstale":"2024-05-09T00:08:28.064602+0000","last_undegraded":"2024-05-09T00:08:28.064602+0000","last_fullsized":"2024-05-09T00:08:28.064602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:06.883167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.058852+0000","last_change":"2024-05-09T00:08:26.055200+0000","last_active":"2024-05-09T00:08:28.058852+0000","last_peered":"2024-05-09T00:08:28.058852+0000","last_clean":"2024-05-09T00:08:28.058852+0000","last_became_active":"2024-05-09T00:08:26.054839+0000","last_became_peered":"2024-05-09T00:08:26.054839+0000","last_unstale":"2024-05-09T00:08:28.058852+0000","last_undegraded":"2024-05-09T00:08:28.058852+0000","last_fullsized":"2024-05-09T00:08:28.058852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:31:40.473061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.476484+0000","last_change":"2024-05-09T00:09:10.476484+0000","last_active":"2024-05-09T00:09:10.476484+0000","last_peered":"2024-05-09T00:09:10.476484+0000","last_clean":"2024-05-09T00:09:10.476484+0000","last_became_active":"2024-05-09T00:09:10.476001+0000","last_became_peered":"2024-05-09T00:09:10.476001+0000","last_unstale":"2024-05-09T00:09:10.476484+0000","last_undegraded":"2024-05-09T00:09:10.476484+0000","last_fullsized":"2024-05-09T00:09:10.476484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:32.018749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:09:07.429459+0000","last_change":"2024-05-09T00:08:23.118322+0000","last_active":"2024-05-09T00:09:07.429459+0000","last_peered":"2024-05-09T00:09:07.429459+0000","last_clean":"2024-05-09T00:09:07.429459+0000","last_became_active":"2024-05-09T00:08:23.117943+0000","last_became_peered":"2024-05-09T00:08:23.117943+0000","last_unstale":"2024-05-09T00:09:07.429459+0000","last_undegraded":"2024-05-09T00:09:07.429459+0000","last_fullsized":"2024-05-09T00:09:07.429459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:07.890437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467948+0000","last_change":"2024-05-09T00:08:26.052294+0000","last_active":"2024-05-09T00:09:10.467948+0000","last_peered":"2024-05-09T00:09:10.467948+0000","last_clean":"2024-05-09T00:09:10.467948+0000","last_became_active":"2024-05-09T00:08:26.051926+0000","last_became_peered":"2024-05-09T00:08:26.051926+0000","last_unstale":"2024-05-09T00:09:10.467948+0000","last_undegraded":"2024-05-09T00:09:10.467948+0000","last_fullsized":"2024-05-09T00:09:10.467948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:53.651401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796281+0000","last_change":"2024-05-09T00:08:23.015007+0000","last_active":"2024-05-09T00:09:10.796281+0000","last_peered":"2024-05-09T00:09:10.796281+0000","last_clean":"2024-05-09T00:09:10.796281+0000","last_became_active":"2024-05-09T00:08:23.014545+0000","last_became_peered":"2024-05-09T00:08:23.014545+0000","last_unstale":"2024-05-09T00:09:10.796281+0000","last_undegraded":"2024-05-09T00:09:10.796281+0000","last_fullsized":"2024-05-09T00:09:10.796281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:19.850200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063181+0000","last_change":"2024-05-09T00:07:00.733233+0000","last_active":"2024-05-09T00:08:28.063181+0000","last_peered":"2024-05-09T00:08:28.063181+0000","last_clean":"2024-05-09T00:08:28.063181+0000","last_became_active":"2024-05-09T00:07:00.727365+0000","last_became_peered":"2024-05-09T00:07:00.727365+0000","last_unstale":"2024-05-09T00:08:28.063181+0000","last_undegraded":"2024-05-09T00:08:28.063181+0000","last_fullsized":"2024-05-09T00:08:28.063181+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_clean_scrub_stamp":"2024-05-09T00:03:33.285220+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:08.597290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.043602+0000","last_change":"2024-05-09T00:08:26.064000+0000","last_active":"2024-05-09T00:08:28.043602+0000","last_peered":"2024-05-09T00:08:28.043602+0000","last_clean":"2024-05-09T00:08:28.043602+0000","last_became_active":"2024-05-09T00:08:26.063636+0000","last_became_peered":"2024-05-09T00:08:26.063636+0000","last_unstale":"2024-05-09T00:08:28.043602+0000","last_undegraded":"2024-05-09T00:08:28.043602+0000","last_fullsized":"2024-05-09T00:08:28.043602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:59.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:41.380397+0000","last_change":"2024-05-09T00:08:23.023459+0000","last_active":"2024-05-09T00:08:41.380397+0000","last_peered":"2024-05-09T00:08:41.380397+0000","last_clean":"2024-05-09T00:08:41.380397+0000","last_became_active":"2024-05-09T00:08:23.023089+0000","last_became_peered":"2024-05-09T00:08:23.023089+0000","last_unstale":"2024-05-09T00:08:41.380397+0000","last_undegraded":"2024-05-09T00:08:41.380397+0000","last_fullsized":"2024-05-09T00:08:41.380397+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:41.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:40.364231+0000","last_change":"2024-05-09T00:08:23.120566+0000","last_active":"2024-05-09T00:08:40.364231+0000","last_peered":"2024-05-09T00:08:40.364231+0000","last_clean":"2024-05-09T00:08:40.364231+0000","last_became_active":"2024-05-09T00:08:23.119861+0000","last_became_peered":"2024-05-09T00:08:23.119861+0000","last_unstale":"2024-05-09T00:08:40.364231+0000","last_undegraded":"2024-05-09T00:08:40.364231+0000","last_fullsized":"2024-05-09T00:08:40.364231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:52.282934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.043882+0000","last_change":"2024-05-09T00:08:26.050685+0000","last_active":"2024-05-09T00:08:28.043882+0000","last_peered":"2024-05-09T00:08:28.043882+0000","last_clean":"2024-05-09T00:08:28.043882+0000","last_became_active":"2024-05-09T00:08:26.050312+0000","last_became_peered":"2024-05-09T00:08:26.050312+0000","last_unstale":"2024-05-09T00:08:28.043882+0000","last_undegraded":"2024-05-09T00:08:28.043882+0000","last_fullsized":"2024-05-09T00:08:28.043882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:31.623182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'11","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469432+0000","last_change":"2024-05-09T00:08:23.019663+0000","last_active":"2024-05-09T00:09:10.469432+0000","last_peered":"2024-05-09T00:09:10.469432+0000","last_clean":"2024-05-09T00:09:10.469432+0000","last_became_active":"2024-05-09T00:08:23.019301+0000","last_became_peered":"2024-05-09T00:08:23.019301+0000","last_unstale":"2024-05-09T00:09:10.469432+0000","last_undegraded":"2024-05-09T00:09:10.469432+0000","last_fullsized":"2024-05-09T00:09:10.469432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:08.291227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468179+0000","last_change":"2024-05-09T00:08:26.058825+0000","last_active":"2024-05-09T00:09:10.468179+0000","last_peered":"2024-05-09T00:09:10.468179+0000","last_clean":"2024-05-09T00:09:10.468179+0000","last_became_active":"2024-05-09T00:08:26.058469+0000","last_became_peered":"2024-05-09T00:08:26.058469+0000","last_unstale":"2024-05-09T00:09:10.468179+0000","last_undegraded":"2024-05-09T00:09:10.468179+0000","last_fullsized":"2024-05-09T00:09:10.468179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:08.118819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796528+0000","last_change":"2024-05-09T00:08:23.054366+0000","last_active":"2024-05-09T00:09:10.796528+0000","last_peered":"2024-05-09T00:09:10.796528+0000","last_clean":"2024-05-09T00:09:10.796528+0000","last_became_active":"2024-05-09T00:08:23.053977+0000","last_became_peered":"2024-05-09T00:08:23.053977+0000","last_unstale":"2024-05-09T00:09:10.796528+0000","last_undegraded":"2024-05-09T00:09:10.796528+0000","last_fullsized":"2024-05-09T00:09:10.796528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:37.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.467190+0000","last_change":"2024-05-09T00:08:26.057864+0000","last_active":"2024-05-09T00:09:09.467190+0000","last_peered":"2024-05-09T00:09:09.467190+0000","last_clean":"2024-05-09T00:09:09.467190+0000","last_became_active":"2024-05-09T00:08:26.057436+0000","last_became_peered":"2024-05-09T00:08:26.057436+0000","last_unstale":"2024-05-09T00:09:09.467190+0000","last_undegraded":"2024-05-09T00:09:09.467190+0000","last_fullsized":"2024-05-09T00:09:09.467190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:19.976265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:40.365037+0000","last_change":"2024-05-09T00:08:23.118803+0000","last_active":"2024-05-09T00:08:40.365037+0000","last_peered":"2024-05-09T00:08:40.365037+0000","last_clean":"2024-05-09T00:08:40.365037+0000","last_became_active":"2024-05-09T00:08:23.118432+0000","last_became_peered":"2024-05-09T00:08:23.118432+0000","last_unstale":"2024-05-09T00:08:40.365037+0000","last_undegraded":"2024-05-09T00:08:40.365037+0000","last_fullsized":"2024-05-09T00:08:40.365037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:50.253699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.059058+0000","last_change":"2024-05-09T00:08:26.054281+0000","last_active":"2024-05-09T00:08:28.059058+0000","last_peered":"2024-05-09T00:08:28.059058+0000","last_clean":"2024-05-09T00:08:28.059058+0000","last_became_active":"2024-05-09T00:08:26.053874+0000","last_became_peered":"2024-05-09T00:08:26.053874+0000","last_unstale":"2024-05-09T00:08:28.059058+0000","last_undegraded":"2024-05-09T00:08:28.059058+0000","last_fullsized":"2024-05-09T00:08:28.059058+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:11.566337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467696+0000","last_change":"2024-05-09T00:08:23.017905+0000","last_active":"2024-05-09T00:09:10.467696+0000","last_peered":"2024-05-09T00:09:10.467696+0000","last_clean":"2024-05-09T00:09:10.467696+0000","last_became_active":"2024-05-09T00:08:23.017481+0000","last_became_peered":"2024-05-09T00:08:23.017481+0000","last_unstale":"2024-05-09T00:09:10.467696+0000","last_undegraded":"2024-05-09T00:09:10.467696+0000","last_fullsized":"2024-05-09T00:09:10.467696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:40.096415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063090+0000","last_change":"2024-05-09T00:08:26.053392+0000","last_active":"2024-05-09T00:08:28.063090+0000","last_peered":"2024-05-09T00:08:28.063090+0000","last_clean":"2024-05-09T00:08:28.063090+0000","last_became_active":"2024-05-09T00:08:26.052628+0000","last_became_peered":"2024-05-09T00:08:26.052628+0000","last_unstale":"2024-05-09T00:08:28.063090+0000","last_undegraded":"2024-05-09T00:08:28.063090+0000","last_fullsized":"2024-05-09T00:08:28.063090+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:28.013955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.060294+0000","last_change":"2024-05-09T00:08:23.020786+0000","last_active":"2024-05-09T00:08:28.060294+0000","last_peered":"2024-05-09T00:08:28.060294+0000","last_clean":"2024-05-09T00:08:28.060294+0000","last_became_active":"2024-05-09T00:08:23.019949+0000","last_became_peered":"2024-05-09T00:08:23.019949+0000","last_unstale":"2024-05-09T00:08:28.060294+0000","last_undegraded":"2024-05-09T00:08:28.060294+0000","last_fullsized":"2024-05-09T00:08:28.060294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:18.366813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467933+0000","last_change":"2024-05-09T00:08:26.047812+0000","last_active":"2024-05-09T00:09:10.467933+0000","last_peered":"2024-05-09T00:09:10.467933+0000","last_clean":"2024-05-09T00:09:10.467933+0000","last_became_active":"2024-05-09T00:08:26.047459+0000","last_became_peered":"2024-05-09T00:08:26.047459+0000","last_unstale":"2024-05-09T00:09:10.467933+0000","last_undegraded":"2024-05-09T00:09:10.467933+0000","last_fullsized":"2024-05-09T00:09:10.467933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.257831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:40.365280+0000","last_change":"2024-05-09T00:08:23.023800+0000","last_active":"2024-05-09T00:08:40.365280+0000","last_peered":"2024-05-09T00:08:40.365280+0000","last_clean":"2024-05-09T00:08:40.365280+0000","last_became_active":"2024-05-09T00:08:23.023387+0000","last_became_peered":"2024-05-09T00:08:23.023387+0000","last_unstale":"2024-05-09T00:08:40.365280+0000","last_undegraded":"2024-05-09T00:08:40.365280+0000","last_fullsized":"2024-05-09T00:08:40.365280+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:33.714683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.064621+0000","last_change":"2024-05-09T00:08:26.056981+0000","last_active":"2024-05-09T00:08:28.064621+0000","last_peered":"2024-05-09T00:08:28.064621+0000","last_clean":"2024-05-09T00:08:28.064621+0000","last_became_active":"2024-05-09T00:08:26.056605+0000","last_became_peered":"2024-05-09T00:08:26.056605+0000","last_unstale":"2024-05-09T00:08:28.064621+0000","last_undegraded":"2024-05-09T00:08:28.064621+0000","last_fullsized":"2024-05-09T00:08:28.064621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:32.897929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473611+0000","last_change":"2024-05-09T00:08:23.120612+0000","last_active":"2024-05-09T00:09:10.473611+0000","last_peered":"2024-05-09T00:09:10.473611+0000","last_clean":"2024-05-09T00:09:10.473611+0000","last_became_active":"2024-05-09T00:08:23.120157+0000","last_became_peered":"2024-05-09T00:08:23.120157+0000","last_unstale":"2024-05-09T00:09:10.473611+0000","last_undegraded":"2024-05-09T00:09:10.473611+0000","last_fullsized":"2024-05-09T00:09:10.473611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:08.027214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063419+0000","last_change":"2024-05-09T00:08:26.061594+0000","last_active":"2024-05-09T00:08:28.063419+0000","last_peered":"2024-05-09T00:08:28.063419+0000","last_clean":"2024-05-09T00:08:28.063419+0000","last_became_active":"2024-05-09T00:08:26.061201+0000","last_became_peered":"2024-05-09T00:08:26.061201+0000","last_unstale":"2024-05-09T00:08:28.063419+0000","last_undegraded":"2024-05-09T00:08:28.063419+0000","last_fullsized":"2024-05-09T00:08:28.063419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:50.183648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469471+0000","last_change":"2024-05-09T00:08:23.017693+0000","last_active":"2024-05-09T00:09:10.469471+0000","last_peered":"2024-05-09T00:09:10.469471+0000","last_clean":"2024-05-09T00:09:10.469471+0000","last_became_active":"2024-05-09T00:08:23.017211+0000","last_became_peered":"2024-05-09T00:08:23.017211+0000","last_unstale":"2024-05-09T00:09:10.469471+0000","last_undegraded":"2024-05-09T00:09:10.469471+0000","last_fullsized":"2024-05-09T00:09:10.469471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:53.569932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.880483+0000","last_change":"2024-05-09T00:08:26.063922+0000","last_active":"2024-05-09T00:08:28.880483+0000","last_peered":"2024-05-09T00:08:28.880483+0000","last_clean":"2024-05-09T00:08:28.880483+0000","last_became_active":"2024-05-09T00:08:26.063554+0000","last_became_peered":"2024-05-09T00:08:26.063554+0000","last_unstale":"2024-05-09T00:08:28.880483+0000","last_undegraded":"2024-05-09T00:08:28.880483+0000","last_fullsized":"2024-05-09T00:08:28.880483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:01.754688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.064971+0000","last_change":"2024-05-09T00:08:23.020367+0000","last_active":"2024-05-09T00:08:28.064971+0000","last_peered":"2024-05-09T00:08:28.064971+0000","last_clean":"2024-05-09T00:08:28.064971+0000","last_became_active":"2024-05-09T00:08:23.019801+0000","last_became_peered":"2024-05-09T00:08:23.019801+0000","last_unstale":"2024-05-09T00:08:28.064971+0000","last_undegraded":"2024-05-09T00:08:28.064971+0000","last_fullsized":"2024-05-09T00:08:28.064971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:14.936021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.065405+0000","last_change":"2024-05-09T00:08:26.063863+0000","last_active":"2024-05-09T00:08:28.065405+0000","last_peered":"2024-05-09T00:08:28.065405+0000","last_clean":"2024-05-09T00:08:28.065405+0000","last_became_active":"2024-05-09T00:08:26.061938+0000","last_became_peered":"2024-05-09T00:08:26.061938+0000","last_unstale":"2024-05-09T00:08:28.065405+0000","last_undegraded":"2024-05-09T00:08:28.065405+0000","last_fullsized":"2024-05-09T00:08:28.065405+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:04.597991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063148+0000","last_change":"2024-05-09T00:08:26.050837+0000","last_active":"2024-05-09T00:08:28.063148+0000","last_peered":"2024-05-09T00:08:28.063148+0000","last_clean":"2024-05-09T00:08:28.063148+0000","last_became_active":"2024-05-09T00:08:26.050424+0000","last_became_peered":"2024-05-09T00:08:26.050424+0000","last_unstale":"2024-05-09T00:08:28.063148+0000","last_undegraded":"2024-05-09T00:08:28.063148+0000","last_fullsized":"2024-05-09T00:08:28.063148+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:45.134811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.549989+0000","last_change":"2024-05-09T00:09:10.549989+0000","last_active":"2024-05-09T00:09:10.549989+0000","last_peered":"2024-05-09T00:09:10.549989+0000","last_clean":"2024-05-09T00:09:10.549989+0000","last_became_active":"2024-05-09T00:09:10.549502+0000","last_became_peered":"2024-05-09T00:09:10.549502+0000","last_unstale":"2024-05-09T00:09:10.549989+0000","last_undegraded":"2024-05-09T00:09:10.549989+0000","last_fullsized":"2024-05-09T00:09:10.549989+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:10.654452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473094+0000","last_change":"2024-05-09T00:08:26.058012+0000","last_active":"2024-05-09T00:09:10.473094+0000","last_peered":"2024-05-09T00:09:10.473094+0000","last_clean":"2024-05-09T00:09:10.473094+0000","last_became_active":"2024-05-09T00:08:26.057282+0000","last_became_peered":"2024-05-09T00:08:26.057282+0000","last_unstale":"2024-05-09T00:09:10.473094+0000","last_undegraded":"2024-05-09T00:09:10.473094+0000","last_fullsized":"2024-05-09T00:09:10.473094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:53.520593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-09T00:09:09.465725+0000","last_change":"2024-05-09T00:08:26.063331+0000","last_active":"2024-05-09T00:09:09.465725+0000","last_peered":"2024-05-09T00:09:09.465725+0000","last_clean":"2024-05-09T00:09:09.465725+0000","last_became_active":"2024-05-09T00:08:26.062866+0000","last_became_peered":"2024-05-09T00:08:26.062866+0000","last_unstale":"2024-05-09T00:09:09.465725+0000","last_undegraded":"2024-05-09T00:09:09.465725+0000","last_fullsized":"2024-05-09T00:09:09.465725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:10.387852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.044038+0000","last_change":"2024-05-09T00:08:26.063735+0000","last_active":"2024-05-09T00:08:28.044038+0000","last_peered":"2024-05-09T00:08:28.044038+0000","last_clean":"2024-05-09T00:08:28.044038+0000","last_became_active":"2024-05-09T00:08:26.063358+0000","last_became_peered":"2024-05-09T00:08:26.063358+0000","last_unstale":"2024-05-09T00:08:28.044038+0000","last_undegraded":"2024-05-09T00:08:28.044038+0000","last_fullsized":"2024-05-09T00:08:28.044038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:06.832071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.063227+0000","last_change":"2024-05-09T00:08:26.052746+0000","last_active":"2024-05-09T00:08:28.063227+0000","last_peered":"2024-05-09T00:08:28.063227+0000","last_clean":"2024-05-09T00:08:28.063227+0000","last_became_active":"2024-05-09T00:08:26.051544+0000","last_became_peered":"2024-05-09T00:08:26.051544+0000","last_unstale":"2024-05-09T00:08:28.063227+0000","last_undegraded":"2024-05-09T00:08:28.063227+0000","last_fullsized":"2024-05-09T00:08:28.063227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:31.963268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473534+0000","last_change":"2024-05-09T00:08:26.058075+0000","last_active":"2024-05-09T00:09:10.473534+0000","last_peered":"2024-05-09T00:09:10.473534+0000","last_clean":"2024-05-09T00:09:10.473534+0000","last_became_active":"2024-05-09T00:08:26.057349+0000","last_became_peered":"2024-05-09T00:08:26.057349+0000","last_unstale":"2024-05-09T00:09:10.473534+0000","last_undegraded":"2024-05-09T00:09:10.473534+0000","last_fullsized":"2024-05-09T00:09:10.473534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:59.322903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468103+0000","last_change":"2024-05-09T00:08:26.056429+0000","last_active":"2024-05-09T00:09:10.468103+0000","last_peered":"2024-05-09T00:09:10.468103+0000","last_clean":"2024-05-09T00:09:10.468103+0000","last_became_active":"2024-05-09T00:08:26.054852+0000","last_became_peered":"2024-05-09T00:08:26.054852+0000","last_unstale":"2024-05-09T00:09:10.468103+0000","last_undegraded":"2024-05-09T00:09:10.468103+0000","last_fullsized":"2024-05-09T00:09:10.468103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:05.337636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.549880+0000","last_change":"2024-05-09T00:09:10.549880+0000","last_active":"2024-05-09T00:09:10.549880+0000","last_peered":"2024-05-09T00:09:10.549880+0000","last_clean":"2024-05-09T00:09:10.549880+0000","last_became_active":"2024-05-09T00:09:10.549448+0000","last_became_peered":"2024-05-09T00:09:10.549448+0000","last_unstale":"2024-05-09T00:09:10.549880+0000","last_undegraded":"2024-05-09T00:09:10.549880+0000","last_fullsized":"2024-05-09T00:09:10.549880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:16.990729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467623+0000","last_change":"2024-05-09T00:08:26.054579+0000","last_active":"2024-05-09T00:09:10.467623+0000","last_peered":"2024-05-09T00:09:10.467623+0000","last_clean":"2024-05-09T00:09:10.467623+0000","last_became_active":"2024-05-09T00:08:26.053186+0000","last_became_peered":"2024-05-09T00:08:26.053186+0000","last_unstale":"2024-05-09T00:09:10.467623+0000","last_undegraded":"2024-05-09T00:09:10.467623+0000","last_fullsized":"2024-05-09T00:09:10.467623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:37.561691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.062036+0000","last_change":"2024-05-09T00:08:26.063608+0000","last_active":"2024-05-09T00:08:28.062036+0000","last_peered":"2024-05-09T00:08:28.062036+0000","last_clean":"2024-05-09T00:08:28.062036+0000","last_became_active":"2024-05-09T00:08:26.063078+0000","last_became_peered":"2024-05-09T00:08:26.063078+0000","last_unstale":"2024-05-09T00:08:28.062036+0000","last_undegraded":"2024-05-09T00:08:28.062036+0000","last_fullsized":"2024-05-09T00:08:28.062036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:06.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.043917+0000","last_change":"2024-05-09T00:08:26.055058+0000","last_active":"2024-05-09T00:08:28.043917+0000","last_peered":"2024-05-09T00:08:28.043917+0000","last_clean":"2024-05-09T00:08:28.043917+0000","last_became_active":"2024-05-09T00:08:26.054695+0000","last_became_peered":"2024-05-09T00:08:26.054695+0000","last_unstale":"2024-05-09T00:08:28.043917+0000","last_undegraded":"2024-05-09T00:08:28.043917+0000","last_fullsized":"2024-05-09T00:08:28.043917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:58.925990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-09T00:08:28.062669+0000","last_change":"2024-05-09T00:08:26.055366+0000","last_active":"2024-05-09T00:08:28.062669+0000","last_peered":"2024-05-09T00:08:28.062669+0000","last_clean":"2024-05-09T00:08:28.062669+0000","last_became_active":"2024-05-09T00:08:26.054904+0000","last_became_peered":"2024-05-09T00:08:26.054904+0000","last_unstale":"2024-05-09T00:08:28.062669+0000","last_undegraded":"2024-05-09T00:08:28.062669+0000","last_fullsized":"2024-05-09T00:08:28.062669+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:40.754878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285241,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59582,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":184,"num_read_kb":151,"num_write":163,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":184991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710743,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":408028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":1055481,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070498,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":425461,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266727,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":1000114,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":843776,"data_stored":407653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":463027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":864256,"data_stored":408924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":481574,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":425161,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":425678,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":1016604,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":55581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:09:15.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T00:09:16.178 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:16.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:16 smithi046 ceph-mon[30151]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:16.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:16 smithi046 ceph-mon[30151]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 8 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T00:09:16.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:16 smithi169 ceph-mon[34777]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:16.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:16 smithi169 ceph-mon[34777]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 8 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T00:09:16.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:16 smithi142 ceph-mon[34533]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:16.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:16 smithi142 ceph-mon[34533]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 8 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T00:09:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.859+0000 7fa270277700 1 -- 172.21.15.46:0/2443096848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa268108f10 msgr2=0x7fa268109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.859+0000 7fa270277700 1 --2- 172.21.15.46:0/2443096848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa268108f10 0x7fa268109320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0099c0 tx=0x7fa25c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.859+0000 7fa270277700 1 -- 172.21.15.46:0/2443096848 shutdown_connections 2024-05-09T00:09:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.859+0000 7fa270277700 1 --2- 172.21.15.46:0/2443096848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa268109860 0x7fa2681111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.859+0000 7fa270277700 1 --2- 172.21.15.46:0/2443096848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2681045e0 0x7fa2681049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.859+0000 7fa270277700 1 --2- 172.21.15.46:0/2443096848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa268108f10 0x7fa268109320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.859+0000 7fa270277700 1 -- 172.21.15.46:0/2443096848 >> 172.21.15.46:0/2443096848 conn(0x7fa2680ffef0 msgr2=0x7fa268102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.859+0000 7fa270277700 1 -- 172.21.15.46:0/2443096848 shutdown_connections 2024-05-09T00:09:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.859+0000 7fa270277700 1 -- 172.21.15.46:0/2443096848 wait complete. 2024-05-09T00:09:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa270277700 1 Processor -- start 2024-05-09T00:09:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa270277700 1 -- start start 2024-05-09T00:09:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa270277700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2681045e0 0x7fa26819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:17.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa270277700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa268108f10 0x7fa26819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:17.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa270277700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa268109860 0x7fa26819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:17.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa270277700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa268114470 con 0x7fa268108f10 2024-05-09T00:09:17.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa270277700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681142f0 con 0x7fa268109860 2024-05-09T00:09:17.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa270277700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2681145f0 con 0x7fa2681045e0 2024-05-09T00:09:17.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa26d812700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa268108f10 0x7fa26819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:17.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa26d812700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa268108f10 0x7fa26819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56016/0 (socket says 172.21.15.46:56016) 2024-05-09T00:09:17.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa26d812700 1 -- 172.21.15.46:0/1545960928 learned_addr learned my addr 172.21.15.46:0/1545960928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:17.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa26e013700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2681045e0 0x7fa26819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:17.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.860+0000 7fa26e814700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa268109860 0x7fa26819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:17.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.861+0000 7fa26d812700 1 -- 172.21.15.46:0/1545960928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2681045e0 msgr2=0x7fa26819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:17.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.861+0000 7fa26d812700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2681045e0 0x7fa26819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.861+0000 7fa26d812700 1 -- 172.21.15.46:0/1545960928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa268109860 msgr2=0x7fa26819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.861+0000 7fa26d812700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa268109860 0x7fa26819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.861+0000 7fa26d812700 1 -- 172.21.15.46:0/1545960928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26819fd10 con 0x7fa268108f10 2024-05-09T00:09:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.861+0000 7fa26d812700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa268108f10 0x7fa26819b3e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa25c006010 tx=0x7fa25c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.861+0000 7fa2677fe700 1 -- 172.21.15.46:0/1545960928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c027070 con 0x7fa268108f10 2024-05-09T00:09:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.861+0000 7fa2677fe700 1 -- 172.21.15.46:0/1545960928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa25c003f20 con 0x7fa268108f10 2024-05-09T00:09:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.861+0000 7fa270277700 1 -- 172.21.15.46:0/1545960928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26819ffa0 con 0x7fa268108f10 2024-05-09T00:09:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.861+0000 7fa2677fe700 1 -- 172.21.15.46:0/1545960928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c022c60 con 0x7fa268108f10 2024-05-09T00:09:17.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.861+0000 7fa270277700 1 -- 172.21.15.46:0/1545960928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2681a0480 con 0x7fa268108f10 2024-05-09T00:09:17.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.862+0000 7fa270277700 1 -- 172.21.15.46:0/1545960928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2680512c0 con 0x7fa268108f10 2024-05-09T00:09:17.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.864+0000 7fa2677fe700 1 -- 172.21.15.46:0/1545960928 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa25c02a070 con 0x7fa268108f10 2024-05-09T00:09:17.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.865+0000 7fa2677fe700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa2500b2910 0x7fa2500b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:17.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.865+0000 7fa26e013700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa2500b2910 0x7fa2500b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:17.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.865+0000 7fa2677fe700 1 -- 172.21.15.46:0/1545960928 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fa25c022e00 con 0x7fa268108f10 2024-05-09T00:09:17.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.866+0000 7fa26e013700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa2500b2910 0x7fa2500b4e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa258009550 tx=0x7fa2580091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:17.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:17.869+0000 7fa2677fe700 1 -- 172.21.15.46:0/1545960928 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa25c013070 con 0x7fa268108f10 2024-05-09T00:09:18.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.081+0000 7fa270277700 1 -- 172.21.15.46:0/1545960928 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa26819c1a0 con 0x7fa2500b2910 2024-05-09T00:09:18.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.089+0000 7fa2677fe700 1 -- 172.21.15.46:0/1545960928 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258959 (secure 0 0 0) 0x7fa26819c1a0 con 0x7fa2500b2910 2024-05-09T00:09:18.085 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:09:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.092+0000 7fa270277700 1 -- 172.21.15.46:0/1545960928 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa2500b2910 msgr2=0x7fa2500b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.092+0000 7fa270277700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa2500b2910 0x7fa2500b4e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa258009550 tx=0x7fa2580091b0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.092+0000 7fa270277700 1 -- 172.21.15.46:0/1545960928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa268108f10 msgr2=0x7fa26819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.092+0000 7fa270277700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa268108f10 0x7fa26819b3e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa25c006010 tx=0x7fa25c0047b0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.092+0000 7fa270277700 1 -- 172.21.15.46:0/1545960928 shutdown_connections 2024-05-09T00:09:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.092+0000 7fa270277700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa2500b2910 0x7fa2500b4e20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.092+0000 7fa270277700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa268109860 0x7fa26819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.092+0000 7fa270277700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2681045e0 0x7fa26819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.092+0000 7fa270277700 1 --2- 172.21.15.46:0/1545960928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa268108f10 0x7fa26819b3e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:18.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.092+0000 7fa270277700 1 -- 172.21.15.46:0/1545960928 >> 172.21.15.46:0/1545960928 conn(0x7fa2680ffef0 msgr2=0x7fa26810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:18.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.092+0000 7fa270277700 1 -- 172.21.15.46:0/1545960928 shutdown_connections 2024-05-09T00:09:18.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:18.092+0000 7fa270277700 1 -- 172.21.15.46:0/1545960928 wait complete. 2024-05-09T00:09:18.091 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T00:09:18.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:18 smithi046 ceph-mon[30151]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 895 B/s wr, 4 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T00:09:18.724 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-05-09T00:09:17.293884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656429,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":285,"num_read_kb":236,"num_write":290,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372372,"kb_used_data":10652,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569452,"statfs":{"total":1151940427776,"available":1151559118848,"internally_reserved":0,"allocated":10907648,"data_stored":7058758,"data_compressed":177897,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6245,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":21,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002087"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471697+0000","last_change":"2024-05-09T00:08:26.060455+0000","last_active":"2024-05-09T00:09:10.471697+0000","last_peered":"2024-05-09T00:09:10.471697+0000","last_clean":"2024-05-09T00:09:10.471697+0000","last_became_active":"2024-05-09T00:08:26.060068+0000","last_became_peered":"2024-05-09T00:08:26.060068+0000","last_unstale":"2024-05-09T00:09:10.471697+0000","last_undegraded":"2024-05-09T00:09:10.471697+0000","last_fullsized":"2024-05-09T00:09:10.471697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:48.354188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471761+0000","last_change":"2024-05-09T00:08:26.055737+0000","last_active":"2024-05-09T00:09:10.471761+0000","last_peered":"2024-05-09T00:09:10.471761+0000","last_clean":"2024-05-09T00:09:10.471761+0000","last_became_active":"2024-05-09T00:08:26.055354+0000","last_became_peered":"2024-05-09T00:08:26.055354+0000","last_unstale":"2024-05-09T00:09:10.471761+0000","last_undegraded":"2024-05-09T00:09:10.471761+0000","last_fullsized":"2024-05-09T00:09:10.471761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:54.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471480+0000","last_change":"2024-05-09T00:08:26.050776+0000","last_active":"2024-05-09T00:09:10.471480+0000","last_peered":"2024-05-09T00:09:10.471480+0000","last_clean":"2024-05-09T00:09:10.471480+0000","last_became_active":"2024-05-09T00:08:26.050393+0000","last_became_peered":"2024-05-09T00:08:26.050393+0000","last_unstale":"2024-05-09T00:09:10.471480+0000","last_undegraded":"2024-05-09T00:09:10.471480+0000","last_fullsized":"2024-05-09T00:09:10.471480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:30.819219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796411+0000","last_change":"2024-05-09T00:08:26.065946+0000","last_active":"2024-05-09T00:09:10.796411+0000","last_peered":"2024-05-09T00:09:10.796411+0000","last_clean":"2024-05-09T00:09:10.796411+0000","last_became_active":"2024-05-09T00:08:26.065498+0000","last_became_peered":"2024-05-09T00:08:26.065498+0000","last_unstale":"2024-05-09T00:09:10.796411+0000","last_undegraded":"2024-05-09T00:09:10.796411+0000","last_fullsized":"2024-05-09T00:09:10.796411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:04.444860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473630+0000","last_change":"2024-05-09T00:08:26.051961+0000","last_active":"2024-05-09T00:09:10.473630+0000","last_peered":"2024-05-09T00:09:10.473630+0000","last_clean":"2024-05-09T00:09:10.473630+0000","last_became_active":"2024-05-09T00:08:26.051548+0000","last_became_peered":"2024-05-09T00:08:26.051548+0000","last_unstale":"2024-05-09T00:09:10.473630+0000","last_undegraded":"2024-05-09T00:09:10.473630+0000","last_fullsized":"2024-05-09T00:09:10.473630+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:07.539117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471811+0000","last_change":"2024-05-09T00:08:26.052012+0000","last_active":"2024-05-09T00:09:10.471811+0000","last_peered":"2024-05-09T00:09:10.471811+0000","last_clean":"2024-05-09T00:09:10.471811+0000","last_became_active":"2024-05-09T00:08:26.050886+0000","last_became_peered":"2024-05-09T00:08:26.050886+0000","last_unstale":"2024-05-09T00:09:10.471811+0000","last_undegraded":"2024-05-09T00:09:10.471811+0000","last_fullsized":"2024-05-09T00:09:10.471811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:14.651888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472730+0000","last_change":"2024-05-09T00:08:26.058418+0000","last_active":"2024-05-09T00:09:10.472730+0000","last_peered":"2024-05-09T00:09:10.472730+0000","last_clean":"2024-05-09T00:09:10.472730+0000","last_became_active":"2024-05-09T00:08:26.058039+0000","last_became_peered":"2024-05-09T00:08:26.058039+0000","last_unstale":"2024-05-09T00:09:10.472730+0000","last_undegraded":"2024-05-09T00:09:10.472730+0000","last_fullsized":"2024-05-09T00:09:10.472730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:13.678123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467742+0000","last_change":"2024-05-09T00:08:26.057602+0000","last_active":"2024-05-09T00:09:10.467742+0000","last_peered":"2024-05-09T00:09:10.467742+0000","last_clean":"2024-05-09T00:09:10.467742+0000","last_became_active":"2024-05-09T00:08:26.057169+0000","last_became_peered":"2024-05-09T00:08:26.057169+0000","last_unstale":"2024-05-09T00:09:10.467742+0000","last_undegraded":"2024-05-09T00:09:10.467742+0000","last_fullsized":"2024-05-09T00:09:10.467742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:11.452083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469166+0000","last_change":"2024-05-09T00:08:26.067639+0000","last_active":"2024-05-09T00:09:10.469166+0000","last_peered":"2024-05-09T00:09:10.469166+0000","last_clean":"2024-05-09T00:09:10.469166+0000","last_became_active":"2024-05-09T00:08:26.067226+0000","last_became_peered":"2024-05-09T00:08:26.067226+0000","last_unstale":"2024-05-09T00:09:10.469166+0000","last_undegraded":"2024-05-09T00:09:10.469166+0000","last_fullsized":"2024-05-09T00:09:10.469166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:12.366500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472299+0000","last_change":"2024-05-09T00:08:26.055234+0000","last_active":"2024-05-09T00:09:10.472299+0000","last_peered":"2024-05-09T00:09:10.472299+0000","last_clean":"2024-05-09T00:09:10.472299+0000","last_became_active":"2024-05-09T00:08:26.054851+0000","last_became_peered":"2024-05-09T00:08:26.054851+0000","last_unstale":"2024-05-09T00:09:10.472299+0000","last_undegraded":"2024-05-09T00:09:10.472299+0000","last_fullsized":"2024-05-09T00:09:10.472299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:54:52.121329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472899+0000","last_change":"2024-05-09T00:08:26.063375+0000","last_active":"2024-05-09T00:09:10.472899+0000","last_peered":"2024-05-09T00:09:10.472899+0000","last_clean":"2024-05-09T00:09:10.472899+0000","last_became_active":"2024-05-09T00:08:26.062962+0000","last_became_peered":"2024-05-09T00:08:26.062962+0000","last_unstale":"2024-05-09T00:09:10.472899+0000","last_undegraded":"2024-05-09T00:09:10.472899+0000","last_fullsized":"2024-05-09T00:09:10.472899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:05.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796257+0000","last_change":"2024-05-09T00:08:26.062034+0000","last_active":"2024-05-09T00:09:10.796257+0000","last_peered":"2024-05-09T00:09:10.796257+0000","last_clean":"2024-05-09T00:09:10.796257+0000","last_became_active":"2024-05-09T00:08:26.060376+0000","last_became_peered":"2024-05-09T00:08:26.060376+0000","last_unstale":"2024-05-09T00:09:10.796257+0000","last_undegraded":"2024-05-09T00:09:10.796257+0000","last_fullsized":"2024-05-09T00:09:10.796257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:15.830024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548685+0000","last_change":"2024-05-09T00:08:26.053227+0000","last_active":"2024-05-09T00:09:10.548685+0000","last_peered":"2024-05-09T00:09:10.548685+0000","last_clean":"2024-05-09T00:09:10.548685+0000","last_became_active":"2024-05-09T00:08:26.052878+0000","last_became_peered":"2024-05-09T00:08:26.052878+0000","last_unstale":"2024-05-09T00:09:10.548685+0000","last_undegraded":"2024-05-09T00:09:10.548685+0000","last_fullsized":"2024-05-09T00:09:10.548685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:24.385026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471290+0000","last_change":"2024-05-09T00:08:26.053041+0000","last_active":"2024-05-09T00:09:10.471290+0000","last_peered":"2024-05-09T00:09:10.471290+0000","last_clean":"2024-05-09T00:09:10.471290+0000","last_became_active":"2024-05-09T00:08:26.052677+0000","last_became_peered":"2024-05-09T00:08:26.052677+0000","last_unstale":"2024-05-09T00:09:10.471290+0000","last_undegraded":"2024-05-09T00:09:10.471290+0000","last_fullsized":"2024-05-09T00:09:10.471290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:24.478270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471141+0000","last_change":"2024-05-09T00:08:26.063391+0000","last_active":"2024-05-09T00:09:10.471141+0000","last_peered":"2024-05-09T00:09:10.471141+0000","last_clean":"2024-05-09T00:09:10.471141+0000","last_became_active":"2024-05-09T00:08:26.063033+0000","last_became_peered":"2024-05-09T00:08:26.063033+0000","last_unstale":"2024-05-09T00:09:10.471141+0000","last_undegraded":"2024-05-09T00:09:10.471141+0000","last_fullsized":"2024-05-09T00:09:10.471141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:41.900453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468575+0000","last_change":"2024-05-09T00:08:26.065018+0000","last_active":"2024-05-09T00:09:10.468575+0000","last_peered":"2024-05-09T00:09:10.468575+0000","last_clean":"2024-05-09T00:09:10.468575+0000","last_became_active":"2024-05-09T00:08:26.064642+0000","last_became_peered":"2024-05-09T00:08:26.064642+0000","last_unstale":"2024-05-09T00:09:10.468575+0000","last_undegraded":"2024-05-09T00:09:10.468575+0000","last_fullsized":"2024-05-09T00:09:10.468575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:02.403704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471869+0000","last_change":"2024-05-09T00:08:26.049770+0000","last_active":"2024-05-09T00:09:10.471869+0000","last_peered":"2024-05-09T00:09:10.471869+0000","last_clean":"2024-05-09T00:09:10.471869+0000","last_became_active":"2024-05-09T00:08:26.048555+0000","last_became_peered":"2024-05-09T00:08:26.048555+0000","last_unstale":"2024-05-09T00:09:10.471869+0000","last_undegraded":"2024-05-09T00:09:10.471869+0000","last_fullsized":"2024-05-09T00:09:10.471869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:16.009200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469538+0000","last_change":"2024-05-09T00:08:26.048587+0000","last_active":"2024-05-09T00:09:10.469538+0000","last_peered":"2024-05-09T00:09:10.469538+0000","last_clean":"2024-05-09T00:09:10.469538+0000","last_became_active":"2024-05-09T00:08:26.048148+0000","last_became_peered":"2024-05-09T00:08:26.048148+0000","last_unstale":"2024-05-09T00:09:10.469538+0000","last_undegraded":"2024-05-09T00:09:10.469538+0000","last_fullsized":"2024-05-09T00:09:10.469538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:42.113829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473434+0000","last_change":"2024-05-09T00:08:26.053607+0000","last_active":"2024-05-09T00:09:10.473434+0000","last_peered":"2024-05-09T00:09:10.473434+0000","last_clean":"2024-05-09T00:09:10.473434+0000","last_became_active":"2024-05-09T00:08:26.053212+0000","last_became_peered":"2024-05-09T00:08:26.053212+0000","last_unstale":"2024-05-09T00:09:10.473434+0000","last_undegraded":"2024-05-09T00:09:10.473434+0000","last_fullsized":"2024-05-09T00:09:10.473434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:32.937049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469272+0000","last_change":"2024-05-09T00:08:23.024481+0000","last_active":"2024-05-09T00:09:10.469272+0000","last_peered":"2024-05-09T00:09:10.469272+0000","last_clean":"2024-05-09T00:09:10.469272+0000","last_became_active":"2024-05-09T00:08:23.024132+0000","last_became_peered":"2024-05-09T00:08:23.024132+0000","last_unstale":"2024-05-09T00:09:10.469272+0000","last_undegraded":"2024-05-09T00:09:10.469272+0000","last_fullsized":"2024-05-09T00:09:10.469272+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:11.188505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468045+0000","last_change":"2024-05-09T00:08:26.054772+0000","last_active":"2024-05-09T00:09:10.468045+0000","last_peered":"2024-05-09T00:09:10.468045+0000","last_clean":"2024-05-09T00:09:10.468045+0000","last_became_active":"2024-05-09T00:08:26.054433+0000","last_became_peered":"2024-05-09T00:08:26.054433+0000","last_unstale":"2024-05-09T00:09:10.468045+0000","last_undegraded":"2024-05-09T00:09:10.468045+0000","last_fullsized":"2024-05-09T00:09:10.468045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:44.283240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468325+0000","last_change":"2024-05-09T00:08:23.118778+0000","last_active":"2024-05-09T00:09:10.468325+0000","last_peered":"2024-05-09T00:09:10.468325+0000","last_clean":"2024-05-09T00:09:10.468325+0000","last_became_active":"2024-05-09T00:08:23.118377+0000","last_became_peered":"2024-05-09T00:08:23.118377+0000","last_unstale":"2024-05-09T00:09:10.468325+0000","last_undegraded":"2024-05-09T00:09:10.468325+0000","last_fullsized":"2024-05-09T00:09:10.468325+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:51.352442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472000+0000","last_change":"2024-05-09T00:08:26.059953+0000","last_active":"2024-05-09T00:09:10.472000+0000","last_peered":"2024-05-09T00:09:10.472000+0000","last_clean":"2024-05-09T00:09:10.472000+0000","last_became_active":"2024-05-09T00:08:26.059571+0000","last_became_peered":"2024-05-09T00:08:26.059571+0000","last_unstale":"2024-05-09T00:09:10.472000+0000","last_undegraded":"2024-05-09T00:09:10.472000+0000","last_fullsized":"2024-05-09T00:09:10.472000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:48.056798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475240+0000","last_change":"2024-05-09T00:08:23.021244+0000","last_active":"2024-05-09T00:09:10.475240+0000","last_peered":"2024-05-09T00:09:10.475240+0000","last_clean":"2024-05-09T00:09:10.475240+0000","last_became_active":"2024-05-09T00:08:23.020597+0000","last_became_peered":"2024-05-09T00:08:23.020597+0000","last_unstale":"2024-05-09T00:09:10.475240+0000","last_undegraded":"2024-05-09T00:09:10.475240+0000","last_fullsized":"2024-05-09T00:09:10.475240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.654261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468788+0000","last_change":"2024-05-09T00:08:26.059834+0000","last_active":"2024-05-09T00:09:10.468788+0000","last_peered":"2024-05-09T00:09:10.468788+0000","last_clean":"2024-05-09T00:09:10.468788+0000","last_became_active":"2024-05-09T00:08:26.059349+0000","last_became_peered":"2024-05-09T00:08:26.059349+0000","last_unstale":"2024-05-09T00:09:10.468788+0000","last_undegraded":"2024-05-09T00:09:10.468788+0000","last_fullsized":"2024-05-09T00:09:10.468788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:42.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474158+0000","last_change":"2024-05-09T00:08:23.019568+0000","last_active":"2024-05-09T00:09:10.474158+0000","last_peered":"2024-05-09T00:09:10.474158+0000","last_clean":"2024-05-09T00:09:10.474158+0000","last_became_active":"2024-05-09T00:08:23.019149+0000","last_became_peered":"2024-05-09T00:08:23.019149+0000","last_unstale":"2024-05-09T00:09:10.474158+0000","last_undegraded":"2024-05-09T00:09:10.474158+0000","last_fullsized":"2024-05-09T00:09:10.474158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:44.569911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469966+0000","last_change":"2024-05-09T00:08:26.061308+0000","last_active":"2024-05-09T00:09:10.469966+0000","last_peered":"2024-05-09T00:09:10.469966+0000","last_clean":"2024-05-09T00:09:10.469966+0000","last_became_active":"2024-05-09T00:08:26.060973+0000","last_became_peered":"2024-05-09T00:08:26.060973+0000","last_unstale":"2024-05-09T00:09:10.469966+0000","last_undegraded":"2024-05-09T00:09:10.469966+0000","last_fullsized":"2024-05-09T00:09:10.469966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:16.845074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:15.442379+0000","last_change":"2024-05-09T00:08:23.020886+0000","last_active":"2024-05-09T00:09:15.442379+0000","last_peered":"2024-05-09T00:09:15.442379+0000","last_clean":"2024-05-09T00:09:15.442379+0000","last_became_active":"2024-05-09T00:08:23.020530+0000","last_became_peered":"2024-05-09T00:08:23.020530+0000","last_unstale":"2024-05-09T00:09:15.442379+0000","last_undegraded":"2024-05-09T00:09:15.442379+0000","last_fullsized":"2024-05-09T00:09:15.442379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:34.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474367+0000","last_change":"2024-05-09T00:08:26.051018+0000","last_active":"2024-05-09T00:09:10.474367+0000","last_peered":"2024-05-09T00:09:10.474367+0000","last_clean":"2024-05-09T00:09:10.474367+0000","last_became_active":"2024-05-09T00:08:26.050495+0000","last_became_peered":"2024-05-09T00:08:26.050495+0000","last_unstale":"2024-05-09T00:09:10.474367+0000","last_undegraded":"2024-05-09T00:09:10.474367+0000","last_fullsized":"2024-05-09T00:09:10.474367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:07.862339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472087+0000","last_change":"2024-05-09T00:08:23.021187+0000","last_active":"2024-05-09T00:09:10.472087+0000","last_peered":"2024-05-09T00:09:10.472087+0000","last_clean":"2024-05-09T00:09:10.472087+0000","last_became_active":"2024-05-09T00:08:23.020770+0000","last_became_peered":"2024-05-09T00:08:23.020770+0000","last_unstale":"2024-05-09T00:09:10.472087+0000","last_undegraded":"2024-05-09T00:09:10.472087+0000","last_fullsized":"2024-05-09T00:09:10.472087+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:08.992770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.797838+0000","last_change":"2024-05-09T00:09:10.797392+0000","last_active":"2024-05-09T00:09:10.797838+0000","last_peered":"2024-05-09T00:09:10.797838+0000","last_clean":"2024-05-09T00:09:10.797838+0000","last_became_active":"2024-05-09T00:09:10.796969+0000","last_became_peered":"2024-05-09T00:09:10.796969+0000","last_unstale":"2024-05-09T00:09:10.797838+0000","last_undegraded":"2024-05-09T00:09:10.797838+0000","last_fullsized":"2024-05-09T00:09:10.797838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:18.852678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548755+0000","last_change":"2024-05-09T00:08:23.012226+0000","last_active":"2024-05-09T00:09:10.548755+0000","last_peered":"2024-05-09T00:09:10.548755+0000","last_clean":"2024-05-09T00:09:10.548755+0000","last_became_active":"2024-05-09T00:08:23.011781+0000","last_became_peered":"2024-05-09T00:08:23.011781+0000","last_unstale":"2024-05-09T00:09:10.548755+0000","last_undegraded":"2024-05-09T00:09:10.548755+0000","last_fullsized":"2024-05-09T00:09:10.548755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:44.452206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467563+0000","last_change":"2024-05-09T00:08:26.043035+0000","last_active":"2024-05-09T00:09:10.467563+0000","last_peered":"2024-05-09T00:09:10.467563+0000","last_clean":"2024-05-09T00:09:10.467563+0000","last_became_active":"2024-05-09T00:08:26.042686+0000","last_became_peered":"2024-05-09T00:08:26.042686+0000","last_unstale":"2024-05-09T00:09:10.467563+0000","last_undegraded":"2024-05-09T00:09:10.467563+0000","last_fullsized":"2024-05-09T00:09:10.467563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:24.511309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469042+0000","last_change":"2024-05-09T00:08:23.022017+0000","last_active":"2024-05-09T00:09:10.469042+0000","last_peered":"2024-05-09T00:09:10.469042+0000","last_clean":"2024-05-09T00:09:10.469042+0000","last_became_active":"2024-05-09T00:08:23.021540+0000","last_became_peered":"2024-05-09T00:08:23.021540+0000","last_unstale":"2024-05-09T00:09:10.469042+0000","last_undegraded":"2024-05-09T00:09:10.469042+0000","last_fullsized":"2024-05-09T00:09:10.469042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:50.773211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469091+0000","last_change":"2024-05-09T00:08:26.048039+0000","last_active":"2024-05-09T00:09:10.469091+0000","last_peered":"2024-05-09T00:09:10.469091+0000","last_clean":"2024-05-09T00:09:10.469091+0000","last_became_active":"2024-05-09T00:08:26.047695+0000","last_became_peered":"2024-05-09T00:08:26.047695+0000","last_unstale":"2024-05-09T00:09:10.469091+0000","last_undegraded":"2024-05-09T00:09:10.469091+0000","last_fullsized":"2024-05-09T00:09:10.469091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:27.363847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":59,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:12.437184+0000","last_change":"2024-05-09T00:08:23.019565+0000","last_active":"2024-05-09T00:09:12.437184+0000","last_peered":"2024-05-09T00:09:12.437184+0000","last_clean":"2024-05-09T00:09:12.437184+0000","last_became_active":"2024-05-09T00:08:23.019113+0000","last_became_peered":"2024-05-09T00:08:23.019113+0000","last_unstale":"2024-05-09T00:09:12.437184+0000","last_undegraded":"2024-05-09T00:09:12.437184+0000","last_fullsized":"2024-05-09T00:09:12.437184+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:45.578934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469070+0000","last_change":"2024-05-09T00:08:26.058869+0000","last_active":"2024-05-09T00:09:10.469070+0000","last_peered":"2024-05-09T00:09:10.469070+0000","last_clean":"2024-05-09T00:09:10.469070+0000","last_became_active":"2024-05-09T00:08:26.058482+0000","last_became_peered":"2024-05-09T00:08:26.058482+0000","last_unstale":"2024-05-09T00:09:10.469070+0000","last_undegraded":"2024-05-09T00:09:10.469070+0000","last_fullsized":"2024-05-09T00:09:10.469070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:33.252208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472150+0000","last_change":"2024-05-09T00:08:23.024425+0000","last_active":"2024-05-09T00:09:10.472150+0000","last_peered":"2024-05-09T00:09:10.472150+0000","last_clean":"2024-05-09T00:09:10.472150+0000","last_became_active":"2024-05-09T00:08:23.023991+0000","last_became_peered":"2024-05-09T00:08:23.023991+0000","last_unstale":"2024-05-09T00:09:10.472150+0000","last_undegraded":"2024-05-09T00:09:10.472150+0000","last_fullsized":"2024-05-09T00:09:10.472150+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:47.907851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475008+0000","last_change":"2024-05-09T00:09:10.475008+0000","last_active":"2024-05-09T00:09:10.475008+0000","last_peered":"2024-05-09T00:09:10.475008+0000","last_clean":"2024-05-09T00:09:10.475008+0000","last_became_active":"2024-05-09T00:09:10.474663+0000","last_became_peered":"2024-05-09T00:09:10.474663+0000","last_unstale":"2024-05-09T00:09:10.475008+0000","last_undegraded":"2024-05-09T00:09:10.475008+0000","last_fullsized":"2024-05-09T00:09:10.475008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:36.990362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470120+0000","last_change":"2024-05-09T00:08:23.118516+0000","last_active":"2024-05-09T00:09:10.470120+0000","last_peered":"2024-05-09T00:09:10.470120+0000","last_clean":"2024-05-09T00:09:10.470120+0000","last_became_active":"2024-05-09T00:08:23.118147+0000","last_became_peered":"2024-05-09T00:08:23.118147+0000","last_unstale":"2024-05-09T00:09:10.470120+0000","last_undegraded":"2024-05-09T00:09:10.470120+0000","last_fullsized":"2024-05-09T00:09:10.470120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:24.205818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468743+0000","last_change":"2024-05-09T00:08:26.067063+0000","last_active":"2024-05-09T00:09:10.468743+0000","last_peered":"2024-05-09T00:09:10.468743+0000","last_clean":"2024-05-09T00:09:10.468743+0000","last_became_active":"2024-05-09T00:08:26.066440+0000","last_became_peered":"2024-05-09T00:08:26.066440+0000","last_unstale":"2024-05-09T00:09:10.468743+0000","last_undegraded":"2024-05-09T00:09:10.468743+0000","last_fullsized":"2024-05-09T00:09:10.468743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:48.648485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":125,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:13.441305+0000","last_change":"2024-05-09T00:08:23.020757+0000","last_active":"2024-05-09T00:09:13.441305+0000","last_peered":"2024-05-09T00:09:13.441305+0000","last_clean":"2024-05-09T00:09:13.441305+0000","last_became_active":"2024-05-09T00:08:23.020355+0000","last_became_peered":"2024-05-09T00:08:23.020355+0000","last_unstale":"2024-05-09T00:09:13.441305+0000","last_undegraded":"2024-05-09T00:09:13.441305+0000","last_fullsized":"2024-05-09T00:09:13.441305+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:00.605691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61826,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":69,"num_read_kb":94,"num_write":31,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475093+0000","last_change":"2024-05-09T00:08:26.057402+0000","last_active":"2024-05-09T00:09:10.475093+0000","last_peered":"2024-05-09T00:09:10.475093+0000","last_clean":"2024-05-09T00:09:10.475093+0000","last_became_active":"2024-05-09T00:08:26.056974+0000","last_became_peered":"2024-05-09T00:08:26.056974+0000","last_unstale":"2024-05-09T00:09:10.475093+0000","last_undegraded":"2024-05-09T00:09:10.475093+0000","last_fullsized":"2024-05-09T00:09:10.475093+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:47.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473890+0000","last_change":"2024-05-09T00:08:23.020208+0000","last_active":"2024-05-09T00:09:10.473890+0000","last_peered":"2024-05-09T00:09:10.473890+0000","last_clean":"2024-05-09T00:09:10.473890+0000","last_became_active":"2024-05-09T00:08:23.019809+0000","last_became_peered":"2024-05-09T00:08:23.019809+0000","last_unstale":"2024-05-09T00:09:10.473890+0000","last_undegraded":"2024-05-09T00:09:10.473890+0000","last_fullsized":"2024-05-09T00:09:10.473890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:47.900731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548422+0000","last_change":"2024-05-09T00:08:26.054273+0000","last_active":"2024-05-09T00:09:10.548422+0000","last_peered":"2024-05-09T00:09:10.548422+0000","last_clean":"2024-05-09T00:09:10.548422+0000","last_became_active":"2024-05-09T00:08:26.053840+0000","last_became_peered":"2024-05-09T00:08:26.053840+0000","last_unstale":"2024-05-09T00:09:10.548422+0000","last_undegraded":"2024-05-09T00:09:10.548422+0000","last_fullsized":"2024-05-09T00:09:10.548422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:27.128582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469240+0000","last_change":"2024-05-09T00:08:23.021594+0000","last_active":"2024-05-09T00:09:10.469240+0000","last_peered":"2024-05-09T00:09:10.469240+0000","last_clean":"2024-05-09T00:09:10.469240+0000","last_became_active":"2024-05-09T00:08:23.021152+0000","last_became_peered":"2024-05-09T00:08:23.021152+0000","last_unstale":"2024-05-09T00:09:10.469240+0000","last_undegraded":"2024-05-09T00:09:10.469240+0000","last_fullsized":"2024-05-09T00:09:10.469240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:58.505781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473341+0000","last_change":"2024-05-09T00:08:26.064464+0000","last_active":"2024-05-09T00:09:10.473341+0000","last_peered":"2024-05-09T00:09:10.473341+0000","last_clean":"2024-05-09T00:09:10.473341+0000","last_became_active":"2024-05-09T00:08:26.064000+0000","last_became_peered":"2024-05-09T00:08:26.064000+0000","last_unstale":"2024-05-09T00:09:10.473341+0000","last_undegraded":"2024-05-09T00:09:10.473341+0000","last_fullsized":"2024-05-09T00:09:10.473341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:01.455979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'4","reported_seq":62,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:11.436334+0000","last_change":"2024-05-09T00:08:23.020823+0000","last_active":"2024-05-09T00:09:11.436334+0000","last_peered":"2024-05-09T00:09:11.436334+0000","last_clean":"2024-05-09T00:09:11.436334+0000","last_became_active":"2024-05-09T00:08:23.020412+0000","last_became_peered":"2024-05-09T00:08:23.020412+0000","last_unstale":"2024-05-09T00:09:11.436334+0000","last_undegraded":"2024-05-09T00:09:11.436334+0000","last_fullsized":"2024-05-09T00:09:11.436334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:05.457452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474540+0000","last_change":"2024-05-09T00:09:10.474540+0000","last_active":"2024-05-09T00:09:10.474540+0000","last_peered":"2024-05-09T00:09:10.474540+0000","last_clean":"2024-05-09T00:09:10.474540+0000","last_became_active":"2024-05-09T00:09:10.474068+0000","last_became_peered":"2024-05-09T00:09:10.474068+0000","last_unstale":"2024-05-09T00:09:10.474540+0000","last_undegraded":"2024-05-09T00:09:10.474540+0000","last_fullsized":"2024-05-09T00:09:10.474540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:37.296140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474161+0000","last_change":"2024-05-09T00:08:23.118857+0000","last_active":"2024-05-09T00:09:10.474161+0000","last_peered":"2024-05-09T00:09:10.474161+0000","last_clean":"2024-05-09T00:09:10.474161+0000","last_became_active":"2024-05-09T00:08:23.118433+0000","last_became_peered":"2024-05-09T00:08:23.118433+0000","last_unstale":"2024-05-09T00:09:10.474161+0000","last_undegraded":"2024-05-09T00:09:10.474161+0000","last_fullsized":"2024-05-09T00:09:10.474161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:10.617987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469212+0000","last_change":"2024-05-09T00:08:26.066911+0000","last_active":"2024-05-09T00:09:10.469212+0000","last_peered":"2024-05-09T00:09:10.469212+0000","last_clean":"2024-05-09T00:09:10.469212+0000","last_became_active":"2024-05-09T00:08:26.066548+0000","last_became_peered":"2024-05-09T00:08:26.066548+0000","last_unstale":"2024-05-09T00:09:10.469212+0000","last_undegraded":"2024-05-09T00:09:10.469212+0000","last_fullsized":"2024-05-09T00:09:10.469212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:32.391390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475008+0000","last_change":"2024-05-09T00:08:23.019829+0000","last_active":"2024-05-09T00:09:10.475008+0000","last_peered":"2024-05-09T00:09:10.475008+0000","last_clean":"2024-05-09T00:09:10.475008+0000","last_became_active":"2024-05-09T00:08:23.019336+0000","last_became_peered":"2024-05-09T00:08:23.019336+0000","last_unstale":"2024-05-09T00:09:10.475008+0000","last_undegraded":"2024-05-09T00:09:10.475008+0000","last_fullsized":"2024-05-09T00:09:10.475008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:33.802738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470640+0000","last_change":"2024-05-09T00:08:26.055975+0000","last_active":"2024-05-09T00:09:10.470640+0000","last_peered":"2024-05-09T00:09:10.470640+0000","last_clean":"2024-05-09T00:09:10.470640+0000","last_became_active":"2024-05-09T00:08:26.055595+0000","last_became_peered":"2024-05-09T00:08:26.055595+0000","last_unstale":"2024-05-09T00:09:10.470640+0000","last_undegraded":"2024-05-09T00:09:10.470640+0000","last_fullsized":"2024-05-09T00:09:10.470640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:20.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469371+0000","last_change":"2024-05-09T00:08:23.120353+0000","last_active":"2024-05-09T00:09:10.469371+0000","last_peered":"2024-05-09T00:09:10.469371+0000","last_clean":"2024-05-09T00:09:10.469371+0000","last_became_active":"2024-05-09T00:08:23.120004+0000","last_became_peered":"2024-05-09T00:08:23.120004+0000","last_unstale":"2024-05-09T00:09:10.469371+0000","last_undegraded":"2024-05-09T00:09:10.469371+0000","last_fullsized":"2024-05-09T00:09:10.469371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:12.168671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471920+0000","last_change":"2024-05-09T00:08:26.046554+0000","last_active":"2024-05-09T00:09:10.471920+0000","last_peered":"2024-05-09T00:09:10.471920+0000","last_clean":"2024-05-09T00:09:10.471920+0000","last_became_active":"2024-05-09T00:08:26.045924+0000","last_became_peered":"2024-05-09T00:08:26.045924+0000","last_unstale":"2024-05-09T00:09:10.471920+0000","last_undegraded":"2024-05-09T00:09:10.471920+0000","last_fullsized":"2024-05-09T00:09:10.471920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:01.208980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472382+0000","last_change":"2024-05-09T00:08:23.019778+0000","last_active":"2024-05-09T00:09:10.472382+0000","last_peered":"2024-05-09T00:09:10.472382+0000","last_clean":"2024-05-09T00:09:10.472382+0000","last_became_active":"2024-05-09T00:08:23.019400+0000","last_became_peered":"2024-05-09T00:08:23.019400+0000","last_unstale":"2024-05-09T00:09:10.472382+0000","last_undegraded":"2024-05-09T00:09:10.472382+0000","last_fullsized":"2024-05-09T00:09:10.472382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:06.883167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474352+0000","last_change":"2024-05-09T00:09:10.474352+0000","last_active":"2024-05-09T00:09:10.474352+0000","last_peered":"2024-05-09T00:09:10.474352+0000","last_clean":"2024-05-09T00:09:10.474352+0000","last_became_active":"2024-05-09T00:09:10.473906+0000","last_became_peered":"2024-05-09T00:09:10.473906+0000","last_unstale":"2024-05-09T00:09:10.474352+0000","last_undegraded":"2024-05-09T00:09:10.474352+0000","last_fullsized":"2024-05-09T00:09:10.474352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:35:18.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.476484+0000","last_change":"2024-05-09T00:09:10.476484+0000","last_active":"2024-05-09T00:09:10.476484+0000","last_peered":"2024-05-09T00:09:10.476484+0000","last_clean":"2024-05-09T00:09:10.476484+0000","last_became_active":"2024-05-09T00:09:10.476001+0000","last_became_peered":"2024-05-09T00:09:10.476001+0000","last_unstale":"2024-05-09T00:09:10.476484+0000","last_undegraded":"2024-05-09T00:09:10.476484+0000","last_fullsized":"2024-05-09T00:09:10.476484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:32.018749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":60,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:12.437133+0000","last_change":"2024-05-09T00:08:23.118322+0000","last_active":"2024-05-09T00:09:12.437133+0000","last_peered":"2024-05-09T00:09:12.437133+0000","last_clean":"2024-05-09T00:09:12.437133+0000","last_became_active":"2024-05-09T00:08:23.117943+0000","last_became_peered":"2024-05-09T00:08:23.117943+0000","last_unstale":"2024-05-09T00:09:12.437133+0000","last_undegraded":"2024-05-09T00:09:12.437133+0000","last_fullsized":"2024-05-09T00:09:12.437133+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:07.890437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467948+0000","last_change":"2024-05-09T00:08:26.052294+0000","last_active":"2024-05-09T00:09:10.467948+0000","last_peered":"2024-05-09T00:09:10.467948+0000","last_clean":"2024-05-09T00:09:10.467948+0000","last_became_active":"2024-05-09T00:08:26.051926+0000","last_became_peered":"2024-05-09T00:08:26.051926+0000","last_unstale":"2024-05-09T00:09:10.467948+0000","last_undegraded":"2024-05-09T00:09:10.467948+0000","last_fullsized":"2024-05-09T00:09:10.467948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:53.651401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796281+0000","last_change":"2024-05-09T00:08:23.015007+0000","last_active":"2024-05-09T00:09:10.796281+0000","last_peered":"2024-05-09T00:09:10.796281+0000","last_clean":"2024-05-09T00:09:10.796281+0000","last_became_active":"2024-05-09T00:08:23.014545+0000","last_became_peered":"2024-05-09T00:08:23.014545+0000","last_unstale":"2024-05-09T00:09:10.796281+0000","last_undegraded":"2024-05-09T00:09:10.796281+0000","last_fullsized":"2024-05-09T00:09:10.796281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:19.850200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472332+0000","last_change":"2024-05-09T00:07:00.733233+0000","last_active":"2024-05-09T00:09:10.472332+0000","last_peered":"2024-05-09T00:09:10.472332+0000","last_clean":"2024-05-09T00:09:10.472332+0000","last_became_active":"2024-05-09T00:07:00.727365+0000","last_became_peered":"2024-05-09T00:07:00.727365+0000","last_unstale":"2024-05-09T00:09:10.472332+0000","last_undegraded":"2024-05-09T00:09:10.472332+0000","last_fullsized":"2024-05-09T00:09:10.472332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_clean_scrub_stamp":"2024-05-09T00:03:33.285220+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:08.597290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548228+0000","last_change":"2024-05-09T00:08:26.064000+0000","last_active":"2024-05-09T00:09:10.548228+0000","last_peered":"2024-05-09T00:09:10.548228+0000","last_clean":"2024-05-09T00:09:10.548228+0000","last_became_active":"2024-05-09T00:08:26.063636+0000","last_became_peered":"2024-05-09T00:08:26.063636+0000","last_unstale":"2024-05-09T00:09:10.548228+0000","last_undegraded":"2024-05-09T00:09:10.548228+0000","last_fullsized":"2024-05-09T00:09:10.548228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:59.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472537+0000","last_change":"2024-05-09T00:08:23.023459+0000","last_active":"2024-05-09T00:09:10.472537+0000","last_peered":"2024-05-09T00:09:10.472537+0000","last_clean":"2024-05-09T00:09:10.472537+0000","last_became_active":"2024-05-09T00:08:23.023089+0000","last_became_peered":"2024-05-09T00:08:23.023089+0000","last_unstale":"2024-05-09T00:09:10.472537+0000","last_undegraded":"2024-05-09T00:09:10.472537+0000","last_fullsized":"2024-05-09T00:09:10.472537+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:41.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.479186+0000","last_change":"2024-05-09T00:09:10.478686+0000","last_active":"2024-05-09T00:09:10.479186+0000","last_peered":"2024-05-09T00:09:10.479186+0000","last_clean":"2024-05-09T00:09:10.479186+0000","last_became_active":"2024-05-09T00:09:10.474890+0000","last_became_peered":"2024-05-09T00:09:10.474890+0000","last_unstale":"2024-05-09T00:09:10.479186+0000","last_undegraded":"2024-05-09T00:09:10.479186+0000","last_fullsized":"2024-05-09T00:09:10.479186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:41.041247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548460+0000","last_change":"2024-05-09T00:08:26.050685+0000","last_active":"2024-05-09T00:09:10.548460+0000","last_peered":"2024-05-09T00:09:10.548460+0000","last_clean":"2024-05-09T00:09:10.548460+0000","last_became_active":"2024-05-09T00:08:26.050312+0000","last_became_peered":"2024-05-09T00:08:26.050312+0000","last_unstale":"2024-05-09T00:09:10.548460+0000","last_undegraded":"2024-05-09T00:09:10.548460+0000","last_fullsized":"2024-05-09T00:09:10.548460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:31.623182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'11","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469432+0000","last_change":"2024-05-09T00:08:23.019663+0000","last_active":"2024-05-09T00:09:10.469432+0000","last_peered":"2024-05-09T00:09:10.469432+0000","last_clean":"2024-05-09T00:09:10.469432+0000","last_became_active":"2024-05-09T00:08:23.019301+0000","last_became_peered":"2024-05-09T00:08:23.019301+0000","last_unstale":"2024-05-09T00:09:10.469432+0000","last_undegraded":"2024-05-09T00:09:10.469432+0000","last_fullsized":"2024-05-09T00:09:10.469432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:08.291227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468179+0000","last_change":"2024-05-09T00:08:26.058825+0000","last_active":"2024-05-09T00:09:10.468179+0000","last_peered":"2024-05-09T00:09:10.468179+0000","last_clean":"2024-05-09T00:09:10.468179+0000","last_became_active":"2024-05-09T00:08:26.058469+0000","last_became_peered":"2024-05-09T00:08:26.058469+0000","last_unstale":"2024-05-09T00:09:10.468179+0000","last_undegraded":"2024-05-09T00:09:10.468179+0000","last_fullsized":"2024-05-09T00:09:10.468179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:08.118819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796528+0000","last_change":"2024-05-09T00:08:23.054366+0000","last_active":"2024-05-09T00:09:10.796528+0000","last_peered":"2024-05-09T00:09:10.796528+0000","last_clean":"2024-05-09T00:09:10.796528+0000","last_became_active":"2024-05-09T00:08:23.053977+0000","last_became_peered":"2024-05-09T00:08:23.053977+0000","last_unstale":"2024-05-09T00:09:10.796528+0000","last_undegraded":"2024-05-09T00:09:10.796528+0000","last_fullsized":"2024-05-09T00:09:10.796528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:37.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471766+0000","last_change":"2024-05-09T00:08:26.057864+0000","last_active":"2024-05-09T00:09:10.471766+0000","last_peered":"2024-05-09T00:09:10.471766+0000","last_clean":"2024-05-09T00:09:10.471766+0000","last_became_active":"2024-05-09T00:08:26.057436+0000","last_became_peered":"2024-05-09T00:08:26.057436+0000","last_unstale":"2024-05-09T00:09:10.471766+0000","last_undegraded":"2024-05-09T00:09:10.471766+0000","last_fullsized":"2024-05-09T00:09:10.471766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:19.976265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468611+0000","last_change":"2024-05-09T00:08:23.118803+0000","last_active":"2024-05-09T00:09:10.468611+0000","last_peered":"2024-05-09T00:09:10.468611+0000","last_clean":"2024-05-09T00:09:10.468611+0000","last_became_active":"2024-05-09T00:08:23.118432+0000","last_became_peered":"2024-05-09T00:08:23.118432+0000","last_unstale":"2024-05-09T00:09:10.468611+0000","last_undegraded":"2024-05-09T00:09:10.468611+0000","last_fullsized":"2024-05-09T00:09:10.468611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:50.253699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473482+0000","last_change":"2024-05-09T00:08:26.054281+0000","last_active":"2024-05-09T00:09:10.473482+0000","last_peered":"2024-05-09T00:09:10.473482+0000","last_clean":"2024-05-09T00:09:10.473482+0000","last_became_active":"2024-05-09T00:08:26.053874+0000","last_became_peered":"2024-05-09T00:08:26.053874+0000","last_unstale":"2024-05-09T00:09:10.473482+0000","last_undegraded":"2024-05-09T00:09:10.473482+0000","last_fullsized":"2024-05-09T00:09:10.473482+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:11.566337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467696+0000","last_change":"2024-05-09T00:08:23.017905+0000","last_active":"2024-05-09T00:09:10.467696+0000","last_peered":"2024-05-09T00:09:10.467696+0000","last_clean":"2024-05-09T00:09:10.467696+0000","last_became_active":"2024-05-09T00:08:23.017481+0000","last_became_peered":"2024-05-09T00:08:23.017481+0000","last_unstale":"2024-05-09T00:09:10.467696+0000","last_undegraded":"2024-05-09T00:09:10.467696+0000","last_fullsized":"2024-05-09T00:09:10.467696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:40.096415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469888+0000","last_change":"2024-05-09T00:08:26.053392+0000","last_active":"2024-05-09T00:09:10.469888+0000","last_peered":"2024-05-09T00:09:10.469888+0000","last_clean":"2024-05-09T00:09:10.469888+0000","last_became_active":"2024-05-09T00:08:26.052628+0000","last_became_peered":"2024-05-09T00:08:26.052628+0000","last_unstale":"2024-05-09T00:09:10.469888+0000","last_undegraded":"2024-05-09T00:09:10.469888+0000","last_fullsized":"2024-05-09T00:09:10.469888+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:28.013955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.481712+0000","last_change":"2024-05-09T00:09:10.481233+0000","last_active":"2024-05-09T00:09:10.481712+0000","last_peered":"2024-05-09T00:09:10.481712+0000","last_clean":"2024-05-09T00:09:10.481712+0000","last_became_active":"2024-05-09T00:09:10.475321+0000","last_became_peered":"2024-05-09T00:09:10.475321+0000","last_unstale":"2024-05-09T00:09:10.481712+0000","last_undegraded":"2024-05-09T00:09:10.481712+0000","last_fullsized":"2024-05-09T00:09:10.481712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:58.859277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467933+0000","last_change":"2024-05-09T00:08:26.047812+0000","last_active":"2024-05-09T00:09:10.467933+0000","last_peered":"2024-05-09T00:09:10.467933+0000","last_clean":"2024-05-09T00:09:10.467933+0000","last_became_active":"2024-05-09T00:08:26.047459+0000","last_became_peered":"2024-05-09T00:08:26.047459+0000","last_unstale":"2024-05-09T00:09:10.467933+0000","last_undegraded":"2024-05-09T00:09:10.467933+0000","last_fullsized":"2024-05-09T00:09:10.467933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.257831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473758+0000","last_change":"2024-05-09T00:08:23.023800+0000","last_active":"2024-05-09T00:09:10.473758+0000","last_peered":"2024-05-09T00:09:10.473758+0000","last_clean":"2024-05-09T00:09:10.473758+0000","last_became_active":"2024-05-09T00:08:23.023387+0000","last_became_peered":"2024-05-09T00:08:23.023387+0000","last_unstale":"2024-05-09T00:09:10.473758+0000","last_undegraded":"2024-05-09T00:09:10.473758+0000","last_fullsized":"2024-05-09T00:09:10.473758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:33.714683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472532+0000","last_change":"2024-05-09T00:08:26.056981+0000","last_active":"2024-05-09T00:09:10.472532+0000","last_peered":"2024-05-09T00:09:10.472532+0000","last_clean":"2024-05-09T00:09:10.472532+0000","last_became_active":"2024-05-09T00:08:26.056605+0000","last_became_peered":"2024-05-09T00:08:26.056605+0000","last_unstale":"2024-05-09T00:09:10.472532+0000","last_undegraded":"2024-05-09T00:09:10.472532+0000","last_fullsized":"2024-05-09T00:09:10.472532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:32.897929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473611+0000","last_change":"2024-05-09T00:08:23.120612+0000","last_active":"2024-05-09T00:09:10.473611+0000","last_peered":"2024-05-09T00:09:10.473611+0000","last_clean":"2024-05-09T00:09:10.473611+0000","last_became_active":"2024-05-09T00:08:23.120157+0000","last_became_peered":"2024-05-09T00:08:23.120157+0000","last_unstale":"2024-05-09T00:09:10.473611+0000","last_undegraded":"2024-05-09T00:09:10.473611+0000","last_fullsized":"2024-05-09T00:09:10.473611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:08.027214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473165+0000","last_change":"2024-05-09T00:08:26.061594+0000","last_active":"2024-05-09T00:09:10.473165+0000","last_peered":"2024-05-09T00:09:10.473165+0000","last_clean":"2024-05-09T00:09:10.473165+0000","last_became_active":"2024-05-09T00:08:26.061201+0000","last_became_peered":"2024-05-09T00:08:26.061201+0000","last_unstale":"2024-05-09T00:09:10.473165+0000","last_undegraded":"2024-05-09T00:09:10.473165+0000","last_fullsized":"2024-05-09T00:09:10.473165+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:50.183648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469471+0000","last_change":"2024-05-09T00:08:23.017693+0000","last_active":"2024-05-09T00:09:10.469471+0000","last_peered":"2024-05-09T00:09:10.469471+0000","last_clean":"2024-05-09T00:09:10.469471+0000","last_became_active":"2024-05-09T00:08:23.017211+0000","last_became_peered":"2024-05-09T00:08:23.017211+0000","last_unstale":"2024-05-09T00:09:10.469471+0000","last_undegraded":"2024-05-09T00:09:10.469471+0000","last_fullsized":"2024-05-09T00:09:10.469471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:53.569932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472752+0000","last_change":"2024-05-09T00:08:26.063922+0000","last_active":"2024-05-09T00:09:10.472752+0000","last_peered":"2024-05-09T00:09:10.472752+0000","last_clean":"2024-05-09T00:09:10.472752+0000","last_became_active":"2024-05-09T00:08:26.063554+0000","last_became_peered":"2024-05-09T00:08:26.063554+0000","last_unstale":"2024-05-09T00:09:10.472752+0000","last_undegraded":"2024-05-09T00:09:10.472752+0000","last_fullsized":"2024-05-09T00:09:10.472752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:01.754688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548572+0000","last_change":"2024-05-09T00:08:23.020367+0000","last_active":"2024-05-09T00:09:10.548572+0000","last_peered":"2024-05-09T00:09:10.548572+0000","last_clean":"2024-05-09T00:09:10.548572+0000","last_became_active":"2024-05-09T00:08:23.019801+0000","last_became_peered":"2024-05-09T00:08:23.019801+0000","last_unstale":"2024-05-09T00:09:10.548572+0000","last_undegraded":"2024-05-09T00:09:10.548572+0000","last_fullsized":"2024-05-09T00:09:10.548572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:14.936021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472767+0000","last_change":"2024-05-09T00:08:26.063863+0000","last_active":"2024-05-09T00:09:10.472767+0000","last_peered":"2024-05-09T00:09:10.472767+0000","last_clean":"2024-05-09T00:09:10.472767+0000","last_became_active":"2024-05-09T00:08:26.061938+0000","last_became_peered":"2024-05-09T00:08:26.061938+0000","last_unstale":"2024-05-09T00:09:10.472767+0000","last_undegraded":"2024-05-09T00:09:10.472767+0000","last_fullsized":"2024-05-09T00:09:10.472767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:04.597991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470033+0000","last_change":"2024-05-09T00:08:26.050837+0000","last_active":"2024-05-09T00:09:10.470033+0000","last_peered":"2024-05-09T00:09:10.470033+0000","last_clean":"2024-05-09T00:09:10.470033+0000","last_became_active":"2024-05-09T00:08:26.050424+0000","last_became_peered":"2024-05-09T00:08:26.050424+0000","last_unstale":"2024-05-09T00:09:10.470033+0000","last_undegraded":"2024-05-09T00:09:10.470033+0000","last_fullsized":"2024-05-09T00:09:10.470033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:45.134811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.549989+0000","last_change":"2024-05-09T00:09:10.549989+0000","last_active":"2024-05-09T00:09:10.549989+0000","last_peered":"2024-05-09T00:09:10.549989+0000","last_clean":"2024-05-09T00:09:10.549989+0000","last_became_active":"2024-05-09T00:09:10.549502+0000","last_became_peered":"2024-05-09T00:09:10.549502+0000","last_unstale":"2024-05-09T00:09:10.549989+0000","last_undegraded":"2024-05-09T00:09:10.549989+0000","last_fullsized":"2024-05-09T00:09:10.549989+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:10.654452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473094+0000","last_change":"2024-05-09T00:08:26.058012+0000","last_active":"2024-05-09T00:09:10.473094+0000","last_peered":"2024-05-09T00:09:10.473094+0000","last_clean":"2024-05-09T00:09:10.473094+0000","last_became_active":"2024-05-09T00:08:26.057282+0000","last_became_peered":"2024-05-09T00:08:26.057282+0000","last_unstale":"2024-05-09T00:09:10.473094+0000","last_undegraded":"2024-05-09T00:09:10.473094+0000","last_fullsized":"2024-05-09T00:09:10.473094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:53.520593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471555+0000","last_change":"2024-05-09T00:08:26.063331+0000","last_active":"2024-05-09T00:09:10.471555+0000","last_peered":"2024-05-09T00:09:10.471555+0000","last_clean":"2024-05-09T00:09:10.471555+0000","last_became_active":"2024-05-09T00:08:26.062866+0000","last_became_peered":"2024-05-09T00:08:26.062866+0000","last_unstale":"2024-05-09T00:09:10.471555+0000","last_undegraded":"2024-05-09T00:09:10.471555+0000","last_fullsized":"2024-05-09T00:09:10.471555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:10.387852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548504+0000","last_change":"2024-05-09T00:08:26.063735+0000","last_active":"2024-05-09T00:09:10.548504+0000","last_peered":"2024-05-09T00:09:10.548504+0000","last_clean":"2024-05-09T00:09:10.548504+0000","last_became_active":"2024-05-09T00:08:26.063358+0000","last_became_peered":"2024-05-09T00:08:26.063358+0000","last_unstale":"2024-05-09T00:09:10.548504+0000","last_undegraded":"2024-05-09T00:09:10.548504+0000","last_fullsized":"2024-05-09T00:09:10.548504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:06.832071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471634+0000","last_change":"2024-05-09T00:08:26.052746+0000","last_active":"2024-05-09T00:09:10.471634+0000","last_peered":"2024-05-09T00:09:10.471634+0000","last_clean":"2024-05-09T00:09:10.471634+0000","last_became_active":"2024-05-09T00:08:26.051544+0000","last_became_peered":"2024-05-09T00:08:26.051544+0000","last_unstale":"2024-05-09T00:09:10.471634+0000","last_undegraded":"2024-05-09T00:09:10.471634+0000","last_fullsized":"2024-05-09T00:09:10.471634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:31.963268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473534+0000","last_change":"2024-05-09T00:08:26.058075+0000","last_active":"2024-05-09T00:09:10.473534+0000","last_peered":"2024-05-09T00:09:10.473534+0000","last_clean":"2024-05-09T00:09:10.473534+0000","last_became_active":"2024-05-09T00:08:26.057349+0000","last_became_peered":"2024-05-09T00:08:26.057349+0000","last_unstale":"2024-05-09T00:09:10.473534+0000","last_undegraded":"2024-05-09T00:09:10.473534+0000","last_fullsized":"2024-05-09T00:09:10.473534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:59.322903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468103+0000","last_change":"2024-05-09T00:08:26.056429+0000","last_active":"2024-05-09T00:09:10.468103+0000","last_peered":"2024-05-09T00:09:10.468103+0000","last_clean":"2024-05-09T00:09:10.468103+0000","last_became_active":"2024-05-09T00:08:26.054852+0000","last_became_peered":"2024-05-09T00:08:26.054852+0000","last_unstale":"2024-05-09T00:09:10.468103+0000","last_undegraded":"2024-05-09T00:09:10.468103+0000","last_fullsized":"2024-05-09T00:09:10.468103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:05.337636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.549880+0000","last_change":"2024-05-09T00:09:10.549880+0000","last_active":"2024-05-09T00:09:10.549880+0000","last_peered":"2024-05-09T00:09:10.549880+0000","last_clean":"2024-05-09T00:09:10.549880+0000","last_became_active":"2024-05-09T00:09:10.549448+0000","last_became_peered":"2024-05-09T00:09:10.549448+0000","last_unstale":"2024-05-09T00:09:10.549880+0000","last_undegraded":"2024-05-09T00:09:10.549880+0000","last_fullsized":"2024-05-09T00:09:10.549880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:16.990729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467623+0000","last_change":"2024-05-09T00:08:26.054579+0000","last_active":"2024-05-09T00:09:10.467623+0000","last_peered":"2024-05-09T00:09:10.467623+0000","last_clean":"2024-05-09T00:09:10.467623+0000","last_became_active":"2024-05-09T00:08:26.053186+0000","last_became_peered":"2024-05-09T00:08:26.053186+0000","last_unstale":"2024-05-09T00:09:10.467623+0000","last_undegraded":"2024-05-09T00:09:10.467623+0000","last_fullsized":"2024-05-09T00:09:10.467623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:37.561691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468682+0000","last_change":"2024-05-09T00:08:26.063608+0000","last_active":"2024-05-09T00:09:10.468682+0000","last_peered":"2024-05-09T00:09:10.468682+0000","last_clean":"2024-05-09T00:09:10.468682+0000","last_became_active":"2024-05-09T00:08:26.063078+0000","last_became_peered":"2024-05-09T00:08:26.063078+0000","last_unstale":"2024-05-09T00:09:10.468682+0000","last_undegraded":"2024-05-09T00:09:10.468682+0000","last_fullsized":"2024-05-09T00:09:10.468682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:06.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548642+0000","last_change":"2024-05-09T00:08:26.055058+0000","last_active":"2024-05-09T00:09:10.548642+0000","last_peered":"2024-05-09T00:09:10.548642+0000","last_clean":"2024-05-09T00:09:10.548642+0000","last_became_active":"2024-05-09T00:08:26.054695+0000","last_became_peered":"2024-05-09T00:08:26.054695+0000","last_unstale":"2024-05-09T00:09:10.548642+0000","last_undegraded":"2024-05-09T00:09:10.548642+0000","last_fullsized":"2024-05-09T00:09:10.548642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:58.925990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470339+0000","last_change":"2024-05-09T00:08:26.055366+0000","last_active":"2024-05-09T00:09:10.470339+0000","last_peered":"2024-05-09T00:09:10.470339+0000","last_clean":"2024-05-09T00:09:10.470339+0000","last_became_active":"2024-05-09T00:08:26.054904+0000","last_became_peered":"2024-05-09T00:08:26.054904+0000","last_unstale":"2024-05-09T00:09:10.470339+0000","last_undegraded":"2024-05-09T00:09:10.470339+0000","last_fullsized":"2024-05-09T00:09:10.470339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:40.754878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285241,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65827,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":209,"num_read_kb":172,"num_write":165,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":197481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710744,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":425247,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906974,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1078923,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070498,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":425461,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266728,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":425359,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":1017311,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":868352,"data_stored":424850,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":486469,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":888832,"data_stored":426121,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":481574,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":425161,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":425678,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":1016604,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:09:18.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph status --format=json 2024-05-09T00:09:18.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:18 smithi169 ceph-mon[34777]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 895 B/s wr, 4 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T00:09:18.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:18 smithi142 ceph-mon[34533]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 895 B/s wr, 4 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T00:09:18.987 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:19.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:19 smithi046 ceph-mon[30151]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:19.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:19 smithi169 ceph-mon[34777]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:19.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:19 smithi142 ceph-mon[34533]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:20.705 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:20 smithi046 ceph-mon[30151]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 728 B/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T00:09:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.720+0000 7f97f9fca700 1 -- 172.21.15.46:0/4064908900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f41034c0 msgr2=0x7f97f41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.720+0000 7f97f9fca700 1 --2- 172.21.15.46:0/4064908900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f41034c0 0x7f97f41038d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f97e40099c0 tx=0x7f97e4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.720+0000 7f97f9fca700 1 -- 172.21.15.46:0/4064908900 shutdown_connections 2024-05-09T00:09:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.720+0000 7f97f9fca700 1 --2- 172.21.15.46:0/4064908900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f4108740 0x7f97f41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.720+0000 7f97f9fca700 1 --2- 172.21.15.46:0/4064908900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f41034c0 0x7f97f41038d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.720+0000 7f97f9fca700 1 --2- 172.21.15.46:0/4064908900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97f4107df0 0x7f97f4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.720+0000 7f97f9fca700 1 -- 172.21.15.46:0/4064908900 >> 172.21.15.46:0/4064908900 conn(0x7f97f40fedc0 msgr2=0x7f97f41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.720+0000 7f97f9fca700 1 -- 172.21.15.46:0/4064908900 shutdown_connections 2024-05-09T00:09:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.720+0000 7f97f9fca700 1 -- 172.21.15.46:0/4064908900 wait complete. 2024-05-09T00:09:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.721+0000 7f97f9fca700 1 Processor -- start 2024-05-09T00:09:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.721+0000 7f97f9fca700 1 -- start start 2024-05-09T00:09:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97f9fca700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97f41034c0 0x7f97f419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97f9fca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4107df0 0x7f97f419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97f9fca700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f4108740 0x7f97f41a6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97f9fca700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4113390 con 0x7f97f4107df0 2024-05-09T00:09:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97f9fca700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4113210 con 0x7f97f41034c0 2024-05-09T00:09:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97f9fca700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4113510 con 0x7f97f4108740 2024-05-09T00:09:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4107df0 0x7f97f419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4107df0 0x7f97f419c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56030/0 (socket says 172.21.15.46:56030) 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97eaffd700 1 -- 172.21.15.46:0/2415274906 learned_addr learned my addr 172.21.15.46:0/2415274906 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97f37fe700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97f41034c0 0x7f97f419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97f3fff700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f4108740 0x7f97f41a6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97eaffd700 1 -- 172.21.15.46:0/2415274906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f4108740 msgr2=0x7f97f41a6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97eaffd700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f4108740 0x7f97f41a6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97eaffd700 1 -- 172.21.15.46:0/2415274906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97f41034c0 msgr2=0x7f97f419bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97eaffd700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97f41034c0 0x7f97f419bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.722+0000 7f97eaffd700 1 -- 172.21.15.46:0/2415274906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f419ca00 con 0x7f97f4107df0 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.723+0000 7f97eaffd700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4107df0 0x7f97f419c150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00d950 tx=0x7f97dc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.723+0000 7f97f17fa700 1 -- 172.21.15.46:0/2415274906 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc016070 con 0x7f97f4107df0 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.723+0000 7f97f17fa700 1 -- 172.21.15.46:0/2415274906 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97dc004830 con 0x7f97f4107df0 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.723+0000 7f97f9fca700 1 -- 172.21.15.46:0/2415274906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f41a72f0 con 0x7f97f4107df0 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.723+0000 7f97f17fa700 1 -- 172.21.15.46:0/2415274906 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc005440 con 0x7f97f4107df0 2024-05-09T00:09:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.723+0000 7f97f9fca700 1 -- 172.21.15.46:0/2415274906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f41a7830 con 0x7f97f4107df0 2024-05-09T00:09:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.724+0000 7f97f9fca700 1 -- 172.21.15.46:0/2415274906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f40512c0 con 0x7f97f4107df0 2024-05-09T00:09:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.726+0000 7f97f17fa700 1 -- 172.21.15.46:0/2415274906 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f97dc0055e0 con 0x7f97f4107df0 2024-05-09T00:09:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.727+0000 7f97f17fa700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97e00b2990 0x7f97e00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.727+0000 7f97f37fe700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97e00b2990 0x7f97e00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.727+0000 7f97f17fa700 1 -- 172.21.15.46:0/2415274906 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f97dc0e25d0 con 0x7f97f4107df0 2024-05-09T00:09:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.728+0000 7f97f37fe700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97e00b2990 0x7f97e00b4ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009990 tx=0x7f97e40036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:20.731+0000 7f97f17fa700 1 -- 172.21.15.46:0/2415274906 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f97dc01f020 con 0x7f97f4107df0 2024-05-09T00:09:20.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:20 smithi169 ceph-mon[34777]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 728 B/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T00:09:20.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:20 smithi142 ceph-mon[34533]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 728 B/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T00:09:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.035+0000 7f97f9fca700 1 -- 172.21.15.46:0/2415274906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f97f402e5f0 con 0x7f97f4107df0 2024-05-09T00:09:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.036+0000 7f97f17fa700 1 -- 172.21.15.46:0/2415274906 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2461 (secure 0 0 0) 0x7f97dc0aa150 con 0x7f97f4107df0 2024-05-09T00:09:21.032 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:09:21.032 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":433,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1715213243,"num_in_osds":12,"osd_in_since":1715213223,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":64,"data_bytes":656429,"bytes_used":381313024,"bytes_avail":1151559114752,"bytes_total":1151940427776,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":2,"recovering_keys_per_sec":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"read_bytes_sec":3747,"write_bytes_sec":728,"read_op_per_sec":6,"write_op_per_sec":0},"fsmap":{"epoch":25,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24446},{"filesystem_id":1,"rank":1,"name":"h","status":"up:standby-replay","gid":24470},{"filesystem_id":1,"rank":0,"name":"f","status":"up:standby-replay","gid":24481},{"filesystem_id":1,"rank":2,"name":"c","status":"up:active","gid":24487},{"filesystem_id":1,"rank":1,"name":"b","status":"up:active","gid":24497},{"filesystem_id":1,"rank":0,"name":"i","status":"up:active","gid":24499}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.46:8443/"}},"servicemap":{"epoch":24,"modified":"2024-05-09T00:09:11.292346+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-09T00:09:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.039+0000 7f97f9fca700 1 -- 172.21.15.46:0/2415274906 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97e00b2990 msgr2=0x7f97e00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.039+0000 7f97f9fca700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97e00b2990 0x7f97e00b4ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009990 tx=0x7f97e40036e0 comp rx=0 tx=0).stop 2024-05-09T00:09:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.039+0000 7f97f9fca700 1 -- 172.21.15.46:0/2415274906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4107df0 msgr2=0x7f97f419c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.039+0000 7f97f9fca700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4107df0 0x7f97f419c150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00d950 tx=0x7f97dc00dc60 comp rx=0 tx=0).stop 2024-05-09T00:09:21.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.039+0000 7f97f9fca700 1 -- 172.21.15.46:0/2415274906 shutdown_connections 2024-05-09T00:09:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.039+0000 7f97f9fca700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97e00b2990 0x7f97e00b4ea0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.039+0000 7f97f9fca700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f4108740 0x7f97f41a6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.039+0000 7f97f9fca700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97f41034c0 0x7f97f419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.039+0000 7f97f9fca700 1 --2- 172.21.15.46:0/2415274906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4107df0 0x7f97f419c150 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.039+0000 7f97f9fca700 1 -- 172.21.15.46:0/2415274906 >> 172.21.15.46:0/2415274906 conn(0x7f97f40fedc0 msgr2=0x7f97f41075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.039+0000 7f97f9fca700 1 -- 172.21.15.46:0/2415274906 shutdown_connections 2024-05-09T00:09:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:21.039+0000 7f97f9fca700 1 -- 172.21.15.46:0/2415274906 wait complete. 2024-05-09T00:09:21.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:21 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2415274906' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-09T00:09:21.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:21 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:09:21.711 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-05-09T00:09:21.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T00:09:21.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:21 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2415274906' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-09T00:09:21.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:21 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:09:21.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:21 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2415274906' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-09T00:09:21.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:21 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:09:22.602 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:22.689 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:22 smithi142 ceph-mon[34533]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 661 B/s wr, 5 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T00:09:22.726 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:22 smithi046 ceph-mon[30151]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 661 B/s wr, 5 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T00:09:22.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:22 smithi169 ceph-mon[34777]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 661 B/s wr, 5 op/s; 2 B/s, 0 objects/s recovering 2024-05-09T00:09:24.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:24.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:24.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:24.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:23 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:24.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:23 smithi046 ceph-mon[30151]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s; 1 B/s, 0 objects/s recovering 2024-05-09T00:09:24.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:24.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:24.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:24.189 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:23 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:24.190 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:23 smithi142 ceph-mon[34533]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s; 1 B/s, 0 objects/s recovering 2024-05-09T00:09:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.331+0000 7f6b64e5c700 1 -- 172.21.15.46:0/699230352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b60108f10 msgr2=0x7f6b60109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.331+0000 7f6b64e5c700 1 --2- 172.21.15.46:0/699230352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b60108f10 0x7f6b60109320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6b480099c0 tx=0x7f6b48009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.331+0000 7f6b64e5c700 1 -- 172.21.15.46:0/699230352 shutdown_connections 2024-05-09T00:09:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.331+0000 7f6b64e5c700 1 --2- 172.21.15.46:0/699230352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b60109860 0x7f6b601111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.331+0000 7f6b64e5c700 1 --2- 172.21.15.46:0/699230352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b601045e0 0x7f6b601049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.331+0000 7f6b64e5c700 1 --2- 172.21.15.46:0/699230352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b60108f10 0x7f6b60109320 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.331+0000 7f6b64e5c700 1 -- 172.21.15.46:0/699230352 >> 172.21.15.46:0/699230352 conn(0x7f6b600ffe70 msgr2=0x7f6b601022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.331+0000 7f6b64e5c700 1 -- 172.21.15.46:0/699230352 shutdown_connections 2024-05-09T00:09:24.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.331+0000 7f6b64e5c700 1 -- 172.21.15.46:0/699230352 wait complete. 2024-05-09T00:09:24.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.332+0000 7f6b64e5c700 1 Processor -- start 2024-05-09T00:09:24.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.332+0000 7f6b64e5c700 1 -- start start 2024-05-09T00:09:24.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.332+0000 7f6b64e5c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b601045e0 0x7f6b6019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:24.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.332+0000 7f6b64e5c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b60108f10 0x7f6b6019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:24.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.332+0000 7f6b64e5c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b60109860 0x7f6b6019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:24.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.332+0000 7f6b64e5c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601144e0 con 0x7f6b60108f10 2024-05-09T00:09:24.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.332+0000 7f6b64e5c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60114360 con 0x7f6b60109860 2024-05-09T00:09:24.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.332+0000 7f6b64e5c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60114660 con 0x7f6b601045e0 2024-05-09T00:09:24.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.332+0000 7f6b55d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b60108f10 0x7f6b6019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:24.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.332+0000 7f6b55d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b60108f10 0x7f6b6019b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56042/0 (socket says 172.21.15.46:56042) 2024-05-09T00:09:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.332+0000 7f6b55d9b700 1 -- 172.21.15.46:0/4205353007 learned_addr learned my addr 172.21.15.46:0/4205353007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b5e59c700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b601045e0 0x7f6b6019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b5ed9d700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b60109860 0x7f6b6019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b55d9b700 1 -- 172.21.15.46:0/4205353007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b601045e0 msgr2=0x7f6b6019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b55d9b700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b601045e0 0x7f6b6019aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b55d9b700 1 -- 172.21.15.46:0/4205353007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b60109860 msgr2=0x7f6b6019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b55d9b700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b60109860 0x7f6b6019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b55d9b700 1 -- 172.21.15.46:0/4205353007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6019fd80 con 0x7f6b60108f10 2024-05-09T00:09:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b55d9b700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b60108f10 0x7f6b6019b450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6b48006010 tx=0x7f6b480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b57fff700 1 -- 172.21.15.46:0/4205353007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48026070 con 0x7f6b60108f10 2024-05-09T00:09:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b64e5c700 1 -- 172.21.15.46:0/4205353007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b601a0010 con 0x7f6b60108f10 2024-05-09T00:09:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b57fff700 1 -- 172.21.15.46:0/4205353007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b480058e0 con 0x7f6b60108f10 2024-05-09T00:09:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b57fff700 1 -- 172.21.15.46:0/4205353007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800fc20 con 0x7f6b60108f10 2024-05-09T00:09:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.333+0000 7f6b64e5c700 1 -- 172.21.15.46:0/4205353007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b601a7990 con 0x7f6b60108f10 2024-05-09T00:09:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.335+0000 7f6b64e5c700 1 -- 172.21.15.46:0/4205353007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b600512c0 con 0x7f6b60108f10 2024-05-09T00:09:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.336+0000 7f6b57fff700 1 -- 172.21.15.46:0/4205353007 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6b48029070 con 0x7f6b60108f10 2024-05-09T00:09:24.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.337+0000 7f6b57fff700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6b4c0b2910 0x7f6b4c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:24.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.338+0000 7f6b5e59c700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6b4c0b2910 0x7f6b4c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.338+0000 7f6b57fff700 1 -- 172.21.15.46:0/4205353007 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6b480e2fa0 con 0x7f6b60108f10 2024-05-09T00:09:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.338+0000 7f6b5e59c700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6b4c0b2910 0x7f6b4c0b4e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000a400 tx=0x7f6b5000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.341+0000 7f6b57fff700 1 -- 172.21.15.46:0/4205353007 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b48013070 con 0x7f6b60108f10 2024-05-09T00:09:24.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:24.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:24.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:24.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:23 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:24.352 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:23 smithi169 ceph-mon[34777]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s; 1 B/s, 0 objects/s recovering 2024-05-09T00:09:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.575+0000 7f6b64e5c700 1 -- 172.21.15.46:0/4205353007 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6b6019c210 con 0x7f6b4c0b2910 2024-05-09T00:09:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.583+0000 7f6b57fff700 1 -- 172.21.15.46:0/4205353007 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258960 (secure 0 0 0) 0x7f6b6019c210 con 0x7f6b4c0b2910 2024-05-09T00:09:24.583 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:09:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.587+0000 7f6b64e5c700 1 -- 172.21.15.46:0/4205353007 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6b4c0b2910 msgr2=0x7f6b4c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.587+0000 7f6b64e5c700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6b4c0b2910 0x7f6b4c0b4e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000a400 tx=0x7f6b5000d040 comp rx=0 tx=0).stop 2024-05-09T00:09:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.587+0000 7f6b64e5c700 1 -- 172.21.15.46:0/4205353007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b60108f10 msgr2=0x7f6b6019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.587+0000 7f6b64e5c700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b60108f10 0x7f6b6019b450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6b48006010 tx=0x7f6b480045e0 comp rx=0 tx=0).stop 2024-05-09T00:09:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.587+0000 7f6b64e5c700 1 -- 172.21.15.46:0/4205353007 shutdown_connections 2024-05-09T00:09:24.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.587+0000 7f6b64e5c700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6b4c0b2910 0x7f6b4c0b4e20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.587+0000 7f6b64e5c700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b60109860 0x7f6b6019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.587+0000 7f6b64e5c700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b601045e0 0x7f6b6019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.587+0000 7f6b64e5c700 1 --2- 172.21.15.46:0/4205353007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b60108f10 0x7f6b6019b450 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.587+0000 7f6b64e5c700 1 -- 172.21.15.46:0/4205353007 >> 172.21.15.46:0/4205353007 conn(0x7f6b600ffe70 msgr2=0x7f6b601086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.587+0000 7f6b64e5c700 1 -- 172.21.15.46:0/4205353007 shutdown_connections 2024-05-09T00:09:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:24.587+0000 7f6b64e5c700 1 -- 172.21.15.46:0/4205353007 wait complete. 2024-05-09T00:09:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T00:09:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-05-09T00:09:23.295530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656429,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":320,"num_read_kb":251,"num_write":290,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372376,"kb_used_data":10656,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569448,"statfs":{"total":1151940427776,"available":1151559114752,"internally_reserved":0,"allocated":10911744,"data_stored":7065003,"data_compressed":177897,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6245,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":36,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003222"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471697+0000","last_change":"2024-05-09T00:08:26.060455+0000","last_active":"2024-05-09T00:09:10.471697+0000","last_peered":"2024-05-09T00:09:10.471697+0000","last_clean":"2024-05-09T00:09:10.471697+0000","last_became_active":"2024-05-09T00:08:26.060068+0000","last_became_peered":"2024-05-09T00:08:26.060068+0000","last_unstale":"2024-05-09T00:09:10.471697+0000","last_undegraded":"2024-05-09T00:09:10.471697+0000","last_fullsized":"2024-05-09T00:09:10.471697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:48.354188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471761+0000","last_change":"2024-05-09T00:08:26.055737+0000","last_active":"2024-05-09T00:09:10.471761+0000","last_peered":"2024-05-09T00:09:10.471761+0000","last_clean":"2024-05-09T00:09:10.471761+0000","last_became_active":"2024-05-09T00:08:26.055354+0000","last_became_peered":"2024-05-09T00:08:26.055354+0000","last_unstale":"2024-05-09T00:09:10.471761+0000","last_undegraded":"2024-05-09T00:09:10.471761+0000","last_fullsized":"2024-05-09T00:09:10.471761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:54.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471480+0000","last_change":"2024-05-09T00:08:26.050776+0000","last_active":"2024-05-09T00:09:10.471480+0000","last_peered":"2024-05-09T00:09:10.471480+0000","last_clean":"2024-05-09T00:09:10.471480+0000","last_became_active":"2024-05-09T00:08:26.050393+0000","last_became_peered":"2024-05-09T00:08:26.050393+0000","last_unstale":"2024-05-09T00:09:10.471480+0000","last_undegraded":"2024-05-09T00:09:10.471480+0000","last_fullsized":"2024-05-09T00:09:10.471480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:30.819219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796411+0000","last_change":"2024-05-09T00:08:26.065946+0000","last_active":"2024-05-09T00:09:10.796411+0000","last_peered":"2024-05-09T00:09:10.796411+0000","last_clean":"2024-05-09T00:09:10.796411+0000","last_became_active":"2024-05-09T00:08:26.065498+0000","last_became_peered":"2024-05-09T00:08:26.065498+0000","last_unstale":"2024-05-09T00:09:10.796411+0000","last_undegraded":"2024-05-09T00:09:10.796411+0000","last_fullsized":"2024-05-09T00:09:10.796411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:04.444860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473630+0000","last_change":"2024-05-09T00:08:26.051961+0000","last_active":"2024-05-09T00:09:10.473630+0000","last_peered":"2024-05-09T00:09:10.473630+0000","last_clean":"2024-05-09T00:09:10.473630+0000","last_became_active":"2024-05-09T00:08:26.051548+0000","last_became_peered":"2024-05-09T00:08:26.051548+0000","last_unstale":"2024-05-09T00:09:10.473630+0000","last_undegraded":"2024-05-09T00:09:10.473630+0000","last_fullsized":"2024-05-09T00:09:10.473630+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:07.539117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471811+0000","last_change":"2024-05-09T00:08:26.052012+0000","last_active":"2024-05-09T00:09:10.471811+0000","last_peered":"2024-05-09T00:09:10.471811+0000","last_clean":"2024-05-09T00:09:10.471811+0000","last_became_active":"2024-05-09T00:08:26.050886+0000","last_became_peered":"2024-05-09T00:08:26.050886+0000","last_unstale":"2024-05-09T00:09:10.471811+0000","last_undegraded":"2024-05-09T00:09:10.471811+0000","last_fullsized":"2024-05-09T00:09:10.471811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:14.651888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472730+0000","last_change":"2024-05-09T00:08:26.058418+0000","last_active":"2024-05-09T00:09:10.472730+0000","last_peered":"2024-05-09T00:09:10.472730+0000","last_clean":"2024-05-09T00:09:10.472730+0000","last_became_active":"2024-05-09T00:08:26.058039+0000","last_became_peered":"2024-05-09T00:08:26.058039+0000","last_unstale":"2024-05-09T00:09:10.472730+0000","last_undegraded":"2024-05-09T00:09:10.472730+0000","last_fullsized":"2024-05-09T00:09:10.472730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:13.678123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467742+0000","last_change":"2024-05-09T00:08:26.057602+0000","last_active":"2024-05-09T00:09:10.467742+0000","last_peered":"2024-05-09T00:09:10.467742+0000","last_clean":"2024-05-09T00:09:10.467742+0000","last_became_active":"2024-05-09T00:08:26.057169+0000","last_became_peered":"2024-05-09T00:08:26.057169+0000","last_unstale":"2024-05-09T00:09:10.467742+0000","last_undegraded":"2024-05-09T00:09:10.467742+0000","last_fullsized":"2024-05-09T00:09:10.467742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:11.452083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469166+0000","last_change":"2024-05-09T00:08:26.067639+0000","last_active":"2024-05-09T00:09:10.469166+0000","last_peered":"2024-05-09T00:09:10.469166+0000","last_clean":"2024-05-09T00:09:10.469166+0000","last_became_active":"2024-05-09T00:08:26.067226+0000","last_became_peered":"2024-05-09T00:08:26.067226+0000","last_unstale":"2024-05-09T00:09:10.469166+0000","last_undegraded":"2024-05-09T00:09:10.469166+0000","last_fullsized":"2024-05-09T00:09:10.469166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:12.366500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472299+0000","last_change":"2024-05-09T00:08:26.055234+0000","last_active":"2024-05-09T00:09:10.472299+0000","last_peered":"2024-05-09T00:09:10.472299+0000","last_clean":"2024-05-09T00:09:10.472299+0000","last_became_active":"2024-05-09T00:08:26.054851+0000","last_became_peered":"2024-05-09T00:08:26.054851+0000","last_unstale":"2024-05-09T00:09:10.472299+0000","last_undegraded":"2024-05-09T00:09:10.472299+0000","last_fullsized":"2024-05-09T00:09:10.472299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:54:52.121329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472899+0000","last_change":"2024-05-09T00:08:26.063375+0000","last_active":"2024-05-09T00:09:10.472899+0000","last_peered":"2024-05-09T00:09:10.472899+0000","last_clean":"2024-05-09T00:09:10.472899+0000","last_became_active":"2024-05-09T00:08:26.062962+0000","last_became_peered":"2024-05-09T00:08:26.062962+0000","last_unstale":"2024-05-09T00:09:10.472899+0000","last_undegraded":"2024-05-09T00:09:10.472899+0000","last_fullsized":"2024-05-09T00:09:10.472899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:05.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796257+0000","last_change":"2024-05-09T00:08:26.062034+0000","last_active":"2024-05-09T00:09:10.796257+0000","last_peered":"2024-05-09T00:09:10.796257+0000","last_clean":"2024-05-09T00:09:10.796257+0000","last_became_active":"2024-05-09T00:08:26.060376+0000","last_became_peered":"2024-05-09T00:08:26.060376+0000","last_unstale":"2024-05-09T00:09:10.796257+0000","last_undegraded":"2024-05-09T00:09:10.796257+0000","last_fullsized":"2024-05-09T00:09:10.796257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:15.830024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548685+0000","last_change":"2024-05-09T00:08:26.053227+0000","last_active":"2024-05-09T00:09:10.548685+0000","last_peered":"2024-05-09T00:09:10.548685+0000","last_clean":"2024-05-09T00:09:10.548685+0000","last_became_active":"2024-05-09T00:08:26.052878+0000","last_became_peered":"2024-05-09T00:08:26.052878+0000","last_unstale":"2024-05-09T00:09:10.548685+0000","last_undegraded":"2024-05-09T00:09:10.548685+0000","last_fullsized":"2024-05-09T00:09:10.548685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:24.385026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471290+0000","last_change":"2024-05-09T00:08:26.053041+0000","last_active":"2024-05-09T00:09:10.471290+0000","last_peered":"2024-05-09T00:09:10.471290+0000","last_clean":"2024-05-09T00:09:10.471290+0000","last_became_active":"2024-05-09T00:08:26.052677+0000","last_became_peered":"2024-05-09T00:08:26.052677+0000","last_unstale":"2024-05-09T00:09:10.471290+0000","last_undegraded":"2024-05-09T00:09:10.471290+0000","last_fullsized":"2024-05-09T00:09:10.471290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:24.478270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471141+0000","last_change":"2024-05-09T00:08:26.063391+0000","last_active":"2024-05-09T00:09:10.471141+0000","last_peered":"2024-05-09T00:09:10.471141+0000","last_clean":"2024-05-09T00:09:10.471141+0000","last_became_active":"2024-05-09T00:08:26.063033+0000","last_became_peered":"2024-05-09T00:08:26.063033+0000","last_unstale":"2024-05-09T00:09:10.471141+0000","last_undegraded":"2024-05-09T00:09:10.471141+0000","last_fullsized":"2024-05-09T00:09:10.471141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:41.900453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468575+0000","last_change":"2024-05-09T00:08:26.065018+0000","last_active":"2024-05-09T00:09:10.468575+0000","last_peered":"2024-05-09T00:09:10.468575+0000","last_clean":"2024-05-09T00:09:10.468575+0000","last_became_active":"2024-05-09T00:08:26.064642+0000","last_became_peered":"2024-05-09T00:08:26.064642+0000","last_unstale":"2024-05-09T00:09:10.468575+0000","last_undegraded":"2024-05-09T00:09:10.468575+0000","last_fullsized":"2024-05-09T00:09:10.468575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:02.403704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471869+0000","last_change":"2024-05-09T00:08:26.049770+0000","last_active":"2024-05-09T00:09:10.471869+0000","last_peered":"2024-05-09T00:09:10.471869+0000","last_clean":"2024-05-09T00:09:10.471869+0000","last_became_active":"2024-05-09T00:08:26.048555+0000","last_became_peered":"2024-05-09T00:08:26.048555+0000","last_unstale":"2024-05-09T00:09:10.471869+0000","last_undegraded":"2024-05-09T00:09:10.471869+0000","last_fullsized":"2024-05-09T00:09:10.471869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:16.009200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469538+0000","last_change":"2024-05-09T00:08:26.048587+0000","last_active":"2024-05-09T00:09:10.469538+0000","last_peered":"2024-05-09T00:09:10.469538+0000","last_clean":"2024-05-09T00:09:10.469538+0000","last_became_active":"2024-05-09T00:08:26.048148+0000","last_became_peered":"2024-05-09T00:08:26.048148+0000","last_unstale":"2024-05-09T00:09:10.469538+0000","last_undegraded":"2024-05-09T00:09:10.469538+0000","last_fullsized":"2024-05-09T00:09:10.469538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:42.113829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473434+0000","last_change":"2024-05-09T00:08:26.053607+0000","last_active":"2024-05-09T00:09:10.473434+0000","last_peered":"2024-05-09T00:09:10.473434+0000","last_clean":"2024-05-09T00:09:10.473434+0000","last_became_active":"2024-05-09T00:08:26.053212+0000","last_became_peered":"2024-05-09T00:08:26.053212+0000","last_unstale":"2024-05-09T00:09:10.473434+0000","last_undegraded":"2024-05-09T00:09:10.473434+0000","last_fullsized":"2024-05-09T00:09:10.473434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:32.937049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469272+0000","last_change":"2024-05-09T00:08:23.024481+0000","last_active":"2024-05-09T00:09:10.469272+0000","last_peered":"2024-05-09T00:09:10.469272+0000","last_clean":"2024-05-09T00:09:10.469272+0000","last_became_active":"2024-05-09T00:08:23.024132+0000","last_became_peered":"2024-05-09T00:08:23.024132+0000","last_unstale":"2024-05-09T00:09:10.469272+0000","last_undegraded":"2024-05-09T00:09:10.469272+0000","last_fullsized":"2024-05-09T00:09:10.469272+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:11.188505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468045+0000","last_change":"2024-05-09T00:08:26.054772+0000","last_active":"2024-05-09T00:09:10.468045+0000","last_peered":"2024-05-09T00:09:10.468045+0000","last_clean":"2024-05-09T00:09:10.468045+0000","last_became_active":"2024-05-09T00:08:26.054433+0000","last_became_peered":"2024-05-09T00:08:26.054433+0000","last_unstale":"2024-05-09T00:09:10.468045+0000","last_undegraded":"2024-05-09T00:09:10.468045+0000","last_fullsized":"2024-05-09T00:09:10.468045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:44.283240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468325+0000","last_change":"2024-05-09T00:08:23.118778+0000","last_active":"2024-05-09T00:09:10.468325+0000","last_peered":"2024-05-09T00:09:10.468325+0000","last_clean":"2024-05-09T00:09:10.468325+0000","last_became_active":"2024-05-09T00:08:23.118377+0000","last_became_peered":"2024-05-09T00:08:23.118377+0000","last_unstale":"2024-05-09T00:09:10.468325+0000","last_undegraded":"2024-05-09T00:09:10.468325+0000","last_fullsized":"2024-05-09T00:09:10.468325+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:51.352442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472000+0000","last_change":"2024-05-09T00:08:26.059953+0000","last_active":"2024-05-09T00:09:10.472000+0000","last_peered":"2024-05-09T00:09:10.472000+0000","last_clean":"2024-05-09T00:09:10.472000+0000","last_became_active":"2024-05-09T00:08:26.059571+0000","last_became_peered":"2024-05-09T00:08:26.059571+0000","last_unstale":"2024-05-09T00:09:10.472000+0000","last_undegraded":"2024-05-09T00:09:10.472000+0000","last_fullsized":"2024-05-09T00:09:10.472000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:48.056798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475240+0000","last_change":"2024-05-09T00:08:23.021244+0000","last_active":"2024-05-09T00:09:10.475240+0000","last_peered":"2024-05-09T00:09:10.475240+0000","last_clean":"2024-05-09T00:09:10.475240+0000","last_became_active":"2024-05-09T00:08:23.020597+0000","last_became_peered":"2024-05-09T00:08:23.020597+0000","last_unstale":"2024-05-09T00:09:10.475240+0000","last_undegraded":"2024-05-09T00:09:10.475240+0000","last_fullsized":"2024-05-09T00:09:10.475240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.654261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468788+0000","last_change":"2024-05-09T00:08:26.059834+0000","last_active":"2024-05-09T00:09:10.468788+0000","last_peered":"2024-05-09T00:09:10.468788+0000","last_clean":"2024-05-09T00:09:10.468788+0000","last_became_active":"2024-05-09T00:08:26.059349+0000","last_became_peered":"2024-05-09T00:08:26.059349+0000","last_unstale":"2024-05-09T00:09:10.468788+0000","last_undegraded":"2024-05-09T00:09:10.468788+0000","last_fullsized":"2024-05-09T00:09:10.468788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:42.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474158+0000","last_change":"2024-05-09T00:08:23.019568+0000","last_active":"2024-05-09T00:09:10.474158+0000","last_peered":"2024-05-09T00:09:10.474158+0000","last_clean":"2024-05-09T00:09:10.474158+0000","last_became_active":"2024-05-09T00:08:23.019149+0000","last_became_peered":"2024-05-09T00:08:23.019149+0000","last_unstale":"2024-05-09T00:09:10.474158+0000","last_undegraded":"2024-05-09T00:09:10.474158+0000","last_fullsized":"2024-05-09T00:09:10.474158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:44.569911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469966+0000","last_change":"2024-05-09T00:08:26.061308+0000","last_active":"2024-05-09T00:09:10.469966+0000","last_peered":"2024-05-09T00:09:10.469966+0000","last_clean":"2024-05-09T00:09:10.469966+0000","last_became_active":"2024-05-09T00:08:26.060973+0000","last_became_peered":"2024-05-09T00:08:26.060973+0000","last_unstale":"2024-05-09T00:09:10.469966+0000","last_undegraded":"2024-05-09T00:09:10.469966+0000","last_fullsized":"2024-05-09T00:09:10.469966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:16.845074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":70,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:20.449726+0000","last_change":"2024-05-09T00:08:23.020886+0000","last_active":"2024-05-09T00:09:20.449726+0000","last_peered":"2024-05-09T00:09:20.449726+0000","last_clean":"2024-05-09T00:09:20.449726+0000","last_became_active":"2024-05-09T00:08:23.020530+0000","last_became_peered":"2024-05-09T00:08:23.020530+0000","last_unstale":"2024-05-09T00:09:20.449726+0000","last_undegraded":"2024-05-09T00:09:20.449726+0000","last_fullsized":"2024-05-09T00:09:20.449726+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:34.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474367+0000","last_change":"2024-05-09T00:08:26.051018+0000","last_active":"2024-05-09T00:09:10.474367+0000","last_peered":"2024-05-09T00:09:10.474367+0000","last_clean":"2024-05-09T00:09:10.474367+0000","last_became_active":"2024-05-09T00:08:26.050495+0000","last_became_peered":"2024-05-09T00:08:26.050495+0000","last_unstale":"2024-05-09T00:09:10.474367+0000","last_undegraded":"2024-05-09T00:09:10.474367+0000","last_fullsized":"2024-05-09T00:09:10.474367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:07.862339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472087+0000","last_change":"2024-05-09T00:08:23.021187+0000","last_active":"2024-05-09T00:09:10.472087+0000","last_peered":"2024-05-09T00:09:10.472087+0000","last_clean":"2024-05-09T00:09:10.472087+0000","last_became_active":"2024-05-09T00:08:23.020770+0000","last_became_peered":"2024-05-09T00:08:23.020770+0000","last_unstale":"2024-05-09T00:09:10.472087+0000","last_undegraded":"2024-05-09T00:09:10.472087+0000","last_fullsized":"2024-05-09T00:09:10.472087+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:08.992770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.797838+0000","last_change":"2024-05-09T00:09:10.797392+0000","last_active":"2024-05-09T00:09:10.797838+0000","last_peered":"2024-05-09T00:09:10.797838+0000","last_clean":"2024-05-09T00:09:10.797838+0000","last_became_active":"2024-05-09T00:09:10.796969+0000","last_became_peered":"2024-05-09T00:09:10.796969+0000","last_unstale":"2024-05-09T00:09:10.797838+0000","last_undegraded":"2024-05-09T00:09:10.797838+0000","last_fullsized":"2024-05-09T00:09:10.797838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:18.852678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548755+0000","last_change":"2024-05-09T00:08:23.012226+0000","last_active":"2024-05-09T00:09:10.548755+0000","last_peered":"2024-05-09T00:09:10.548755+0000","last_clean":"2024-05-09T00:09:10.548755+0000","last_became_active":"2024-05-09T00:08:23.011781+0000","last_became_peered":"2024-05-09T00:08:23.011781+0000","last_unstale":"2024-05-09T00:09:10.548755+0000","last_undegraded":"2024-05-09T00:09:10.548755+0000","last_fullsized":"2024-05-09T00:09:10.548755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:44.452206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467563+0000","last_change":"2024-05-09T00:08:26.043035+0000","last_active":"2024-05-09T00:09:10.467563+0000","last_peered":"2024-05-09T00:09:10.467563+0000","last_clean":"2024-05-09T00:09:10.467563+0000","last_became_active":"2024-05-09T00:08:26.042686+0000","last_became_peered":"2024-05-09T00:08:26.042686+0000","last_unstale":"2024-05-09T00:09:10.467563+0000","last_undegraded":"2024-05-09T00:09:10.467563+0000","last_fullsized":"2024-05-09T00:09:10.467563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:24.511309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469042+0000","last_change":"2024-05-09T00:08:23.022017+0000","last_active":"2024-05-09T00:09:10.469042+0000","last_peered":"2024-05-09T00:09:10.469042+0000","last_clean":"2024-05-09T00:09:10.469042+0000","last_became_active":"2024-05-09T00:08:23.021540+0000","last_became_peered":"2024-05-09T00:08:23.021540+0000","last_unstale":"2024-05-09T00:09:10.469042+0000","last_undegraded":"2024-05-09T00:09:10.469042+0000","last_fullsized":"2024-05-09T00:09:10.469042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:50.773211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469091+0000","last_change":"2024-05-09T00:08:26.048039+0000","last_active":"2024-05-09T00:09:10.469091+0000","last_peered":"2024-05-09T00:09:10.469091+0000","last_clean":"2024-05-09T00:09:10.469091+0000","last_became_active":"2024-05-09T00:08:26.047695+0000","last_became_peered":"2024-05-09T00:08:26.047695+0000","last_unstale":"2024-05-09T00:09:10.469091+0000","last_undegraded":"2024-05-09T00:09:10.469091+0000","last_fullsized":"2024-05-09T00:09:10.469091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:27.363847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":64,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:17.444693+0000","last_change":"2024-05-09T00:08:23.019565+0000","last_active":"2024-05-09T00:09:17.444693+0000","last_peered":"2024-05-09T00:09:17.444693+0000","last_clean":"2024-05-09T00:09:17.444693+0000","last_became_active":"2024-05-09T00:08:23.019113+0000","last_became_peered":"2024-05-09T00:08:23.019113+0000","last_unstale":"2024-05-09T00:09:17.444693+0000","last_undegraded":"2024-05-09T00:09:17.444693+0000","last_fullsized":"2024-05-09T00:09:17.444693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:45.578934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469070+0000","last_change":"2024-05-09T00:08:26.058869+0000","last_active":"2024-05-09T00:09:10.469070+0000","last_peered":"2024-05-09T00:09:10.469070+0000","last_clean":"2024-05-09T00:09:10.469070+0000","last_became_active":"2024-05-09T00:08:26.058482+0000","last_became_peered":"2024-05-09T00:08:26.058482+0000","last_unstale":"2024-05-09T00:09:10.469070+0000","last_undegraded":"2024-05-09T00:09:10.469070+0000","last_fullsized":"2024-05-09T00:09:10.469070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:33.252208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472150+0000","last_change":"2024-05-09T00:08:23.024425+0000","last_active":"2024-05-09T00:09:10.472150+0000","last_peered":"2024-05-09T00:09:10.472150+0000","last_clean":"2024-05-09T00:09:10.472150+0000","last_became_active":"2024-05-09T00:08:23.023991+0000","last_became_peered":"2024-05-09T00:08:23.023991+0000","last_unstale":"2024-05-09T00:09:10.472150+0000","last_undegraded":"2024-05-09T00:09:10.472150+0000","last_fullsized":"2024-05-09T00:09:10.472150+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:47.907851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475008+0000","last_change":"2024-05-09T00:09:10.475008+0000","last_active":"2024-05-09T00:09:10.475008+0000","last_peered":"2024-05-09T00:09:10.475008+0000","last_clean":"2024-05-09T00:09:10.475008+0000","last_became_active":"2024-05-09T00:09:10.474663+0000","last_became_peered":"2024-05-09T00:09:10.474663+0000","last_unstale":"2024-05-09T00:09:10.475008+0000","last_undegraded":"2024-05-09T00:09:10.475008+0000","last_fullsized":"2024-05-09T00:09:10.475008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:36.990362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470120+0000","last_change":"2024-05-09T00:08:23.118516+0000","last_active":"2024-05-09T00:09:10.470120+0000","last_peered":"2024-05-09T00:09:10.470120+0000","last_clean":"2024-05-09T00:09:10.470120+0000","last_became_active":"2024-05-09T00:08:23.118147+0000","last_became_peered":"2024-05-09T00:08:23.118147+0000","last_unstale":"2024-05-09T00:09:10.470120+0000","last_undegraded":"2024-05-09T00:09:10.470120+0000","last_fullsized":"2024-05-09T00:09:10.470120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:24.205818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468743+0000","last_change":"2024-05-09T00:08:26.067063+0000","last_active":"2024-05-09T00:09:10.468743+0000","last_peered":"2024-05-09T00:09:10.468743+0000","last_clean":"2024-05-09T00:09:10.468743+0000","last_became_active":"2024-05-09T00:08:26.066440+0000","last_became_peered":"2024-05-09T00:08:26.066440+0000","last_unstale":"2024-05-09T00:09:10.468743+0000","last_undegraded":"2024-05-09T00:09:10.468743+0000","last_fullsized":"2024-05-09T00:09:10.468743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:48.648485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":135,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:18.447905+0000","last_change":"2024-05-09T00:08:23.020757+0000","last_active":"2024-05-09T00:09:18.447905+0000","last_peered":"2024-05-09T00:09:18.447905+0000","last_clean":"2024-05-09T00:09:18.447905+0000","last_became_active":"2024-05-09T00:08:23.020355+0000","last_became_peered":"2024-05-09T00:08:23.020355+0000","last_unstale":"2024-05-09T00:09:18.447905+0000","last_undegraded":"2024-05-09T00:09:18.447905+0000","last_fullsized":"2024-05-09T00:09:18.447905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:00.605691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61826,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":79,"num_read_kb":99,"num_write":31,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475093+0000","last_change":"2024-05-09T00:08:26.057402+0000","last_active":"2024-05-09T00:09:10.475093+0000","last_peered":"2024-05-09T00:09:10.475093+0000","last_clean":"2024-05-09T00:09:10.475093+0000","last_became_active":"2024-05-09T00:08:26.056974+0000","last_became_peered":"2024-05-09T00:08:26.056974+0000","last_unstale":"2024-05-09T00:09:10.475093+0000","last_undegraded":"2024-05-09T00:09:10.475093+0000","last_fullsized":"2024-05-09T00:09:10.475093+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:47.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473890+0000","last_change":"2024-05-09T00:08:23.020208+0000","last_active":"2024-05-09T00:09:10.473890+0000","last_peered":"2024-05-09T00:09:10.473890+0000","last_clean":"2024-05-09T00:09:10.473890+0000","last_became_active":"2024-05-09T00:08:23.019809+0000","last_became_peered":"2024-05-09T00:08:23.019809+0000","last_unstale":"2024-05-09T00:09:10.473890+0000","last_undegraded":"2024-05-09T00:09:10.473890+0000","last_fullsized":"2024-05-09T00:09:10.473890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:47.900731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548422+0000","last_change":"2024-05-09T00:08:26.054273+0000","last_active":"2024-05-09T00:09:10.548422+0000","last_peered":"2024-05-09T00:09:10.548422+0000","last_clean":"2024-05-09T00:09:10.548422+0000","last_became_active":"2024-05-09T00:08:26.053840+0000","last_became_peered":"2024-05-09T00:08:26.053840+0000","last_unstale":"2024-05-09T00:09:10.548422+0000","last_undegraded":"2024-05-09T00:09:10.548422+0000","last_fullsized":"2024-05-09T00:09:10.548422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:27.128582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469240+0000","last_change":"2024-05-09T00:08:23.021594+0000","last_active":"2024-05-09T00:09:10.469240+0000","last_peered":"2024-05-09T00:09:10.469240+0000","last_clean":"2024-05-09T00:09:10.469240+0000","last_became_active":"2024-05-09T00:08:23.021152+0000","last_became_peered":"2024-05-09T00:08:23.021152+0000","last_unstale":"2024-05-09T00:09:10.469240+0000","last_undegraded":"2024-05-09T00:09:10.469240+0000","last_fullsized":"2024-05-09T00:09:10.469240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:58.505781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473341+0000","last_change":"2024-05-09T00:08:26.064464+0000","last_active":"2024-05-09T00:09:10.473341+0000","last_peered":"2024-05-09T00:09:10.473341+0000","last_clean":"2024-05-09T00:09:10.473341+0000","last_became_active":"2024-05-09T00:08:26.064000+0000","last_became_peered":"2024-05-09T00:08:26.064000+0000","last_unstale":"2024-05-09T00:09:10.473341+0000","last_undegraded":"2024-05-09T00:09:10.473341+0000","last_fullsized":"2024-05-09T00:09:10.473341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:01.455979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'4","reported_seq":72,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:21.450557+0000","last_change":"2024-05-09T00:08:23.020823+0000","last_active":"2024-05-09T00:09:21.450557+0000","last_peered":"2024-05-09T00:09:21.450557+0000","last_clean":"2024-05-09T00:09:21.450557+0000","last_became_active":"2024-05-09T00:08:23.020412+0000","last_became_peered":"2024-05-09T00:08:23.020412+0000","last_unstale":"2024-05-09T00:09:21.450557+0000","last_undegraded":"2024-05-09T00:09:21.450557+0000","last_fullsized":"2024-05-09T00:09:21.450557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:05.457452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474540+0000","last_change":"2024-05-09T00:09:10.474540+0000","last_active":"2024-05-09T00:09:10.474540+0000","last_peered":"2024-05-09T00:09:10.474540+0000","last_clean":"2024-05-09T00:09:10.474540+0000","last_became_active":"2024-05-09T00:09:10.474068+0000","last_became_peered":"2024-05-09T00:09:10.474068+0000","last_unstale":"2024-05-09T00:09:10.474540+0000","last_undegraded":"2024-05-09T00:09:10.474540+0000","last_fullsized":"2024-05-09T00:09:10.474540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:37.296140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474161+0000","last_change":"2024-05-09T00:08:23.118857+0000","last_active":"2024-05-09T00:09:10.474161+0000","last_peered":"2024-05-09T00:09:10.474161+0000","last_clean":"2024-05-09T00:09:10.474161+0000","last_became_active":"2024-05-09T00:08:23.118433+0000","last_became_peered":"2024-05-09T00:08:23.118433+0000","last_unstale":"2024-05-09T00:09:10.474161+0000","last_undegraded":"2024-05-09T00:09:10.474161+0000","last_fullsized":"2024-05-09T00:09:10.474161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:10.617987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469212+0000","last_change":"2024-05-09T00:08:26.066911+0000","last_active":"2024-05-09T00:09:10.469212+0000","last_peered":"2024-05-09T00:09:10.469212+0000","last_clean":"2024-05-09T00:09:10.469212+0000","last_became_active":"2024-05-09T00:08:26.066548+0000","last_became_peered":"2024-05-09T00:08:26.066548+0000","last_unstale":"2024-05-09T00:09:10.469212+0000","last_undegraded":"2024-05-09T00:09:10.469212+0000","last_fullsized":"2024-05-09T00:09:10.469212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:32.391390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475008+0000","last_change":"2024-05-09T00:08:23.019829+0000","last_active":"2024-05-09T00:09:10.475008+0000","last_peered":"2024-05-09T00:09:10.475008+0000","last_clean":"2024-05-09T00:09:10.475008+0000","last_became_active":"2024-05-09T00:08:23.019336+0000","last_became_peered":"2024-05-09T00:08:23.019336+0000","last_unstale":"2024-05-09T00:09:10.475008+0000","last_undegraded":"2024-05-09T00:09:10.475008+0000","last_fullsized":"2024-05-09T00:09:10.475008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:33.802738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470640+0000","last_change":"2024-05-09T00:08:26.055975+0000","last_active":"2024-05-09T00:09:10.470640+0000","last_peered":"2024-05-09T00:09:10.470640+0000","last_clean":"2024-05-09T00:09:10.470640+0000","last_became_active":"2024-05-09T00:08:26.055595+0000","last_became_peered":"2024-05-09T00:08:26.055595+0000","last_unstale":"2024-05-09T00:09:10.470640+0000","last_undegraded":"2024-05-09T00:09:10.470640+0000","last_fullsized":"2024-05-09T00:09:10.470640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:20.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469371+0000","last_change":"2024-05-09T00:08:23.120353+0000","last_active":"2024-05-09T00:09:10.469371+0000","last_peered":"2024-05-09T00:09:10.469371+0000","last_clean":"2024-05-09T00:09:10.469371+0000","last_became_active":"2024-05-09T00:08:23.120004+0000","last_became_peered":"2024-05-09T00:08:23.120004+0000","last_unstale":"2024-05-09T00:09:10.469371+0000","last_undegraded":"2024-05-09T00:09:10.469371+0000","last_fullsized":"2024-05-09T00:09:10.469371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:12.168671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471920+0000","last_change":"2024-05-09T00:08:26.046554+0000","last_active":"2024-05-09T00:09:10.471920+0000","last_peered":"2024-05-09T00:09:10.471920+0000","last_clean":"2024-05-09T00:09:10.471920+0000","last_became_active":"2024-05-09T00:08:26.045924+0000","last_became_peered":"2024-05-09T00:08:26.045924+0000","last_unstale":"2024-05-09T00:09:10.471920+0000","last_undegraded":"2024-05-09T00:09:10.471920+0000","last_fullsized":"2024-05-09T00:09:10.471920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:01.208980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472382+0000","last_change":"2024-05-09T00:08:23.019778+0000","last_active":"2024-05-09T00:09:10.472382+0000","last_peered":"2024-05-09T00:09:10.472382+0000","last_clean":"2024-05-09T00:09:10.472382+0000","last_became_active":"2024-05-09T00:08:23.019400+0000","last_became_peered":"2024-05-09T00:08:23.019400+0000","last_unstale":"2024-05-09T00:09:10.472382+0000","last_undegraded":"2024-05-09T00:09:10.472382+0000","last_fullsized":"2024-05-09T00:09:10.472382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:06.883167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474352+0000","last_change":"2024-05-09T00:09:10.474352+0000","last_active":"2024-05-09T00:09:10.474352+0000","last_peered":"2024-05-09T00:09:10.474352+0000","last_clean":"2024-05-09T00:09:10.474352+0000","last_became_active":"2024-05-09T00:09:10.473906+0000","last_became_peered":"2024-05-09T00:09:10.473906+0000","last_unstale":"2024-05-09T00:09:10.474352+0000","last_undegraded":"2024-05-09T00:09:10.474352+0000","last_fullsized":"2024-05-09T00:09:10.474352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:35:18.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.476484+0000","last_change":"2024-05-09T00:09:10.476484+0000","last_active":"2024-05-09T00:09:10.476484+0000","last_peered":"2024-05-09T00:09:10.476484+0000","last_clean":"2024-05-09T00:09:10.476484+0000","last_became_active":"2024-05-09T00:09:10.476001+0000","last_became_peered":"2024-05-09T00:09:10.476001+0000","last_unstale":"2024-05-09T00:09:10.476484+0000","last_undegraded":"2024-05-09T00:09:10.476484+0000","last_fullsized":"2024-05-09T00:09:10.476484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:32.018749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:17.444097+0000","last_change":"2024-05-09T00:08:23.118322+0000","last_active":"2024-05-09T00:09:17.444097+0000","last_peered":"2024-05-09T00:09:17.444097+0000","last_clean":"2024-05-09T00:09:17.444097+0000","last_became_active":"2024-05-09T00:08:23.117943+0000","last_became_peered":"2024-05-09T00:08:23.117943+0000","last_unstale":"2024-05-09T00:09:17.444097+0000","last_undegraded":"2024-05-09T00:09:17.444097+0000","last_fullsized":"2024-05-09T00:09:17.444097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:07.890437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467948+0000","last_change":"2024-05-09T00:08:26.052294+0000","last_active":"2024-05-09T00:09:10.467948+0000","last_peered":"2024-05-09T00:09:10.467948+0000","last_clean":"2024-05-09T00:09:10.467948+0000","last_became_active":"2024-05-09T00:08:26.051926+0000","last_became_peered":"2024-05-09T00:08:26.051926+0000","last_unstale":"2024-05-09T00:09:10.467948+0000","last_undegraded":"2024-05-09T00:09:10.467948+0000","last_fullsized":"2024-05-09T00:09:10.467948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:53.651401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796281+0000","last_change":"2024-05-09T00:08:23.015007+0000","last_active":"2024-05-09T00:09:10.796281+0000","last_peered":"2024-05-09T00:09:10.796281+0000","last_clean":"2024-05-09T00:09:10.796281+0000","last_became_active":"2024-05-09T00:08:23.014545+0000","last_became_peered":"2024-05-09T00:08:23.014545+0000","last_unstale":"2024-05-09T00:09:10.796281+0000","last_undegraded":"2024-05-09T00:09:10.796281+0000","last_fullsized":"2024-05-09T00:09:10.796281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:19.850200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472332+0000","last_change":"2024-05-09T00:07:00.733233+0000","last_active":"2024-05-09T00:09:10.472332+0000","last_peered":"2024-05-09T00:09:10.472332+0000","last_clean":"2024-05-09T00:09:10.472332+0000","last_became_active":"2024-05-09T00:07:00.727365+0000","last_became_peered":"2024-05-09T00:07:00.727365+0000","last_unstale":"2024-05-09T00:09:10.472332+0000","last_undegraded":"2024-05-09T00:09:10.472332+0000","last_fullsized":"2024-05-09T00:09:10.472332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_clean_scrub_stamp":"2024-05-09T00:03:33.285220+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:08.597290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548228+0000","last_change":"2024-05-09T00:08:26.064000+0000","last_active":"2024-05-09T00:09:10.548228+0000","last_peered":"2024-05-09T00:09:10.548228+0000","last_clean":"2024-05-09T00:09:10.548228+0000","last_became_active":"2024-05-09T00:08:26.063636+0000","last_became_peered":"2024-05-09T00:08:26.063636+0000","last_unstale":"2024-05-09T00:09:10.548228+0000","last_undegraded":"2024-05-09T00:09:10.548228+0000","last_fullsized":"2024-05-09T00:09:10.548228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:59.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472537+0000","last_change":"2024-05-09T00:08:23.023459+0000","last_active":"2024-05-09T00:09:10.472537+0000","last_peered":"2024-05-09T00:09:10.472537+0000","last_clean":"2024-05-09T00:09:10.472537+0000","last_became_active":"2024-05-09T00:08:23.023089+0000","last_became_peered":"2024-05-09T00:08:23.023089+0000","last_unstale":"2024-05-09T00:09:10.472537+0000","last_undegraded":"2024-05-09T00:09:10.472537+0000","last_fullsized":"2024-05-09T00:09:10.472537+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:41.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.479186+0000","last_change":"2024-05-09T00:09:10.478686+0000","last_active":"2024-05-09T00:09:10.479186+0000","last_peered":"2024-05-09T00:09:10.479186+0000","last_clean":"2024-05-09T00:09:10.479186+0000","last_became_active":"2024-05-09T00:09:10.474890+0000","last_became_peered":"2024-05-09T00:09:10.474890+0000","last_unstale":"2024-05-09T00:09:10.479186+0000","last_undegraded":"2024-05-09T00:09:10.479186+0000","last_fullsized":"2024-05-09T00:09:10.479186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:41.041247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548460+0000","last_change":"2024-05-09T00:08:26.050685+0000","last_active":"2024-05-09T00:09:10.548460+0000","last_peered":"2024-05-09T00:09:10.548460+0000","last_clean":"2024-05-09T00:09:10.548460+0000","last_became_active":"2024-05-09T00:08:26.050312+0000","last_became_peered":"2024-05-09T00:08:26.050312+0000","last_unstale":"2024-05-09T00:09:10.548460+0000","last_undegraded":"2024-05-09T00:09:10.548460+0000","last_fullsized":"2024-05-09T00:09:10.548460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:31.623182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'11","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469432+0000","last_change":"2024-05-09T00:08:23.019663+0000","last_active":"2024-05-09T00:09:10.469432+0000","last_peered":"2024-05-09T00:09:10.469432+0000","last_clean":"2024-05-09T00:09:10.469432+0000","last_became_active":"2024-05-09T00:08:23.019301+0000","last_became_peered":"2024-05-09T00:08:23.019301+0000","last_unstale":"2024-05-09T00:09:10.469432+0000","last_undegraded":"2024-05-09T00:09:10.469432+0000","last_fullsized":"2024-05-09T00:09:10.469432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:08.291227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468179+0000","last_change":"2024-05-09T00:08:26.058825+0000","last_active":"2024-05-09T00:09:10.468179+0000","last_peered":"2024-05-09T00:09:10.468179+0000","last_clean":"2024-05-09T00:09:10.468179+0000","last_became_active":"2024-05-09T00:08:26.058469+0000","last_became_peered":"2024-05-09T00:08:26.058469+0000","last_unstale":"2024-05-09T00:09:10.468179+0000","last_undegraded":"2024-05-09T00:09:10.468179+0000","last_fullsized":"2024-05-09T00:09:10.468179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:08.118819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796528+0000","last_change":"2024-05-09T00:08:23.054366+0000","last_active":"2024-05-09T00:09:10.796528+0000","last_peered":"2024-05-09T00:09:10.796528+0000","last_clean":"2024-05-09T00:09:10.796528+0000","last_became_active":"2024-05-09T00:08:23.053977+0000","last_became_peered":"2024-05-09T00:08:23.053977+0000","last_unstale":"2024-05-09T00:09:10.796528+0000","last_undegraded":"2024-05-09T00:09:10.796528+0000","last_fullsized":"2024-05-09T00:09:10.796528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:37.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471766+0000","last_change":"2024-05-09T00:08:26.057864+0000","last_active":"2024-05-09T00:09:10.471766+0000","last_peered":"2024-05-09T00:09:10.471766+0000","last_clean":"2024-05-09T00:09:10.471766+0000","last_became_active":"2024-05-09T00:08:26.057436+0000","last_became_peered":"2024-05-09T00:08:26.057436+0000","last_unstale":"2024-05-09T00:09:10.471766+0000","last_undegraded":"2024-05-09T00:09:10.471766+0000","last_fullsized":"2024-05-09T00:09:10.471766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:19.976265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468611+0000","last_change":"2024-05-09T00:08:23.118803+0000","last_active":"2024-05-09T00:09:10.468611+0000","last_peered":"2024-05-09T00:09:10.468611+0000","last_clean":"2024-05-09T00:09:10.468611+0000","last_became_active":"2024-05-09T00:08:23.118432+0000","last_became_peered":"2024-05-09T00:08:23.118432+0000","last_unstale":"2024-05-09T00:09:10.468611+0000","last_undegraded":"2024-05-09T00:09:10.468611+0000","last_fullsized":"2024-05-09T00:09:10.468611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:50.253699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473482+0000","last_change":"2024-05-09T00:08:26.054281+0000","last_active":"2024-05-09T00:09:10.473482+0000","last_peered":"2024-05-09T00:09:10.473482+0000","last_clean":"2024-05-09T00:09:10.473482+0000","last_became_active":"2024-05-09T00:08:26.053874+0000","last_became_peered":"2024-05-09T00:08:26.053874+0000","last_unstale":"2024-05-09T00:09:10.473482+0000","last_undegraded":"2024-05-09T00:09:10.473482+0000","last_fullsized":"2024-05-09T00:09:10.473482+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:11.566337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467696+0000","last_change":"2024-05-09T00:08:23.017905+0000","last_active":"2024-05-09T00:09:10.467696+0000","last_peered":"2024-05-09T00:09:10.467696+0000","last_clean":"2024-05-09T00:09:10.467696+0000","last_became_active":"2024-05-09T00:08:23.017481+0000","last_became_peered":"2024-05-09T00:08:23.017481+0000","last_unstale":"2024-05-09T00:09:10.467696+0000","last_undegraded":"2024-05-09T00:09:10.467696+0000","last_fullsized":"2024-05-09T00:09:10.467696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:40.096415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469888+0000","last_change":"2024-05-09T00:08:26.053392+0000","last_active":"2024-05-09T00:09:10.469888+0000","last_peered":"2024-05-09T00:09:10.469888+0000","last_clean":"2024-05-09T00:09:10.469888+0000","last_became_active":"2024-05-09T00:08:26.052628+0000","last_became_peered":"2024-05-09T00:08:26.052628+0000","last_unstale":"2024-05-09T00:09:10.469888+0000","last_undegraded":"2024-05-09T00:09:10.469888+0000","last_fullsized":"2024-05-09T00:09:10.469888+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:28.013955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.481712+0000","last_change":"2024-05-09T00:09:10.481233+0000","last_active":"2024-05-09T00:09:10.481712+0000","last_peered":"2024-05-09T00:09:10.481712+0000","last_clean":"2024-05-09T00:09:10.481712+0000","last_became_active":"2024-05-09T00:09:10.475321+0000","last_became_peered":"2024-05-09T00:09:10.475321+0000","last_unstale":"2024-05-09T00:09:10.481712+0000","last_undegraded":"2024-05-09T00:09:10.481712+0000","last_fullsized":"2024-05-09T00:09:10.481712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:58.859277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467933+0000","last_change":"2024-05-09T00:08:26.047812+0000","last_active":"2024-05-09T00:09:10.467933+0000","last_peered":"2024-05-09T00:09:10.467933+0000","last_clean":"2024-05-09T00:09:10.467933+0000","last_became_active":"2024-05-09T00:08:26.047459+0000","last_became_peered":"2024-05-09T00:08:26.047459+0000","last_unstale":"2024-05-09T00:09:10.467933+0000","last_undegraded":"2024-05-09T00:09:10.467933+0000","last_fullsized":"2024-05-09T00:09:10.467933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.257831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473758+0000","last_change":"2024-05-09T00:08:23.023800+0000","last_active":"2024-05-09T00:09:10.473758+0000","last_peered":"2024-05-09T00:09:10.473758+0000","last_clean":"2024-05-09T00:09:10.473758+0000","last_became_active":"2024-05-09T00:08:23.023387+0000","last_became_peered":"2024-05-09T00:08:23.023387+0000","last_unstale":"2024-05-09T00:09:10.473758+0000","last_undegraded":"2024-05-09T00:09:10.473758+0000","last_fullsized":"2024-05-09T00:09:10.473758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:33.714683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472532+0000","last_change":"2024-05-09T00:08:26.056981+0000","last_active":"2024-05-09T00:09:10.472532+0000","last_peered":"2024-05-09T00:09:10.472532+0000","last_clean":"2024-05-09T00:09:10.472532+0000","last_became_active":"2024-05-09T00:08:26.056605+0000","last_became_peered":"2024-05-09T00:08:26.056605+0000","last_unstale":"2024-05-09T00:09:10.472532+0000","last_undegraded":"2024-05-09T00:09:10.472532+0000","last_fullsized":"2024-05-09T00:09:10.472532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:32.897929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473611+0000","last_change":"2024-05-09T00:08:23.120612+0000","last_active":"2024-05-09T00:09:10.473611+0000","last_peered":"2024-05-09T00:09:10.473611+0000","last_clean":"2024-05-09T00:09:10.473611+0000","last_became_active":"2024-05-09T00:08:23.120157+0000","last_became_peered":"2024-05-09T00:08:23.120157+0000","last_unstale":"2024-05-09T00:09:10.473611+0000","last_undegraded":"2024-05-09T00:09:10.473611+0000","last_fullsized":"2024-05-09T00:09:10.473611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:08.027214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473165+0000","last_change":"2024-05-09T00:08:26.061594+0000","last_active":"2024-05-09T00:09:10.473165+0000","last_peered":"2024-05-09T00:09:10.473165+0000","last_clean":"2024-05-09T00:09:10.473165+0000","last_became_active":"2024-05-09T00:08:26.061201+0000","last_became_peered":"2024-05-09T00:08:26.061201+0000","last_unstale":"2024-05-09T00:09:10.473165+0000","last_undegraded":"2024-05-09T00:09:10.473165+0000","last_fullsized":"2024-05-09T00:09:10.473165+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:50.183648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469471+0000","last_change":"2024-05-09T00:08:23.017693+0000","last_active":"2024-05-09T00:09:10.469471+0000","last_peered":"2024-05-09T00:09:10.469471+0000","last_clean":"2024-05-09T00:09:10.469471+0000","last_became_active":"2024-05-09T00:08:23.017211+0000","last_became_peered":"2024-05-09T00:08:23.017211+0000","last_unstale":"2024-05-09T00:09:10.469471+0000","last_undegraded":"2024-05-09T00:09:10.469471+0000","last_fullsized":"2024-05-09T00:09:10.469471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:53.569932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472752+0000","last_change":"2024-05-09T00:08:26.063922+0000","last_active":"2024-05-09T00:09:10.472752+0000","last_peered":"2024-05-09T00:09:10.472752+0000","last_clean":"2024-05-09T00:09:10.472752+0000","last_became_active":"2024-05-09T00:08:26.063554+0000","last_became_peered":"2024-05-09T00:08:26.063554+0000","last_unstale":"2024-05-09T00:09:10.472752+0000","last_undegraded":"2024-05-09T00:09:10.472752+0000","last_fullsized":"2024-05-09T00:09:10.472752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:01.754688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548572+0000","last_change":"2024-05-09T00:08:23.020367+0000","last_active":"2024-05-09T00:09:10.548572+0000","last_peered":"2024-05-09T00:09:10.548572+0000","last_clean":"2024-05-09T00:09:10.548572+0000","last_became_active":"2024-05-09T00:08:23.019801+0000","last_became_peered":"2024-05-09T00:08:23.019801+0000","last_unstale":"2024-05-09T00:09:10.548572+0000","last_undegraded":"2024-05-09T00:09:10.548572+0000","last_fullsized":"2024-05-09T00:09:10.548572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:14.936021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472767+0000","last_change":"2024-05-09T00:08:26.063863+0000","last_active":"2024-05-09T00:09:10.472767+0000","last_peered":"2024-05-09T00:09:10.472767+0000","last_clean":"2024-05-09T00:09:10.472767+0000","last_became_active":"2024-05-09T00:08:26.061938+0000","last_became_peered":"2024-05-09T00:08:26.061938+0000","last_unstale":"2024-05-09T00:09:10.472767+0000","last_undegraded":"2024-05-09T00:09:10.472767+0000","last_fullsized":"2024-05-09T00:09:10.472767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:04.597991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470033+0000","last_change":"2024-05-09T00:08:26.050837+0000","last_active":"2024-05-09T00:09:10.470033+0000","last_peered":"2024-05-09T00:09:10.470033+0000","last_clean":"2024-05-09T00:09:10.470033+0000","last_became_active":"2024-05-09T00:08:26.050424+0000","last_became_peered":"2024-05-09T00:08:26.050424+0000","last_unstale":"2024-05-09T00:09:10.470033+0000","last_undegraded":"2024-05-09T00:09:10.470033+0000","last_fullsized":"2024-05-09T00:09:10.470033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:45.134811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.549989+0000","last_change":"2024-05-09T00:09:10.549989+0000","last_active":"2024-05-09T00:09:10.549989+0000","last_peered":"2024-05-09T00:09:10.549989+0000","last_clean":"2024-05-09T00:09:10.549989+0000","last_became_active":"2024-05-09T00:09:10.549502+0000","last_became_peered":"2024-05-09T00:09:10.549502+0000","last_unstale":"2024-05-09T00:09:10.549989+0000","last_undegraded":"2024-05-09T00:09:10.549989+0000","last_fullsized":"2024-05-09T00:09:10.549989+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:10.654452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473094+0000","last_change":"2024-05-09T00:08:26.058012+0000","last_active":"2024-05-09T00:09:10.473094+0000","last_peered":"2024-05-09T00:09:10.473094+0000","last_clean":"2024-05-09T00:09:10.473094+0000","last_became_active":"2024-05-09T00:08:26.057282+0000","last_became_peered":"2024-05-09T00:08:26.057282+0000","last_unstale":"2024-05-09T00:09:10.473094+0000","last_undegraded":"2024-05-09T00:09:10.473094+0000","last_fullsized":"2024-05-09T00:09:10.473094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:53.520593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471555+0000","last_change":"2024-05-09T00:08:26.063331+0000","last_active":"2024-05-09T00:09:10.471555+0000","last_peered":"2024-05-09T00:09:10.471555+0000","last_clean":"2024-05-09T00:09:10.471555+0000","last_became_active":"2024-05-09T00:08:26.062866+0000","last_became_peered":"2024-05-09T00:08:26.062866+0000","last_unstale":"2024-05-09T00:09:10.471555+0000","last_undegraded":"2024-05-09T00:09:10.471555+0000","last_fullsized":"2024-05-09T00:09:10.471555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:10.387852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548504+0000","last_change":"2024-05-09T00:08:26.063735+0000","last_active":"2024-05-09T00:09:10.548504+0000","last_peered":"2024-05-09T00:09:10.548504+0000","last_clean":"2024-05-09T00:09:10.548504+0000","last_became_active":"2024-05-09T00:08:26.063358+0000","last_became_peered":"2024-05-09T00:08:26.063358+0000","last_unstale":"2024-05-09T00:09:10.548504+0000","last_undegraded":"2024-05-09T00:09:10.548504+0000","last_fullsized":"2024-05-09T00:09:10.548504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:06.832071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471634+0000","last_change":"2024-05-09T00:08:26.052746+0000","last_active":"2024-05-09T00:09:10.471634+0000","last_peered":"2024-05-09T00:09:10.471634+0000","last_clean":"2024-05-09T00:09:10.471634+0000","last_became_active":"2024-05-09T00:08:26.051544+0000","last_became_peered":"2024-05-09T00:08:26.051544+0000","last_unstale":"2024-05-09T00:09:10.471634+0000","last_undegraded":"2024-05-09T00:09:10.471634+0000","last_fullsized":"2024-05-09T00:09:10.471634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:31.963268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473534+0000","last_change":"2024-05-09T00:08:26.058075+0000","last_active":"2024-05-09T00:09:10.473534+0000","last_peered":"2024-05-09T00:09:10.473534+0000","last_clean":"2024-05-09T00:09:10.473534+0000","last_became_active":"2024-05-09T00:08:26.057349+0000","last_became_peered":"2024-05-09T00:08:26.057349+0000","last_unstale":"2024-05-09T00:09:10.473534+0000","last_undegraded":"2024-05-09T00:09:10.473534+0000","last_fullsized":"2024-05-09T00:09:10.473534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:59.322903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468103+0000","last_change":"2024-05-09T00:08:26.056429+0000","last_active":"2024-05-09T00:09:10.468103+0000","last_peered":"2024-05-09T00:09:10.468103+0000","last_clean":"2024-05-09T00:09:10.468103+0000","last_became_active":"2024-05-09T00:08:26.054852+0000","last_became_peered":"2024-05-09T00:08:26.054852+0000","last_unstale":"2024-05-09T00:09:10.468103+0000","last_undegraded":"2024-05-09T00:09:10.468103+0000","last_fullsized":"2024-05-09T00:09:10.468103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:05.337636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.549880+0000","last_change":"2024-05-09T00:09:10.549880+0000","last_active":"2024-05-09T00:09:10.549880+0000","last_peered":"2024-05-09T00:09:10.549880+0000","last_clean":"2024-05-09T00:09:10.549880+0000","last_became_active":"2024-05-09T00:09:10.549448+0000","last_became_peered":"2024-05-09T00:09:10.549448+0000","last_unstale":"2024-05-09T00:09:10.549880+0000","last_undegraded":"2024-05-09T00:09:10.549880+0000","last_fullsized":"2024-05-09T00:09:10.549880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:16.990729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467623+0000","last_change":"2024-05-09T00:08:26.054579+0000","last_active":"2024-05-09T00:09:10.467623+0000","last_peered":"2024-05-09T00:09:10.467623+0000","last_clean":"2024-05-09T00:09:10.467623+0000","last_became_active":"2024-05-09T00:08:26.053186+0000","last_became_peered":"2024-05-09T00:08:26.053186+0000","last_unstale":"2024-05-09T00:09:10.467623+0000","last_undegraded":"2024-05-09T00:09:10.467623+0000","last_fullsized":"2024-05-09T00:09:10.467623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:37.561691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468682+0000","last_change":"2024-05-09T00:08:26.063608+0000","last_active":"2024-05-09T00:09:10.468682+0000","last_peered":"2024-05-09T00:09:10.468682+0000","last_clean":"2024-05-09T00:09:10.468682+0000","last_became_active":"2024-05-09T00:08:26.063078+0000","last_became_peered":"2024-05-09T00:08:26.063078+0000","last_unstale":"2024-05-09T00:09:10.468682+0000","last_undegraded":"2024-05-09T00:09:10.468682+0000","last_fullsized":"2024-05-09T00:09:10.468682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:06.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548642+0000","last_change":"2024-05-09T00:08:26.055058+0000","last_active":"2024-05-09T00:09:10.548642+0000","last_peered":"2024-05-09T00:09:10.548642+0000","last_clean":"2024-05-09T00:09:10.548642+0000","last_became_active":"2024-05-09T00:08:26.054695+0000","last_became_peered":"2024-05-09T00:08:26.054695+0000","last_unstale":"2024-05-09T00:09:10.548642+0000","last_undegraded":"2024-05-09T00:09:10.548642+0000","last_fullsized":"2024-05-09T00:09:10.548642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:58.925990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470339+0000","last_change":"2024-05-09T00:08:26.055366+0000","last_active":"2024-05-09T00:09:10.470339+0000","last_peered":"2024-05-09T00:09:10.470339+0000","last_clean":"2024-05-09T00:09:10.470339+0000","last_became_active":"2024-05-09T00:08:26.054904+0000","last_became_peered":"2024-05-09T00:08:26.054904+0000","last_unstale":"2024-05-09T00:09:10.470339+0000","last_undegraded":"2024-05-09T00:09:10.470339+0000","last_fullsized":"2024-05-09T00:09:10.470339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:40.754878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285241,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65827,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":244,"num_read_kb":187,"num_write":165,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":197481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710745,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":425247,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1078923,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070500,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":425461,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266729,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":425359,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462958,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":1017311,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":868352,"data_stored":424850,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":486469,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":888832,"data_stored":426121,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":487819,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":425161,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":425678,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822713,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":1016604,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:09:25.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:25.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:25.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:25 smithi046 ceph-mon[30151]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:25.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:09:25.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:09:25.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:25 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:25.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:25.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:25.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:25 smithi169 ceph-mon[34777]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:25.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:09:25.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:09:25.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:25 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:25.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:25.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:25.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:25 smithi142 ceph-mon[34533]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:25.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:09:25.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:09:25.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:25 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:09:26.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:26 smithi046 ceph-mon[30151]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s; 1 B/s, 0 objects/s recovering 2024-05-09T00:09:26.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:26 smithi169 ceph-mon[34777]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s; 1 B/s, 0 objects/s recovering 2024-05-09T00:09:26.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:26 smithi142 ceph-mon[34533]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s; 1 B/s, 0 objects/s recovering 2024-05-09T00:09:28.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T00:09:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:28.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:28 smithi046 ceph-mon[30151]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:28.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:28 smithi169 ceph-mon[34777]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:28.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:28 smithi142 ceph-mon[34533]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:30.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.283+0000 7f6e3bde2700 1 -- 172.21.15.46:0/2185597444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e341034c0 msgr2=0x7f6e341038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:30.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.283+0000 7f6e3bde2700 1 --2- 172.21.15.46:0/2185597444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e341034c0 0x7f6e341038d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6e300099c0 tx=0x7f6e30009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:30.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.283+0000 7f6e3bde2700 1 -- 172.21.15.46:0/2185597444 shutdown_connections 2024-05-09T00:09:30.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.283+0000 7f6e3bde2700 1 --2- 172.21.15.46:0/2185597444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e34108740 0x7f6e341100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:30.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.283+0000 7f6e3bde2700 1 --2- 172.21.15.46:0/2185597444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e341034c0 0x7f6e341038d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:30.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.283+0000 7f6e3bde2700 1 --2- 172.21.15.46:0/2185597444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e34107df0 0x7f6e34108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:30.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.283+0000 7f6e3bde2700 1 -- 172.21.15.46:0/2185597444 >> 172.21.15.46:0/2185597444 conn(0x7f6e340fedf0 msgr2=0x7f6e34101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:30.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.283+0000 7f6e3bde2700 1 -- 172.21.15.46:0/2185597444 shutdown_connections 2024-05-09T00:09:30.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.283+0000 7f6e3bde2700 1 -- 172.21.15.46:0/2185597444 wait complete. 2024-05-09T00:09:30.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.284+0000 7f6e3bde2700 1 Processor -- start 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.284+0000 7f6e3bde2700 1 -- start start 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.284+0000 7f6e3bde2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e341034c0 0x7f6e3419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3bde2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e34107df0 0x7f6e3419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3bde2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e34108740 0x7f6e3419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3bde2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6e341144e0 con 0x7f6e34108740 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3bde2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34114360 con 0x7f6e341034c0 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3bde2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34114660 con 0x7f6e34107df0 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3a37f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e34108740 0x7f6e3419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3a37f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e34108740 0x7f6e3419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50036/0 (socket says 172.21.15.46:50036) 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3a37f700 1 -- 172.21.15.46:0/3806577841 learned_addr learned my addr 172.21.15.46:0/3806577841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e39b7e700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e341034c0 0x7f6e3419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3937d700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e34107df0 0x7f6e3419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3a37f700 1 -- 172.21.15.46:0/3806577841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e34107df0 msgr2=0x7f6e3419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3a37f700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e34107df0 0x7f6e3419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3a37f700 1 -- 172.21.15.46:0/3806577841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e341034c0 msgr2=0x7f6e3419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3a37f700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e341034c0 0x7f6e3419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3a37f700 1 -- 172.21.15.46:0/3806577841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3419fd00 con 0x7f6e34108740 2024-05-09T00:09:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e3a37f700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e34108740 0x7f6e3419f660 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c007ee0 tx=0x7f6e2c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.285+0000 7f6e2affd700 1 -- 172.21.15.46:0/3806577841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c00dbd0 con 0x7f6e34108740 2024-05-09T00:09:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.286+0000 7f6e2affd700 1 -- 172.21.15.46:0/3806577841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e2c00dd70 con 0x7f6e34108740 2024-05-09T00:09:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.286+0000 7f6e3bde2700 1 -- 172.21.15.46:0/3806577841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3419fff0 con 0x7f6e34108740 2024-05-09T00:09:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.286+0000 7f6e2affd700 1 -- 172.21.15.46:0/3806577841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c005990 con 0x7f6e34108740 2024-05-09T00:09:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.286+0000 7f6e3bde2700 1 -- 172.21.15.46:0/3806577841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e341a7910 con 0x7f6e34108740 2024-05-09T00:09:30.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.287+0000 7f6e3bde2700 1 -- 172.21.15.46:0/3806577841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e340512c0 con 0x7f6e34108740 2024-05-09T00:09:30.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.288+0000 7f6e2affd700 1 -- 172.21.15.46:0/3806577841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6e2c01a020 con 0x7f6e34108740 2024-05-09T00:09:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.289+0000 7f6e2affd700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6e200b2910 0x7f6e200b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.289+0000 7f6e39b7e700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6e200b2910 0x7f6e200b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.289+0000 7f6e2affd700 1 -- 172.21.15.46:0/3806577841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6e2c0e1b60 con 0x7f6e34108740 2024-05-09T00:09:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.289+0000 7f6e39b7e700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6e200b2910 0x7f6e200b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009990 tx=0x7f6e300036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.294+0000 7f6e2affd700 1 -- 172.21.15.46:0/3806577841 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6e2c010070 con 0x7f6e34108740 2024-05-09T00:09:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.506+0000 7f6e3bde2700 1 -- 172.21.15.46:0/3806577841 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e3419c190 con 0x7f6e200b2910 2024-05-09T00:09:30.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.513+0000 7f6e2affd700 1 -- 172.21.15.46:0/3806577841 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258957 (secure 0 0 0) 0x7f6e300673a0 con 0x7f6e200b2910 2024-05-09T00:09:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.517+0000 7f6e3bde2700 1 -- 172.21.15.46:0/3806577841 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6e200b2910 msgr2=0x7f6e200b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.517+0000 7f6e3bde2700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6e200b2910 0x7f6e200b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009990 tx=0x7f6e300036e0 comp rx=0 tx=0).stop 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.517+0000 7f6e3bde2700 1 -- 172.21.15.46:0/3806577841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e34108740 msgr2=0x7f6e3419f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.517+0000 7f6e3bde2700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e34108740 0x7f6e3419f660 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c007ee0 tx=0x7f6e2c00d3b0 comp rx=0 tx=0).stop 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.517+0000 7f6e3bde2700 1 -- 172.21.15.46:0/3806577841 shutdown_connections 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.517+0000 7f6e3bde2700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6e200b2910 0x7f6e200b4e20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.517+0000 7f6e3bde2700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e34108740 0x7f6e3419f660 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.517+0000 7f6e3bde2700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e341034c0 0x7f6e3419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.517+0000 7f6e3bde2700 1 --2- 172.21.15.46:0/3806577841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e34107df0 0x7f6e3419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.517+0000 7f6e3bde2700 1 -- 172.21.15.46:0/3806577841 >> 172.21.15.46:0/3806577841 conn(0x7f6e340fedf0 msgr2=0x7f6e34101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.517+0000 7f6e3bde2700 1 -- 172.21.15.46:0/3806577841 shutdown_connections 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:30.517+0000 7f6e3bde2700 1 -- 172.21.15.46:0/3806577841 wait complete. 2024-05-09T00:09:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T00:09:30.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:30 smithi046 ceph-mon[30151]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:09:30.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:30 smithi169 ceph-mon[34777]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:09:30.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:30 smithi142 ceph-mon[34533]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:09:31.244 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-05-09T00:09:29.297255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656429,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":370,"num_read_kb":281,"num_write":290,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372376,"kb_used_data":10656,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569448,"statfs":{"total":1151940427776,"available":1151559114752,"internally_reserved":0,"allocated":10911744,"data_stored":7065003,"data_compressed":177897,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003371"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471697+0000","last_change":"2024-05-09T00:08:26.060455+0000","last_active":"2024-05-09T00:09:10.471697+0000","last_peered":"2024-05-09T00:09:10.471697+0000","last_clean":"2024-05-09T00:09:10.471697+0000","last_became_active":"2024-05-09T00:08:26.060068+0000","last_became_peered":"2024-05-09T00:08:26.060068+0000","last_unstale":"2024-05-09T00:09:10.471697+0000","last_undegraded":"2024-05-09T00:09:10.471697+0000","last_fullsized":"2024-05-09T00:09:10.471697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:48.354188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471761+0000","last_change":"2024-05-09T00:08:26.055737+0000","last_active":"2024-05-09T00:09:10.471761+0000","last_peered":"2024-05-09T00:09:10.471761+0000","last_clean":"2024-05-09T00:09:10.471761+0000","last_became_active":"2024-05-09T00:08:26.055354+0000","last_became_peered":"2024-05-09T00:08:26.055354+0000","last_unstale":"2024-05-09T00:09:10.471761+0000","last_undegraded":"2024-05-09T00:09:10.471761+0000","last_fullsized":"2024-05-09T00:09:10.471761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:54.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471480+0000","last_change":"2024-05-09T00:08:26.050776+0000","last_active":"2024-05-09T00:09:10.471480+0000","last_peered":"2024-05-09T00:09:10.471480+0000","last_clean":"2024-05-09T00:09:10.471480+0000","last_became_active":"2024-05-09T00:08:26.050393+0000","last_became_peered":"2024-05-09T00:08:26.050393+0000","last_unstale":"2024-05-09T00:09:10.471480+0000","last_undegraded":"2024-05-09T00:09:10.471480+0000","last_fullsized":"2024-05-09T00:09:10.471480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:30.819219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796411+0000","last_change":"2024-05-09T00:08:26.065946+0000","last_active":"2024-05-09T00:09:10.796411+0000","last_peered":"2024-05-09T00:09:10.796411+0000","last_clean":"2024-05-09T00:09:10.796411+0000","last_became_active":"2024-05-09T00:08:26.065498+0000","last_became_peered":"2024-05-09T00:08:26.065498+0000","last_unstale":"2024-05-09T00:09:10.796411+0000","last_undegraded":"2024-05-09T00:09:10.796411+0000","last_fullsized":"2024-05-09T00:09:10.796411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:04.444860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473630+0000","last_change":"2024-05-09T00:08:26.051961+0000","last_active":"2024-05-09T00:09:10.473630+0000","last_peered":"2024-05-09T00:09:10.473630+0000","last_clean":"2024-05-09T00:09:10.473630+0000","last_became_active":"2024-05-09T00:08:26.051548+0000","last_became_peered":"2024-05-09T00:08:26.051548+0000","last_unstale":"2024-05-09T00:09:10.473630+0000","last_undegraded":"2024-05-09T00:09:10.473630+0000","last_fullsized":"2024-05-09T00:09:10.473630+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:07.539117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471811+0000","last_change":"2024-05-09T00:08:26.052012+0000","last_active":"2024-05-09T00:09:10.471811+0000","last_peered":"2024-05-09T00:09:10.471811+0000","last_clean":"2024-05-09T00:09:10.471811+0000","last_became_active":"2024-05-09T00:08:26.050886+0000","last_became_peered":"2024-05-09T00:08:26.050886+0000","last_unstale":"2024-05-09T00:09:10.471811+0000","last_undegraded":"2024-05-09T00:09:10.471811+0000","last_fullsized":"2024-05-09T00:09:10.471811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:14.651888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472730+0000","last_change":"2024-05-09T00:08:26.058418+0000","last_active":"2024-05-09T00:09:10.472730+0000","last_peered":"2024-05-09T00:09:10.472730+0000","last_clean":"2024-05-09T00:09:10.472730+0000","last_became_active":"2024-05-09T00:08:26.058039+0000","last_became_peered":"2024-05-09T00:08:26.058039+0000","last_unstale":"2024-05-09T00:09:10.472730+0000","last_undegraded":"2024-05-09T00:09:10.472730+0000","last_fullsized":"2024-05-09T00:09:10.472730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:13.678123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467742+0000","last_change":"2024-05-09T00:08:26.057602+0000","last_active":"2024-05-09T00:09:10.467742+0000","last_peered":"2024-05-09T00:09:10.467742+0000","last_clean":"2024-05-09T00:09:10.467742+0000","last_became_active":"2024-05-09T00:08:26.057169+0000","last_became_peered":"2024-05-09T00:08:26.057169+0000","last_unstale":"2024-05-09T00:09:10.467742+0000","last_undegraded":"2024-05-09T00:09:10.467742+0000","last_fullsized":"2024-05-09T00:09:10.467742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:11.452083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469166+0000","last_change":"2024-05-09T00:08:26.067639+0000","last_active":"2024-05-09T00:09:10.469166+0000","last_peered":"2024-05-09T00:09:10.469166+0000","last_clean":"2024-05-09T00:09:10.469166+0000","last_became_active":"2024-05-09T00:08:26.067226+0000","last_became_peered":"2024-05-09T00:08:26.067226+0000","last_unstale":"2024-05-09T00:09:10.469166+0000","last_undegraded":"2024-05-09T00:09:10.469166+0000","last_fullsized":"2024-05-09T00:09:10.469166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:12.366500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472299+0000","last_change":"2024-05-09T00:08:26.055234+0000","last_active":"2024-05-09T00:09:10.472299+0000","last_peered":"2024-05-09T00:09:10.472299+0000","last_clean":"2024-05-09T00:09:10.472299+0000","last_became_active":"2024-05-09T00:08:26.054851+0000","last_became_peered":"2024-05-09T00:08:26.054851+0000","last_unstale":"2024-05-09T00:09:10.472299+0000","last_undegraded":"2024-05-09T00:09:10.472299+0000","last_fullsized":"2024-05-09T00:09:10.472299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:54:52.121329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472899+0000","last_change":"2024-05-09T00:08:26.063375+0000","last_active":"2024-05-09T00:09:10.472899+0000","last_peered":"2024-05-09T00:09:10.472899+0000","last_clean":"2024-05-09T00:09:10.472899+0000","last_became_active":"2024-05-09T00:08:26.062962+0000","last_became_peered":"2024-05-09T00:08:26.062962+0000","last_unstale":"2024-05-09T00:09:10.472899+0000","last_undegraded":"2024-05-09T00:09:10.472899+0000","last_fullsized":"2024-05-09T00:09:10.472899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:05.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796257+0000","last_change":"2024-05-09T00:08:26.062034+0000","last_active":"2024-05-09T00:09:10.796257+0000","last_peered":"2024-05-09T00:09:10.796257+0000","last_clean":"2024-05-09T00:09:10.796257+0000","last_became_active":"2024-05-09T00:08:26.060376+0000","last_became_peered":"2024-05-09T00:08:26.060376+0000","last_unstale":"2024-05-09T00:09:10.796257+0000","last_undegraded":"2024-05-09T00:09:10.796257+0000","last_fullsized":"2024-05-09T00:09:10.796257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:15.830024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548685+0000","last_change":"2024-05-09T00:08:26.053227+0000","last_active":"2024-05-09T00:09:10.548685+0000","last_peered":"2024-05-09T00:09:10.548685+0000","last_clean":"2024-05-09T00:09:10.548685+0000","last_became_active":"2024-05-09T00:08:26.052878+0000","last_became_peered":"2024-05-09T00:08:26.052878+0000","last_unstale":"2024-05-09T00:09:10.548685+0000","last_undegraded":"2024-05-09T00:09:10.548685+0000","last_fullsized":"2024-05-09T00:09:10.548685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:24.385026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471290+0000","last_change":"2024-05-09T00:08:26.053041+0000","last_active":"2024-05-09T00:09:10.471290+0000","last_peered":"2024-05-09T00:09:10.471290+0000","last_clean":"2024-05-09T00:09:10.471290+0000","last_became_active":"2024-05-09T00:08:26.052677+0000","last_became_peered":"2024-05-09T00:08:26.052677+0000","last_unstale":"2024-05-09T00:09:10.471290+0000","last_undegraded":"2024-05-09T00:09:10.471290+0000","last_fullsized":"2024-05-09T00:09:10.471290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:24.478270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471141+0000","last_change":"2024-05-09T00:08:26.063391+0000","last_active":"2024-05-09T00:09:10.471141+0000","last_peered":"2024-05-09T00:09:10.471141+0000","last_clean":"2024-05-09T00:09:10.471141+0000","last_became_active":"2024-05-09T00:08:26.063033+0000","last_became_peered":"2024-05-09T00:08:26.063033+0000","last_unstale":"2024-05-09T00:09:10.471141+0000","last_undegraded":"2024-05-09T00:09:10.471141+0000","last_fullsized":"2024-05-09T00:09:10.471141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:41.900453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468575+0000","last_change":"2024-05-09T00:08:26.065018+0000","last_active":"2024-05-09T00:09:10.468575+0000","last_peered":"2024-05-09T00:09:10.468575+0000","last_clean":"2024-05-09T00:09:10.468575+0000","last_became_active":"2024-05-09T00:08:26.064642+0000","last_became_peered":"2024-05-09T00:08:26.064642+0000","last_unstale":"2024-05-09T00:09:10.468575+0000","last_undegraded":"2024-05-09T00:09:10.468575+0000","last_fullsized":"2024-05-09T00:09:10.468575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:02.403704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471869+0000","last_change":"2024-05-09T00:08:26.049770+0000","last_active":"2024-05-09T00:09:10.471869+0000","last_peered":"2024-05-09T00:09:10.471869+0000","last_clean":"2024-05-09T00:09:10.471869+0000","last_became_active":"2024-05-09T00:08:26.048555+0000","last_became_peered":"2024-05-09T00:08:26.048555+0000","last_unstale":"2024-05-09T00:09:10.471869+0000","last_undegraded":"2024-05-09T00:09:10.471869+0000","last_fullsized":"2024-05-09T00:09:10.471869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:16.009200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469538+0000","last_change":"2024-05-09T00:08:26.048587+0000","last_active":"2024-05-09T00:09:10.469538+0000","last_peered":"2024-05-09T00:09:10.469538+0000","last_clean":"2024-05-09T00:09:10.469538+0000","last_became_active":"2024-05-09T00:08:26.048148+0000","last_became_peered":"2024-05-09T00:08:26.048148+0000","last_unstale":"2024-05-09T00:09:10.469538+0000","last_undegraded":"2024-05-09T00:09:10.469538+0000","last_fullsized":"2024-05-09T00:09:10.469538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:42.113829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473434+0000","last_change":"2024-05-09T00:08:26.053607+0000","last_active":"2024-05-09T00:09:10.473434+0000","last_peered":"2024-05-09T00:09:10.473434+0000","last_clean":"2024-05-09T00:09:10.473434+0000","last_became_active":"2024-05-09T00:08:26.053212+0000","last_became_peered":"2024-05-09T00:08:26.053212+0000","last_unstale":"2024-05-09T00:09:10.473434+0000","last_undegraded":"2024-05-09T00:09:10.473434+0000","last_fullsized":"2024-05-09T00:09:10.473434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:32.937049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469272+0000","last_change":"2024-05-09T00:08:23.024481+0000","last_active":"2024-05-09T00:09:10.469272+0000","last_peered":"2024-05-09T00:09:10.469272+0000","last_clean":"2024-05-09T00:09:10.469272+0000","last_became_active":"2024-05-09T00:08:23.024132+0000","last_became_peered":"2024-05-09T00:08:23.024132+0000","last_unstale":"2024-05-09T00:09:10.469272+0000","last_undegraded":"2024-05-09T00:09:10.469272+0000","last_fullsized":"2024-05-09T00:09:10.469272+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:11.188505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468045+0000","last_change":"2024-05-09T00:08:26.054772+0000","last_active":"2024-05-09T00:09:10.468045+0000","last_peered":"2024-05-09T00:09:10.468045+0000","last_clean":"2024-05-09T00:09:10.468045+0000","last_became_active":"2024-05-09T00:08:26.054433+0000","last_became_peered":"2024-05-09T00:08:26.054433+0000","last_unstale":"2024-05-09T00:09:10.468045+0000","last_undegraded":"2024-05-09T00:09:10.468045+0000","last_fullsized":"2024-05-09T00:09:10.468045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:44.283240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468325+0000","last_change":"2024-05-09T00:08:23.118778+0000","last_active":"2024-05-09T00:09:10.468325+0000","last_peered":"2024-05-09T00:09:10.468325+0000","last_clean":"2024-05-09T00:09:10.468325+0000","last_became_active":"2024-05-09T00:08:23.118377+0000","last_became_peered":"2024-05-09T00:08:23.118377+0000","last_unstale":"2024-05-09T00:09:10.468325+0000","last_undegraded":"2024-05-09T00:09:10.468325+0000","last_fullsized":"2024-05-09T00:09:10.468325+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:51.352442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472000+0000","last_change":"2024-05-09T00:08:26.059953+0000","last_active":"2024-05-09T00:09:10.472000+0000","last_peered":"2024-05-09T00:09:10.472000+0000","last_clean":"2024-05-09T00:09:10.472000+0000","last_became_active":"2024-05-09T00:08:26.059571+0000","last_became_peered":"2024-05-09T00:08:26.059571+0000","last_unstale":"2024-05-09T00:09:10.472000+0000","last_undegraded":"2024-05-09T00:09:10.472000+0000","last_fullsized":"2024-05-09T00:09:10.472000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:48.056798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475240+0000","last_change":"2024-05-09T00:08:23.021244+0000","last_active":"2024-05-09T00:09:10.475240+0000","last_peered":"2024-05-09T00:09:10.475240+0000","last_clean":"2024-05-09T00:09:10.475240+0000","last_became_active":"2024-05-09T00:08:23.020597+0000","last_became_peered":"2024-05-09T00:08:23.020597+0000","last_unstale":"2024-05-09T00:09:10.475240+0000","last_undegraded":"2024-05-09T00:09:10.475240+0000","last_fullsized":"2024-05-09T00:09:10.475240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.654261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468788+0000","last_change":"2024-05-09T00:08:26.059834+0000","last_active":"2024-05-09T00:09:10.468788+0000","last_peered":"2024-05-09T00:09:10.468788+0000","last_clean":"2024-05-09T00:09:10.468788+0000","last_became_active":"2024-05-09T00:08:26.059349+0000","last_became_peered":"2024-05-09T00:08:26.059349+0000","last_unstale":"2024-05-09T00:09:10.468788+0000","last_undegraded":"2024-05-09T00:09:10.468788+0000","last_fullsized":"2024-05-09T00:09:10.468788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:42.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474158+0000","last_change":"2024-05-09T00:08:23.019568+0000","last_active":"2024-05-09T00:09:10.474158+0000","last_peered":"2024-05-09T00:09:10.474158+0000","last_clean":"2024-05-09T00:09:10.474158+0000","last_became_active":"2024-05-09T00:08:23.019149+0000","last_became_peered":"2024-05-09T00:08:23.019149+0000","last_unstale":"2024-05-09T00:09:10.474158+0000","last_undegraded":"2024-05-09T00:09:10.474158+0000","last_fullsized":"2024-05-09T00:09:10.474158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:44.569911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469966+0000","last_change":"2024-05-09T00:08:26.061308+0000","last_active":"2024-05-09T00:09:10.469966+0000","last_peered":"2024-05-09T00:09:10.469966+0000","last_clean":"2024-05-09T00:09:10.469966+0000","last_became_active":"2024-05-09T00:08:26.060973+0000","last_became_peered":"2024-05-09T00:08:26.060973+0000","last_unstale":"2024-05-09T00:09:10.469966+0000","last_undegraded":"2024-05-09T00:09:10.469966+0000","last_fullsized":"2024-05-09T00:09:10.469966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:16.845074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":75,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:25.455614+0000","last_change":"2024-05-09T00:08:23.020886+0000","last_active":"2024-05-09T00:09:25.455614+0000","last_peered":"2024-05-09T00:09:25.455614+0000","last_clean":"2024-05-09T00:09:25.455614+0000","last_became_active":"2024-05-09T00:08:23.020530+0000","last_became_peered":"2024-05-09T00:08:23.020530+0000","last_unstale":"2024-05-09T00:09:25.455614+0000","last_undegraded":"2024-05-09T00:09:25.455614+0000","last_fullsized":"2024-05-09T00:09:25.455614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:34.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474367+0000","last_change":"2024-05-09T00:08:26.051018+0000","last_active":"2024-05-09T00:09:10.474367+0000","last_peered":"2024-05-09T00:09:10.474367+0000","last_clean":"2024-05-09T00:09:10.474367+0000","last_became_active":"2024-05-09T00:08:26.050495+0000","last_became_peered":"2024-05-09T00:08:26.050495+0000","last_unstale":"2024-05-09T00:09:10.474367+0000","last_undegraded":"2024-05-09T00:09:10.474367+0000","last_fullsized":"2024-05-09T00:09:10.474367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:07.862339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472087+0000","last_change":"2024-05-09T00:08:23.021187+0000","last_active":"2024-05-09T00:09:10.472087+0000","last_peered":"2024-05-09T00:09:10.472087+0000","last_clean":"2024-05-09T00:09:10.472087+0000","last_became_active":"2024-05-09T00:08:23.020770+0000","last_became_peered":"2024-05-09T00:08:23.020770+0000","last_unstale":"2024-05-09T00:09:10.472087+0000","last_undegraded":"2024-05-09T00:09:10.472087+0000","last_fullsized":"2024-05-09T00:09:10.472087+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:08.992770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.797838+0000","last_change":"2024-05-09T00:09:10.797392+0000","last_active":"2024-05-09T00:09:10.797838+0000","last_peered":"2024-05-09T00:09:10.797838+0000","last_clean":"2024-05-09T00:09:10.797838+0000","last_became_active":"2024-05-09T00:09:10.796969+0000","last_became_peered":"2024-05-09T00:09:10.796969+0000","last_unstale":"2024-05-09T00:09:10.797838+0000","last_undegraded":"2024-05-09T00:09:10.797838+0000","last_fullsized":"2024-05-09T00:09:10.797838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:18.852678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548755+0000","last_change":"2024-05-09T00:08:23.012226+0000","last_active":"2024-05-09T00:09:10.548755+0000","last_peered":"2024-05-09T00:09:10.548755+0000","last_clean":"2024-05-09T00:09:10.548755+0000","last_became_active":"2024-05-09T00:08:23.011781+0000","last_became_peered":"2024-05-09T00:08:23.011781+0000","last_unstale":"2024-05-09T00:09:10.548755+0000","last_undegraded":"2024-05-09T00:09:10.548755+0000","last_fullsized":"2024-05-09T00:09:10.548755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:44.452206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467563+0000","last_change":"2024-05-09T00:08:26.043035+0000","last_active":"2024-05-09T00:09:10.467563+0000","last_peered":"2024-05-09T00:09:10.467563+0000","last_clean":"2024-05-09T00:09:10.467563+0000","last_became_active":"2024-05-09T00:08:26.042686+0000","last_became_peered":"2024-05-09T00:08:26.042686+0000","last_unstale":"2024-05-09T00:09:10.467563+0000","last_undegraded":"2024-05-09T00:09:10.467563+0000","last_fullsized":"2024-05-09T00:09:10.467563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:24.511309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469042+0000","last_change":"2024-05-09T00:08:23.022017+0000","last_active":"2024-05-09T00:09:10.469042+0000","last_peered":"2024-05-09T00:09:10.469042+0000","last_clean":"2024-05-09T00:09:10.469042+0000","last_became_active":"2024-05-09T00:08:23.021540+0000","last_became_peered":"2024-05-09T00:08:23.021540+0000","last_unstale":"2024-05-09T00:09:10.469042+0000","last_undegraded":"2024-05-09T00:09:10.469042+0000","last_fullsized":"2024-05-09T00:09:10.469042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:50.773211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469091+0000","last_change":"2024-05-09T00:08:26.048039+0000","last_active":"2024-05-09T00:09:10.469091+0000","last_peered":"2024-05-09T00:09:10.469091+0000","last_clean":"2024-05-09T00:09:10.469091+0000","last_became_active":"2024-05-09T00:08:26.047695+0000","last_became_peered":"2024-05-09T00:08:26.047695+0000","last_unstale":"2024-05-09T00:09:10.469091+0000","last_undegraded":"2024-05-09T00:09:10.469091+0000","last_fullsized":"2024-05-09T00:09:10.469091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:27.363847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":74,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:27.457809+0000","last_change":"2024-05-09T00:08:23.019565+0000","last_active":"2024-05-09T00:09:27.457809+0000","last_peered":"2024-05-09T00:09:27.457809+0000","last_clean":"2024-05-09T00:09:27.457809+0000","last_became_active":"2024-05-09T00:08:23.019113+0000","last_became_peered":"2024-05-09T00:08:23.019113+0000","last_unstale":"2024-05-09T00:09:27.457809+0000","last_undegraded":"2024-05-09T00:09:27.457809+0000","last_fullsized":"2024-05-09T00:09:27.457809+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:45.578934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469070+0000","last_change":"2024-05-09T00:08:26.058869+0000","last_active":"2024-05-09T00:09:10.469070+0000","last_peered":"2024-05-09T00:09:10.469070+0000","last_clean":"2024-05-09T00:09:10.469070+0000","last_became_active":"2024-05-09T00:08:26.058482+0000","last_became_peered":"2024-05-09T00:08:26.058482+0000","last_unstale":"2024-05-09T00:09:10.469070+0000","last_undegraded":"2024-05-09T00:09:10.469070+0000","last_fullsized":"2024-05-09T00:09:10.469070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:33.252208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472150+0000","last_change":"2024-05-09T00:08:23.024425+0000","last_active":"2024-05-09T00:09:10.472150+0000","last_peered":"2024-05-09T00:09:10.472150+0000","last_clean":"2024-05-09T00:09:10.472150+0000","last_became_active":"2024-05-09T00:08:23.023991+0000","last_became_peered":"2024-05-09T00:08:23.023991+0000","last_unstale":"2024-05-09T00:09:10.472150+0000","last_undegraded":"2024-05-09T00:09:10.472150+0000","last_fullsized":"2024-05-09T00:09:10.472150+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:47.907851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475008+0000","last_change":"2024-05-09T00:09:10.475008+0000","last_active":"2024-05-09T00:09:10.475008+0000","last_peered":"2024-05-09T00:09:10.475008+0000","last_clean":"2024-05-09T00:09:10.475008+0000","last_became_active":"2024-05-09T00:09:10.474663+0000","last_became_peered":"2024-05-09T00:09:10.474663+0000","last_unstale":"2024-05-09T00:09:10.475008+0000","last_undegraded":"2024-05-09T00:09:10.475008+0000","last_fullsized":"2024-05-09T00:09:10.475008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:36.990362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470120+0000","last_change":"2024-05-09T00:08:23.118516+0000","last_active":"2024-05-09T00:09:10.470120+0000","last_peered":"2024-05-09T00:09:10.470120+0000","last_clean":"2024-05-09T00:09:10.470120+0000","last_became_active":"2024-05-09T00:08:23.118147+0000","last_became_peered":"2024-05-09T00:08:23.118147+0000","last_unstale":"2024-05-09T00:09:10.470120+0000","last_undegraded":"2024-05-09T00:09:10.470120+0000","last_fullsized":"2024-05-09T00:09:10.470120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:24.205818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468743+0000","last_change":"2024-05-09T00:08:26.067063+0000","last_active":"2024-05-09T00:09:10.468743+0000","last_peered":"2024-05-09T00:09:10.468743+0000","last_clean":"2024-05-09T00:09:10.468743+0000","last_became_active":"2024-05-09T00:08:26.066440+0000","last_became_peered":"2024-05-09T00:08:26.066440+0000","last_unstale":"2024-05-09T00:09:10.468743+0000","last_undegraded":"2024-05-09T00:09:10.468743+0000","last_fullsized":"2024-05-09T00:09:10.468743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:48.648485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":155,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:28.459371+0000","last_change":"2024-05-09T00:08:23.020757+0000","last_active":"2024-05-09T00:09:28.459371+0000","last_peered":"2024-05-09T00:09:28.459371+0000","last_clean":"2024-05-09T00:09:28.459371+0000","last_became_active":"2024-05-09T00:08:23.020355+0000","last_became_peered":"2024-05-09T00:08:23.020355+0000","last_unstale":"2024-05-09T00:09:28.459371+0000","last_undegraded":"2024-05-09T00:09:28.459371+0000","last_fullsized":"2024-05-09T00:09:28.459371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:00.605691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61826,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":99,"num_read_kb":109,"num_write":31,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475093+0000","last_change":"2024-05-09T00:08:26.057402+0000","last_active":"2024-05-09T00:09:10.475093+0000","last_peered":"2024-05-09T00:09:10.475093+0000","last_clean":"2024-05-09T00:09:10.475093+0000","last_became_active":"2024-05-09T00:08:26.056974+0000","last_became_peered":"2024-05-09T00:08:26.056974+0000","last_unstale":"2024-05-09T00:09:10.475093+0000","last_undegraded":"2024-05-09T00:09:10.475093+0000","last_fullsized":"2024-05-09T00:09:10.475093+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:47.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473890+0000","last_change":"2024-05-09T00:08:23.020208+0000","last_active":"2024-05-09T00:09:10.473890+0000","last_peered":"2024-05-09T00:09:10.473890+0000","last_clean":"2024-05-09T00:09:10.473890+0000","last_became_active":"2024-05-09T00:08:23.019809+0000","last_became_peered":"2024-05-09T00:08:23.019809+0000","last_unstale":"2024-05-09T00:09:10.473890+0000","last_undegraded":"2024-05-09T00:09:10.473890+0000","last_fullsized":"2024-05-09T00:09:10.473890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:47.900731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548422+0000","last_change":"2024-05-09T00:08:26.054273+0000","last_active":"2024-05-09T00:09:10.548422+0000","last_peered":"2024-05-09T00:09:10.548422+0000","last_clean":"2024-05-09T00:09:10.548422+0000","last_became_active":"2024-05-09T00:08:26.053840+0000","last_became_peered":"2024-05-09T00:08:26.053840+0000","last_unstale":"2024-05-09T00:09:10.548422+0000","last_undegraded":"2024-05-09T00:09:10.548422+0000","last_fullsized":"2024-05-09T00:09:10.548422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:27.128582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469240+0000","last_change":"2024-05-09T00:08:23.021594+0000","last_active":"2024-05-09T00:09:10.469240+0000","last_peered":"2024-05-09T00:09:10.469240+0000","last_clean":"2024-05-09T00:09:10.469240+0000","last_became_active":"2024-05-09T00:08:23.021152+0000","last_became_peered":"2024-05-09T00:08:23.021152+0000","last_unstale":"2024-05-09T00:09:10.469240+0000","last_undegraded":"2024-05-09T00:09:10.469240+0000","last_fullsized":"2024-05-09T00:09:10.469240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:58.505781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473341+0000","last_change":"2024-05-09T00:08:26.064464+0000","last_active":"2024-05-09T00:09:10.473341+0000","last_peered":"2024-05-09T00:09:10.473341+0000","last_clean":"2024-05-09T00:09:10.473341+0000","last_became_active":"2024-05-09T00:08:26.064000+0000","last_became_peered":"2024-05-09T00:08:26.064000+0000","last_unstale":"2024-05-09T00:09:10.473341+0000","last_undegraded":"2024-05-09T00:09:10.473341+0000","last_fullsized":"2024-05-09T00:09:10.473341+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:01.455979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'4","reported_seq":77,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:26.457435+0000","last_change":"2024-05-09T00:08:23.020823+0000","last_active":"2024-05-09T00:09:26.457435+0000","last_peered":"2024-05-09T00:09:26.457435+0000","last_clean":"2024-05-09T00:09:26.457435+0000","last_became_active":"2024-05-09T00:08:23.020412+0000","last_became_peered":"2024-05-09T00:08:23.020412+0000","last_unstale":"2024-05-09T00:09:26.457435+0000","last_undegraded":"2024-05-09T00:09:26.457435+0000","last_fullsized":"2024-05-09T00:09:26.457435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:05.457452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474540+0000","last_change":"2024-05-09T00:09:10.474540+0000","last_active":"2024-05-09T00:09:10.474540+0000","last_peered":"2024-05-09T00:09:10.474540+0000","last_clean":"2024-05-09T00:09:10.474540+0000","last_became_active":"2024-05-09T00:09:10.474068+0000","last_became_peered":"2024-05-09T00:09:10.474068+0000","last_unstale":"2024-05-09T00:09:10.474540+0000","last_undegraded":"2024-05-09T00:09:10.474540+0000","last_fullsized":"2024-05-09T00:09:10.474540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:37.296140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474161+0000","last_change":"2024-05-09T00:08:23.118857+0000","last_active":"2024-05-09T00:09:10.474161+0000","last_peered":"2024-05-09T00:09:10.474161+0000","last_clean":"2024-05-09T00:09:10.474161+0000","last_became_active":"2024-05-09T00:08:23.118433+0000","last_became_peered":"2024-05-09T00:08:23.118433+0000","last_unstale":"2024-05-09T00:09:10.474161+0000","last_undegraded":"2024-05-09T00:09:10.474161+0000","last_fullsized":"2024-05-09T00:09:10.474161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:10.617987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469212+0000","last_change":"2024-05-09T00:08:26.066911+0000","last_active":"2024-05-09T00:09:10.469212+0000","last_peered":"2024-05-09T00:09:10.469212+0000","last_clean":"2024-05-09T00:09:10.469212+0000","last_became_active":"2024-05-09T00:08:26.066548+0000","last_became_peered":"2024-05-09T00:08:26.066548+0000","last_unstale":"2024-05-09T00:09:10.469212+0000","last_undegraded":"2024-05-09T00:09:10.469212+0000","last_fullsized":"2024-05-09T00:09:10.469212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:32.391390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475008+0000","last_change":"2024-05-09T00:08:23.019829+0000","last_active":"2024-05-09T00:09:10.475008+0000","last_peered":"2024-05-09T00:09:10.475008+0000","last_clean":"2024-05-09T00:09:10.475008+0000","last_became_active":"2024-05-09T00:08:23.019336+0000","last_became_peered":"2024-05-09T00:08:23.019336+0000","last_unstale":"2024-05-09T00:09:10.475008+0000","last_undegraded":"2024-05-09T00:09:10.475008+0000","last_fullsized":"2024-05-09T00:09:10.475008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:33.802738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470640+0000","last_change":"2024-05-09T00:08:26.055975+0000","last_active":"2024-05-09T00:09:10.470640+0000","last_peered":"2024-05-09T00:09:10.470640+0000","last_clean":"2024-05-09T00:09:10.470640+0000","last_became_active":"2024-05-09T00:08:26.055595+0000","last_became_peered":"2024-05-09T00:08:26.055595+0000","last_unstale":"2024-05-09T00:09:10.470640+0000","last_undegraded":"2024-05-09T00:09:10.470640+0000","last_fullsized":"2024-05-09T00:09:10.470640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:20.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469371+0000","last_change":"2024-05-09T00:08:23.120353+0000","last_active":"2024-05-09T00:09:10.469371+0000","last_peered":"2024-05-09T00:09:10.469371+0000","last_clean":"2024-05-09T00:09:10.469371+0000","last_became_active":"2024-05-09T00:08:23.120004+0000","last_became_peered":"2024-05-09T00:08:23.120004+0000","last_unstale":"2024-05-09T00:09:10.469371+0000","last_undegraded":"2024-05-09T00:09:10.469371+0000","last_fullsized":"2024-05-09T00:09:10.469371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:12.168671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471920+0000","last_change":"2024-05-09T00:08:26.046554+0000","last_active":"2024-05-09T00:09:10.471920+0000","last_peered":"2024-05-09T00:09:10.471920+0000","last_clean":"2024-05-09T00:09:10.471920+0000","last_became_active":"2024-05-09T00:08:26.045924+0000","last_became_peered":"2024-05-09T00:08:26.045924+0000","last_unstale":"2024-05-09T00:09:10.471920+0000","last_undegraded":"2024-05-09T00:09:10.471920+0000","last_fullsized":"2024-05-09T00:09:10.471920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:01.208980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472382+0000","last_change":"2024-05-09T00:08:23.019778+0000","last_active":"2024-05-09T00:09:10.472382+0000","last_peered":"2024-05-09T00:09:10.472382+0000","last_clean":"2024-05-09T00:09:10.472382+0000","last_became_active":"2024-05-09T00:08:23.019400+0000","last_became_peered":"2024-05-09T00:08:23.019400+0000","last_unstale":"2024-05-09T00:09:10.472382+0000","last_undegraded":"2024-05-09T00:09:10.472382+0000","last_fullsized":"2024-05-09T00:09:10.472382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:06.883167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474352+0000","last_change":"2024-05-09T00:09:10.474352+0000","last_active":"2024-05-09T00:09:10.474352+0000","last_peered":"2024-05-09T00:09:10.474352+0000","last_clean":"2024-05-09T00:09:10.474352+0000","last_became_active":"2024-05-09T00:09:10.473906+0000","last_became_peered":"2024-05-09T00:09:10.473906+0000","last_unstale":"2024-05-09T00:09:10.474352+0000","last_undegraded":"2024-05-09T00:09:10.474352+0000","last_fullsized":"2024-05-09T00:09:10.474352+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:35:18.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.476484+0000","last_change":"2024-05-09T00:09:10.476484+0000","last_active":"2024-05-09T00:09:10.476484+0000","last_peered":"2024-05-09T00:09:10.476484+0000","last_clean":"2024-05-09T00:09:10.476484+0000","last_became_active":"2024-05-09T00:09:10.476001+0000","last_became_peered":"2024-05-09T00:09:10.476001+0000","last_unstale":"2024-05-09T00:09:10.476484+0000","last_undegraded":"2024-05-09T00:09:10.476484+0000","last_fullsized":"2024-05-09T00:09:10.476484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:32.018749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":75,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:27.458215+0000","last_change":"2024-05-09T00:08:23.118322+0000","last_active":"2024-05-09T00:09:27.458215+0000","last_peered":"2024-05-09T00:09:27.458215+0000","last_clean":"2024-05-09T00:09:27.458215+0000","last_became_active":"2024-05-09T00:08:23.117943+0000","last_became_peered":"2024-05-09T00:08:23.117943+0000","last_unstale":"2024-05-09T00:09:27.458215+0000","last_undegraded":"2024-05-09T00:09:27.458215+0000","last_fullsized":"2024-05-09T00:09:27.458215+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:07.890437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467948+0000","last_change":"2024-05-09T00:08:26.052294+0000","last_active":"2024-05-09T00:09:10.467948+0000","last_peered":"2024-05-09T00:09:10.467948+0000","last_clean":"2024-05-09T00:09:10.467948+0000","last_became_active":"2024-05-09T00:08:26.051926+0000","last_became_peered":"2024-05-09T00:08:26.051926+0000","last_unstale":"2024-05-09T00:09:10.467948+0000","last_undegraded":"2024-05-09T00:09:10.467948+0000","last_fullsized":"2024-05-09T00:09:10.467948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:53.651401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796281+0000","last_change":"2024-05-09T00:08:23.015007+0000","last_active":"2024-05-09T00:09:10.796281+0000","last_peered":"2024-05-09T00:09:10.796281+0000","last_clean":"2024-05-09T00:09:10.796281+0000","last_became_active":"2024-05-09T00:08:23.014545+0000","last_became_peered":"2024-05-09T00:08:23.014545+0000","last_unstale":"2024-05-09T00:09:10.796281+0000","last_undegraded":"2024-05-09T00:09:10.796281+0000","last_fullsized":"2024-05-09T00:09:10.796281+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:19.850200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472332+0000","last_change":"2024-05-09T00:07:00.733233+0000","last_active":"2024-05-09T00:09:10.472332+0000","last_peered":"2024-05-09T00:09:10.472332+0000","last_clean":"2024-05-09T00:09:10.472332+0000","last_became_active":"2024-05-09T00:07:00.727365+0000","last_became_peered":"2024-05-09T00:07:00.727365+0000","last_unstale":"2024-05-09T00:09:10.472332+0000","last_undegraded":"2024-05-09T00:09:10.472332+0000","last_fullsized":"2024-05-09T00:09:10.472332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_clean_scrub_stamp":"2024-05-09T00:03:33.285220+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:08.597290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548228+0000","last_change":"2024-05-09T00:08:26.064000+0000","last_active":"2024-05-09T00:09:10.548228+0000","last_peered":"2024-05-09T00:09:10.548228+0000","last_clean":"2024-05-09T00:09:10.548228+0000","last_became_active":"2024-05-09T00:08:26.063636+0000","last_became_peered":"2024-05-09T00:08:26.063636+0000","last_unstale":"2024-05-09T00:09:10.548228+0000","last_undegraded":"2024-05-09T00:09:10.548228+0000","last_fullsized":"2024-05-09T00:09:10.548228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:59.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472537+0000","last_change":"2024-05-09T00:08:23.023459+0000","last_active":"2024-05-09T00:09:10.472537+0000","last_peered":"2024-05-09T00:09:10.472537+0000","last_clean":"2024-05-09T00:09:10.472537+0000","last_became_active":"2024-05-09T00:08:23.023089+0000","last_became_peered":"2024-05-09T00:08:23.023089+0000","last_unstale":"2024-05-09T00:09:10.472537+0000","last_undegraded":"2024-05-09T00:09:10.472537+0000","last_fullsized":"2024-05-09T00:09:10.472537+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:41.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.479186+0000","last_change":"2024-05-09T00:09:10.478686+0000","last_active":"2024-05-09T00:09:10.479186+0000","last_peered":"2024-05-09T00:09:10.479186+0000","last_clean":"2024-05-09T00:09:10.479186+0000","last_became_active":"2024-05-09T00:09:10.474890+0000","last_became_peered":"2024-05-09T00:09:10.474890+0000","last_unstale":"2024-05-09T00:09:10.479186+0000","last_undegraded":"2024-05-09T00:09:10.479186+0000","last_fullsized":"2024-05-09T00:09:10.479186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:41.041247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548460+0000","last_change":"2024-05-09T00:08:26.050685+0000","last_active":"2024-05-09T00:09:10.548460+0000","last_peered":"2024-05-09T00:09:10.548460+0000","last_clean":"2024-05-09T00:09:10.548460+0000","last_became_active":"2024-05-09T00:08:26.050312+0000","last_became_peered":"2024-05-09T00:08:26.050312+0000","last_unstale":"2024-05-09T00:09:10.548460+0000","last_undegraded":"2024-05-09T00:09:10.548460+0000","last_fullsized":"2024-05-09T00:09:10.548460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:31.623182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'11","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469432+0000","last_change":"2024-05-09T00:08:23.019663+0000","last_active":"2024-05-09T00:09:10.469432+0000","last_peered":"2024-05-09T00:09:10.469432+0000","last_clean":"2024-05-09T00:09:10.469432+0000","last_became_active":"2024-05-09T00:08:23.019301+0000","last_became_peered":"2024-05-09T00:08:23.019301+0000","last_unstale":"2024-05-09T00:09:10.469432+0000","last_undegraded":"2024-05-09T00:09:10.469432+0000","last_fullsized":"2024-05-09T00:09:10.469432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:08.291227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468179+0000","last_change":"2024-05-09T00:08:26.058825+0000","last_active":"2024-05-09T00:09:10.468179+0000","last_peered":"2024-05-09T00:09:10.468179+0000","last_clean":"2024-05-09T00:09:10.468179+0000","last_became_active":"2024-05-09T00:08:26.058469+0000","last_became_peered":"2024-05-09T00:08:26.058469+0000","last_unstale":"2024-05-09T00:09:10.468179+0000","last_undegraded":"2024-05-09T00:09:10.468179+0000","last_fullsized":"2024-05-09T00:09:10.468179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:08.118819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796528+0000","last_change":"2024-05-09T00:08:23.054366+0000","last_active":"2024-05-09T00:09:10.796528+0000","last_peered":"2024-05-09T00:09:10.796528+0000","last_clean":"2024-05-09T00:09:10.796528+0000","last_became_active":"2024-05-09T00:08:23.053977+0000","last_became_peered":"2024-05-09T00:08:23.053977+0000","last_unstale":"2024-05-09T00:09:10.796528+0000","last_undegraded":"2024-05-09T00:09:10.796528+0000","last_fullsized":"2024-05-09T00:09:10.796528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:37.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471766+0000","last_change":"2024-05-09T00:08:26.057864+0000","last_active":"2024-05-09T00:09:10.471766+0000","last_peered":"2024-05-09T00:09:10.471766+0000","last_clean":"2024-05-09T00:09:10.471766+0000","last_became_active":"2024-05-09T00:08:26.057436+0000","last_became_peered":"2024-05-09T00:08:26.057436+0000","last_unstale":"2024-05-09T00:09:10.471766+0000","last_undegraded":"2024-05-09T00:09:10.471766+0000","last_fullsized":"2024-05-09T00:09:10.471766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:19.976265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468611+0000","last_change":"2024-05-09T00:08:23.118803+0000","last_active":"2024-05-09T00:09:10.468611+0000","last_peered":"2024-05-09T00:09:10.468611+0000","last_clean":"2024-05-09T00:09:10.468611+0000","last_became_active":"2024-05-09T00:08:23.118432+0000","last_became_peered":"2024-05-09T00:08:23.118432+0000","last_unstale":"2024-05-09T00:09:10.468611+0000","last_undegraded":"2024-05-09T00:09:10.468611+0000","last_fullsized":"2024-05-09T00:09:10.468611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:50.253699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473482+0000","last_change":"2024-05-09T00:08:26.054281+0000","last_active":"2024-05-09T00:09:10.473482+0000","last_peered":"2024-05-09T00:09:10.473482+0000","last_clean":"2024-05-09T00:09:10.473482+0000","last_became_active":"2024-05-09T00:08:26.053874+0000","last_became_peered":"2024-05-09T00:08:26.053874+0000","last_unstale":"2024-05-09T00:09:10.473482+0000","last_undegraded":"2024-05-09T00:09:10.473482+0000","last_fullsized":"2024-05-09T00:09:10.473482+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:11.566337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467696+0000","last_change":"2024-05-09T00:08:23.017905+0000","last_active":"2024-05-09T00:09:10.467696+0000","last_peered":"2024-05-09T00:09:10.467696+0000","last_clean":"2024-05-09T00:09:10.467696+0000","last_became_active":"2024-05-09T00:08:23.017481+0000","last_became_peered":"2024-05-09T00:08:23.017481+0000","last_unstale":"2024-05-09T00:09:10.467696+0000","last_undegraded":"2024-05-09T00:09:10.467696+0000","last_fullsized":"2024-05-09T00:09:10.467696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:40.096415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469888+0000","last_change":"2024-05-09T00:08:26.053392+0000","last_active":"2024-05-09T00:09:10.469888+0000","last_peered":"2024-05-09T00:09:10.469888+0000","last_clean":"2024-05-09T00:09:10.469888+0000","last_became_active":"2024-05-09T00:08:26.052628+0000","last_became_peered":"2024-05-09T00:08:26.052628+0000","last_unstale":"2024-05-09T00:09:10.469888+0000","last_undegraded":"2024-05-09T00:09:10.469888+0000","last_fullsized":"2024-05-09T00:09:10.469888+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:28.013955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.481712+0000","last_change":"2024-05-09T00:09:10.481233+0000","last_active":"2024-05-09T00:09:10.481712+0000","last_peered":"2024-05-09T00:09:10.481712+0000","last_clean":"2024-05-09T00:09:10.481712+0000","last_became_active":"2024-05-09T00:09:10.475321+0000","last_became_peered":"2024-05-09T00:09:10.475321+0000","last_unstale":"2024-05-09T00:09:10.481712+0000","last_undegraded":"2024-05-09T00:09:10.481712+0000","last_fullsized":"2024-05-09T00:09:10.481712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:58.859277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467933+0000","last_change":"2024-05-09T00:08:26.047812+0000","last_active":"2024-05-09T00:09:10.467933+0000","last_peered":"2024-05-09T00:09:10.467933+0000","last_clean":"2024-05-09T00:09:10.467933+0000","last_became_active":"2024-05-09T00:08:26.047459+0000","last_became_peered":"2024-05-09T00:08:26.047459+0000","last_unstale":"2024-05-09T00:09:10.467933+0000","last_undegraded":"2024-05-09T00:09:10.467933+0000","last_fullsized":"2024-05-09T00:09:10.467933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.257831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473758+0000","last_change":"2024-05-09T00:08:23.023800+0000","last_active":"2024-05-09T00:09:10.473758+0000","last_peered":"2024-05-09T00:09:10.473758+0000","last_clean":"2024-05-09T00:09:10.473758+0000","last_became_active":"2024-05-09T00:08:23.023387+0000","last_became_peered":"2024-05-09T00:08:23.023387+0000","last_unstale":"2024-05-09T00:09:10.473758+0000","last_undegraded":"2024-05-09T00:09:10.473758+0000","last_fullsized":"2024-05-09T00:09:10.473758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:33.714683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472532+0000","last_change":"2024-05-09T00:08:26.056981+0000","last_active":"2024-05-09T00:09:10.472532+0000","last_peered":"2024-05-09T00:09:10.472532+0000","last_clean":"2024-05-09T00:09:10.472532+0000","last_became_active":"2024-05-09T00:08:26.056605+0000","last_became_peered":"2024-05-09T00:08:26.056605+0000","last_unstale":"2024-05-09T00:09:10.472532+0000","last_undegraded":"2024-05-09T00:09:10.472532+0000","last_fullsized":"2024-05-09T00:09:10.472532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:32.897929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473611+0000","last_change":"2024-05-09T00:08:23.120612+0000","last_active":"2024-05-09T00:09:10.473611+0000","last_peered":"2024-05-09T00:09:10.473611+0000","last_clean":"2024-05-09T00:09:10.473611+0000","last_became_active":"2024-05-09T00:08:23.120157+0000","last_became_peered":"2024-05-09T00:08:23.120157+0000","last_unstale":"2024-05-09T00:09:10.473611+0000","last_undegraded":"2024-05-09T00:09:10.473611+0000","last_fullsized":"2024-05-09T00:09:10.473611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:08.027214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473165+0000","last_change":"2024-05-09T00:08:26.061594+0000","last_active":"2024-05-09T00:09:10.473165+0000","last_peered":"2024-05-09T00:09:10.473165+0000","last_clean":"2024-05-09T00:09:10.473165+0000","last_became_active":"2024-05-09T00:08:26.061201+0000","last_became_peered":"2024-05-09T00:08:26.061201+0000","last_unstale":"2024-05-09T00:09:10.473165+0000","last_undegraded":"2024-05-09T00:09:10.473165+0000","last_fullsized":"2024-05-09T00:09:10.473165+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:50.183648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469471+0000","last_change":"2024-05-09T00:08:23.017693+0000","last_active":"2024-05-09T00:09:10.469471+0000","last_peered":"2024-05-09T00:09:10.469471+0000","last_clean":"2024-05-09T00:09:10.469471+0000","last_became_active":"2024-05-09T00:08:23.017211+0000","last_became_peered":"2024-05-09T00:08:23.017211+0000","last_unstale":"2024-05-09T00:09:10.469471+0000","last_undegraded":"2024-05-09T00:09:10.469471+0000","last_fullsized":"2024-05-09T00:09:10.469471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:53.569932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472752+0000","last_change":"2024-05-09T00:08:26.063922+0000","last_active":"2024-05-09T00:09:10.472752+0000","last_peered":"2024-05-09T00:09:10.472752+0000","last_clean":"2024-05-09T00:09:10.472752+0000","last_became_active":"2024-05-09T00:08:26.063554+0000","last_became_peered":"2024-05-09T00:08:26.063554+0000","last_unstale":"2024-05-09T00:09:10.472752+0000","last_undegraded":"2024-05-09T00:09:10.472752+0000","last_fullsized":"2024-05-09T00:09:10.472752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:01.754688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548572+0000","last_change":"2024-05-09T00:08:23.020367+0000","last_active":"2024-05-09T00:09:10.548572+0000","last_peered":"2024-05-09T00:09:10.548572+0000","last_clean":"2024-05-09T00:09:10.548572+0000","last_became_active":"2024-05-09T00:08:23.019801+0000","last_became_peered":"2024-05-09T00:08:23.019801+0000","last_unstale":"2024-05-09T00:09:10.548572+0000","last_undegraded":"2024-05-09T00:09:10.548572+0000","last_fullsized":"2024-05-09T00:09:10.548572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:14.936021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472767+0000","last_change":"2024-05-09T00:08:26.063863+0000","last_active":"2024-05-09T00:09:10.472767+0000","last_peered":"2024-05-09T00:09:10.472767+0000","last_clean":"2024-05-09T00:09:10.472767+0000","last_became_active":"2024-05-09T00:08:26.061938+0000","last_became_peered":"2024-05-09T00:08:26.061938+0000","last_unstale":"2024-05-09T00:09:10.472767+0000","last_undegraded":"2024-05-09T00:09:10.472767+0000","last_fullsized":"2024-05-09T00:09:10.472767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:04.597991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470033+0000","last_change":"2024-05-09T00:08:26.050837+0000","last_active":"2024-05-09T00:09:10.470033+0000","last_peered":"2024-05-09T00:09:10.470033+0000","last_clean":"2024-05-09T00:09:10.470033+0000","last_became_active":"2024-05-09T00:08:26.050424+0000","last_became_peered":"2024-05-09T00:08:26.050424+0000","last_unstale":"2024-05-09T00:09:10.470033+0000","last_undegraded":"2024-05-09T00:09:10.470033+0000","last_fullsized":"2024-05-09T00:09:10.470033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:45.134811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.549989+0000","last_change":"2024-05-09T00:09:10.549989+0000","last_active":"2024-05-09T00:09:10.549989+0000","last_peered":"2024-05-09T00:09:10.549989+0000","last_clean":"2024-05-09T00:09:10.549989+0000","last_became_active":"2024-05-09T00:09:10.549502+0000","last_became_peered":"2024-05-09T00:09:10.549502+0000","last_unstale":"2024-05-09T00:09:10.549989+0000","last_undegraded":"2024-05-09T00:09:10.549989+0000","last_fullsized":"2024-05-09T00:09:10.549989+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:10.654452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473094+0000","last_change":"2024-05-09T00:08:26.058012+0000","last_active":"2024-05-09T00:09:10.473094+0000","last_peered":"2024-05-09T00:09:10.473094+0000","last_clean":"2024-05-09T00:09:10.473094+0000","last_became_active":"2024-05-09T00:08:26.057282+0000","last_became_peered":"2024-05-09T00:08:26.057282+0000","last_unstale":"2024-05-09T00:09:10.473094+0000","last_undegraded":"2024-05-09T00:09:10.473094+0000","last_fullsized":"2024-05-09T00:09:10.473094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:53.520593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471555+0000","last_change":"2024-05-09T00:08:26.063331+0000","last_active":"2024-05-09T00:09:10.471555+0000","last_peered":"2024-05-09T00:09:10.471555+0000","last_clean":"2024-05-09T00:09:10.471555+0000","last_became_active":"2024-05-09T00:08:26.062866+0000","last_became_peered":"2024-05-09T00:08:26.062866+0000","last_unstale":"2024-05-09T00:09:10.471555+0000","last_undegraded":"2024-05-09T00:09:10.471555+0000","last_fullsized":"2024-05-09T00:09:10.471555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:10.387852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548504+0000","last_change":"2024-05-09T00:08:26.063735+0000","last_active":"2024-05-09T00:09:10.548504+0000","last_peered":"2024-05-09T00:09:10.548504+0000","last_clean":"2024-05-09T00:09:10.548504+0000","last_became_active":"2024-05-09T00:08:26.063358+0000","last_became_peered":"2024-05-09T00:08:26.063358+0000","last_unstale":"2024-05-09T00:09:10.548504+0000","last_undegraded":"2024-05-09T00:09:10.548504+0000","last_fullsized":"2024-05-09T00:09:10.548504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:06.832071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.471634+0000","last_change":"2024-05-09T00:08:26.052746+0000","last_active":"2024-05-09T00:09:10.471634+0000","last_peered":"2024-05-09T00:09:10.471634+0000","last_clean":"2024-05-09T00:09:10.471634+0000","last_became_active":"2024-05-09T00:08:26.051544+0000","last_became_peered":"2024-05-09T00:08:26.051544+0000","last_unstale":"2024-05-09T00:09:10.471634+0000","last_undegraded":"2024-05-09T00:09:10.471634+0000","last_fullsized":"2024-05-09T00:09:10.471634+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:31.963268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473534+0000","last_change":"2024-05-09T00:08:26.058075+0000","last_active":"2024-05-09T00:09:10.473534+0000","last_peered":"2024-05-09T00:09:10.473534+0000","last_clean":"2024-05-09T00:09:10.473534+0000","last_became_active":"2024-05-09T00:08:26.057349+0000","last_became_peered":"2024-05-09T00:08:26.057349+0000","last_unstale":"2024-05-09T00:09:10.473534+0000","last_undegraded":"2024-05-09T00:09:10.473534+0000","last_fullsized":"2024-05-09T00:09:10.473534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:59.322903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468103+0000","last_change":"2024-05-09T00:08:26.056429+0000","last_active":"2024-05-09T00:09:10.468103+0000","last_peered":"2024-05-09T00:09:10.468103+0000","last_clean":"2024-05-09T00:09:10.468103+0000","last_became_active":"2024-05-09T00:08:26.054852+0000","last_became_peered":"2024-05-09T00:08:26.054852+0000","last_unstale":"2024-05-09T00:09:10.468103+0000","last_undegraded":"2024-05-09T00:09:10.468103+0000","last_fullsized":"2024-05-09T00:09:10.468103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:05.337636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.549880+0000","last_change":"2024-05-09T00:09:10.549880+0000","last_active":"2024-05-09T00:09:10.549880+0000","last_peered":"2024-05-09T00:09:10.549880+0000","last_clean":"2024-05-09T00:09:10.549880+0000","last_became_active":"2024-05-09T00:09:10.549448+0000","last_became_peered":"2024-05-09T00:09:10.549448+0000","last_unstale":"2024-05-09T00:09:10.549880+0000","last_undegraded":"2024-05-09T00:09:10.549880+0000","last_fullsized":"2024-05-09T00:09:10.549880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:16.990729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467623+0000","last_change":"2024-05-09T00:08:26.054579+0000","last_active":"2024-05-09T00:09:10.467623+0000","last_peered":"2024-05-09T00:09:10.467623+0000","last_clean":"2024-05-09T00:09:10.467623+0000","last_became_active":"2024-05-09T00:08:26.053186+0000","last_became_peered":"2024-05-09T00:08:26.053186+0000","last_unstale":"2024-05-09T00:09:10.467623+0000","last_undegraded":"2024-05-09T00:09:10.467623+0000","last_fullsized":"2024-05-09T00:09:10.467623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:37.561691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468682+0000","last_change":"2024-05-09T00:08:26.063608+0000","last_active":"2024-05-09T00:09:10.468682+0000","last_peered":"2024-05-09T00:09:10.468682+0000","last_clean":"2024-05-09T00:09:10.468682+0000","last_became_active":"2024-05-09T00:08:26.063078+0000","last_became_peered":"2024-05-09T00:08:26.063078+0000","last_unstale":"2024-05-09T00:09:10.468682+0000","last_undegraded":"2024-05-09T00:09:10.468682+0000","last_fullsized":"2024-05-09T00:09:10.468682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:06.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548642+0000","last_change":"2024-05-09T00:08:26.055058+0000","last_active":"2024-05-09T00:09:10.548642+0000","last_peered":"2024-05-09T00:09:10.548642+0000","last_clean":"2024-05-09T00:09:10.548642+0000","last_became_active":"2024-05-09T00:08:26.054695+0000","last_became_peered":"2024-05-09T00:08:26.054695+0000","last_unstale":"2024-05-09T00:09:10.548642+0000","last_undegraded":"2024-05-09T00:09:10.548642+0000","last_fullsized":"2024-05-09T00:09:10.548642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:58.925990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470339+0000","last_change":"2024-05-09T00:08:26.055366+0000","last_active":"2024-05-09T00:09:10.470339+0000","last_peered":"2024-05-09T00:09:10.470339+0000","last_clean":"2024-05-09T00:09:10.470339+0000","last_became_active":"2024-05-09T00:08:26.054904+0000","last_became_peered":"2024-05-09T00:08:26.054904+0000","last_unstale":"2024-05-09T00:09:10.470339+0000","last_undegraded":"2024-05-09T00:09:10.470339+0000","last_fullsized":"2024-05-09T00:09:10.470339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:40.754878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285241,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65827,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":294,"num_read_kb":217,"num_write":165,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":197481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710746,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":425247,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906977,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1078923,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070501,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":425461,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266731,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":425359,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462960,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":1017311,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":868352,"data_stored":424850,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":486469,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":888832,"data_stored":426121,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":487819,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":425161,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986240,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":425678,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822714,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":1016604,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:09:31.248 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T00:09:31.249 INFO:tasks.cephadm:Setting up client nodes... 2024-05-09T00:09:31.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-09T00:09:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:31.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:31 smithi046 ceph-mon[30151]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:31.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:31 smithi169 ceph-mon[34777]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:31.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:31 smithi142 ceph-mon[34533]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:09:32.756 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:32 smithi046 ceph-mon[30151]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:09:32.841 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:32 smithi142 ceph-mon[34533]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:09:32.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:32 smithi169 ceph-mon[34777]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:09:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.335+0000 7f010f949700 1 -- 172.21.15.46:0/2052346635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0108103230 msgr2=0x7f0108103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.335+0000 7f010f949700 1 --2- 172.21.15.46:0/2052346635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0108103230 0x7f0108103640 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0104009990 tx=0x7f0104009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.335+0000 7f010f949700 1 -- 172.21.15.46:0/2052346635 shutdown_connections 2024-05-09T00:09:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.335+0000 7f010f949700 1 --2- 172.21.15.46:0/2052346635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0108104d00 0x7f010810b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.335+0000 7f010f949700 1 --2- 172.21.15.46:0/2052346635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0108103230 0x7f0108103640 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.335+0000 7f010f949700 1 --2- 172.21.15.46:0/2052346635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0108104360 0x7f0108104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:33.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.335+0000 7f010f949700 1 -- 172.21.15.46:0/2052346635 >> 172.21.15.46:0/2052346635 conn(0x7f01080fe870 msgr2=0x7f0108100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:33.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.336+0000 7f010f949700 1 -- 172.21.15.46:0/2052346635 shutdown_connections 2024-05-09T00:09:33.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.336+0000 7f010f949700 1 -- 172.21.15.46:0/2052346635 wait complete. 2024-05-09T00:09:33.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010f949700 1 Processor -- start 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010f949700 1 -- start start 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010f949700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0108103230 0x7f010819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010f949700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0108104360 0x7f010819af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010f949700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0108104d00 0x7f01081a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010f949700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f010810ecf0 con 0x7f0108104360 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010f949700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f010810eb70 con 0x7f0108103230 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010f949700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f010810ee70 con 0x7f0108104d00 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010cee4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0108104360 0x7f010819af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010cee4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0108104360 0x7f010819af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50058/0 (socket says 172.21.15.46:50058) 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010cee4700 1 -- 172.21.15.46:0/736178655 learned_addr learned my addr 172.21.15.46:0/736178655 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010dee6700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0108104d00 0x7f01081a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.337+0000 7f010d6e5700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0108103230 0x7f010819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:33.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.338+0000 7f010cee4700 1 -- 172.21.15.46:0/736178655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0108104d00 msgr2=0x7f01081a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.338+0000 7f010cee4700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0108104d00 0x7f01081a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.338+0000 7f010cee4700 1 -- 172.21.15.46:0/736178655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0108103230 msgr2=0x7f010819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.338+0000 7f010cee4700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0108103230 0x7f010819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.338+0000 7f010cee4700 1 -- 172.21.15.46:0/736178655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01081a2530 con 0x7f0108104360 2024-05-09T00:09:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.338+0000 7f010cee4700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0108104360 0x7f010819af10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f00f800b530 tx=0x7f00f800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.338+0000 7f00fe7fc700 1 -- 172.21.15.46:0/736178655 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8015020 con 0x7f0108104360 2024-05-09T00:09:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.338+0000 7f00fe7fc700 1 -- 172.21.15.46:0/736178655 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f8011ce0 con 0x7f0108104360 2024-05-09T00:09:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.338+0000 7f010f949700 1 -- 172.21.15.46:0/736178655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01081a2820 con 0x7f0108104360 2024-05-09T00:09:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.338+0000 7f00fe7fc700 1 -- 172.21.15.46:0/736178655 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8010dd0 con 0x7f0108104360 2024-05-09T00:09:33.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.338+0000 7f010f949700 1 -- 172.21.15.46:0/736178655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01081a2de0 con 0x7f0108104360 2024-05-09T00:09:33.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.339+0000 7f010f949700 1 -- 172.21.15.46:0/736178655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01080512c0 con 0x7f0108104360 2024-05-09T00:09:33.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.341+0000 7f00fe7fc700 1 -- 172.21.15.46:0/736178655 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f00f8018020 con 0x7f0108104360 2024-05-09T00:09:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.342+0000 7f00fe7fc700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f00f40b2910 0x7f00f40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.342+0000 7f010d6e5700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f00f40b2910 0x7f00f40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.342+0000 7f00fe7fc700 1 -- 172.21.15.46:0/736178655 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f00f80e1f70 con 0x7f0108104360 2024-05-09T00:09:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.342+0000 7f010d6e5700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f00f40b2910 0x7f00f40b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f01040038a0 tx=0x7f010401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.345+0000 7f00fe7fc700 1 -- 172.21.15.46:0/736178655 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00f801f020 con 0x7f0108104360 2024-05-09T00:09:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.651+0000 7f010f949700 1 -- 172.21.15.46:0/736178655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f01081a3210 con 0x7f0108104360 2024-05-09T00:09:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.654+0000 7f00fe7fc700 1 -- 172.21.15.46:0/736178655 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f00f80a9c00 con 0x7f0108104360 2024-05-09T00:09:33.650 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-05-09T00:09:33.651 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQA9FDxmQFniJhAAULoNAWCPXP6gfd/FjnxFaA== 2024-05-09T00:09:33.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.657+0000 7f010f949700 1 -- 172.21.15.46:0/736178655 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f00f40b2910 msgr2=0x7f00f40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:33.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.657+0000 7f010f949700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f00f40b2910 0x7f00f40b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f01040038a0 tx=0x7f010401a040 comp rx=0 tx=0).stop 2024-05-09T00:09:33.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.657+0000 7f010f949700 1 -- 172.21.15.46:0/736178655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0108104360 msgr2=0x7f010819af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:33.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.657+0000 7f010f949700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0108104360 0x7f010819af10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f00f800b530 tx=0x7f00f800b840 comp rx=0 tx=0).stop 2024-05-09T00:09:33.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.657+0000 7f010f949700 1 -- 172.21.15.46:0/736178655 shutdown_connections 2024-05-09T00:09:33.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.657+0000 7f010f949700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f00f40b2910 0x7f00f40b4e20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.657+0000 7f010f949700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0108104d00 0x7f01081a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.657+0000 7f010f949700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0108103230 0x7f010819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.657+0000 7f010f949700 1 --2- 172.21.15.46:0/736178655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0108104360 0x7f010819af10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.657+0000 7f010f949700 1 -- 172.21.15.46:0/736178655 >> 172.21.15.46:0/736178655 conn(0x7f01080fe870 msgr2=0x7f0108100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.657+0000 7f010f949700 1 -- 172.21.15.46:0/736178655 shutdown_connections 2024-05-09T00:09:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:33.657+0000 7f010f949700 1 -- 172.21.15.46:0/736178655 wait complete. 2024-05-09T00:09:34.282 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T00:09:34.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-09T00:09:34.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T00:09:34.324 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T00:09:34.325 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-09T00:09:34.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-09T00:09:34.619 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:34.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:34 smithi046 ceph-mon[30151]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:34.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:34 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/736178655' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T00:09:34.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:34 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/736178655' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T00:09:34.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:34 smithi169 ceph-mon[34777]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:34.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:34 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/736178655' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T00:09:34.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:34 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/736178655' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T00:09:34.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:34 smithi142 ceph-mon[34533]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:34.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:34 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/736178655' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T00:09:34.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:34 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/736178655' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T00:09:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.335+0000 7f705dda1700 1 -- 172.21.15.46:0/3564754600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7058104400 msgr2=0x7f7058104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.335+0000 7f705dda1700 1 --2- 172.21.15.46:0/3564754600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7058104400 0x7f7058104810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f704c0099c0 tx=0x7f704c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.335+0000 7f705dda1700 1 -- 172.21.15.46:0/3564754600 shutdown_connections 2024-05-09T00:09:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.335+0000 7f705dda1700 1 --2- 172.21.15.46:0/3564754600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7058109680 0x7f7058110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.335+0000 7f705dda1700 1 --2- 172.21.15.46:0/3564754600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7058104400 0x7f7058104810 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.335+0000 7f705dda1700 1 --2- 172.21.15.46:0/3564754600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7058108d30 0x7f7058109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.335+0000 7f705dda1700 1 -- 172.21.15.46:0/3564754600 >> 172.21.15.46:0/3564754600 conn(0x7f7058079d10 msgr2=0x7f705807a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.335+0000 7f705dda1700 1 -- 172.21.15.46:0/3564754600 shutdown_connections 2024-05-09T00:09:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.335+0000 7f705dda1700 1 -- 172.21.15.46:0/3564754600 wait complete. 2024-05-09T00:09:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f705dda1700 1 Processor -- start 2024-05-09T00:09:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f705dda1700 1 -- start start 2024-05-09T00:09:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f705dda1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7058104400 0x7f70581a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f705dda1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7058108d30 0x7f70581a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f705dda1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7058109680 0x7f705819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f705dda1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7058114320 con 0x7f7058109680 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f705dda1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f70581141a0 con 0x7f7058104400 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f705dda1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f70581144a0 con 0x7f7058108d30 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f705d5a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7058109680 0x7f705819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f705d5a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7058109680 0x7f705819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54836/0 (socket says 172.21.15.46:54836) 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f705d5a0700 1 -- 172.21.15.46:0/121988894 learned_addr learned my addr 172.21.15.46:0/121988894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f705cd9f700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7058104400 0x7f70581a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.336+0000 7f7057fff700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7058108d30 0x7f70581a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.337+0000 7f705d5a0700 1 -- 172.21.15.46:0/121988894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7058108d30 msgr2=0x7f70581a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.337+0000 7f705d5a0700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7058108d30 0x7f70581a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.337+0000 7f705d5a0700 1 -- 172.21.15.46:0/121988894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7058104400 msgr2=0x7f70581a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.337+0000 7f705d5a0700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7058104400 0x7f70581a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.337+0000 7f705d5a0700 1 -- 172.21.15.46:0/121988894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705819b130 con 0x7f7058109680 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.337+0000 7f705d5a0700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7058109680 0x7f705819aa80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7050007ee0 tx=0x7f705000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.337+0000 7f7055ffb700 1 -- 172.21.15.46:0/121988894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705000dbd0 con 0x7f7058109680 2024-05-09T00:09:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.337+0000 7f7055ffb700 1 -- 172.21.15.46:0/121988894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f705000dd70 con 0x7f7058109680 2024-05-09T00:09:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.337+0000 7f705dda1700 1 -- 172.21.15.46:0/121988894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705819b420 con 0x7f7058109680 2024-05-09T00:09:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.337+0000 7f7055ffb700 1 -- 172.21.15.46:0/121988894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050005990 con 0x7f7058109680 2024-05-09T00:09:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.337+0000 7f705dda1700 1 -- 172.21.15.46:0/121988894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70581a76f0 con 0x7f7058109680 2024-05-09T00:09:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.338+0000 7f705dda1700 1 -- 172.21.15.46:0/121988894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70580512c0 con 0x7f7058109680 2024-05-09T00:09:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.341+0000 7f7055ffb700 1 -- 172.21.15.46:0/121988894 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f705001a020 con 0x7f7058109680 2024-05-09T00:09:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.342+0000 7f7055ffb700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f70480b2910 0x7f70480b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.342+0000 7f705cd9f700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f70480b2910 0x7f70480b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.342+0000 7f7055ffb700 1 -- 172.21.15.46:0/121988894 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f70500e2270 con 0x7f7058109680 2024-05-09T00:09:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.342+0000 7f705cd9f700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f70480b2910 0x7f70480b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f705819bbc0 tx=0x7f704c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.345+0000 7f7055ffb700 1 -- 172.21.15.46:0/121988894 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7050010070 con 0x7f7058109680 2024-05-09T00:09:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.622+0000 7f705dda1700 1 -- 172.21.15.46:0/121988894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f705802e5f0 con 0x7f7058109680 2024-05-09T00:09:36.619 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:36 smithi046 ceph-mon[30151]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:09:36.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.627+0000 7f7055ffb700 1 -- 172.21.15.46:0/121988894 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288641 (secure 0 0 0) 0x7f70500b0e20 con 0x7f7058109680 2024-05-09T00:09:36.628 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:09:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.631+0000 7f705dda1700 1 -- 172.21.15.46:0/121988894 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f70480b2910 msgr2=0x7f70480b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.631+0000 7f705dda1700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f70480b2910 0x7f70480b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f705819bbc0 tx=0x7f704c01a040 comp rx=0 tx=0).stop 2024-05-09T00:09:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.631+0000 7f705dda1700 1 -- 172.21.15.46:0/121988894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7058109680 msgr2=0x7f705819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.631+0000 7f705dda1700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7058109680 0x7f705819aa80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7050007ee0 tx=0x7f705000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:09:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.631+0000 7f705dda1700 1 -- 172.21.15.46:0/121988894 shutdown_connections 2024-05-09T00:09:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.631+0000 7f705dda1700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f70480b2910 0x7f70480b4e20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.631+0000 7f705dda1700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7058109680 0x7f705819aa80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.631+0000 7f705dda1700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7058104400 0x7f70581a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.631+0000 7f705dda1700 1 --2- 172.21.15.46:0/121988894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7058108d30 0x7f70581a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.631+0000 7f705dda1700 1 -- 172.21.15.46:0/121988894 >> 172.21.15.46:0/121988894 conn(0x7f7058079d10 msgr2=0x7f7058102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.631+0000 7f705dda1700 1 -- 172.21.15.46:0/121988894 shutdown_connections 2024-05-09T00:09:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:36.631+0000 7f705dda1700 1 -- 172.21.15.46:0/121988894 wait complete. 2024-05-09T00:09:36.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:36 smithi169 ceph-mon[34777]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:09:36.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:36 smithi142 ceph-mon[34533]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:09:37.389 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":286117730},{"type":"v1","addr":"172.21.15.46:6801","nonce":286117730}]},"active_addr":"172.21.15.46:6801/286117730","active_change":"2024-05-09T00:01:09.191408+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.46:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":3613578525}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":291802359}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":1678263194}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":429688555}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":793214818}]}]} 2024-05-09T00:09:37.395 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-09T00:09:37.395 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-09T00:09:37.395 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:09:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:37.773 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:37 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/121988894' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T00:09:37.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:37 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/121988894' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T00:09:37.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:37 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/121988894' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T00:09:38.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:38 smithi169 ceph-mon[34777]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:38.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:38 smithi142 ceph-mon[34533]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:38.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:38 smithi046 ceph-mon[30151]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:39.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.407+0000 7f9e2391b700 1 -- 172.21.15.46:0/1649809948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1c075b60 msgr2=0x7f9e1c075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.407+0000 7f9e2391b700 1 --2- 172.21.15.46:0/1649809948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1c075b60 0x7f9e1c075f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9e180099c0 tx=0x7f9e18009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.407+0000 7f9e2391b700 1 -- 172.21.15.46:0/1649809948 shutdown_connections 2024-05-09T00:09:39.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.407+0000 7f9e2391b700 1 --2- 172.21.15.46:0/1649809948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e1c1153d0 0x7f9e1c1177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.407+0000 7f9e2391b700 1 --2- 172.21.15.46:0/1649809948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1c075b60 0x7f9e1c075f70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.407+0000 7f9e2391b700 1 --2- 172.21.15.46:0/1649809948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c0764b0 0x7f9e1c114e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.407+0000 7f9e2391b700 1 -- 172.21.15.46:0/1649809948 >> 172.21.15.46:0/1649809948 conn(0x7f9e1c079f00 msgr2=0x7f9e1c07a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.407+0000 7f9e2391b700 1 -- 172.21.15.46:0/1649809948 shutdown_connections 2024-05-09T00:09:39.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.407+0000 7f9e2391b700 1 -- 172.21.15.46:0/1649809948 wait complete. 2024-05-09T00:09:39.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e2391b700 1 Processor -- start 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e2391b700 1 -- start start 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e2391b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c075b60 0x7f9e1c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e2391b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e1c0764b0 0x7f9e1c1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e2391b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1c1153d0 0x7f9e1c1a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e2391b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c11aa90 con 0x7f9e1c1153d0 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e2391b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c11a910 con 0x7f9e1c0764b0 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e2391b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c11ac10 con 0x7f9e1c075b60 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e21eb8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1c1153d0 0x7f9e1c1a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e21eb8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1c1153d0 0x7f9e1c1a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54856/0 (socket says 172.21.15.46:54856) 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e21eb8700 1 -- 172.21.15.46:0/2264459528 learned_addr learned my addr 172.21.15.46:0/2264459528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e20eb6700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e1c0764b0 0x7f9e1c1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.408+0000 7f9e216b7700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c075b60 0x7f9e1c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.409+0000 7f9e21eb8700 1 -- 172.21.15.46:0/2264459528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c075b60 msgr2=0x7f9e1c1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.409+0000 7f9e21eb8700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c075b60 0x7f9e1c1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.409+0000 7f9e21eb8700 1 -- 172.21.15.46:0/2264459528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e1c0764b0 msgr2=0x7f9e1c1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.409+0000 7f9e21eb8700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e1c0764b0 0x7f9e1c1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.409+0000 7f9e21eb8700 1 -- 172.21.15.46:0/2264459528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c1a6290 con 0x7f9e1c1153d0 2024-05-09T00:09:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.409+0000 7f9e216b7700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c075b60 0x7f9e1c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:09:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.409+0000 7f9e21eb8700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1c1153d0 0x7f9e1c1a5bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9e14007ee0 tx=0x7f9e1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.409+0000 7f9e127fc700 1 -- 172.21.15.46:0/2264459528 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1400dbd0 con 0x7f9e1c1153d0 2024-05-09T00:09:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.409+0000 7f9e2391b700 1 -- 172.21.15.46:0/2264459528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c1a6580 con 0x7f9e1c1153d0 2024-05-09T00:09:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.409+0000 7f9e127fc700 1 -- 172.21.15.46:0/2264459528 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e1400dd70 con 0x7f9e1c1153d0 2024-05-09T00:09:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.409+0000 7f9e127fc700 1 -- 172.21.15.46:0/2264459528 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14005990 con 0x7f9e1c1153d0 2024-05-09T00:09:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.409+0000 7f9e2391b700 1 -- 172.21.15.46:0/2264459528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c1adea0 con 0x7f9e1c1153d0 2024-05-09T00:09:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.410+0000 7f9e2391b700 1 -- 172.21.15.46:0/2264459528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1c077250 con 0x7f9e1c1153d0 2024-05-09T00:09:39.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.412+0000 7f9e127fc700 1 -- 172.21.15.46:0/2264459528 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9e1401a020 con 0x7f9e1c1153d0 2024-05-09T00:09:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.413+0000 7f9e127fc700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9e080b2910 0x7f9e080b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.413+0000 7f9e216b7700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9e080b2910 0x7f9e080b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.413+0000 7f9e127fc700 1 -- 172.21.15.46:0/2264459528 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9e140e21e0 con 0x7f9e1c1153d0 2024-05-09T00:09:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.414+0000 7f9e216b7700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9e080b2910 0x7f9e080b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9e18000c00 tx=0x7f9e1801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.417+0000 7f9e127fc700 1 -- 172.21.15.46:0/2264459528 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9e14010070 con 0x7f9e1c1153d0 2024-05-09T00:09:39.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.632+0000 7f9e2391b700 1 -- 172.21.15.46:0/2264459528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e1c02e5f0 con 0x7f9e1c1153d0 2024-05-09T00:09:39.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.634+0000 7f9e127fc700 1 -- 172.21.15.46:0/2264459528 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22694 (secure 0 0 0) 0x7f9e140a9d60 con 0x7f9e1c1153d0 2024-05-09T00:09:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:09:39.631 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","created":"2024-05-09T00:00:14.356524+0000","modified":"2024-05-09T00:09:10.458026+0000","last_up_change":"2024-05-09T00:07:23.332796+0000","last_in_change":"2024-05-09T00:07:03.238539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:03:33.278910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:08:21.014416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:08:24.922968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"740d6388-5c1f-45eb-93b0-7583bfa5a225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6803","nonce":3727342125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6805","nonce":3727342125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6809","nonce":3727342125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6807","nonce":3727342125}]},"public_addr":"172.21.15.46:6803/3727342125","cluster_addr":"172.21.15.46:6805/3727342125","heartbeat_back_addr":"172.21.15.46:6809/3727342125","heartbeat_front_addr":"172.21.15.46:6807/3727342125","state":["exists","up"]},{"osd":1,"uuid":"85fbf3ae-0be8-4504-84d6-66fae0d2169d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6801","nonce":3156042469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6803","nonce":3156042469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6807","nonce":3156042469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6805","nonce":3156042469}]},"public_addr":"172.21.15.142:6801/3156042469","cluster_addr":"172.21.15.142:6803/3156042469","heartbeat_back_addr":"172.21.15.142:6807/3156042469","heartbeat_front_addr":"172.21.15.142:6805/3156042469","state":["exists","up"]},{"osd":2,"uuid":"05c070e7-83ad-4949-9834-4ce51ac0a2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6801","nonce":2680673064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6803","nonce":2680673064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6807","nonce":2680673064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6805","nonce":2680673064}]},"public_addr":"172.21.15.169:6801/2680673064","cluster_addr":"172.21.15.169:6803/2680673064","heartbeat_back_addr":"172.21.15.169:6807/2680673064","heartbeat_front_addr":"172.21.15.169:6805/2680673064","state":["exists","up"]},{"osd":3,"uuid":"09d7bbe1-58b6-4407-bef2-920f179dfb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6811","nonce":1158536261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6813","nonce":1158536261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6817","nonce":1158536261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6815","nonce":1158536261}]},"public_addr":"172.21.15.46:6811/1158536261","cluster_addr":"172.21.15.46:6813/1158536261","heartbeat_back_addr":"172.21.15.46:6817/1158536261","heartbeat_front_addr":"172.21.15.46:6815/1158536261","state":["exists","up"]},{"osd":4,"uuid":"66c9644b-6b0b-4fe7-aa63-b9d87bff7153","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6809","nonce":2299616865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6811","nonce":2299616865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6815","nonce":2299616865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6813","nonce":2299616865}]},"public_addr":"172.21.15.142:6809/2299616865","cluster_addr":"172.21.15.142:6811/2299616865","heartbeat_back_addr":"172.21.15.142:6815/2299616865","heartbeat_front_addr":"172.21.15.142:6813/2299616865","state":["exists","up"]},{"osd":5,"uuid":"96a72e01-1c5f-418e-bb9b-b790d9b3059b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6809","nonce":445044713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6811","nonce":445044713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6815","nonce":445044713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6813","nonce":445044713}]},"public_addr":"172.21.15.169:6809/445044713","cluster_addr":"172.21.15.169:6811/445044713","heartbeat_back_addr":"172.21.15.169:6815/445044713","heartbeat_front_addr":"172.21.15.169:6813/445044713","state":["exists","up"]},{"osd":6,"uuid":"6b459542-0e56-4369-b542-e36c46a18f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6819","nonce":1394035482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6821","nonce":1394035482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6825","nonce":1394035482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6823","nonce":1394035482}]},"public_addr":"172.21.15.46:6819/1394035482","cluster_addr":"172.21.15.46:6821/1394035482","heartbeat_back_addr":"172.21.15.46:6825/1394035482","heartbeat_front_addr":"172.21.15.46:6823/1394035482","state":["exists","up"]},{"osd":7,"uuid":"786b4517-4cbd-4fa3-b5a0-39cccf839893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6817","nonce":596805607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6819","nonce":596805607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6823","nonce":596805607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6821","nonce":596805607}]},"public_addr":"172.21.15.142:6817/596805607","cluster_addr":"172.21.15.142:6819/596805607","heartbeat_back_addr":"172.21.15.142:6823/596805607","heartbeat_front_addr":"172.21.15.142:6821/596805607","state":["exists","up"]},{"osd":8,"uuid":"b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6817","nonce":2650652376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6819","nonce":2650652376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6823","nonce":2650652376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6821","nonce":2650652376}]},"public_addr":"172.21.15.169:6817/2650652376","cluster_addr":"172.21.15.169:6819/2650652376","heartbeat_back_addr":"172.21.15.169:6823/2650652376","heartbeat_front_addr":"172.21.15.169:6821/2650652376","state":["exists","up"]},{"osd":9,"uuid":"32df0dab-5875-4beb-b733-60a915c94a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6827","nonce":374861248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6829","nonce":374861248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6833","nonce":374861248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6831","nonce":374861248}]},"public_addr":"172.21.15.46:6827/374861248","cluster_addr":"172.21.15.46:6829/374861248","heartbeat_back_addr":"172.21.15.46:6833/374861248","heartbeat_front_addr":"172.21.15.46:6831/374861248","state":["exists","up"]},{"osd":10,"uuid":"380a4f47-e528-4258-8836-32eac0e6e630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6825","nonce":1364891982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6827","nonce":1364891982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6831","nonce":1364891982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6829","nonce":1364891982}]},"public_addr":"172.21.15.142:6825/1364891982","cluster_addr":"172.21.15.142:6827/1364891982","heartbeat_back_addr":"172.21.15.142:6831/1364891982","heartbeat_front_addr":"172.21.15.142:6829/1364891982","state":["exists","up"]},{"osd":11,"uuid":"56b04b15-6fcd-497a-aa66-19da9624b656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6825","nonce":4262478976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6827","nonce":4262478976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6831","nonce":4262478976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6829","nonce":4262478976}]},"public_addr":"172.21.15.169:6825/4262478976","cluster_addr":"172.21.15.169:6827/4262478976","heartbeat_back_addr":"172.21.15.169:6831/4262478976","heartbeat_front_addr":"172.21.15.169:6829/4262478976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:02:42.159298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:06.053328+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:29.587137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:54.573782+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:20.276073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:45.320565+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:10.985522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:36.806137+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:02.266736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:29.788773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:55.827639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:07:21.762382+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/1072138393":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2786891364":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/7723180":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/626888490":"2024-05-10T00:01:09.191239+0000","172.21.15.46:6800/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/4236543921":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2168313126":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6800/953818590":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6801/953818590":"2024-05-10T00:00:41.833323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:09:39.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.636+0000 7f9e2391b700 1 -- 172.21.15.46:0/2264459528 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9e080b2910 msgr2=0x7f9e080b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.636+0000 7f9e2391b700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9e080b2910 0x7f9e080b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9e18000c00 tx=0x7f9e1801a040 comp rx=0 tx=0).stop 2024-05-09T00:09:39.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.636+0000 7f9e2391b700 1 -- 172.21.15.46:0/2264459528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1c1153d0 msgr2=0x7f9e1c1a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:39.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.636+0000 7f9e2391b700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1c1153d0 0x7f9e1c1a5bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9e14007ee0 tx=0x7f9e1400d3b0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.636+0000 7f9e2391b700 1 -- 172.21.15.46:0/2264459528 shutdown_connections 2024-05-09T00:09:39.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.636+0000 7f9e2391b700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9e080b2910 0x7f9e080b4e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.636+0000 7f9e2391b700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1c1153d0 0x7f9e1c1a5bf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.636+0000 7f9e2391b700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e1c075b60 0x7f9e1c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.636+0000 7f9e2391b700 1 --2- 172.21.15.46:0/2264459528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e1c0764b0 0x7f9e1c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:39.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.636+0000 7f9e2391b700 1 -- 172.21.15.46:0/2264459528 >> 172.21.15.46:0/2264459528 conn(0x7f9e1c079f00 msgr2=0x7f9e1c1020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:39.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.636+0000 7f9e2391b700 1 -- 172.21.15.46:0/2264459528 shutdown_connections 2024-05-09T00:09:39.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:39.636+0000 7f9e2391b700 1 -- 172.21.15.46:0/2264459528 wait complete. 2024-05-09T00:09:40.325 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-09T00:09:40.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:09:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:40.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:40 smithi169 ceph-mon[34777]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:09:40.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:40 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2264459528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:09:40.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:40 smithi046 ceph-mon[30151]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:09:40.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:40 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2264459528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:09:40.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:40 smithi142 ceph-mon[34533]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:09:40.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:40 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2264459528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:09:42.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.360+0000 7f9600ae7700 1 -- 172.21.15.46:0/1237406742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95fc103310 msgr2=0x7f95fc10f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.360+0000 7f9600ae7700 1 --2- 172.21.15.46:0/1237406742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95fc103310 0x7f95fc10f890 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009990 tx=0x7f95f4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.360+0000 7f9600ae7700 1 -- 172.21.15.46:0/1237406742 shutdown_connections 2024-05-09T00:09:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.360+0000 7f9600ae7700 1 --2- 172.21.15.46:0/1237406742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95fc103310 0x7f95fc10f890 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.360+0000 7f9600ae7700 1 --2- 172.21.15.46:0/1237406742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95fc1089f0 0x7f95fc108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.360+0000 7f9600ae7700 1 --2- 172.21.15.46:0/1237406742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95fc1029c0 0x7f95fc102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.360+0000 7f9600ae7700 1 -- 172.21.15.46:0/1237406742 >> 172.21.15.46:0/1237406742 conn(0x7f95fc0fe4e0 msgr2=0x7f95fc100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.360+0000 7f9600ae7700 1 -- 172.21.15.46:0/1237406742 shutdown_connections 2024-05-09T00:09:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.360+0000 7f9600ae7700 1 -- 172.21.15.46:0/1237406742 wait complete. 2024-05-09T00:09:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.361+0000 7f9600ae7700 1 Processor -- start 2024-05-09T00:09:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.361+0000 7f9600ae7700 1 -- start start 2024-05-09T00:09:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.361+0000 7f9600ae7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95fc1029c0 0x7f95fc0fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.361+0000 7f9600ae7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95fc103310 0x7f95fc0fef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.361+0000 7f9600ae7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95fc1089f0 0x7f95fc0ff440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.361+0000 7f9600ae7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc1122e0 con 0x7f95fc1029c0 2024-05-09T00:09:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.361+0000 7f9600ae7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc112160 con 0x7f95fc103310 2024-05-09T00:09:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.361+0000 7f9600ae7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc112460 con 0x7f95fc1089f0 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.361+0000 7f95fa59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95fc1029c0 0x7f95fc0fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.361+0000 7f95fa59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95fc1029c0 0x7f95fc0fe9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54876/0 (socket says 172.21.15.46:54876) 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.361+0000 7f95fa59c700 1 -- 172.21.15.46:0/1424979556 learned_addr learned my addr 172.21.15.46:0/1424979556 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.361+0000 7f95f9d9b700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95fc103310 0x7f95fc0fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.362+0000 7f95fad9d700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95fc1089f0 0x7f95fc0ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.362+0000 7f95fa59c700 1 -- 172.21.15.46:0/1424979556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95fc1089f0 msgr2=0x7f95fc0ff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.362+0000 7f95fa59c700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95fc1089f0 0x7f95fc0ff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.362+0000 7f95fa59c700 1 -- 172.21.15.46:0/1424979556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95fc103310 msgr2=0x7f95fc0fef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.362+0000 7f95fa59c700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95fc103310 0x7f95fc0fef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.362+0000 7f95fa59c700 1 -- 172.21.15.46:0/1424979556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc1a9120 con 0x7f95fc1029c0 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.362+0000 7f95fa59c700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95fc1029c0 0x7f95fc0fe9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00d3a0 tx=0x7f95ec00d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.362+0000 7f95eb7fe700 1 -- 172.21.15.46:0/1424979556 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00a2d0 con 0x7f95fc1029c0 2024-05-09T00:09:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.362+0000 7f9600ae7700 1 -- 172.21.15.46:0/1424979556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95fc1a93b0 con 0x7f95fc1029c0 2024-05-09T00:09:42.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.362+0000 7f95eb7fe700 1 -- 172.21.15.46:0/1424979556 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95ec018070 con 0x7f95fc1029c0 2024-05-09T00:09:42.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.362+0000 7f95eb7fe700 1 -- 172.21.15.46:0/1424979556 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec013800 con 0x7f95fc1029c0 2024-05-09T00:09:42.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.362+0000 7f9600ae7700 1 -- 172.21.15.46:0/1424979556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95fc1a9940 con 0x7f95fc1029c0 2024-05-09T00:09:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.363+0000 7f9600ae7700 1 -- 172.21.15.46:0/1424979556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95fc0512c0 con 0x7f95fc1029c0 2024-05-09T00:09:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.365+0000 7f95eb7fe700 1 -- 172.21.15.46:0/1424979556 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f95ec012070 con 0x7f95fc1029c0 2024-05-09T00:09:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.365+0000 7f95eb7fe700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95e40b2910 0x7f95e40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.366+0000 7f95f9d9b700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95e40b2910 0x7f95e40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.366+0000 7f95eb7fe700 1 -- 172.21.15.46:0/1424979556 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f95ec0e22c0 con 0x7f95fc1029c0 2024-05-09T00:09:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.366+0000 7f95f9d9b700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95e40b2910 0x7f95e40b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f95fc106710 tx=0x7f95f000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:42.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.371+0000 7f95eb7fe700 1 -- 172.21.15.46:0/1424979556 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f95ec0a9e80 con 0x7f95fc1029c0 2024-05-09T00:09:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.586+0000 7f9600ae7700 1 -- 172.21.15.46:0/1424979556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95fc02e5f0 con 0x7f95fc1029c0 2024-05-09T00:09:42.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.587+0000 7f95eb7fe700 1 -- 172.21.15.46:0/1424979556 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22694 (secure 0 0 0) 0x7f95ec0a9780 con 0x7f95fc1029c0 2024-05-09T00:09:42.585 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:09:42.585 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","created":"2024-05-09T00:00:14.356524+0000","modified":"2024-05-09T00:09:10.458026+0000","last_up_change":"2024-05-09T00:07:23.332796+0000","last_in_change":"2024-05-09T00:07:03.238539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:03:33.278910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:08:21.014416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:08:24.922968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"740d6388-5c1f-45eb-93b0-7583bfa5a225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6803","nonce":3727342125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6805","nonce":3727342125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6809","nonce":3727342125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6807","nonce":3727342125}]},"public_addr":"172.21.15.46:6803/3727342125","cluster_addr":"172.21.15.46:6805/3727342125","heartbeat_back_addr":"172.21.15.46:6809/3727342125","heartbeat_front_addr":"172.21.15.46:6807/3727342125","state":["exists","up"]},{"osd":1,"uuid":"85fbf3ae-0be8-4504-84d6-66fae0d2169d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6801","nonce":3156042469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6803","nonce":3156042469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6807","nonce":3156042469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6805","nonce":3156042469}]},"public_addr":"172.21.15.142:6801/3156042469","cluster_addr":"172.21.15.142:6803/3156042469","heartbeat_back_addr":"172.21.15.142:6807/3156042469","heartbeat_front_addr":"172.21.15.142:6805/3156042469","state":["exists","up"]},{"osd":2,"uuid":"05c070e7-83ad-4949-9834-4ce51ac0a2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6801","nonce":2680673064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6803","nonce":2680673064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6807","nonce":2680673064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6805","nonce":2680673064}]},"public_addr":"172.21.15.169:6801/2680673064","cluster_addr":"172.21.15.169:6803/2680673064","heartbeat_back_addr":"172.21.15.169:6807/2680673064","heartbeat_front_addr":"172.21.15.169:6805/2680673064","state":["exists","up"]},{"osd":3,"uuid":"09d7bbe1-58b6-4407-bef2-920f179dfb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6811","nonce":1158536261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6813","nonce":1158536261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6817","nonce":1158536261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6815","nonce":1158536261}]},"public_addr":"172.21.15.46:6811/1158536261","cluster_addr":"172.21.15.46:6813/1158536261","heartbeat_back_addr":"172.21.15.46:6817/1158536261","heartbeat_front_addr":"172.21.15.46:6815/1158536261","state":["exists","up"]},{"osd":4,"uuid":"66c9644b-6b0b-4fe7-aa63-b9d87bff7153","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6809","nonce":2299616865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6811","nonce":2299616865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6815","nonce":2299616865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6813","nonce":2299616865}]},"public_addr":"172.21.15.142:6809/2299616865","cluster_addr":"172.21.15.142:6811/2299616865","heartbeat_back_addr":"172.21.15.142:6815/2299616865","heartbeat_front_addr":"172.21.15.142:6813/2299616865","state":["exists","up"]},{"osd":5,"uuid":"96a72e01-1c5f-418e-bb9b-b790d9b3059b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6809","nonce":445044713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6811","nonce":445044713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6815","nonce":445044713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6813","nonce":445044713}]},"public_addr":"172.21.15.169:6809/445044713","cluster_addr":"172.21.15.169:6811/445044713","heartbeat_back_addr":"172.21.15.169:6815/445044713","heartbeat_front_addr":"172.21.15.169:6813/445044713","state":["exists","up"]},{"osd":6,"uuid":"6b459542-0e56-4369-b542-e36c46a18f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6819","nonce":1394035482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6821","nonce":1394035482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6825","nonce":1394035482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6823","nonce":1394035482}]},"public_addr":"172.21.15.46:6819/1394035482","cluster_addr":"172.21.15.46:6821/1394035482","heartbeat_back_addr":"172.21.15.46:6825/1394035482","heartbeat_front_addr":"172.21.15.46:6823/1394035482","state":["exists","up"]},{"osd":7,"uuid":"786b4517-4cbd-4fa3-b5a0-39cccf839893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6817","nonce":596805607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6819","nonce":596805607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6823","nonce":596805607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6821","nonce":596805607}]},"public_addr":"172.21.15.142:6817/596805607","cluster_addr":"172.21.15.142:6819/596805607","heartbeat_back_addr":"172.21.15.142:6823/596805607","heartbeat_front_addr":"172.21.15.142:6821/596805607","state":["exists","up"]},{"osd":8,"uuid":"b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6817","nonce":2650652376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6819","nonce":2650652376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6823","nonce":2650652376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6821","nonce":2650652376}]},"public_addr":"172.21.15.169:6817/2650652376","cluster_addr":"172.21.15.169:6819/2650652376","heartbeat_back_addr":"172.21.15.169:6823/2650652376","heartbeat_front_addr":"172.21.15.169:6821/2650652376","state":["exists","up"]},{"osd":9,"uuid":"32df0dab-5875-4beb-b733-60a915c94a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6827","nonce":374861248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6829","nonce":374861248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6833","nonce":374861248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6831","nonce":374861248}]},"public_addr":"172.21.15.46:6827/374861248","cluster_addr":"172.21.15.46:6829/374861248","heartbeat_back_addr":"172.21.15.46:6833/374861248","heartbeat_front_addr":"172.21.15.46:6831/374861248","state":["exists","up"]},{"osd":10,"uuid":"380a4f47-e528-4258-8836-32eac0e6e630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6825","nonce":1364891982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6827","nonce":1364891982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6831","nonce":1364891982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6829","nonce":1364891982}]},"public_addr":"172.21.15.142:6825/1364891982","cluster_addr":"172.21.15.142:6827/1364891982","heartbeat_back_addr":"172.21.15.142:6831/1364891982","heartbeat_front_addr":"172.21.15.142:6829/1364891982","state":["exists","up"]},{"osd":11,"uuid":"56b04b15-6fcd-497a-aa66-19da9624b656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6825","nonce":4262478976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6827","nonce":4262478976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6831","nonce":4262478976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6829","nonce":4262478976}]},"public_addr":"172.21.15.169:6825/4262478976","cluster_addr":"172.21.15.169:6827/4262478976","heartbeat_back_addr":"172.21.15.169:6831/4262478976","heartbeat_front_addr":"172.21.15.169:6829/4262478976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:02:42.159298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:06.053328+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:29.587137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:54.573782+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:20.276073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:45.320565+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:10.985522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:36.806137+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:02.266736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:29.788773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:55.827639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:07:21.762382+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/1072138393":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2786891364":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/7723180":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/626888490":"2024-05-10T00:01:09.191239+0000","172.21.15.46:6800/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/4236543921":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2168313126":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6800/953818590":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6801/953818590":"2024-05-10T00:00:41.833323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:09:42.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.590+0000 7f9600ae7700 1 -- 172.21.15.46:0/1424979556 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95e40b2910 msgr2=0x7f95e40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:42.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.590+0000 7f9600ae7700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95e40b2910 0x7f95e40b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f95fc106710 tx=0x7f95f000a380 comp rx=0 tx=0).stop 2024-05-09T00:09:42.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.590+0000 7f9600ae7700 1 -- 172.21.15.46:0/1424979556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95fc1029c0 msgr2=0x7f95fc0fe9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:42.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.590+0000 7f9600ae7700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95fc1029c0 0x7f95fc0fe9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00d3a0 tx=0x7f95ec00d6b0 comp rx=0 tx=0).stop 2024-05-09T00:09:42.587 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:42 smithi046 ceph-mon[30151]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:09:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.590+0000 7f9600ae7700 1 -- 172.21.15.46:0/1424979556 shutdown_connections 2024-05-09T00:09:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.590+0000 7f9600ae7700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95e40b2910 0x7f95e40b4e20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.590+0000 7f9600ae7700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95fc1089f0 0x7f95fc0ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.590+0000 7f9600ae7700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95fc1029c0 0x7f95fc0fe9c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.590+0000 7f9600ae7700 1 --2- 172.21.15.46:0/1424979556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95fc103310 0x7f95fc0fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.590+0000 7f9600ae7700 1 -- 172.21.15.46:0/1424979556 >> 172.21.15.46:0/1424979556 conn(0x7f95fc0fe4e0 msgr2=0x7f95fc105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.590+0000 7f9600ae7700 1 -- 172.21.15.46:0/1424979556 shutdown_connections 2024-05-09T00:09:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:42.590+0000 7f9600ae7700 1 -- 172.21.15.46:0/1424979556 wait complete. 2024-05-09T00:09:42.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:42 smithi169 ceph-mon[34777]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:09:42.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:42 smithi142 ceph-mon[34533]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:09:43.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-09T00:09:43.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-09T00:09:43.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-09T00:09:43.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-09T00:09:43.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-09T00:09:43.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-09T00:09:43.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-09T00:09:43.250 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-09T00:09:43.250 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-09T00:09:43.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-09T00:09:43.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-09T00:09:43.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-09T00:09:43.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:43 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1424979556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:09:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:43.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:43 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1424979556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:09:43.941 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:43.944 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:43.949 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:44.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:43 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1424979556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:09:44.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:44 smithi046 ceph-mon[30151]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:44.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:44 smithi169 ceph-mon[34777]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:44.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:44 smithi142 ceph-mon[34533]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:46.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:46 smithi046 ceph-mon[30151]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:09:46.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:46 smithi169 ceph-mon[34777]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:09:46.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:46 smithi142 ceph-mon[34533]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:09:48.725 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:48 smithi046 ceph-mon[30151]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:48.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:48 smithi169 ceph-mon[34777]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:48.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:48 smithi142 ceph-mon[34533]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:09:50.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:50 smithi046 ceph-mon[30151]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:09:50.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:50 smithi169 ceph-mon[34777]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:09:50.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:50 smithi142 ceph-mon[34533]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:09:52.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.462+0000 7f1186865700 1 -- 172.21.15.46:0/261949421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1180103550 msgr2=0x7f1180103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.462+0000 7f1186865700 1 --2- 172.21.15.46:0/261949421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1180103550 0x7f1180103960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f11700099c0 tx=0x7f1170009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.462+0000 7f1186865700 1 -- 172.21.15.46:0/261949421 shutdown_connections 2024-05-09T00:09:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.463+0000 7f1186865700 1 --2- 172.21.15.46:0/261949421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1180108780 0x7f11801100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.463+0000 7f1186865700 1 --2- 172.21.15.46:0/261949421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1180103550 0x7f1180103960 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.463+0000 7f1186865700 1 --2- 172.21.15.46:0/261949421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1180107e30 0x7f1180108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.463+0000 7f1186865700 1 -- 172.21.15.46:0/261949421 >> 172.21.15.46:0/261949421 conn(0x7f1180078e00 msgr2=0x7f1180079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.463+0000 7f1186865700 1 -- 172.21.15.46:0/261949421 shutdown_connections 2024-05-09T00:09:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.463+0000 7f1186865700 1 -- 172.21.15.46:0/261949421 wait complete. 2024-05-09T00:09:52.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.463+0000 7f1186865700 1 Processor -- start 2024-05-09T00:09:52.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.463+0000 7f1186865700 1 -- start start 2024-05-09T00:09:52.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f1186865700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1180103550 0x7f118019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:52.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f1186865700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1180107e30 0x7f118019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:52.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f1186865700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1180108780 0x7f118019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:52.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f1186865700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f11801133b0 con 0x7f1180103550 2024-05-09T00:09:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f1186865700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1180113230 con 0x7f1180107e30 2024-05-09T00:09:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f1186865700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1180113530 con 0x7f1180108780 2024-05-09T00:09:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f117ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1180103550 0x7f118019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f117ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1180103550 0x7f118019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53014/0 (socket says 172.21.15.46:53014) 2024-05-09T00:09:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f117ffff700 1 -- 172.21.15.46:0/2308615570 learned_addr learned my addr 172.21.15.46:0/2308615570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f1177fff700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1180107e30 0x7f118019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f1184e02700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1180108780 0x7f118019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f117ffff700 1 -- 172.21.15.46:0/2308615570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1180108780 msgr2=0x7f118019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f117ffff700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1180108780 0x7f118019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f117ffff700 1 -- 172.21.15.46:0/2308615570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1180107e30 msgr2=0x7f118019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f117ffff700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1180107e30 0x7f118019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f117ffff700 1 -- 172.21.15.46:0/2308615570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11801a6ff0 con 0x7f1180103550 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.464+0000 7f117ffff700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1180103550 0x7f118019bd10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f11700098e0 tx=0x7f11700057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.465+0000 7f117dffb700 1 -- 172.21.15.46:0/2308615570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170027070 con 0x7f1180103550 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.465+0000 7f117dffb700 1 -- 172.21.15.46:0/2308615570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1170003f20 con 0x7f1180103550 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.465+0000 7f1186865700 1 -- 172.21.15.46:0/2308615570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11801a7280 con 0x7f1180103550 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.465+0000 7f117dffb700 1 -- 172.21.15.46:0/2308615570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170022d70 con 0x7f1180103550 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.465+0000 7f1186865700 1 -- 172.21.15.46:0/2308615570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11801a7730 con 0x7f1180103550 2024-05-09T00:09:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.465+0000 7f1186865700 1 -- 172.21.15.46:0/2308615570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1180063ed0 con 0x7f1180103550 2024-05-09T00:09:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.467+0000 7f117dffb700 1 -- 172.21.15.46:0/2308615570 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f117002a070 con 0x7f1180103550 2024-05-09T00:09:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.468+0000 7f117dffb700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f116c0b2990 0x7f116c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.468+0000 7f1177fff700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f116c0b2990 0x7f116c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.468+0000 7f117dffb700 1 -- 172.21.15.46:0/2308615570 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f11700e3ec0 con 0x7f1180103550 2024-05-09T00:09:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.468+0000 7f117dffb700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469] conn(0x7f116c0bc5a0 0x7f116c0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.469+0000 7f117dffb700 1 -- 172.21.15.46:0/2308615570 --> [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f117000fe50 con 0x7f116c0bc5a0 2024-05-09T00:09:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.469+0000 7f117dffb700 1 -- 172.21.15.46:0/2308615570 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f11700e42f0 con 0x7f1180103550 2024-05-09T00:09:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.469+0000 7f1177fff700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f116c0b2990 0x7f116c0b4ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1180079470 tx=0x7f116800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.469+0000 7f1184e02700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469] conn(0x7f116c0bc5a0 0x7f116c0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.469+0000 7f1184e02700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469] conn(0x7f116c0bc5a0 0x7f116c0bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.470+0000 7f117dffb700 1 -- 172.21.15.46:0/2308615570 <== osd.1 v2:172.21.15.142:6800/3156042469 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f117000fe50 con 0x7f116c0bc5a0 2024-05-09T00:09:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.487+0000 7f1186865700 1 -- 172.21.15.46:0/2308615570 --> [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1180064300 con 0x7f116c0bc5a0 2024-05-09T00:09:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.487+0000 7f117dffb700 1 -- 172.21.15.46:0/2308615570 <== osd.1 v2:172.21.15.142:6800/3156042469 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1180064300 con 0x7f116c0bc5a0 2024-05-09T00:09:52.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.488+0000 7f1186865700 1 -- 172.21.15.46:0/2308615570 >> [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469] conn(0x7f116c0bc5a0 msgr2=0x7f116c0bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:52.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.488+0000 7f1186865700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469] conn(0x7f116c0bc5a0 0x7f116c0bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:52.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.488+0000 7f1186865700 1 -- 172.21.15.46:0/2308615570 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f116c0b2990 msgr2=0x7f116c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:52.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.488+0000 7f1186865700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f116c0b2990 0x7f116c0b4ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1180079470 tx=0x7f116800a300 comp rx=0 tx=0).stop 2024-05-09T00:09:52.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.488+0000 7f1186865700 1 -- 172.21.15.46:0/2308615570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1180103550 msgr2=0x7f118019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:52.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.488+0000 7f1186865700 1 --2- 172.21.15.46:0/2308615570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1180103550 0x7f118019bd10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f11700098e0 tx=0x7f11700057c0 comp rx=0 tx=0).stop 2024-05-09T00:09:52.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.488+0000 7f1184e02700 1 -- 172.21.15.46:0/2308615570 reap_dead start 2024-05-09T00:09:52.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.488+0000 7f1186865700 1 -- 172.21.15.46:0/2308615570 shutdown_connections 2024-05-09T00:09:52.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.488+0000 7f1186865700 1 -- 172.21.15.46:0/2308615570 >> 172.21.15.46:0/2308615570 conn(0x7f1180078e00 msgr2=0x7f1180101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:52.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.488+0000 7f1186865700 1 -- 172.21.15.46:0/2308615570 shutdown_connections 2024-05-09T00:09:52.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:52.488+0000 7f1186865700 1 -- 172.21.15.46:0/2308615570 wait complete. 2024-05-09T00:09:52.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:52 smithi046 ceph-mon[30151]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:09:52.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:52 smithi169 ceph-mon[34777]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:09:52.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:52 smithi142 ceph-mon[34533]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:09:54.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.722+0000 7f02611c3700 1 -- 172.21.15.46:0/1714643912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c1034c0 msgr2=0x7f025c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.722+0000 7f02611c3700 1 --2- 172.21.15.46:0/1714643912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c1034c0 0x7f025c1038d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f024c0099c0 tx=0x7f024c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.722+0000 7f02611c3700 1 -- 172.21.15.46:0/1714643912 shutdown_connections 2024-05-09T00:09:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.722+0000 7f02611c3700 1 --2- 172.21.15.46:0/1714643912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f025c108740 0x7f025c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.722+0000 7f02611c3700 1 --2- 172.21.15.46:0/1714643912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c1034c0 0x7f025c1038d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.722+0000 7f02611c3700 1 --2- 172.21.15.46:0/1714643912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c107df0 0x7f025c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.722+0000 7f02611c3700 1 -- 172.21.15.46:0/1714643912 >> 172.21.15.46:0/1714643912 conn(0x7f025c0fedc0 msgr2=0x7f025c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.722+0000 7f02611c3700 1 -- 172.21.15.46:0/1714643912 shutdown_connections 2024-05-09T00:09:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.722+0000 7f02611c3700 1 -- 172.21.15.46:0/1714643912 wait complete. 2024-05-09T00:09:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.723+0000 7f02611c3700 1 Processor -- start 2024-05-09T00:09:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.723+0000 7f02611c3700 1 -- start start 2024-05-09T00:09:54.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f02611c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c1034c0 0x7f025c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f02611c3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f025c107df0 0x7f025c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f02611c3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c108740 0x7f025c1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f02611c3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f025c113370 con 0x7f025c1034c0 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f02611c3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1131f0 con 0x7f025c107df0 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f02611c3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1134f0 con 0x7f025c108740 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f025ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c1034c0 0x7f025c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f025a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f025c107df0 0x7f025c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f025b59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c108740 0x7f025c1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f025a59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f025c107df0 0x7f025c19d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:38208/0 (socket says 172.21.15.46:38208) 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f025a59c700 1 -- 172.21.15.46:0/2287717672 learned_addr learned my addr 172.21.15.46:0/2287717672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f025ad9d700 1 -- 172.21.15.46:0/2287717672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c108740 msgr2=0x7f025c1a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f025ad9d700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f025c108740 0x7f025c1a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f025ad9d700 1 -- 172.21.15.46:0/2287717672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f025c107df0 msgr2=0x7f025c19d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f025ad9d700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f025c107df0 0x7f025c19d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f025ad9d700 1 -- 172.21.15.46:0/2287717672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c1a1ea0 con 0x7f025c1034c0 2024-05-09T00:09:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.724+0000 7f025a59c700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f025c107df0 0x7f025c19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:09:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.725+0000 7f025ad9d700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c1034c0 0x7f025c19cfd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f024c0098e0 tx=0x7f024c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.725+0000 7f0243fff700 1 -- 172.21.15.46:0/2287717672 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c027070 con 0x7f025c1034c0 2024-05-09T00:09:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.725+0000 7f02611c3700 1 -- 172.21.15.46:0/2287717672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c1a2130 con 0x7f025c1034c0 2024-05-09T00:09:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.725+0000 7f0243fff700 1 -- 172.21.15.46:0/2287717672 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f024c003740 con 0x7f025c1034c0 2024-05-09T00:09:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.725+0000 7f0243fff700 1 -- 172.21.15.46:0/2287717672 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c022b20 con 0x7f025c1034c0 2024-05-09T00:09:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.725+0000 7f02611c3700 1 -- 172.21.15.46:0/2287717672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c1a9ab0 con 0x7f025c1034c0 2024-05-09T00:09:54.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.726+0000 7f02611c3700 1 -- 172.21.15.46:0/2287717672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f025c1a2660 con 0x7f025c1034c0 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.729+0000 7f0243fff700 1 -- 172.21.15.46:0/2287717672 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f024c02a070 con 0x7f025c1034c0 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.730+0000 7f0243fff700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f02440b2910 0x7f02440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.730+0000 7f025a59c700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f02440b2910 0x7f02440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.730+0000 7f0243fff700 1 -- 172.21.15.46:0/2287717672 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f024c0e3a20 con 0x7f025c1034c0 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.730+0000 7f0243fff700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125] conn(0x7f02440bc520 0x7f02440bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.730+0000 7f0243fff700 1 -- 172.21.15.46:0/2287717672 --> [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f024c00b0b0 con 0x7f02440bc520 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.730+0000 7f0243fff700 1 -- 172.21.15.46:0/2287717672 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f024c0e3e50 con 0x7f025c1034c0 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.730+0000 7f025b59e700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125] conn(0x7f02440bc520 0x7f02440bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.730+0000 7f025a59c700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f02440b2910 0x7f02440b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f025c1027c0 tx=0x7f025000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.731+0000 7f025b59e700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125] conn(0x7f02440bc520 0x7f02440bea10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.732+0000 7f0243fff700 1 -- 172.21.15.46:0/2287717672 <== osd.0 v2:172.21.15.46:6802/3727342125 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f024c00b0b0 con 0x7f02440bc520 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.749+0000 7f02611c3700 1 -- 172.21.15.46:0/2287717672 --> [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f025c109c00 con 0x7f02440bc520 2024-05-09T00:09:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.749+0000 7f0243fff700 1 -- 172.21.15.46:0/2287717672 <== osd.0 v2:172.21.15.46:6802/3727342125 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f025c109c00 con 0x7f02440bc520 2024-05-09T00:09:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.749+0000 7f02611c3700 1 -- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125] conn(0x7f02440bc520 msgr2=0x7f02440bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.749+0000 7f02611c3700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125] conn(0x7f02440bc520 0x7f02440bea10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.749+0000 7f02611c3700 1 -- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f02440b2910 msgr2=0x7f02440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.749+0000 7f02611c3700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f02440b2910 0x7f02440b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f025c1027c0 tx=0x7f025000b480 comp rx=0 tx=0).stop 2024-05-09T00:09:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.749+0000 7f02611c3700 1 -- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c1034c0 msgr2=0x7f025c19cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.749+0000 7f02611c3700 1 --2- 172.21.15.46:0/2287717672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c1034c0 0x7f025c19cfd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f024c0098e0 tx=0x7f024c0047b0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.749+0000 7f025b59e700 1 -- 172.21.15.46:0/2287717672 reap_dead start 2024-05-09T00:09:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.750+0000 7f02611c3700 1 -- 172.21.15.46:0/2287717672 shutdown_connections 2024-05-09T00:09:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.750+0000 7f02611c3700 1 -- 172.21.15.46:0/2287717672 >> 172.21.15.46:0/2287717672 conn(0x7f025c0fedc0 msgr2=0x7f025c107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.750+0000 7f02611c3700 1 -- 172.21.15.46:0/2287717672 shutdown_connections 2024-05-09T00:09:54.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:51.750+0000 7f02611c3700 1 -- 172.21.15.46:0/2287717672 wait complete. 2024-05-09T00:09:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.662+0000 7f4b60dec700 1 -- 172.21.15.46:0/234025043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c103540 msgr2=0x7f4b5c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.662+0000 7f4b60dec700 1 --2- 172.21.15.46:0/234025043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c103540 0x7f4b5c103950 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0099c0 tx=0x7f4b4c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.662+0000 7f4b60dec700 1 -- 172.21.15.46:0/234025043 shutdown_connections 2024-05-09T00:09:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.662+0000 7f4b60dec700 1 --2- 172.21.15.46:0/234025043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b5c108770 0x7f4b5c1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.662+0000 7f4b60dec700 1 --2- 172.21.15.46:0/234025043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c103540 0x7f4b5c103950 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.662+0000 7f4b60dec700 1 --2- 172.21.15.46:0/234025043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b5c107e20 0x7f4b5c108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.662+0000 7f4b60dec700 1 -- 172.21.15.46:0/234025043 >> 172.21.15.46:0/234025043 conn(0x7f4b5c078e50 msgr2=0x7f4b5c079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.662+0000 7f4b60dec700 1 -- 172.21.15.46:0/234025043 shutdown_connections 2024-05-09T00:09:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.662+0000 7f4b60dec700 1 -- 172.21.15.46:0/234025043 wait complete. 2024-05-09T00:09:54.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.663+0000 7f4b60dec700 1 Processor -- start 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.663+0000 7f4b60dec700 1 -- start start 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.663+0000 7f4b60dec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c103540 0x7f4b5c19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.663+0000 7f4b60dec700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b5c107e20 0x7f4b5c19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.663+0000 7f4b60dec700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b5c108770 0x7f4b5c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.663+0000 7f4b60dec700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c1133a0 con 0x7f4b5c103540 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.663+0000 7f4b60dec700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c113220 con 0x7f4b5c108770 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.663+0000 7f4b60dec700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c113520 con 0x7f4b5c107e20 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.663+0000 7f4b5a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c103540 0x7f4b5c19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.663+0000 7f4b5a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c103540 0x7f4b5c19bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53034/0 (socket says 172.21.15.46:53034) 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.663+0000 7f4b5a59c700 1 -- 172.21.15.46:0/3235207441 learned_addr learned my addr 172.21.15.46:0/3235207441 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b5ad9d700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b5c108770 0x7f4b5c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b51d9b700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b5c107e20 0x7f4b5c19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b5a59c700 1 -- 172.21.15.46:0/3235207441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b5c107e20 msgr2=0x7f4b5c19c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b5a59c700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b5c107e20 0x7f4b5c19c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b5a59c700 1 -- 172.21.15.46:0/3235207441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b5c108770 msgr2=0x7f4b5c19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b5a59c700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b5c108770 0x7f4b5c19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b5a59c700 1 -- 172.21.15.46:0/3235207441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c1a6fb0 con 0x7f4b5c103540 2024-05-09T00:09:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b51d9b700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b5c107e20 0x7f4b5c19c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:09:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b5a59c700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c103540 0x7f4b5c19bc60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b580 tx=0x7f4b4c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b53fff700 1 -- 172.21.15.46:0/3235207441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c027070 con 0x7f4b5c103540 2024-05-09T00:09:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b60dec700 1 -- 172.21.15.46:0/3235207441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c1a7240 con 0x7f4b5c103540 2024-05-09T00:09:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b53fff700 1 -- 172.21.15.46:0/3235207441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b4c003f20 con 0x7f4b5c103540 2024-05-09T00:09:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b53fff700 1 -- 172.21.15.46:0/3235207441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c022d70 con 0x7f4b5c103540 2024-05-09T00:09:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.664+0000 7f4b60dec700 1 -- 172.21.15.46:0/3235207441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c1a76a0 con 0x7f4b5c103540 2024-05-09T00:09:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.665+0000 7f4b60dec700 1 -- 172.21.15.46:0/3235207441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4b5c063ed0 con 0x7f4b5c103540 2024-05-09T00:09:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.667+0000 7f4b53fff700 1 -- 172.21.15.46:0/3235207441 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4b4c022500 con 0x7f4b5c103540 2024-05-09T00:09:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.668+0000 7f4b53fff700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4b480b2910 0x7f4b480b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.668+0000 7f4b51d9b700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4b480b2910 0x7f4b480b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.668+0000 7f4b53fff700 1 -- 172.21.15.46:0/3235207441 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4b4c0e3770 con 0x7f4b5c103540 2024-05-09T00:09:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.668+0000 7f4b53fff700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482] conn(0x7f4b480bc520 0x7f4b480bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.668+0000 7f4b53fff700 1 -- 172.21.15.46:0/3235207441 --> [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b4c0043d0 con 0x7f4b480bc520 2024-05-09T00:09:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.668+0000 7f4b53fff700 1 -- 172.21.15.46:0/3235207441 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f4b4c0e3ba0 con 0x7f4b5c103540 2024-05-09T00:09:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.668+0000 7f4b5ad9d700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482] conn(0x7f4b480bc520 0x7f4b480bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.669+0000 7f4b51d9b700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4b480b2910 0x7f4b480b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4b44005950 tx=0x7f4b440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:54.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.669+0000 7f4b5ad9d700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482] conn(0x7f4b480bc520 0x7f4b480bea10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:54.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.670+0000 7f4b53fff700 1 -- 172.21.15.46:0/3235207441 <== osd.6 v2:172.21.15.46:6818/1394035482 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b4c0043d0 con 0x7f4b480bc520 2024-05-09T00:09:54.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.686+0000 7f4b60dec700 1 -- 172.21.15.46:0/3235207441 --> [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4b5c064350 con 0x7f4b480bc520 2024-05-09T00:09:54.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.686+0000 7f4b53fff700 1 -- 172.21.15.46:0/3235207441 <== osd.6 v2:172.21.15.46:6818/1394035482 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4b5c064350 con 0x7f4b480bc520 2024-05-09T00:09:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.687+0000 7f4b60dec700 1 -- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482] conn(0x7f4b480bc520 msgr2=0x7f4b480bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.687+0000 7f4b60dec700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482] conn(0x7f4b480bc520 0x7f4b480bea10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.687+0000 7f4b60dec700 1 -- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4b480b2910 msgr2=0x7f4b480b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.687+0000 7f4b60dec700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4b480b2910 0x7f4b480b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4b44005950 tx=0x7f4b440058e0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.687+0000 7f4b60dec700 1 -- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c103540 msgr2=0x7f4b5c19bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:54.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.687+0000 7f4b60dec700 1 --2- 172.21.15.46:0/3235207441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c103540 0x7f4b5c19bc60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b580 tx=0x7f4b4c0047b0 comp rx=0 tx=0).stop 2024-05-09T00:09:54.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.687+0000 7f4b5ad9d700 1 -- 172.21.15.46:0/3235207441 reap_dead start 2024-05-09T00:09:54.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.687+0000 7f4b60dec700 1 -- 172.21.15.46:0/3235207441 shutdown_connections 2024-05-09T00:09:54.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.687+0000 7f4b60dec700 1 -- 172.21.15.46:0/3235207441 >> 172.21.15.46:0/3235207441 conn(0x7f4b5c078e50 msgr2=0x7f4b5c101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:54.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.687+0000 7f4b60dec700 1 -- 172.21.15.46:0/3235207441 shutdown_connections 2024-05-09T00:09:54.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:54.687+0000 7f4b60dec700 1 -- 172.21.15.46:0/3235207441 wait complete. 2024-05-09T00:09:54.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:54 smithi046 ceph-mon[30151]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:54.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:54 smithi169 ceph-mon[34777]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:54.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:54 smithi142 ceph-mon[34533]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:09:55.441 INFO:teuthology.orchestra.run.smithi046.stdout:55834574932 2024-05-09T00:09:55.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-09T00:09:56.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:56 smithi046 ceph-mon[30151]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-05-09T00:09:56.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:56 smithi169 ceph-mon[34777]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-05-09T00:09:56.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:56 smithi142 ceph-mon[34533]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-05-09T00:09:57.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.923+0000 7f2f345a7700 1 -- 172.21.15.46:0/955085611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f2c102790 msgr2=0x7f2f2c102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:57.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.923+0000 7f2f345a7700 1 --2- 172.21.15.46:0/955085611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f2c102790 0x7f2f2c102ba0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009990 tx=0x7f2f1c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:09:57.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.923+0000 7f2f345a7700 1 -- 172.21.15.46:0/955085611 shutdown_connections 2024-05-09T00:09:57.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.923+0000 7f2f345a7700 1 --2- 172.21.15.46:0/955085611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f2c1030e0 0x7f2f2c111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:57.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.923+0000 7f2f345a7700 1 --2- 172.21.15.46:0/955085611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f2c109300 0x7f2f2c109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:57.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.923+0000 7f2f345a7700 1 --2- 172.21.15.46:0/955085611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f2c102790 0x7f2f2c102ba0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:57.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.923+0000 7f2f345a7700 1 -- 172.21.15.46:0/955085611 >> 172.21.15.46:0/955085611 conn(0x7f2f2c0fe4e0 msgr2=0x7f2f2c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:57.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.923+0000 7f2f345a7700 1 -- 172.21.15.46:0/955085611 shutdown_connections 2024-05-09T00:09:57.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.923+0000 7f2f345a7700 1 -- 172.21.15.46:0/955085611 wait complete. 2024-05-09T00:09:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.924+0000 7f2f345a7700 1 Processor -- start 2024-05-09T00:09:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.924+0000 7f2f345a7700 1 -- start start 2024-05-09T00:09:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f345a7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f2c102790 0x7f2f2c103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f345a7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f2c1030e0 0x7f2f2c103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f345a7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f2c109300 0x7f2f2c1043f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f345a7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c1145a0 con 0x7f2f2c109300 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f345a7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c114420 con 0x7f2f2c102790 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f345a7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c114720 con 0x7f2f2c1030e0 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f32b44700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f2c109300 0x7f2f2c1043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f32b44700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f2c109300 0x7f2f2c1043f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35516/0 (socket says 172.21.15.46:35516) 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f32b44700 1 -- 172.21.15.46:0/670625924 learned_addr learned my addr 172.21.15.46:0/670625924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f31b42700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f2c1030e0 0x7f2f2c103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f32343700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f2c102790 0x7f2f2c103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f32b44700 1 -- 172.21.15.46:0/670625924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f2c1030e0 msgr2=0x7f2f2c103eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f32b44700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f2c1030e0 0x7f2f2c103eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f32b44700 1 -- 172.21.15.46:0/670625924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f2c102790 msgr2=0x7f2f2c103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f32b44700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f2c102790 0x7f2f2c103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f32b44700 1 -- 172.21.15.46:0/670625924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c1a91e0 con 0x7f2f2c109300 2024-05-09T00:09:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f32343700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f2c102790 0x7f2f2c103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:09:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.925+0000 7f2f32b44700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f2c109300 0x7f2f2c1043f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2f24007d70 tx=0x7f2f2400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.926+0000 7f2f237fe700 1 -- 172.21.15.46:0/670625924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f24016020 con 0x7f2f2c109300 2024-05-09T00:09:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.926+0000 7f2f345a7700 1 -- 172.21.15.46:0/670625924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2c1a94d0 con 0x7f2f2c109300 2024-05-09T00:09:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.926+0000 7f2f237fe700 1 -- 172.21.15.46:0/670625924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f2400ad40 con 0x7f2f2c109300 2024-05-09T00:09:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.926+0000 7f2f237fe700 1 -- 172.21.15.46:0/670625924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f240115e0 con 0x7f2f2c109300 2024-05-09T00:09:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.926+0000 7f2f345a7700 1 -- 172.21.15.46:0/670625924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2c1a99e0 con 0x7f2f2c109300 2024-05-09T00:09:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.927+0000 7f2f345a7700 1 -- 172.21.15.46:0/670625924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f2c063ed0 con 0x7f2f2c109300 2024-05-09T00:09:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.928+0000 7f2f237fe700 1 -- 172.21.15.46:0/670625924 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2f24019020 con 0x7f2f2c109300 2024-05-09T00:09:57.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.929+0000 7f2f237fe700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2f180b2910 0x7f2f180b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.929+0000 7f2f32343700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2f180b2910 0x7f2f180b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.929+0000 7f2f237fe700 1 -- 172.21.15.46:0/670625924 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f2f240e28d0 con 0x7f2f2c109300 2024-05-09T00:09:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.929+0000 7f2f237fe700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976] conn(0x7f2f180bc520 0x7f2f180bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.929+0000 7f2f237fe700 1 -- 172.21.15.46:0/670625924 --> [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f2401ce50 con 0x7f2f180bc520 2024-05-09T00:09:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.929+0000 7f2f237fe700 1 -- 172.21.15.46:0/670625924 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f2f240e2d00 con 0x7f2f2c109300 2024-05-09T00:09:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.930+0000 7f2f32343700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2f180b2910 0x7f2f180b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2f280059d0 tx=0x7f2f28005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.930+0000 7f2f31b42700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976] conn(0x7f2f180bc520 0x7f2f180bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.930+0000 7f2f31b42700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976] conn(0x7f2f180bc520 0x7f2f180bea10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.931+0000 7f2f237fe700 1 -- 172.21.15.46:0/670625924 <== osd.11 v2:172.21.15.169:6824/4262478976 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2f2401ce50 con 0x7f2f180bc520 2024-05-09T00:09:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.948+0000 7f2f345a7700 1 -- 172.21.15.46:0/670625924 --> [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f2c0642d0 con 0x7f2f180bc520 2024-05-09T00:09:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.948+0000 7f2f237fe700 1 -- 172.21.15.46:0/670625924 <== osd.11 v2:172.21.15.169:6824/4262478976 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2f2c0642d0 con 0x7f2f180bc520 2024-05-09T00:09:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.948+0000 7f2f345a7700 1 -- 172.21.15.46:0/670625924 >> [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976] conn(0x7f2f180bc520 msgr2=0x7f2f180bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.948+0000 7f2f345a7700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976] conn(0x7f2f180bc520 0x7f2f180bea10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.948+0000 7f2f345a7700 1 -- 172.21.15.46:0/670625924 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2f180b2910 msgr2=0x7f2f180b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.948+0000 7f2f345a7700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2f180b2910 0x7f2f180b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2f280059d0 tx=0x7f2f28005900 comp rx=0 tx=0).stop 2024-05-09T00:09:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.948+0000 7f2f345a7700 1 -- 172.21.15.46:0/670625924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f2c109300 msgr2=0x7f2f2c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.948+0000 7f2f345a7700 1 --2- 172.21.15.46:0/670625924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f2c109300 0x7f2f2c1043f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2f24007d70 tx=0x7f2f2400a340 comp rx=0 tx=0).stop 2024-05-09T00:09:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.948+0000 7f2f32b44700 1 -- 172.21.15.46:0/670625924 reap_dead start 2024-05-09T00:09:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.949+0000 7f2f345a7700 1 -- 172.21.15.46:0/670625924 shutdown_connections 2024-05-09T00:09:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.949+0000 7f2f345a7700 1 -- 172.21.15.46:0/670625924 >> 172.21.15.46:0/670625924 conn(0x7f2f2c0fe4e0 msgr2=0x7f2f2c107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.949+0000 7f2f345a7700 1 -- 172.21.15.46:0/670625924 shutdown_connections 2024-05-09T00:09:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:57.949+0000 7f2f345a7700 1 -- 172.21.15.46:0/670625924 wait complete. 2024-05-09T00:09:58.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:09:58 smithi046 ceph-mon[30151]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T00:09:58.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:09:58 smithi169 ceph-mon[34777]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T00:09:58.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:09:58 smithi142 ceph-mon[34533]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T00:09:59.271 INFO:teuthology.orchestra.run.smithi046.stdout:180388626492 2024-05-09T00:09:59.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-09T00:09:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.466+0000 7f95187e4700 1 -- 172.21.15.46:0/1532274834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9510109870 msgr2=0x7f95101111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.466+0000 7f95187e4700 1 --2- 172.21.15.46:0/1532274834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9510109870 0x7f95101111d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f95080099c0 tx=0x7f9508009cd0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.466+0000 7f95187e4700 1 -- 172.21.15.46:0/1532274834 shutdown_connections 2024-05-09T00:09:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.466+0000 7f95187e4700 1 --2- 172.21.15.46:0/1532274834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9510109870 0x7f95101111d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.466+0000 7f95187e4700 1 --2- 172.21.15.46:0/1532274834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95101045f0 0x7f9510104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.466+0000 7f95187e4700 1 --2- 172.21.15.46:0/1532274834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9510108f20 0x7f9510109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.466+0000 7f95187e4700 1 -- 172.21.15.46:0/1532274834 >> 172.21.15.46:0/1532274834 conn(0x7f9510100040 msgr2=0x7f9510102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.466+0000 7f95187e4700 1 -- 172.21.15.46:0/1532274834 shutdown_connections 2024-05-09T00:09:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.466+0000 7f95187e4700 1 -- 172.21.15.46:0/1532274834 wait complete. 2024-05-09T00:09:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.467+0000 7f95187e4700 1 Processor -- start 2024-05-09T00:09:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.467+0000 7f95187e4700 1 -- start start 2024-05-09T00:09:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f95187e4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95101045f0 0x7f951019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f95187e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9510108f20 0x7f951019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f95187e4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9510109870 0x7f951019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f95187e4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9510114510 con 0x7f9510108f20 2024-05-09T00:09:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f95187e4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9510114390 con 0x7f9510109870 2024-05-09T00:09:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f95187e4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9510114690 con 0x7f95101045f0 2024-05-09T00:09:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9515d7f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9510108f20 0x7f951019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9515d7f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9510108f20 0x7f951019b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35530/0 (socket says 172.21.15.46:35530) 2024-05-09T00:09:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9515d7f700 1 -- 172.21.15.46:0/696326971 learned_addr learned my addr 172.21.15.46:0/696326971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9516580700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95101045f0 0x7f951019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9516d81700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9510109870 0x7f951019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9515d7f700 1 -- 172.21.15.46:0/696326971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95101045f0 msgr2=0x7f951019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9515d7f700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95101045f0 0x7f951019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9515d7f700 1 -- 172.21.15.46:0/696326971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9510109870 msgr2=0x7f951019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9515d7f700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9510109870 0x7f951019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9515d7f700 1 -- 172.21.15.46:0/696326971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951019fd30 con 0x7f9510108f20 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9516d81700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9510109870 0x7f951019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9516580700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95101045f0 0x7f951019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f9515d7f700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9510108f20 0x7f951019b400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f950000b500 tx=0x7f950000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.468+0000 7f95077fe700 1 -- 172.21.15.46:0/696326971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500015020 con 0x7f9510108f20 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.469+0000 7f95077fe700 1 -- 172.21.15.46:0/696326971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9500011ce0 con 0x7f9510108f20 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.469+0000 7f95077fe700 1 -- 172.21.15.46:0/696326971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500010d40 con 0x7f9510108f20 2024-05-09T00:09:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.469+0000 7f95187e4700 1 -- 172.21.15.46:0/696326971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95101a0020 con 0x7f9510108f20 2024-05-09T00:09:59.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.469+0000 7f95187e4700 1 -- 172.21.15.46:0/696326971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95101a7940 con 0x7f9510108f20 2024-05-09T00:09:59.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.470+0000 7f95187e4700 1 -- 172.21.15.46:0/696326971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9510063f50 con 0x7f9510108f20 2024-05-09T00:09:59.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.471+0000 7f95077fe700 1 -- 172.21.15.46:0/696326971 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9500018020 con 0x7f9510108f20 2024-05-09T00:09:59.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.472+0000 7f95077fe700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94fc0b2990 0x7f94fc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.472+0000 7f9516580700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94fc0b2990 0x7f94fc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.472+0000 7f95077fe700 1 -- 172.21.15.46:0/696326971 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f95000e2140 con 0x7f9510108f20 2024-05-09T00:09:59.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.473+0000 7f95077fe700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982] conn(0x7f94fc0bc5a0 0x7f94fc0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:09:59.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.473+0000 7f95077fe700 1 -- 172.21.15.46:0/696326971 --> [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94fc0bf160 con 0x7f94fc0bc5a0 2024-05-09T00:09:59.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.473+0000 7f95077fe700 1 -- 172.21.15.46:0/696326971 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f95000e2570 con 0x7f9510108f20 2024-05-09T00:09:59.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.473+0000 7f9516580700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94fc0b2990 0x7f94fc0b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f950c009dd0 tx=0x7f950c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:59.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.473+0000 7f9516d81700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982] conn(0x7f94fc0bc5a0 0x7f94fc0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:09:59.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.474+0000 7f9516d81700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982] conn(0x7f94fc0bc5a0 0x7f94fc0bea90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:09:59.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.475+0000 7f95077fe700 1 -- 172.21.15.46:0/696326971 <== osd.10 v2:172.21.15.142:6824/1364891982 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94fc0bf160 con 0x7f94fc0bc5a0 2024-05-09T00:09:59.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.492+0000 7f95187e4700 1 -- 172.21.15.46:0/696326971 --> [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9510064350 con 0x7f94fc0bc5a0 2024-05-09T00:09:59.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.493+0000 7f95077fe700 1 -- 172.21.15.46:0/696326971 <== osd.10 v2:172.21.15.142:6824/1364891982 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9510064350 con 0x7f94fc0bc5a0 2024-05-09T00:09:59.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.493+0000 7f95187e4700 1 -- 172.21.15.46:0/696326971 >> [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982] conn(0x7f94fc0bc5a0 msgr2=0x7f94fc0bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.493+0000 7f95187e4700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982] conn(0x7f94fc0bc5a0 0x7f94fc0bea90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.493+0000 7f95187e4700 1 -- 172.21.15.46:0/696326971 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94fc0b2990 msgr2=0x7f94fc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.493+0000 7f95187e4700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94fc0b2990 0x7f94fc0b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f950c009dd0 tx=0x7f950c009040 comp rx=0 tx=0).stop 2024-05-09T00:09:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.493+0000 7f95187e4700 1 -- 172.21.15.46:0/696326971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9510108f20 msgr2=0x7f951019b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:09:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.493+0000 7f95187e4700 1 --2- 172.21.15.46:0/696326971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9510108f20 0x7f951019b400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f950000b500 tx=0x7f950000b8c0 comp rx=0 tx=0).stop 2024-05-09T00:09:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.493+0000 7f9516d81700 1 -- 172.21.15.46:0/696326971 reap_dead start 2024-05-09T00:09:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.493+0000 7f95187e4700 1 -- 172.21.15.46:0/696326971 shutdown_connections 2024-05-09T00:09:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.493+0000 7f95187e4700 1 -- 172.21.15.46:0/696326971 >> 172.21.15.46:0/696326971 conn(0x7f9510100040 msgr2=0x7f9510106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:09:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.493+0000 7f95187e4700 1 -- 172.21.15.46:0/696326971 shutdown_connections 2024-05-09T00:09:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:09:59.493+0000 7f95187e4700 1 -- 172.21.15.46:0/696326971 wait complete. 2024-05-09T00:10:00.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:00 smithi142 ceph-mon[34533]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:10:00.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:00 smithi142 ceph-mon[34533]: overall HEALTH_OK 2024-05-09T00:10:01.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:00 smithi046 ceph-mon[30151]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:10:01.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:00 smithi046 ceph-mon[30151]: overall HEALTH_OK 2024-05-09T00:10:01.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:00 smithi169 ceph-mon[34777]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:10:01.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:00 smithi169 ceph-mon[34777]: overall HEALTH_OK 2024-05-09T00:10:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.920+0000 7f4aff340700 1 -- 172.21.15.46:0/2756969647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8107e30 msgr2=0x7f4af8108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.920+0000 7f4aff340700 1 --2- 172.21.15.46:0/2756969647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8107e30 0x7f4af8108240 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009990 tx=0x7f4aec009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.920+0000 7f4aff340700 1 -- 172.21.15.46:0/2756969647 shutdown_connections 2024-05-09T00:10:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.920+0000 7f4aff340700 1 --2- 172.21.15.46:0/2756969647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4af8108780 0x7f4af81100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.920+0000 7f4aff340700 1 --2- 172.21.15.46:0/2756969647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4af8103550 0x7f4af8103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.920+0000 7f4aff340700 1 --2- 172.21.15.46:0/2756969647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8107e30 0x7f4af8108240 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.920+0000 7f4aff340700 1 -- 172.21.15.46:0/2756969647 >> 172.21.15.46:0/2756969647 conn(0x7f4af8078e00 msgr2=0x7f4af8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.920+0000 7f4aff340700 1 -- 172.21.15.46:0/2756969647 shutdown_connections 2024-05-09T00:10:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.921+0000 7f4aff340700 1 -- 172.21.15.46:0/2756969647 wait complete. 2024-05-09T00:10:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.921+0000 7f4aff340700 1 Processor -- start 2024-05-09T00:10:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.921+0000 7f4aff340700 1 -- start start 2024-05-09T00:10:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.921+0000 7f4aff340700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8103550 0x7f4af819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4aff340700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4af8107e30 0x7f4af819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4aff340700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4af8108780 0x7f4af819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4aff340700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8114510 con 0x7f4af8103550 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4aff340700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8114390 con 0x7f4af8107e30 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4aff340700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8114690 con 0x7f4af8108780 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4afd0dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8103550 0x7f4af819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4afd0dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8103550 0x7f4af819ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35538/0 (socket says 172.21.15.46:35538) 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4afd0dc700 1 -- 172.21.15.46:0/245081071 learned_addr learned my addr 172.21.15.46:0/245081071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4afc8db700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4af8107e30 0x7f4af819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4afd0dc700 1 -- 172.21.15.46:0/245081071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4af8108780 msgr2=0x7f4af819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4afd0dc700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4af8108780 0x7f4af819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4afd0dc700 1 -- 172.21.15.46:0/245081071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4af8107e30 msgr2=0x7f4af819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4afd0dc700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4af8107e30 0x7f4af819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4afd0dc700 1 -- 172.21.15.46:0/245081071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af819fcb0 con 0x7f4af8103550 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4afc8db700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4af8107e30 0x7f4af819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4afd0dc700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8103550 0x7f4af819ade0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80060b0 tx=0x7f4ae8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4af67fc700 1 -- 172.21.15.46:0/245081071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae801a070 con 0x7f4af8103550 2024-05-09T00:10:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4af67fc700 1 -- 172.21.15.46:0/245081071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae800de20 con 0x7f4af8103550 2024-05-09T00:10:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4aff340700 1 -- 172.21.15.46:0/245081071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af819ffa0 con 0x7f4af8103550 2024-05-09T00:10:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4af67fc700 1 -- 172.21.15.46:0/245081071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8014720 con 0x7f4af8103550 2024-05-09T00:10:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.922+0000 7f4aff340700 1 -- 172.21.15.46:0/245081071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af81a78c0 con 0x7f4af8103550 2024-05-09T00:10:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.923+0000 7f4aff340700 1 -- 172.21.15.46:0/245081071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4af8063f50 con 0x7f4af8103550 2024-05-09T00:10:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.925+0000 7f4af67fc700 1 -- 172.21.15.46:0/245081071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4ae801d070 con 0x7f4af8103550 2024-05-09T00:10:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.926+0000 7f4af67fc700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4ae00b2990 0x7f4ae00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.926+0000 7f4afc8db700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4ae00b2990 0x7f4ae00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.926+0000 7f4af67fc700 1 -- 172.21.15.46:0/245081071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4ae80e2480 con 0x7f4af8103550 2024-05-09T00:10:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.926+0000 7f4af67fc700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713] conn(0x7f4ae00bc5a0 0x7f4ae00bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.926+0000 7f4af67fc700 1 -- 172.21.15.46:0/245081071 --> [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ae8019ec0 con 0x7f4ae00bc5a0 2024-05-09T00:10:01.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.926+0000 7f4af67fc700 1 -- 172.21.15.46:0/245081071 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f4ae80e2830 con 0x7f4af8103550 2024-05-09T00:10:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.926+0000 7f4afc8db700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4ae00b2990 0x7f4ae00b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4aec006010 tx=0x7f4aec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.926+0000 7f4afd8dd700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713] conn(0x7f4ae00bc5a0 0x7f4ae00bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.927+0000 7f4afd8dd700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713] conn(0x7f4ae00bc5a0 0x7f4ae00bea90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:01.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.928+0000 7f4af67fc700 1 -- 172.21.15.46:0/245081071 <== osd.5 v2:172.21.15.169:6808/445044713 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ae8019ec0 con 0x7f4ae00bc5a0 2024-05-09T00:10:01.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.944+0000 7f4aff340700 1 -- 172.21.15.46:0/245081071 --> [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4af8064350 con 0x7f4ae00bc5a0 2024-05-09T00:10:01.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.945+0000 7f4af67fc700 1 -- 172.21.15.46:0/245081071 <== osd.5 v2:172.21.15.169:6808/445044713 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4af8064350 con 0x7f4ae00bc5a0 2024-05-09T00:10:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.945+0000 7f4aff340700 1 -- 172.21.15.46:0/245081071 >> [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713] conn(0x7f4ae00bc5a0 msgr2=0x7f4ae00bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.945+0000 7f4aff340700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713] conn(0x7f4ae00bc5a0 0x7f4ae00bea90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.945+0000 7f4aff340700 1 -- 172.21.15.46:0/245081071 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4ae00b2990 msgr2=0x7f4ae00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.945+0000 7f4aff340700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4ae00b2990 0x7f4ae00b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4aec006010 tx=0x7f4aec01a040 comp rx=0 tx=0).stop 2024-05-09T00:10:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.945+0000 7f4aff340700 1 -- 172.21.15.46:0/245081071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8103550 msgr2=0x7f4af819ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.945+0000 7f4aff340700 1 --2- 172.21.15.46:0/245081071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4af8103550 0x7f4af819ade0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80060b0 tx=0x7f4ae8007f90 comp rx=0 tx=0).stop 2024-05-09T00:10:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.945+0000 7f4afd8dd700 1 -- 172.21.15.46:0/245081071 reap_dead start 2024-05-09T00:10:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.946+0000 7f4aff340700 1 -- 172.21.15.46:0/245081071 shutdown_connections 2024-05-09T00:10:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.946+0000 7f4aff340700 1 -- 172.21.15.46:0/245081071 >> 172.21.15.46:0/245081071 conn(0x7f4af8078e00 msgr2=0x7f4af8101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.946+0000 7f4aff340700 1 -- 172.21.15.46:0/245081071 shutdown_connections 2024-05-09T00:10:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:01.946+0000 7f4aff340700 1 -- 172.21.15.46:0/245081071 wait complete. 2024-05-09T00:10:02.762 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:02 smithi046 ceph-mon[30151]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:10:02.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:02 smithi142 ceph-mon[34533]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:10:03.094 INFO:teuthology.orchestra.run.smithi046.stdout:34359738457 2024-05-09T00:10:03.095 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-09T00:10:03.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:02 smithi169 ceph-mon[34777]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:10:03.414 INFO:teuthology.orchestra.run.smithi046.stdout:300647710753 2024-05-09T00:10:03.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-09T00:10:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.608+0000 7f3492ef3700 1 -- 172.21.15.46:0/311027361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f348c108740 msgr2=0x7f348c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.608+0000 7f3492ef3700 1 --2- 172.21.15.46:0/311027361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f348c108740 0x7f348c1100a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f348400d360 tx=0x7f348400d670 comp rx=0 tx=0).stop 2024-05-09T00:10:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.608+0000 7f3492ef3700 1 -- 172.21.15.46:0/311027361 shutdown_connections 2024-05-09T00:10:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.608+0000 7f3492ef3700 1 --2- 172.21.15.46:0/311027361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f348c108740 0x7f348c1100a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.608+0000 7f3492ef3700 1 --2- 172.21.15.46:0/311027361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c1034c0 0x7f348c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.608+0000 7f3492ef3700 1 --2- 172.21.15.46:0/311027361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f348c107df0 0x7f348c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.608+0000 7f3492ef3700 1 -- 172.21.15.46:0/311027361 >> 172.21.15.46:0/311027361 conn(0x7f348c0fedc0 msgr2=0x7f348c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.608+0000 7f3492ef3700 1 -- 172.21.15.46:0/311027361 shutdown_connections 2024-05-09T00:10:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.608+0000 7f3492ef3700 1 -- 172.21.15.46:0/311027361 wait complete. 2024-05-09T00:10:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.609+0000 7f3492ef3700 1 Processor -- start 2024-05-09T00:10:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.609+0000 7f3492ef3700 1 -- start start 2024-05-09T00:10:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.609+0000 7f3492ef3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f348c1034c0 0x7f348c19d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.609+0000 7f3492ef3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f348c107df0 0x7f348c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.609+0000 7f3492ef3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c108740 0x7f348c1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.609+0000 7f3492ef3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f348c1123c0 con 0x7f348c107df0 2024-05-09T00:10:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.609+0000 7f3492ef3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f348c112240 con 0x7f348c1034c0 2024-05-09T00:10:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.609+0000 7f3492ef3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f348c112540 con 0x7f348c108740 2024-05-09T00:10:03.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.610+0000 7f3491490700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c108740 0x7f348c1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:03.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.610+0000 7f3491490700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c108740 0x7f348c1a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:33550/0 (socket says 172.21.15.46:33550) 2024-05-09T00:10:03.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.610+0000 7f3491490700 1 -- 172.21.15.46:0/1640296750 learned_addr learned my addr 172.21.15.46:0/1640296750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:03.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.610+0000 7f348bfff700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f348c107df0 0x7f348c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.610+0000 7f3490c8f700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f348c1034c0 0x7f348c19d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.610+0000 7f3491490700 1 -- 172.21.15.46:0/1640296750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f348c1034c0 msgr2=0x7f348c19d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.610+0000 7f3491490700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f348c1034c0 0x7f348c19d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.610+0000 7f3491490700 1 -- 172.21.15.46:0/1640296750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f348c107df0 msgr2=0x7f348c19d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.610+0000 7f3491490700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f348c107df0 0x7f348c19d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.610+0000 7f3491490700 1 -- 172.21.15.46:0/1640296750 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c1a2070 con 0x7f348c108740 2024-05-09T00:10:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.610+0000 7f3491490700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c108740 0x7f348c1a19d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f348400d330 tx=0x7f34840127b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.611+0000 7f3489ffb700 1 -- 172.21.15.46:0/1640296750 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484015040 con 0x7f348c108740 2024-05-09T00:10:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.611+0000 7f3492ef3700 1 -- 172.21.15.46:0/1640296750 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348c1a2300 con 0x7f348c108740 2024-05-09T00:10:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.611+0000 7f3489ffb700 1 -- 172.21.15.46:0/1640296750 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3484009480 con 0x7f348c108740 2024-05-09T00:10:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.611+0000 7f3489ffb700 1 -- 172.21.15.46:0/1640296750 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348401ae20 con 0x7f348c108740 2024-05-09T00:10:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.611+0000 7f3492ef3700 1 -- 172.21.15.46:0/1640296750 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348c1a9c80 con 0x7f348c108740 2024-05-09T00:10:03.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.612+0000 7f3492ef3700 1 -- 172.21.15.46:0/1640296750 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f348c063f50 con 0x7f348c108740 2024-05-09T00:10:03.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.617+0000 7f3489ffb700 1 -- 172.21.15.46:0/1640296750 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3484022020 con 0x7f348c108740 2024-05-09T00:10:03.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.618+0000 7f3489ffb700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f347c0b2990 0x7f347c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:03.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.618+0000 7f3490c8f700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f347c0b2990 0x7f347c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:03.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.618+0000 7f3489ffb700 1 -- 172.21.15.46:0/1640296750 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f34840e3bd0 con 0x7f348c108740 2024-05-09T00:10:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.618+0000 7f3489ffb700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064] conn(0x7f347c0bc5a0 0x7f347c0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.618+0000 7f3489ffb700 1 -- 172.21.15.46:0/1640296750 --> [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f347c0bf160 con 0x7f347c0bc5a0 2024-05-09T00:10:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.618+0000 7f3489ffb700 1 -- 172.21.15.46:0/1640296750 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f34840e4000 con 0x7f348c108740 2024-05-09T00:10:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.619+0000 7f348bfff700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064] conn(0x7f347c0bc5a0 0x7f347c0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:03.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.619+0000 7f3490c8f700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f347c0b2990 0x7f347c0b4ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3480009910 tx=0x7f3480008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:03.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.619+0000 7f348bfff700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064] conn(0x7f347c0bc5a0 0x7f347c0bea90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:03.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.620+0000 7f3489ffb700 1 -- 172.21.15.46:0/1640296750 <== osd.2 v2:172.21.15.169:6800/2680673064 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f347c0bf160 con 0x7f347c0bc5a0 2024-05-09T00:10:03.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.639+0000 7f3492ef3700 1 -- 172.21.15.46:0/1640296750 --> [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f348c064380 con 0x7f347c0bc5a0 2024-05-09T00:10:03.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.640+0000 7f3489ffb700 1 -- 172.21.15.46:0/1640296750 <== osd.2 v2:172.21.15.169:6800/2680673064 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f348c064380 con 0x7f347c0bc5a0 2024-05-09T00:10:03.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.640+0000 7f3492ef3700 1 -- 172.21.15.46:0/1640296750 >> [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064] conn(0x7f347c0bc5a0 msgr2=0x7f347c0bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:03.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.640+0000 7f3492ef3700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064] conn(0x7f347c0bc5a0 0x7f347c0bea90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:03.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.641+0000 7f3492ef3700 1 -- 172.21.15.46:0/1640296750 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f347c0b2990 msgr2=0x7f347c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:03.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.641+0000 7f3492ef3700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f347c0b2990 0x7f347c0b4ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3480009910 tx=0x7f3480008040 comp rx=0 tx=0).stop 2024-05-09T00:10:03.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.641+0000 7f3492ef3700 1 -- 172.21.15.46:0/1640296750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c108740 msgr2=0x7f348c1a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:03.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.641+0000 7f3492ef3700 1 --2- 172.21.15.46:0/1640296750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c108740 0x7f348c1a19d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f348400d330 tx=0x7f34840127b0 comp rx=0 tx=0).stop 2024-05-09T00:10:03.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.641+0000 7f3491490700 1 -- 172.21.15.46:0/1640296750 reap_dead start 2024-05-09T00:10:03.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.641+0000 7f3492ef3700 1 -- 172.21.15.46:0/1640296750 shutdown_connections 2024-05-09T00:10:03.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.641+0000 7f3492ef3700 1 -- 172.21.15.46:0/1640296750 >> 172.21.15.46:0/1640296750 conn(0x7f348c0fedc0 msgr2=0x7f348c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:03.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.641+0000 7f3492ef3700 1 -- 172.21.15.46:0/1640296750 shutdown_connections 2024-05-09T00:10:03.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:03.641+0000 7f3492ef3700 1 -- 172.21.15.46:0/1640296750 wait complete. 2024-05-09T00:10:04.772 INFO:teuthology.orchestra.run.smithi046.stdout:274877906984 2024-05-09T00:10:04.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-09T00:10:04.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:04 smithi142 ceph-mon[34533]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:04.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:04 smithi046 ceph-mon[30151]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:04.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.957+0000 7f13d7594700 1 -- 172.21.15.46:0/3595158712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0108740 msgr2=0x7f13d01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.957+0000 7f13d7594700 1 --2- 172.21.15.46:0/3595158712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0108740 0x7f13d01100a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13c800b0d0 tx=0x7f13c800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:10:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.957+0000 7f13d7594700 1 -- 172.21.15.46:0/3595158712 shutdown_connections 2024-05-09T00:10:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.957+0000 7f13d7594700 1 --2- 172.21.15.46:0/3595158712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0108740 0x7f13d01100a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.957+0000 7f13d7594700 1 --2- 172.21.15.46:0/3595158712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d01034c0 0x7f13d01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.957+0000 7f13d7594700 1 --2- 172.21.15.46:0/3595158712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d0107df0 0x7f13d0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.957+0000 7f13d7594700 1 -- 172.21.15.46:0/3595158712 >> 172.21.15.46:0/3595158712 conn(0x7f13d00fedc0 msgr2=0x7f13d01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.957+0000 7f13d7594700 1 -- 172.21.15.46:0/3595158712 shutdown_connections 2024-05-09T00:10:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.957+0000 7f13d7594700 1 -- 172.21.15.46:0/3595158712 wait complete. 2024-05-09T00:10:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.958+0000 7f13d7594700 1 Processor -- start 2024-05-09T00:10:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.958+0000 7f13d7594700 1 -- start start 2024-05-09T00:10:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d7594700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d01034c0 0x7f13d019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d7594700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d0107df0 0x7f13d019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d7594700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d0108740 0x7f13d019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d7594700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0113390 con 0x7f13d01034c0 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d7594700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0113210 con 0x7f13d0107df0 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d7594700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0113510 con 0x7f13d0108740 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d5330700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d01034c0 0x7f13d019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d5330700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d01034c0 0x7f13d019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35574/0 (socket says 172.21.15.46:35574) 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d5330700 1 -- 172.21.15.46:0/1837766698 learned_addr learned my addr 172.21.15.46:0/1837766698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d4b2f700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d0107df0 0x7f13d019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d5b31700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d0108740 0x7f13d019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d5330700 1 -- 172.21.15.46:0/1837766698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d0108740 msgr2=0x7f13d019c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d5330700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d0108740 0x7f13d019c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d5330700 1 -- 172.21.15.46:0/1837766698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d0107df0 msgr2=0x7f13d019c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d5330700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13d0107df0 0x7f13d019c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d5330700 1 -- 172.21.15.46:0/1837766698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d01a7020 con 0x7f13d01034c0 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.959+0000 7f13d5330700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d01034c0 0x7f13d019bcb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00d950 tx=0x7f13cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.960+0000 7f13c27fc700 1 -- 172.21.15.46:0/1837766698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc016070 con 0x7f13d01034c0 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.960+0000 7f13d7594700 1 -- 172.21.15.46:0/1837766698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d01a7310 con 0x7f13d01034c0 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.960+0000 7f13c27fc700 1 -- 172.21.15.46:0/1837766698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13cc00fd40 con 0x7f13d01034c0 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.960+0000 7f13c27fc700 1 -- 172.21.15.46:0/1837766698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc00e3b0 con 0x7f13d01034c0 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.960+0000 7f13d7594700 1 -- 172.21.15.46:0/1837766698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d01a78a0 con 0x7f13d01034c0 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.961+0000 7f13d7594700 1 -- 172.21.15.46:0/1837766698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13d0063ed0 con 0x7f13d01034c0 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.964+0000 7f13c27fc700 1 -- 172.21.15.46:0/1837766698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f13cc019070 con 0x7f13d01034c0 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.965+0000 7f13c27fc700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f13bc0b2910 0x7f13bc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.965+0000 7f13d4b2f700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f13bc0b2910 0x7f13bc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.965+0000 7f13c27fc700 1 -- 172.21.15.46:0/1837766698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f13cc0e2400 con 0x7f13d01034c0 2024-05-09T00:10:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.965+0000 7f13c27fc700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376] conn(0x7f13bc0bc520 0x7f13bc0bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.965+0000 7f13c27fc700 1 -- 172.21.15.46:0/1837766698 --> [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13bc0bf0e0 con 0x7f13bc0bc520 2024-05-09T00:10:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.965+0000 7f13c27fc700 1 -- 172.21.15.46:0/1837766698 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f13cc0e27b0 con 0x7f13d01034c0 2024-05-09T00:10:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.966+0000 7f13d4b2f700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f13bc0b2910 0x7f13bc0b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f13d01066c0 tx=0x7f13c400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.966+0000 7f13d5b31700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376] conn(0x7f13bc0bc520 0x7f13bc0bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.966+0000 7f13d5b31700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376] conn(0x7f13bc0bc520 0x7f13bc0bea10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.967+0000 7f13c27fc700 1 -- 172.21.15.46:0/1837766698 <== osd.8 v2:172.21.15.169:6816/2650652376 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13bc0bf0e0 con 0x7f13bc0bc520 2024-05-09T00:10:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.986+0000 7f13d7594700 1 -- 172.21.15.46:0/1837766698 --> [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f13d00642d0 con 0x7f13bc0bc520 2024-05-09T00:10:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.987+0000 7f13c27fc700 1 -- 172.21.15.46:0/1837766698 <== osd.8 v2:172.21.15.169:6816/2650652376 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f13d00642d0 con 0x7f13bc0bc520 2024-05-09T00:10:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.987+0000 7f13d7594700 1 -- 172.21.15.46:0/1837766698 >> [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376] conn(0x7f13bc0bc520 msgr2=0x7f13bc0bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.987+0000 7f13d7594700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376] conn(0x7f13bc0bc520 0x7f13bc0bea10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.987+0000 7f13d7594700 1 -- 172.21.15.46:0/1837766698 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f13bc0b2910 msgr2=0x7f13bc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.987+0000 7f13d7594700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f13bc0b2910 0x7f13bc0b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f13d01066c0 tx=0x7f13c400b3c0 comp rx=0 tx=0).stop 2024-05-09T00:10:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.987+0000 7f13d7594700 1 -- 172.21.15.46:0/1837766698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d01034c0 msgr2=0x7f13d019bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.987+0000 7f13d7594700 1 --2- 172.21.15.46:0/1837766698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d01034c0 0x7f13d019bcb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00d950 tx=0x7f13cc00dc60 comp rx=0 tx=0).stop 2024-05-09T00:10:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.987+0000 7f13d5b31700 1 -- 172.21.15.46:0/1837766698 reap_dead start 2024-05-09T00:10:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.987+0000 7f13d7594700 1 -- 172.21.15.46:0/1837766698 shutdown_connections 2024-05-09T00:10:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.987+0000 7f13d7594700 1 -- 172.21.15.46:0/1837766698 >> 172.21.15.46:0/1837766698 conn(0x7f13d00fedc0 msgr2=0x7f13d0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.987+0000 7f13d7594700 1 -- 172.21.15.46:0/1837766698 shutdown_connections 2024-05-09T00:10:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:04.987+0000 7f13d7594700 1 -- 172.21.15.46:0/1837766698 wait complete. 2024-05-09T00:10:05.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:04 smithi169 ceph-mon[34777]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:06.231 INFO:teuthology.orchestra.run.smithi046.stdout:154618822721 2024-05-09T00:10:06.232 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-09T00:10:06.645 INFO:teuthology.orchestra.run.smithi046.stdout:77309411410 2024-05-09T00:10:06.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-09T00:10:06.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.897+0000 7fac6ff33700 1 -- 172.21.15.46:0/919619864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac68109850 msgr2=0x7fac681111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:06.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.897+0000 7fac6ff33700 1 --2- 172.21.15.46:0/919619864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac68109850 0x7fac681111b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fac6000b0d0 tx=0x7fac6000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:10:06.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.897+0000 7fac6ff33700 1 -- 172.21.15.46:0/919619864 shutdown_connections 2024-05-09T00:10:06.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.897+0000 7fac6ff33700 1 --2- 172.21.15.46:0/919619864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac68109850 0x7fac681111b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:06.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.897+0000 7fac6ff33700 1 --2- 172.21.15.46:0/919619864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac681045d0 0x7fac681049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:06.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.897+0000 7fac6ff33700 1 --2- 172.21.15.46:0/919619864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac68108f00 0x7fac68109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:06.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.897+0000 7fac6ff33700 1 -- 172.21.15.46:0/919619864 >> 172.21.15.46:0/919619864 conn(0x7fac680ffe80 msgr2=0x7fac681022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:06.896 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:06 smithi046 ceph-mon[30151]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.898+0000 7fac6ff33700 1 -- 172.21.15.46:0/919619864 shutdown_connections 2024-05-09T00:10:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.898+0000 7fac6ff33700 1 -- 172.21.15.46:0/919619864 wait complete. 2024-05-09T00:10:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.898+0000 7fac6ff33700 1 Processor -- start 2024-05-09T00:10:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.898+0000 7fac6ff33700 1 -- start start 2024-05-09T00:10:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6ff33700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac681045d0 0x7fac6819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6ff33700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac68108f00 0x7fac6819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6ff33700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac68109850 0x7fac6819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6ff33700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac68114480 con 0x7fac68108f00 2024-05-09T00:10:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6ff33700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fac68114300 con 0x7fac681045d0 2024-05-09T00:10:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6ff33700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fac68114600 con 0x7fac68109850 2024-05-09T00:10:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6d4ce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac68108f00 0x7fac6819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6dccf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac681045d0 0x7fac6819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6d4ce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac68108f00 0x7fac6819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47412/0 (socket says 172.21.15.46:47412) 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6d4ce700 1 -- 172.21.15.46:0/182455371 learned_addr learned my addr 172.21.15.46:0/182455371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6e4d0700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac68109850 0x7fac6819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6d4ce700 1 -- 172.21.15.46:0/182455371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac68109850 msgr2=0x7fac6819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6d4ce700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac68109850 0x7fac6819f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6d4ce700 1 -- 172.21.15.46:0/182455371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac681045d0 msgr2=0x7fac6819ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6d4ce700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac681045d0 0x7fac6819ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6d4ce700 1 -- 172.21.15.46:0/182455371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6819fbb0 con 0x7fac68108f00 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.899+0000 7fac6dccf700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac681045d0 0x7fac6819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.900+0000 7fac6d4ce700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac68108f00 0x7fac6819b330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fac6400e970 tx=0x7fac6400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.900+0000 7fac5affd700 1 -- 172.21.15.46:0/182455371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6400cbd0 con 0x7fac68108f00 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.900+0000 7fac5affd700 1 -- 172.21.15.46:0/182455371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac6400cd70 con 0x7fac68108f00 2024-05-09T00:10:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.900+0000 7fac5affd700 1 -- 172.21.15.46:0/182455371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64005680 con 0x7fac68108f00 2024-05-09T00:10:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.900+0000 7fac6ff33700 1 -- 172.21.15.46:0/182455371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6819fea0 con 0x7fac68108f00 2024-05-09T00:10:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.900+0000 7fac6ff33700 1 -- 172.21.15.46:0/182455371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac681a0390 con 0x7fac68108f00 2024-05-09T00:10:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.901+0000 7fac6ff33700 1 -- 172.21.15.46:0/182455371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac68063f10 con 0x7fac68108f00 2024-05-09T00:10:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.903+0000 7fac5affd700 1 -- 172.21.15.46:0/182455371 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fac64004750 con 0x7fac68108f00 2024-05-09T00:10:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.903+0000 7fac5affd700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fac540b2910 0x7fac540b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.904+0000 7fac6dccf700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fac540b2910 0x7fac540b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.904+0000 7fac5affd700 1 -- 172.21.15.46:0/182455371 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fac640e2d30 con 0x7fac68108f00 2024-05-09T00:10:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.904+0000 7fac5affd700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865] conn(0x7fac540bc520 0x7fac540bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.904+0000 7fac5affd700 1 -- 172.21.15.46:0/182455371 --> [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac540bf0e0 con 0x7fac540bc520 2024-05-09T00:10:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.904+0000 7fac5affd700 1 -- 172.21.15.46:0/182455371 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fac640e3160 con 0x7fac68108f00 2024-05-09T00:10:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.904+0000 7fac6e4d0700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865] conn(0x7fac540bc520 0x7fac540bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:06.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.904+0000 7fac6dccf700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fac540b2910 0x7fac540b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00afd0 tx=0x7fac5c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:06.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.905+0000 7fac6e4d0700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865] conn(0x7fac540bc520 0x7fac540bea10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.906+0000 7fac5affd700 1 -- 172.21.15.46:0/182455371 <== osd.4 v2:172.21.15.142:6808/2299616865 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac540bf0e0 con 0x7fac540bc520 2024-05-09T00:10:06.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.922+0000 7fac6ff33700 1 -- 172.21.15.46:0/182455371 --> [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac68064390 con 0x7fac540bc520 2024-05-09T00:10:06.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.923+0000 7fac5affd700 1 -- 172.21.15.46:0/182455371 <== osd.4 v2:172.21.15.142:6808/2299616865 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fac68064390 con 0x7fac540bc520 2024-05-09T00:10:06.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.923+0000 7fac6ff33700 1 -- 172.21.15.46:0/182455371 >> [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865] conn(0x7fac540bc520 msgr2=0x7fac540bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:06.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.923+0000 7fac6ff33700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865] conn(0x7fac540bc520 0x7fac540bea10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:06.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.923+0000 7fac6ff33700 1 -- 172.21.15.46:0/182455371 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fac540b2910 msgr2=0x7fac540b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:06.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.923+0000 7fac6ff33700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fac540b2910 0x7fac540b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00afd0 tx=0x7fac5c00a380 comp rx=0 tx=0).stop 2024-05-09T00:10:06.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.923+0000 7fac6ff33700 1 -- 172.21.15.46:0/182455371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac68108f00 msgr2=0x7fac6819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:06.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.923+0000 7fac6ff33700 1 --2- 172.21.15.46:0/182455371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac68108f00 0x7fac6819b330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fac6400e970 tx=0x7fac6400ec80 comp rx=0 tx=0).stop 2024-05-09T00:10:06.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.923+0000 7fac6e4d0700 1 -- 172.21.15.46:0/182455371 reap_dead start 2024-05-09T00:10:06.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.923+0000 7fac6ff33700 1 -- 172.21.15.46:0/182455371 shutdown_connections 2024-05-09T00:10:06.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.923+0000 7fac6ff33700 1 -- 172.21.15.46:0/182455371 >> 172.21.15.46:0/182455371 conn(0x7fac680ffe80 msgr2=0x7fac68108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:06.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.923+0000 7fac6ff33700 1 -- 172.21.15.46:0/182455371 shutdown_connections 2024-05-09T00:10:06.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:06.923+0000 7fac6ff33700 1 -- 172.21.15.46:0/182455371 wait complete. 2024-05-09T00:10:06.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:06 smithi142 ceph-mon[34533]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:07.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:06 smithi169 ceph-mon[34777]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:08.136 INFO:teuthology.orchestra.run.smithi046.stdout:133143986248 2024-05-09T00:10:08.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-09T00:10:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.345+0000 7f93e524e700 1 -- 172.21.15.46:0/2110469698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e0103550 msgr2=0x7f93e0103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.345+0000 7f93e524e700 1 --2- 172.21.15.46:0/2110469698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e0103550 0x7f93e0103960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93d00099c0 tx=0x7f93d0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.345+0000 7f93e524e700 1 -- 172.21.15.46:0/2110469698 shutdown_connections 2024-05-09T00:10:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.345+0000 7f93e524e700 1 --2- 172.21.15.46:0/2110469698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e0108780 0x7f93e01100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.345+0000 7f93e524e700 1 --2- 172.21.15.46:0/2110469698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e0103550 0x7f93e0103960 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.345+0000 7f93e524e700 1 --2- 172.21.15.46:0/2110469698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0107e30 0x7f93e0108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.345+0000 7f93e524e700 1 -- 172.21.15.46:0/2110469698 >> 172.21.15.46:0/2110469698 conn(0x7f93e0078e00 msgr2=0x7f93e0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.345+0000 7f93e524e700 1 -- 172.21.15.46:0/2110469698 shutdown_connections 2024-05-09T00:10:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.345+0000 7f93e524e700 1 -- 172.21.15.46:0/2110469698 wait complete. 2024-05-09T00:10:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.346+0000 7f93e524e700 1 Processor -- start 2024-05-09T00:10:08.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.346+0000 7f93e524e700 1 -- start start 2024-05-09T00:10:08.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.346+0000 7f93e524e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e0103550 0x7f93e019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.346+0000 7f93e524e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e0107e30 0x7f93e019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.346+0000 7f93e524e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0108780 0x7f93e019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.346+0000 7f93e524e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93e01134a0 con 0x7f93e0107e30 2024-05-09T00:10:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.346+0000 7f93e524e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f93e0113320 con 0x7f93e0103550 2024-05-09T00:10:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.346+0000 7f93e524e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93e0113620 con 0x7f93e0108780 2024-05-09T00:10:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.346+0000 7f93de59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e0107e30 0x7f93e019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.346+0000 7f93de59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e0107e30 0x7f93e019b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47434/0 (socket says 172.21.15.46:47434) 2024-05-09T00:10:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.346+0000 7f93de59c700 1 -- 172.21.15.46:0/1350625120 learned_addr learned my addr 172.21.15.46:0/1350625120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93ded9d700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e0103550 0x7f93e019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93df59e700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0108780 0x7f93e019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93de59c700 1 -- 172.21.15.46:0/1350625120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0108780 msgr2=0x7f93e019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93de59c700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0108780 0x7f93e019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93de59c700 1 -- 172.21.15.46:0/1350625120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e0103550 msgr2=0x7f93e019aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93de59c700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e0103550 0x7f93e019aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93de59c700 1 -- 172.21.15.46:0/1350625120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e019fd70 con 0x7f93e0107e30 2024-05-09T00:10:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93df59e700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93e0108780 0x7f93e019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93de59c700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e0107e30 0x7f93e019b440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f93d400b500 tx=0x7f93d400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93c7fff700 1 -- 172.21.15.46:0/1350625120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4015020 con 0x7f93e0107e30 2024-05-09T00:10:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93c7fff700 1 -- 172.21.15.46:0/1350625120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d4011ce0 con 0x7f93e0107e30 2024-05-09T00:10:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93e524e700 1 -- 172.21.15.46:0/1350625120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e01a0060 con 0x7f93e0107e30 2024-05-09T00:10:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93c7fff700 1 -- 172.21.15.46:0/1350625120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4010d40 con 0x7f93e0107e30 2024-05-09T00:10:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.347+0000 7f93e524e700 1 -- 172.21.15.46:0/1350625120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e01a7980 con 0x7f93e0107e30 2024-05-09T00:10:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.348+0000 7f93c5ffb700 1 -- 172.21.15.46:0/1350625120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93e0063f50 con 0x7f93e0107e30 2024-05-09T00:10:08.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.350+0000 7f93c7fff700 1 -- 172.21.15.46:0/1350625120 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f93d4018020 con 0x7f93e0107e30 2024-05-09T00:10:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.351+0000 7f93c7fff700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f93c80b2990 0x7f93c80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.351+0000 7f93ded9d700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f93c80b2990 0x7f93c80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.351+0000 7f93c7fff700 1 -- 172.21.15.46:0/1350625120 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f93d40e21b0 con 0x7f93e0107e30 2024-05-09T00:10:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.351+0000 7f93c7fff700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607] conn(0x7f93c80bc5a0 0x7f93c80bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.352+0000 7f93c7fff700 1 -- 172.21.15.46:0/1350625120 --> [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93d40044e0 con 0x7f93c80bc5a0 2024-05-09T00:10:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.352+0000 7f93c7fff700 1 -- 172.21.15.46:0/1350625120 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f93d40e2560 con 0x7f93e0107e30 2024-05-09T00:10:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.352+0000 7f93ded9d700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f93c80b2990 0x7f93c80b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f93d0003800 tx=0x7f93d001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:08.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.352+0000 7f93df59e700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607] conn(0x7f93c80bc5a0 0x7f93c80bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:08.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.352+0000 7f93df59e700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607] conn(0x7f93c80bc5a0 0x7f93c80bea90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:08.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.353+0000 7f93c7fff700 1 -- 172.21.15.46:0/1350625120 <== osd.7 v2:172.21.15.142:6816/596805607 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93d40044e0 con 0x7f93c80bc5a0 2024-05-09T00:10:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.370+0000 7f93c5ffb700 1 -- 172.21.15.46:0/1350625120 --> [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f93e0064350 con 0x7f93c80bc5a0 2024-05-09T00:10:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.370+0000 7f93c7fff700 1 -- 172.21.15.46:0/1350625120 <== osd.7 v2:172.21.15.142:6816/596805607 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f93e0064350 con 0x7f93c80bc5a0 2024-05-09T00:10:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.371+0000 7f93c5ffb700 1 -- 172.21.15.46:0/1350625120 >> [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607] conn(0x7f93c80bc5a0 msgr2=0x7f93c80bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.371+0000 7f93c5ffb700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607] conn(0x7f93c80bc5a0 0x7f93c80bea90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.371+0000 7f93c5ffb700 1 -- 172.21.15.46:0/1350625120 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f93c80b2990 msgr2=0x7f93c80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.371+0000 7f93c5ffb700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f93c80b2990 0x7f93c80b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f93d0003800 tx=0x7f93d001a040 comp rx=0 tx=0).stop 2024-05-09T00:10:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.371+0000 7f93c5ffb700 1 -- 172.21.15.46:0/1350625120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e0107e30 msgr2=0x7f93e019b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.371+0000 7f93c5ffb700 1 --2- 172.21.15.46:0/1350625120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e0107e30 0x7f93e019b440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f93d400b500 tx=0x7f93d400b8c0 comp rx=0 tx=0).stop 2024-05-09T00:10:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.371+0000 7f93df59e700 1 -- 172.21.15.46:0/1350625120 reap_dead start 2024-05-09T00:10:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.371+0000 7f93c5ffb700 1 -- 172.21.15.46:0/1350625120 shutdown_connections 2024-05-09T00:10:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.371+0000 7f93c5ffb700 1 -- 172.21.15.46:0/1350625120 >> 172.21.15.46:0/1350625120 conn(0x7f93e0078e00 msgr2=0x7f93e0101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.371+0000 7f93c5ffb700 1 -- 172.21.15.46:0/1350625120 shutdown_connections 2024-05-09T00:10:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:08.371+0000 7f93c5ffb700 1 -- 172.21.15.46:0/1350625120 wait complete. 2024-05-09T00:10:08.669 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:08 smithi046 ceph-mon[30151]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-09T00:10:08.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:08 smithi142 ceph-mon[34533]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-09T00:10:08.989 INFO:teuthology.orchestra.run.smithi046.stdout:227633266739 2024-05-09T00:10:08.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-09T00:10:09.004 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:09.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:08 smithi169 ceph-mon[34777]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-09T00:10:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.751+0000 7efede098700 1 -- 172.21.15.46:0/1639340827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8104650 msgr2=0x7efed8104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.751+0000 7efede098700 1 --2- 172.21.15.46:0/1639340827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8104650 0x7efed8104a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efec80099c0 tx=0x7efec8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.751+0000 7efede098700 1 -- 172.21.15.46:0/1639340827 shutdown_connections 2024-05-09T00:10:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.751+0000 7efede098700 1 --2- 172.21.15.46:0/1639340827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efed8109880 0x7efed81111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.751+0000 7efede098700 1 --2- 172.21.15.46:0/1639340827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8104650 0x7efed8104a60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.751+0000 7efede098700 1 --2- 172.21.15.46:0/1639340827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efed8108f30 0x7efed8109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.751+0000 7efede098700 1 -- 172.21.15.46:0/1639340827 >> 172.21.15.46:0/1639340827 conn(0x7efed8079f00 msgr2=0x7efed807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.751+0000 7efede098700 1 -- 172.21.15.46:0/1639340827 shutdown_connections 2024-05-09T00:10:09.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.751+0000 7efede098700 1 -- 172.21.15.46:0/1639340827 wait complete. 2024-05-09T00:10:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.752+0000 7efede098700 1 Processor -- start 2024-05-09T00:10:09.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.752+0000 7efede098700 1 -- start start 2024-05-09T00:10:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.752+0000 7efede098700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efed8104650 0x7efed819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.752+0000 7efede098700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efed8108f30 0x7efed819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.752+0000 7efede098700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8109880 0x7efed819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.752+0000 7efede098700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efed81144b0 con 0x7efed8109880 2024-05-09T00:10:09.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.752+0000 7efede098700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efed8114330 con 0x7efed8108f30 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.752+0000 7efede098700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efed8114630 con 0x7efed8104650 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8109880 0x7efed819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efeceffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efed8108f30 0x7efed819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8109880 0x7efed819f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47454/0 (socket says 172.21.15.46:47454) 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efeceffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efed8108f30 0x7efed819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:56852/0 (socket says 172.21.15.46:56852) 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed7fff700 1 -- 172.21.15.46:0/513625032 learned_addr learned my addr 172.21.15.46:0/513625032 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed77fe700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efed8104650 0x7efed819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed7fff700 1 -- 172.21.15.46:0/513625032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efed8104650 msgr2=0x7efed819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed7fff700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efed8104650 0x7efed819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed7fff700 1 -- 172.21.15.46:0/513625032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efed8108f30 msgr2=0x7efed819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed7fff700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efed8108f30 0x7efed819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed7fff700 1 -- 172.21.15.46:0/513625032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed819fc40 con 0x7efed8109880 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed7fff700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8109880 0x7efed819f650 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efed0007d70 tx=0x7efed000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed57fa700 1 -- 172.21.15.46:0/513625032 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0016020 con 0x7efed8109880 2024-05-09T00:10:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed57fa700 1 -- 172.21.15.46:0/513625032 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed000ac90 con 0x7efed8109880 2024-05-09T00:10:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efede098700 1 -- 172.21.15.46:0/513625032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed819ff30 con 0x7efed8109880 2024-05-09T00:10:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efed57fa700 1 -- 172.21.15.46:0/513625032 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed00115e0 con 0x7efed8109880 2024-05-09T00:10:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.753+0000 7efede098700 1 -- 172.21.15.46:0/513625032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed81a0470 con 0x7efed8109880 2024-05-09T00:10:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.754+0000 7efede098700 1 -- 172.21.15.46:0/513625032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efed8063f50 con 0x7efed8109880 2024-05-09T00:10:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.756+0000 7efed57fa700 1 -- 172.21.15.46:0/513625032 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efed0019020 con 0x7efed8109880 2024-05-09T00:10:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.757+0000 7efed57fa700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7efec40b2900 0x7efec40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.757+0000 7efed77fe700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7efec40b2900 0x7efec40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.757+0000 7efed57fa700 1 -- 172.21.15.46:0/513625032 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7efed00e2660 con 0x7efed8109880 2024-05-09T00:10:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.757+0000 7efed57fa700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261] conn(0x7efec40bc510 0x7efec40bea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.757+0000 7efed57fa700 1 -- 172.21.15.46:0/513625032 --> [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efed0017ec0 con 0x7efec40bc510 2024-05-09T00:10:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.757+0000 7efed57fa700 1 -- 172.21.15.46:0/513625032 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7efed00e2a10 con 0x7efed8109880 2024-05-09T00:10:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.757+0000 7efeceffd700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261] conn(0x7efec40bc510 0x7efec40bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.757+0000 7efed77fe700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7efec40b2900 0x7efec40b4e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efec8009990 tx=0x7efec8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.758+0000 7efeceffd700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261] conn(0x7efec40bc510 0x7efec40bea00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.758+0000 7efed57fa700 1 -- 172.21.15.46:0/513625032 <== osd.3 v2:172.21.15.46:6810/1158536261 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efed0017ec0 con 0x7efec40bc510 2024-05-09T00:10:09.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.775+0000 7efede098700 1 -- 172.21.15.46:0/513625032 --> [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efed8064350 con 0x7efec40bc510 2024-05-09T00:10:09.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.775+0000 7efed57fa700 1 -- 172.21.15.46:0/513625032 <== osd.3 v2:172.21.15.46:6810/1158536261 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efed8064350 con 0x7efec40bc510 2024-05-09T00:10:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.776+0000 7efede098700 1 -- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261] conn(0x7efec40bc510 msgr2=0x7efec40bea00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.776+0000 7efede098700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261] conn(0x7efec40bc510 0x7efec40bea00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.776+0000 7efede098700 1 -- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7efec40b2900 msgr2=0x7efec40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.776+0000 7efede098700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7efec40b2900 0x7efec40b4e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efec8009990 tx=0x7efec8000bc0 comp rx=0 tx=0).stop 2024-05-09T00:10:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.776+0000 7efede098700 1 -- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8109880 msgr2=0x7efed819f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.776+0000 7efede098700 1 --2- 172.21.15.46:0/513625032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8109880 0x7efed819f650 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efed0007d70 tx=0x7efed000a340 comp rx=0 tx=0).stop 2024-05-09T00:10:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.776+0000 7efed7fff700 1 -- 172.21.15.46:0/513625032 reap_dead start 2024-05-09T00:10:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.776+0000 7efede098700 1 -- 172.21.15.46:0/513625032 shutdown_connections 2024-05-09T00:10:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.776+0000 7efede098700 1 -- 172.21.15.46:0/513625032 >> 172.21.15.46:0/513625032 conn(0x7efed8079f00 msgr2=0x7efed81020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.776+0000 7efede098700 1 -- 172.21.15.46:0/513625032 shutdown_connections 2024-05-09T00:10:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:09.776+0000 7efede098700 1 -- 172.21.15.46:0/513625032 wait complete. 2024-05-09T00:10:10.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.080+0000 7f978660d700 1 -- 172.21.15.46:0/3152670762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97801109a0 msgr2=0x7f9780110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:10.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.080+0000 7f978660d700 1 --2- 172.21.15.46:0/3152670762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97801109a0 0x7f9780110db0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9770009990 tx=0x7f9770009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:10.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.080+0000 7f978660d700 1 -- 172.21.15.46:0/3152670762 shutdown_connections 2024-05-09T00:10:10.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.080+0000 7f978660d700 1 --2- 172.21.15.46:0/3152670762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f978010b2c0 0x7f9780117840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:10.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.080+0000 7f978660d700 1 --2- 172.21.15.46:0/3152670762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97801109a0 0x7f9780110db0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:10.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.080+0000 7f978660d700 1 --2- 172.21.15.46:0/3152670762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f978010a970 0x7f978010ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:10.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.080+0000 7f978660d700 1 -- 172.21.15.46:0/3152670762 >> 172.21.15.46:0/3152670762 conn(0x7f9780076cc0 msgr2=0x7f97800770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:10.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.081+0000 7f978660d700 1 -- 172.21.15.46:0/3152670762 shutdown_connections 2024-05-09T00:10:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.082+0000 7f978660d700 1 -- 172.21.15.46:0/3152670762 wait complete. 2024-05-09T00:10:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.083+0000 7f978660d700 1 Processor -- start 2024-05-09T00:10:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.083+0000 7f978660d700 1 -- start start 2024-05-09T00:10:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.084+0000 7f978660d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f978010a970 0x7f97801a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.084+0000 7f978660d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f978010b2c0 0x7f97801a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.084+0000 7f978660d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97801109a0 0x7f97801a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.084+0000 7f978660d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f978011aae0 con 0x7f978010b2c0 2024-05-09T00:10:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.084+0000 7f978660d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f978011a960 con 0x7f978010a970 2024-05-09T00:10:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.084+0000 7f978660d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f978011ac60 con 0x7f97801109a0 2024-05-09T00:10:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.085+0000 7f977f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f978010b2c0 0x7f97801a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.085+0000 7f977f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f978010b2c0 0x7f97801a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47476/0 (socket says 172.21.15.46:47476) 2024-05-09T00:10:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.085+0000 7f977f7fe700 1 -- 172.21.15.46:0/3402070175 learned_addr learned my addr 172.21.15.46:0/3402070175 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.085+0000 7f9784baa700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97801109a0 0x7f97801a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.085+0000 7f977f7fe700 1 -- 172.21.15.46:0/3402070175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97801109a0 msgr2=0x7f97801a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.085+0000 7f977f7fe700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97801109a0 0x7f97801a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.085+0000 7f977f7fe700 1 -- 172.21.15.46:0/3402070175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f978010a970 msgr2=0x7f97801a1410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:10:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.085+0000 7f977f7fe700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f978010a970 0x7f97801a1410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.085+0000 7f977f7fe700 1 -- 172.21.15.46:0/3402070175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97801a62e0 con 0x7f978010b2c0 2024-05-09T00:10:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.086+0000 7f977f7fe700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f978010b2c0 0x7f97801a19b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f977800efc0 tx=0x7f977800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.086+0000 7f977d7fa700 1 -- 172.21.15.46:0/3402070175 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977800ca70 con 0x7f978010b2c0 2024-05-09T00:10:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.086+0000 7f977d7fa700 1 -- 172.21.15.46:0/3402070175 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9778019070 con 0x7f978010b2c0 2024-05-09T00:10:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.086+0000 7f977d7fa700 1 -- 172.21.15.46:0/3402070175 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97780146e0 con 0x7f978010b2c0 2024-05-09T00:10:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.086+0000 7f978660d700 1 -- 172.21.15.46:0/3402070175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97801a6570 con 0x7f978010b2c0 2024-05-09T00:10:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.087+0000 7f978660d700 1 -- 172.21.15.46:0/3402070175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97801a6a60 con 0x7f978010b2c0 2024-05-09T00:10:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.088+0000 7f978660d700 1 -- 172.21.15.46:0/3402070175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f976c001030 con 0x7f978010b2c0 2024-05-09T00:10:10.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.091+0000 7f977d7fa700 1 -- 172.21.15.46:0/3402070175 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9778012050 con 0x7f978010b2c0 2024-05-09T00:10:10.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.092+0000 7f977d7fa700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97680b29d0 0x7f97680b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.093+0000 7f977ffff700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97680b29d0 0x7f97680b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.093+0000 7f977d7fa700 1 -- 172.21.15.46:0/3402070175 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f97780e20a0 con 0x7f978010b2c0 2024-05-09T00:10:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.093+0000 7f977d7fa700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248] conn(0x7f97680bc5e0 0x7f97680bead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.093+0000 7f977d7fa700 1 -- 172.21.15.46:0/3402070175 --> [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97680bf1a0 con 0x7f97680bc5e0 2024-05-09T00:10:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.093+0000 7f977d7fa700 1 -- 172.21.15.46:0/3402070175 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f97780e9050 con 0x7f978010b2c0 2024-05-09T00:10:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.094+0000 7f977ffff700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97680b29d0 0x7f97680b4ee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f97700059d0 tx=0x7f9770005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.094+0000 7f9784baa700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248] conn(0x7f97680bc5e0 0x7f97680bead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.094+0000 7f9784baa700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248] conn(0x7f97680bc5e0 0x7f97680bead0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.096+0000 7f977d7fa700 1 -- 172.21.15.46:0/3402070175 <== osd.9 v2:172.21.15.46:6826/374861248 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f97680bf1a0 con 0x7f97680bc5e0 2024-05-09T00:10:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.127+0000 7f978660d700 1 -- 172.21.15.46:0/3402070175 --> [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f976c002e60 con 0x7f97680bc5e0 2024-05-09T00:10:10.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.128+0000 7f977d7fa700 1 -- 172.21.15.46:0/3402070175 <== osd.9 v2:172.21.15.46:6826/374861248 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f976c002e60 con 0x7f97680bc5e0 2024-05-09T00:10:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.128+0000 7f978660d700 1 -- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248] conn(0x7f97680bc5e0 msgr2=0x7f97680bead0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.128+0000 7f978660d700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248] conn(0x7f97680bc5e0 0x7f97680bead0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.128+0000 7f978660d700 1 -- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97680b29d0 msgr2=0x7f97680b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.128+0000 7f978660d700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97680b29d0 0x7f97680b4ee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f97700059d0 tx=0x7f9770005900 comp rx=0 tx=0).stop 2024-05-09T00:10:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.128+0000 7f978660d700 1 -- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f978010b2c0 msgr2=0x7f97801a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.128+0000 7f978660d700 1 --2- 172.21.15.46:0/3402070175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f978010b2c0 0x7f97801a19b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f977800efc0 tx=0x7f977800c3a0 comp rx=0 tx=0).stop 2024-05-09T00:10:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.128+0000 7f9784baa700 1 -- 172.21.15.46:0/3402070175 reap_dead start 2024-05-09T00:10:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.128+0000 7f978660d700 1 -- 172.21.15.46:0/3402070175 shutdown_connections 2024-05-09T00:10:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.128+0000 7f978660d700 1 -- 172.21.15.46:0/3402070175 >> 172.21.15.46:0/3402070175 conn(0x7f9780076cc0 msgr2=0x7f9780109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.129+0000 7f978660d700 1 -- 172.21.15.46:0/3402070175 shutdown_connections 2024-05-09T00:10:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:10.129+0000 7f978660d700 1 -- 172.21.15.46:0/3402070175 wait complete. 2024-05-09T00:10:10.637 INFO:teuthology.orchestra.run.smithi046.stdout:201863462968 2024-05-09T00:10:10.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-09T00:10:10.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:10 smithi142 ceph-mon[34533]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:10.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:10 smithi142 ceph-mon[34533]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T00:10:10.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:10 smithi046 ceph-mon[30151]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:10.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:10 smithi046 ceph-mon[30151]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T00:10:11.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:10 smithi169 ceph-mon[34777]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:11.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:10 smithi169 ceph-mon[34777]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T00:10:11.351 INFO:teuthology.orchestra.run.smithi046.stdout:107374182478 2024-05-09T00:10:11.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-09T00:10:11.569 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:11.577 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:12.059 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:12.096 INFO:teuthology.orchestra.run.smithi046.stdout:253403070510 2024-05-09T00:10:12.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-09T00:10:12.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:12 smithi142 ceph-mon[34533]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-09T00:10:13.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:12 smithi046 ceph-mon[30151]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-09T00:10:13.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:12 smithi169 ceph-mon[34777]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-09T00:10:13.708 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:13.708 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:14.137 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:14.887 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:14 smithi046 ceph-mon[30151]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-09T00:10:14.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:14 smithi142 ceph-mon[34533]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-09T00:10:15.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:14 smithi169 ceph-mon[34777]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-09T00:10:16.728 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:16 smithi046 ceph-mon[30151]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T00:10:16.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:16 smithi142 ceph-mon[34533]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T00:10:17.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:16 smithi169 ceph-mon[34777]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T00:10:17.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.747+0000 7f02afc16700 1 -- 172.21.15.46:0/263324970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a81034e0 msgr2=0x7f02a81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:17.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.747+0000 7f02afc16700 1 --2- 172.21.15.46:0/263324970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a81034e0 0x7f02a81038f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f02a40099c0 tx=0x7f02a4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:17.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.747+0000 7f02afc16700 1 -- 172.21.15.46:0/263324970 shutdown_connections 2024-05-09T00:10:17.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.747+0000 7f02afc16700 1 --2- 172.21.15.46:0/263324970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02a8108760 0x7f02a81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:17.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.747+0000 7f02afc16700 1 --2- 172.21.15.46:0/263324970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a81034e0 0x7f02a81038f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:17.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.747+0000 7f02afc16700 1 --2- 172.21.15.46:0/263324970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a8107e10 0x7f02a8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:17.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.747+0000 7f02afc16700 1 -- 172.21.15.46:0/263324970 >> 172.21.15.46:0/263324970 conn(0x7f02a80fef30 msgr2=0x7f02a8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:17.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.747+0000 7f02afc16700 1 -- 172.21.15.46:0/263324970 shutdown_connections 2024-05-09T00:10:17.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.748+0000 7f02afc16700 1 -- 172.21.15.46:0/263324970 wait complete. 2024-05-09T00:10:17.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.748+0000 7f02afc16700 1 Processor -- start 2024-05-09T00:10:17.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.748+0000 7f02afc16700 1 -- start start 2024-05-09T00:10:17.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02afc16700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a81034e0 0x7f02a819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:17.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02afc16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8107e10 0x7f02a819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:17.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02afc16700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02a8108760 0x7f02a819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:17.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02afc16700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02a8113390 con 0x7f02a8107e10 2024-05-09T00:10:17.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02afc16700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02a8113210 con 0x7f02a8108760 2024-05-09T00:10:17.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02afc16700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02a8113510 con 0x7f02a81034e0 2024-05-09T00:10:17.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02ad1b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8107e10 0x7f02a819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:17.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02ad1b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8107e10 0x7f02a819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37596/0 (socket says 172.21.15.46:37596) 2024-05-09T00:10:17.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02ad1b1700 1 -- 172.21.15.46:0/2192196088 learned_addr learned my addr 172.21.15.46:0/2192196088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02ae1b3700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02a8108760 0x7f02a819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02ad9b2700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a81034e0 0x7f02a819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02ad1b1700 1 -- 172.21.15.46:0/2192196088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a81034e0 msgr2=0x7f02a819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02ad1b1700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a81034e0 0x7f02a819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02ad1b1700 1 -- 172.21.15.46:0/2192196088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02a8108760 msgr2=0x7f02a819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02ad1b1700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02a8108760 0x7f02a819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02ad1b1700 1 -- 172.21.15.46:0/2192196088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a819fd20 con 0x7f02a8107e10 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.749+0000 7f02ad1b1700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8107e10 0x7f02a819b3f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f029800e970 tx=0x7f029800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.750+0000 7f029effd700 1 -- 172.21.15.46:0/2192196088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298004a30 con 0x7f02a8107e10 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.750+0000 7f02afc16700 1 -- 172.21.15.46:0/2192196088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a81a0010 con 0x7f02a8107e10 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.750+0000 7f029effd700 1 -- 172.21.15.46:0/2192196088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f029800cc90 con 0x7f02a8107e10 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.750+0000 7f029effd700 1 -- 172.21.15.46:0/2192196088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298005610 con 0x7f02a8107e10 2024-05-09T00:10:17.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.750+0000 7f02afc16700 1 -- 172.21.15.46:0/2192196088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a81a7930 con 0x7f02a8107e10 2024-05-09T00:10:17.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.751+0000 7f02afc16700 1 -- 172.21.15.46:0/2192196088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a80512c0 con 0x7f02a8107e10 2024-05-09T00:10:17.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.752+0000 7f029effd700 1 -- 172.21.15.46:0/2192196088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f029800c7e0 con 0x7f02a8107e10 2024-05-09T00:10:17.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.753+0000 7f029effd700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f02940b2880 0x7f02940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:17.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.753+0000 7f02ad9b2700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f02940b2880 0x7f02940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:17.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.753+0000 7f029effd700 1 -- 172.21.15.46:0/2192196088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f02980e2390 con 0x7f02a8107e10 2024-05-09T00:10:17.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.753+0000 7f02ad9b2700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f02940b2880 0x7f02940b4d90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f02a4000c00 tx=0x7f02a401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:17.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.758+0000 7f029effd700 1 -- 172.21.15.46:0/2192196088 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02980aa0b0 con 0x7f02a8107e10 2024-05-09T00:10:17.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.974+0000 7f02afc16700 1 -- 172.21.15.46:0/2192196088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f02a819c1b0 con 0x7f02a8107e10 2024-05-09T00:10:17.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.975+0000 7f029effd700 1 -- 172.21.15.46:0/2192196088 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f02980a99b0 con 0x7f02a8107e10 2024-05-09T00:10:17.972 INFO:teuthology.orchestra.run.smithi046.stdout:300647710757 2024-05-09T00:10:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.977+0000 7f02afc16700 1 -- 172.21.15.46:0/2192196088 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f02940b2880 msgr2=0x7f02940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.977+0000 7f02afc16700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f02940b2880 0x7f02940b4d90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f02a4000c00 tx=0x7f02a401a040 comp rx=0 tx=0).stop 2024-05-09T00:10:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.977+0000 7f02afc16700 1 -- 172.21.15.46:0/2192196088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8107e10 msgr2=0x7f02a819b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.977+0000 7f02afc16700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8107e10 0x7f02a819b3f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f029800e970 tx=0x7f029800ec80 comp rx=0 tx=0).stop 2024-05-09T00:10:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.977+0000 7f02afc16700 1 -- 172.21.15.46:0/2192196088 shutdown_connections 2024-05-09T00:10:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.977+0000 7f02afc16700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f02940b2880 0x7f02940b4d90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.977+0000 7f02afc16700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02a8108760 0x7f02a819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.977+0000 7f02afc16700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a81034e0 0x7f02a819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.977+0000 7f02afc16700 1 --2- 172.21.15.46:0/2192196088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a8107e10 0x7f02a819b3f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:17.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.977+0000 7f02afc16700 1 -- 172.21.15.46:0/2192196088 >> 172.21.15.46:0/2192196088 conn(0x7f02a80fef30 msgr2=0x7f02a81063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:17.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.978+0000 7f02afc16700 1 -- 172.21.15.46:0/2192196088 shutdown_connections 2024-05-09T00:10:17.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:17.978+0000 7f02afc16700 1 -- 172.21.15.46:0/2192196088 wait complete. 2024-05-09T00:10:18.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:18 smithi046 ceph-mon[30151]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T00:10:18.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:18 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2192196088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T00:10:18.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:18 smithi142 ceph-mon[34533]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T00:10:18.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:18 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2192196088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T00:10:19.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:18 smithi169 ceph-mon[34777]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T00:10:19.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:18 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2192196088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T00:10:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.651+0000 7f176d175700 1 -- 172.21.15.46:0/126243477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1768107df0 msgr2=0x7f1768108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.651+0000 7f176d175700 1 --2- 172.21.15.46:0/126243477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1768107df0 0x7f1768108200 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f175c0099c0 tx=0x7f175c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.651+0000 7f176d175700 1 -- 172.21.15.46:0/126243477 shutdown_connections 2024-05-09T00:10:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.651+0000 7f176d175700 1 --2- 172.21.15.46:0/126243477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1768108740 0x7f17681100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.651+0000 7f176d175700 1 --2- 172.21.15.46:0/126243477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17681034c0 0x7f17681038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.651+0000 7f176d175700 1 --2- 172.21.15.46:0/126243477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1768107df0 0x7f1768108200 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.651+0000 7f176d175700 1 -- 172.21.15.46:0/126243477 >> 172.21.15.46:0/126243477 conn(0x7f17680fedc0 msgr2=0x7f17681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.651+0000 7f176d175700 1 -- 172.21.15.46:0/126243477 shutdown_connections 2024-05-09T00:10:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.651+0000 7f176d175700 1 -- 172.21.15.46:0/126243477 wait complete. 2024-05-09T00:10:19.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.652+0000 7f176d175700 1 Processor -- start 2024-05-09T00:10:19.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.652+0000 7f176d175700 1 -- start start 2024-05-09T00:10:19.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.652+0000 7f176d175700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17681034c0 0x7f176819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.652+0000 7f176d175700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1768107df0 0x7f176819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.652+0000 7f176d175700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1768108740 0x7f176819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.652+0000 7f176d175700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1768113350 con 0x7f17681034c0 2024-05-09T00:10:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.652+0000 7f176d175700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f17681131d0 con 0x7f1768107df0 2024-05-09T00:10:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.652+0000 7f176d175700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17681134d0 con 0x7f1768108740 2024-05-09T00:10:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.652+0000 7f1766d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17681034c0 0x7f176819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.652+0000 7f1766d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17681034c0 0x7f176819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37640/0 (socket says 172.21.15.46:37640) 2024-05-09T00:10:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.652+0000 7f1766d9d700 1 -- 172.21.15.46:0/2916284837 learned_addr learned my addr 172.21.15.46:0/2916284837 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.652+0000 7f176659c700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1768107df0 0x7f176819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.653+0000 7f176759e700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1768108740 0x7f176819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.653+0000 7f1766d9d700 1 -- 172.21.15.46:0/2916284837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1768108740 msgr2=0x7f176819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.653+0000 7f1766d9d700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1768108740 0x7f176819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.653+0000 7f1766d9d700 1 -- 172.21.15.46:0/2916284837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1768107df0 msgr2=0x7f176819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.653+0000 7f1766d9d700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1768107df0 0x7f176819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.653+0000 7f1766d9d700 1 -- 172.21.15.46:0/2916284837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176819fca0 con 0x7f17681034c0 2024-05-09T00:10:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.653+0000 7f1766d9d700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17681034c0 0x7f176819add0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f175800d950 tx=0x7f175800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.653+0000 7f174ffff700 1 -- 172.21.15.46:0/2916284837 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758016070 con 0x7f17681034c0 2024-05-09T00:10:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.653+0000 7f176d175700 1 -- 172.21.15.46:0/2916284837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176819ff90 con 0x7f17681034c0 2024-05-09T00:10:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.653+0000 7f174ffff700 1 -- 172.21.15.46:0/2916284837 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1758009df0 con 0x7f17681034c0 2024-05-09T00:10:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.653+0000 7f174ffff700 1 -- 172.21.15.46:0/2916284837 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175800f3c0 con 0x7f17681034c0 2024-05-09T00:10:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.653+0000 7f176d175700 1 -- 172.21.15.46:0/2916284837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17681a78b0 con 0x7f17681034c0 2024-05-09T00:10:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.654+0000 7f176d175700 1 -- 172.21.15.46:0/2916284837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17680512c0 con 0x7f17681034c0 2024-05-09T00:10:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.656+0000 7f174ffff700 1 -- 172.21.15.46:0/2916284837 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1758019070 con 0x7f17681034c0 2024-05-09T00:10:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.656+0000 7f174ffff700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17500b2880 0x7f17500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.657+0000 7f176659c700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17500b2880 0x7f17500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.657+0000 7f174ffff700 1 -- 172.21.15.46:0/2916284837 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f17580e1e00 con 0x7f17681034c0 2024-05-09T00:10:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.657+0000 7f176659c700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17500b2880 0x7f17500b4d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f175c006010 tx=0x7f175c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:19.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.660+0000 7f174ffff700 1 -- 172.21.15.46:0/2916284837 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f17580a9b20 con 0x7f17681034c0 2024-05-09T00:10:19.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:19 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/435824864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T00:10:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.883+0000 7f176d175700 1 -- 172.21.15.46:0/2916284837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f176802e5f0 con 0x7f17681034c0 2024-05-09T00:10:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.883+0000 7f174ffff700 1 -- 172.21.15.46:0/2916284837 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f17580a9420 con 0x7f17681034c0 2024-05-09T00:10:19.880 INFO:teuthology.orchestra.run.smithi046.stdout:133143986251 2024-05-09T00:10:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.885+0000 7f176d175700 1 -- 172.21.15.46:0/2916284837 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17500b2880 msgr2=0x7f17500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.885+0000 7f176d175700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17500b2880 0x7f17500b4d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f175c006010 tx=0x7f175c000bc0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.885+0000 7f176d175700 1 -- 172.21.15.46:0/2916284837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17681034c0 msgr2=0x7f176819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.885+0000 7f176d175700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17681034c0 0x7f176819add0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f175800d950 tx=0x7f175800dc60 comp rx=0 tx=0).stop 2024-05-09T00:10:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.885+0000 7f176d175700 1 -- 172.21.15.46:0/2916284837 shutdown_connections 2024-05-09T00:10:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.885+0000 7f176d175700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17500b2880 0x7f17500b4d90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.885+0000 7f176d175700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1768108740 0x7f176819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.885+0000 7f176d175700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17681034c0 0x7f176819add0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.885+0000 7f176d175700 1 --2- 172.21.15.46:0/2916284837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1768107df0 0x7f176819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.885+0000 7f176d175700 1 -- 172.21.15.46:0/2916284837 >> 172.21.15.46:0/2916284837 conn(0x7f17680fedc0 msgr2=0x7f176806b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.886+0000 7f176d175700 1 -- 172.21.15.46:0/2916284837 shutdown_connections 2024-05-09T00:10:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.886+0000 7f176d175700 1 -- 172.21.15.46:0/2916284837 wait complete. 2024-05-09T00:10:19.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:19 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/435824864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T00:10:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.155+0000 7f173725e700 1 -- 172.21.15.46:0/3299396827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 msgr2=0x7f17301111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.155+0000 7f173725e700 1 --2- 172.21.15.46:0/3299396827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f17301111d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f172800b0d0 tx=0x7f172800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.155+0000 7f173725e700 1 -- 172.21.15.46:0/3299396827 shutdown_connections 2024-05-09T00:10:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.155+0000 7f173725e700 1 --2- 172.21.15.46:0/3299396827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1730109870 0x7f17301111d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.155+0000 7f173725e700 1 --2- 172.21.15.46:0/3299396827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17301045f0 0x7f1730104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.155+0000 7f173725e700 1 --2- 172.21.15.46:0/3299396827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1730108f20 0x7f1730109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.155+0000 7f173725e700 1 -- 172.21.15.46:0/3299396827 >> 172.21.15.46:0/3299396827 conn(0x7f1730100040 msgr2=0x7f1730102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.156+0000 7f173725e700 1 -- 172.21.15.46:0/3299396827 shutdown_connections 2024-05-09T00:10:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.156+0000 7f173725e700 1 -- 172.21.15.46:0/3299396827 wait complete. 2024-05-09T00:10:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.156+0000 7f173725e700 1 Processor -- start 2024-05-09T00:10:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.156+0000 7f173725e700 1 -- start start 2024-05-09T00:10:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f173725e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17301045f0 0x7f173019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f173725e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1730108f20 0x7f173019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f173725e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1730109870 0x7f173019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f173725e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17301144a0 con 0x7f17301045f0 2024-05-09T00:10:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f173725e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1730114320 con 0x7f1730108f20 2024-05-09T00:10:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f173725e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1730114620 con 0x7f1730109870 2024-05-09T00:10:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f1734ffa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17301045f0 0x7f173019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f1734ffa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17301045f0 0x7f173019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37618/0 (socket says 172.21.15.46:37618) 2024-05-09T00:10:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f1734ffa700 1 -- 172.21.15.46:0/435824864 learned_addr learned my addr 172.21.15.46:0/435824864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f1727fff700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1730108f20 0x7f173019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f17357fb700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1730109870 0x7f173019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f1734ffa700 1 -- 172.21.15.46:0/435824864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1730109870 msgr2=0x7f173019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f1734ffa700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1730109870 0x7f173019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f1734ffa700 1 -- 172.21.15.46:0/435824864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1730108f20 msgr2=0x7f173019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f1734ffa700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1730108f20 0x7f173019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f1734ffa700 1 -- 172.21.15.46:0/435824864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173019fbd0 con 0x7f17301045f0 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.157+0000 7f1734ffa700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17301045f0 0x7f173019adb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f172c00d980 tx=0x7f172c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.158+0000 7f1725ffb700 1 -- 172.21.15.46:0/435824864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c017030 con 0x7f17301045f0 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.158+0000 7f1725ffb700 1 -- 172.21.15.46:0/435824864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f172c004ce0 con 0x7f17301045f0 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.158+0000 7f173725e700 1 -- 172.21.15.46:0/435824864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173019fec0 con 0x7f17301045f0 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.159+0000 7f1725ffb700 1 -- 172.21.15.46:0/435824864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c00f8f0 con 0x7f17301045f0 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.159+0000 7f173725e700 1 -- 172.21.15.46:0/435824864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17301a0400 con 0x7f17301045f0 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.160+0000 7f173725e700 1 -- 172.21.15.46:0/435824864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17300512c0 con 0x7f17301045f0 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.161+0000 7f1725ffb700 1 -- 172.21.15.46:0/435824864 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f172c01a070 con 0x7f17301045f0 2024-05-09T00:10:19.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.161+0000 7f1725ffb700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17180b2900 0x7f17180b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:19.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.162+0000 7f1727fff700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17180b2900 0x7f17180b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:19.962 INFO:teuthology.orchestra.run.smithi046.stdout:55834574937 2024-05-09T00:10:19.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.162+0000 7f1725ffb700 1 -- 172.21.15.46:0/435824864 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f172c0e1a80 con 0x7f17301045f0 2024-05-09T00:10:19.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.162+0000 7f1727fff700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17180b2900 0x7f17180b4e10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1730103990 tx=0x7f172000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:19.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.166+0000 7f1725ffb700 1 -- 172.21.15.46:0/435824864 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f172c0a97a0 con 0x7f17301045f0 2024-05-09T00:10:19.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.381+0000 7f173725e700 1 -- 172.21.15.46:0/435824864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f173019c110 con 0x7f17301045f0 2024-05-09T00:10:19.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.382+0000 7f1725ffb700 1 -- 172.21.15.46:0/435824864 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f172c0a90a0 con 0x7f17301045f0 2024-05-09T00:10:19.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.384+0000 7f173725e700 1 -- 172.21.15.46:0/435824864 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17180b2900 msgr2=0x7f17180b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:19.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.384+0000 7f173725e700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17180b2900 0x7f17180b4e10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1730103990 tx=0x7f172000a380 comp rx=0 tx=0).stop 2024-05-09T00:10:19.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.384+0000 7f173725e700 1 -- 172.21.15.46:0/435824864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17301045f0 msgr2=0x7f173019adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:19.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.384+0000 7f173725e700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17301045f0 0x7f173019adb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f172c00d980 tx=0x7f172c00dc90 comp rx=0 tx=0).stop 2024-05-09T00:10:19.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.384+0000 7f173725e700 1 -- 172.21.15.46:0/435824864 shutdown_connections 2024-05-09T00:10:19.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.384+0000 7f173725e700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17180b2900 0x7f17180b4e10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.384+0000 7f173725e700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1730109870 0x7f173019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.384+0000 7f173725e700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17301045f0 0x7f173019adb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.384+0000 7f173725e700 1 --2- 172.21.15.46:0/435824864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1730108f20 0x7f173019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:19.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.384+0000 7f173725e700 1 -- 172.21.15.46:0/435824864 >> 172.21.15.46:0/435824864 conn(0x7f1730100040 msgr2=0x7f1730108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:19.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.384+0000 7f173725e700 1 -- 172.21.15.46:0/435824864 shutdown_connections 2024-05-09T00:10:19.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:19.384+0000 7f173725e700 1 -- 172.21.15.46:0/435824864 wait complete. 2024-05-09T00:10:20.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:19 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/435824864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T00:10:20.554 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574932 got 55834574937 for osd.1 2024-05-09T00:10:20.555 DEBUG:teuthology.parallel:result is None 2024-05-09T00:10:20.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:20 smithi046 ceph-mon[30151]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T00:10:20.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:20 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2916284837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T00:10:20.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:20 smithi142 ceph-mon[34533]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T00:10:20.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:20 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2916284837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T00:10:21.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:20 smithi169 ceph-mon[34777]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T00:10:21.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:20 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2916284837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T00:10:22.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:22 smithi046 ceph-mon[30151]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-09T00:10:22.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:22 smithi142 ceph-mon[34533]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-09T00:10:23.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:22 smithi169 ceph-mon[34777]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-09T00:10:23.242 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710753 got 300647710757 for osd.11 2024-05-09T00:10:23.242 DEBUG:teuthology.parallel:result is None 2024-05-09T00:10:24.024 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986248 got 133143986251 for osd.4 2024-05-09T00:10:24.024 DEBUG:teuthology.parallel:result is None 2024-05-09T00:10:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.699+0000 7f949cb92700 1 -- 172.21.15.46:0/1118728583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9498109850 msgr2=0x7f94981111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.699+0000 7f949cb92700 1 --2- 172.21.15.46:0/1118728583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9498109850 0x7f94981111b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f949000b0d0 tx=0x7f949000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:10:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.699+0000 7f949cb92700 1 -- 172.21.15.46:0/1118728583 shutdown_connections 2024-05-09T00:10:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.699+0000 7f949cb92700 1 --2- 172.21.15.46:0/1118728583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9498109850 0x7f94981111b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.699+0000 7f949cb92700 1 --2- 172.21.15.46:0/1118728583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94981045d0 0x7f94981049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.699+0000 7f949cb92700 1 --2- 172.21.15.46:0/1118728583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9498108f00 0x7f9498109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.699+0000 7f949cb92700 1 -- 172.21.15.46:0/1118728583 >> 172.21.15.46:0/1118728583 conn(0x7f94980ffea0 msgr2=0x7f94981022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.699+0000 7f949cb92700 1 -- 172.21.15.46:0/1118728583 shutdown_connections 2024-05-09T00:10:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.699+0000 7f949cb92700 1 -- 172.21.15.46:0/1118728583 wait complete. 2024-05-09T00:10:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f949cb92700 1 Processor -- start 2024-05-09T00:10:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f949cb92700 1 -- start start 2024-05-09T00:10:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f949cb92700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94981045d0 0x7f949819af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f949cb92700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9498108f00 0x7f949819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f949cb92700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9498109850 0x7f949819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f949cb92700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f94981145c0 con 0x7f9498109850 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f949cb92700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9498114440 con 0x7f94981045d0 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f949cb92700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9498114740 con 0x7f9498108f00 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f9496d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9498109850 0x7f949819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f949659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94981045d0 0x7f949819af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f9496d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9498109850 0x7f949819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37666/0 (socket says 172.21.15.46:37666) 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f949659c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94981045d0 0x7f949819af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:33336/0 (socket says 172.21.15.46:33336) 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f9496d9d700 1 -- 172.21.15.46:0/2105906116 learned_addr learned my addr 172.21.15.46:0/2105906116 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.700+0000 7f9495d9b700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9498108f00 0x7f949819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.701+0000 7f9496d9d700 1 -- 172.21.15.46:0/2105906116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9498108f00 msgr2=0x7f949819b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.701+0000 7f9496d9d700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9498108f00 0x7f949819b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.701+0000 7f9496d9d700 1 -- 172.21.15.46:0/2105906116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94981045d0 msgr2=0x7f949819af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.701+0000 7f9496d9d700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94981045d0 0x7f949819af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.701+0000 7f9496d9d700 1 -- 172.21.15.46:0/2105906116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949819fe40 con 0x7f9498109850 2024-05-09T00:10:24.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.701+0000 7f9496d9d700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9498109850 0x7f949819f7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f94900079a0 tx=0x7f94900079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:24.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.701+0000 7f94877fe700 1 -- 172.21.15.46:0/2105906116 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949001f070 con 0x7f9498109850 2024-05-09T00:10:24.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.701+0000 7f949cb92700 1 -- 172.21.15.46:0/2105906116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94981a00d0 con 0x7f9498109850 2024-05-09T00:10:24.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.701+0000 7f94877fe700 1 -- 172.21.15.46:0/2105906116 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9490005070 con 0x7f9498109850 2024-05-09T00:10:24.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.701+0000 7f94877fe700 1 -- 172.21.15.46:0/2105906116 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949001b680 con 0x7f9498109850 2024-05-09T00:10:24.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.701+0000 7f949cb92700 1 -- 172.21.15.46:0/2105906116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94981a7a50 con 0x7f9498109850 2024-05-09T00:10:24.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.702+0000 7f949cb92700 1 -- 172.21.15.46:0/2105906116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9498105a80 con 0x7f9498109850 2024-05-09T00:10:24.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.704+0000 7f94877fe700 1 -- 172.21.15.46:0/2105906116 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9490022070 con 0x7f9498109850 2024-05-09T00:10:24.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.704+0000 7f94877fe700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94800b2880 0x7f94800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:24.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.705+0000 7f949659c700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94800b2880 0x7f94800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:24.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.705+0000 7f94877fe700 1 -- 172.21.15.46:0/2105906116 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f949000f020 con 0x7f9498109850 2024-05-09T00:10:24.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.705+0000 7f949659c700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94800b2880 0x7f94800b4d90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9488009a10 tx=0x7f9488009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:24.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.710+0000 7f94877fe700 1 -- 172.21.15.46:0/2105906116 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9490029020 con 0x7f9498109850 2024-05-09T00:10:24.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.925+0000 7f949cb92700 1 -- 172.21.15.46:0/2105906116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f949819c2d0 con 0x7f9498109850 2024-05-09T00:10:24.922 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:24 smithi046 ceph-mon[30151]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T00:10:24.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.925+0000 7f94877fe700 1 -- 172.21.15.46:0/2105906116 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f94900ab180 con 0x7f9498109850 2024-05-09T00:10:24.923 INFO:teuthology.orchestra.run.smithi046.stdout:154618822726 2024-05-09T00:10:24.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.928+0000 7f949cb92700 1 -- 172.21.15.46:0/2105906116 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94800b2880 msgr2=0x7f94800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.928+0000 7f949cb92700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94800b2880 0x7f94800b4d90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9488009a10 tx=0x7f9488009190 comp rx=0 tx=0).stop 2024-05-09T00:10:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.928+0000 7f949cb92700 1 -- 172.21.15.46:0/2105906116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9498109850 msgr2=0x7f949819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.928+0000 7f949cb92700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9498109850 0x7f949819f7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f94900079a0 tx=0x7f94900079d0 comp rx=0 tx=0).stop 2024-05-09T00:10:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.928+0000 7f949cb92700 1 -- 172.21.15.46:0/2105906116 shutdown_connections 2024-05-09T00:10:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.928+0000 7f949cb92700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94800b2880 0x7f94800b4d90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.928+0000 7f949cb92700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9498109850 0x7f949819f7a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.928+0000 7f949cb92700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94981045d0 0x7f949819af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.928+0000 7f949cb92700 1 --2- 172.21.15.46:0/2105906116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9498108f00 0x7f949819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.928+0000 7f949cb92700 1 -- 172.21.15.46:0/2105906116 >> 172.21.15.46:0/2105906116 conn(0x7f94980ffea0 msgr2=0x7f9498108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.928+0000 7f949cb92700 1 -- 172.21.15.46:0/2105906116 shutdown_connections 2024-05-09T00:10:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:24.928+0000 7f949cb92700 1 -- 172.21.15.46:0/2105906116 wait complete. 2024-05-09T00:10:24.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:24 smithi142 ceph-mon[34533]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T00:10:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:25.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:24 smithi169 ceph-mon[34777]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T00:10:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.777+0000 7fb121b1e700 1 -- 172.21.15.46:0/4163404633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb11c105460 msgr2=0x7fb11c105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.777+0000 7fb121b1e700 1 --2- 172.21.15.46:0/4163404633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb11c105460 0x7fb11c105870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009990 tx=0x7fb10c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.777+0000 7fb121b1e700 1 -- 172.21.15.46:0/4163404633 shutdown_connections 2024-05-09T00:10:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.777+0000 7fb121b1e700 1 --2- 172.21.15.46:0/4163404633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb11c10e8d0 0x7fb11c110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.777+0000 7fb121b1e700 1 --2- 172.21.15.46:0/4163404633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb11c105460 0x7fb11c105870 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.777+0000 7fb121b1e700 1 --2- 172.21.15.46:0/4163404633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c105db0 0x7fb11c10e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.777+0000 7fb121b1e700 1 -- 172.21.15.46:0/4163404633 >> 172.21.15.46:0/4163404633 conn(0x7fb11c0fedc0 msgr2=0x7fb11c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.778+0000 7fb121b1e700 1 -- 172.21.15.46:0/4163404633 shutdown_connections 2024-05-09T00:10:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.778+0000 7fb121b1e700 1 -- 172.21.15.46:0/4163404633 wait complete. 2024-05-09T00:10:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.779+0000 7fb121b1e700 1 Processor -- start 2024-05-09T00:10:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.779+0000 7fb121b1e700 1 -- start start 2024-05-09T00:10:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.779+0000 7fb121b1e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c105460 0x7fb11c19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.779+0000 7fb121b1e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb11c105db0 0x7fb11c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.779+0000 7fb121b1e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb11c10e8d0 0x7fb11c19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.779+0000 7fb121b1e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c114420 con 0x7fb11c105db0 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.779+0000 7fb121b1e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c1142a0 con 0x7fb11c105460 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.779+0000 7fb121b1e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c1145a0 con 0x7fb11c10e8d0 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.779+0000 7fb112ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb11c105db0 0x7fb11c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.779+0000 7fb112ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb11c105db0 0x7fb11c19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37690/0 (socket says 172.21.15.46:37690) 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.779+0000 7fb112ffd700 1 -- 172.21.15.46:0/169640916 learned_addr learned my addr 172.21.15.46:0/169640916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.779+0000 7fb11b7fe700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c105460 0x7fb11c19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.780+0000 7fb11bfff700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb11c10e8d0 0x7fb11c19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.780+0000 7fb11b7fe700 1 -- 172.21.15.46:0/169640916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb11c10e8d0 msgr2=0x7fb11c19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.780+0000 7fb11b7fe700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb11c10e8d0 0x7fb11c19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.780+0000 7fb11b7fe700 1 -- 172.21.15.46:0/169640916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb11c105db0 msgr2=0x7fb11c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.780+0000 7fb11b7fe700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb11c105db0 0x7fb11c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.780+0000 7fb11b7fe700 1 -- 172.21.15.46:0/169640916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c19fc60 con 0x7fb11c105460 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.780+0000 7fb112ffd700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb11c105db0 0x7fb11c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.780+0000 7fb11b7fe700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c105460 0x7fb11c19ad90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0059d0 tx=0x7fb10c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.780+0000 7fb1197fa700 1 -- 172.21.15.46:0/169640916 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c027070 con 0x7fb11c105460 2024-05-09T00:10:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.780+0000 7fb121b1e700 1 -- 172.21.15.46:0/169640916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c19fef0 con 0x7fb11c105460 2024-05-09T00:10:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.780+0000 7fb1197fa700 1 -- 172.21.15.46:0/169640916 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb10c003f20 con 0x7fb11c105460 2024-05-09T00:10:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.780+0000 7fb121b1e700 1 -- 172.21.15.46:0/169640916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c1a03f0 con 0x7fb11c105460 2024-05-09T00:10:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.781+0000 7fb1197fa700 1 -- 172.21.15.46:0/169640916 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c022a20 con 0x7fb11c105460 2024-05-09T00:10:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.782+0000 7fb121b1e700 1 -- 172.21.15.46:0/169640916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0fc005330 con 0x7fb11c105460 2024-05-09T00:10:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.783+0000 7fb1197fa700 1 -- 172.21.15.46:0/169640916 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb10c02a070 con 0x7fb11c105460 2024-05-09T00:10:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.784+0000 7fb1197fa700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb1080b2880 0x7fb1080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.784+0000 7fb112ffd700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb1080b2880 0x7fb1080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.784+0000 7fb1197fa700 1 -- 172.21.15.46:0/169640916 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb10c0e3a10 con 0x7fb11c105460 2024-05-09T00:10:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.785+0000 7fb112ffd700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb1080b2880 0x7fb1080b4d90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb11c102270 tx=0x7fb10400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:25.788+0000 7fb1197fa700 1 -- 172.21.15.46:0/169640916 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb11c1a03f0 con 0x7fb11c105460 2024-05-09T00:10:25.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:25 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2105906116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T00:10:25.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:25 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2105906116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T00:10:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.007+0000 7fb121b1e700 1 -- 172.21.15.46:0/169640916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb0fc0058f0 con 0x7fb11c105460 2024-05-09T00:10:26.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.008+0000 7fb1197fa700 1 -- 172.21.15.46:0/169640916 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb10c013070 con 0x7fb11c105460 2024-05-09T00:10:26.004 INFO:teuthology.orchestra.run.smithi046.stdout:34359738464 2024-05-09T00:10:26.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.010+0000 7fb121b1e700 1 -- 172.21.15.46:0/169640916 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb1080b2880 msgr2=0x7fb1080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:26.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.010+0000 7fb121b1e700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb1080b2880 0x7fb1080b4d90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb11c102270 tx=0x7fb10400b500 comp rx=0 tx=0).stop 2024-05-09T00:10:26.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.010+0000 7fb121b1e700 1 -- 172.21.15.46:0/169640916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c105460 msgr2=0x7fb11c19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:26.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.010+0000 7fb121b1e700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c105460 0x7fb11c19ad90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0059d0 tx=0x7fb10c0047b0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.010+0000 7fb121b1e700 1 -- 172.21.15.46:0/169640916 shutdown_connections 2024-05-09T00:10:26.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.010+0000 7fb121b1e700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb1080b2880 0x7fb1080b4d90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.010+0000 7fb121b1e700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb11c10e8d0 0x7fb11c19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.010+0000 7fb121b1e700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c105460 0x7fb11c19ad90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.010+0000 7fb121b1e700 1 --2- 172.21.15.46:0/169640916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb11c105db0 0x7fb11c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:26.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.010+0000 7fb121b1e700 1 -- 172.21.15.46:0/169640916 >> 172.21.15.46:0/169640916 conn(0x7fb11c0fedc0 msgr2=0x7fb11c0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:26.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.010+0000 7fb121b1e700 1 -- 172.21.15.46:0/169640916 shutdown_connections 2024-05-09T00:10:26.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:26.010+0000 7fb121b1e700 1 -- 172.21.15.46:0/169640916 wait complete. 2024-05-09T00:10:26.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:25 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2105906116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T00:10:26.377 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822721 got 154618822726 for osd.5 2024-05-09T00:10:26.377 DEBUG:teuthology.parallel:result is None 2024-05-09T00:10:26.803 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:26 smithi142 ceph-mon[34533]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-09T00:10:26.803 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:26 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:10:26.803 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:26 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/169640916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T00:10:26.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:26 smithi169 ceph-mon[34777]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-09T00:10:26.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:26 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:10:26.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:26 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/169640916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T00:10:26.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:26 smithi046 ceph-mon[30151]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-09T00:10:26.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:26 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:10:26.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:26 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/169640916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T00:10:27.684 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738464 for osd.0 2024-05-09T00:10:27.684 DEBUG:teuthology.parallel:result is None 2024-05-09T00:10:28.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:28 smithi142 ceph-mon[34533]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:28.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:28 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:28.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:28 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:28.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:28 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:28.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:28 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:28.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:28 smithi046 ceph-mon[30151]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:28.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:28 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:28.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:28 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:28.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:28 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:28.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:28 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:28.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:28 smithi169 ceph-mon[34777]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:28.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:28 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:28.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:28 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:28.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:28 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:28.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:28 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:29.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:29 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3482065109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T00:10:29.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:29 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3482065109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T00:10:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.168+0000 7f229e3eb700 1 -- 172.21.15.46:0/2093002279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2298108740 msgr2=0x7f22981100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.168+0000 7f229e3eb700 1 --2- 172.21.15.46:0/2093002279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2298108740 0x7f22981100a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f229000b0d0 tx=0x7f229000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:10:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.168+0000 7f229e3eb700 1 -- 172.21.15.46:0/2093002279 shutdown_connections 2024-05-09T00:10:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.168+0000 7f229e3eb700 1 --2- 172.21.15.46:0/2093002279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2298108740 0x7f22981100a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.168+0000 7f229e3eb700 1 --2- 172.21.15.46:0/2093002279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22981034c0 0x7f22981038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.168+0000 7f229e3eb700 1 --2- 172.21.15.46:0/2093002279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2298107df0 0x7f2298108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.168+0000 7f229e3eb700 1 -- 172.21.15.46:0/2093002279 >> 172.21.15.46:0/2093002279 conn(0x7f22980fedc0 msgr2=0x7f22981011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.168+0000 7f229e3eb700 1 -- 172.21.15.46:0/2093002279 shutdown_connections 2024-05-09T00:10:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.168+0000 7f229e3eb700 1 -- 172.21.15.46:0/2093002279 wait complete. 2024-05-09T00:10:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.168+0000 7f229e3eb700 1 Processor -- start 2024-05-09T00:10:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f229e3eb700 1 -- start start 2024-05-09T00:10:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f229e3eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22981034c0 0x7f22980ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f229e3eb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2298107df0 0x7f22980ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f229e3eb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2298108740 0x7f22980ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f229e3eb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2298113390 con 0x7f22981034c0 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f229e3eb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2298113210 con 0x7f2298108740 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f229e3eb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2298113510 con 0x7f2298107df0 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f2297fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22981034c0 0x7f22980ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f2297fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22981034c0 0x7f22980ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55078/0 (socket says 172.21.15.46:55078) 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f2297fff700 1 -- 172.21.15.46:0/3482065109 learned_addr learned my addr 172.21.15.46:0/3482065109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f228ffff700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2298107df0 0x7f22980ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f2297fff700 1 -- 172.21.15.46:0/3482065109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2298107df0 msgr2=0x7f22980ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f2297fff700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2298107df0 0x7f22980ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f2297fff700 1 -- 172.21.15.46:0/3482065109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2298108740 msgr2=0x7f22980ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f229c988700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2298108740 0x7f22980ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f2297fff700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2298108740 0x7f22980ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f2297fff700 1 -- 172.21.15.46:0/3482065109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22981a9170 con 0x7f22981034c0 2024-05-09T00:10:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f228ffff700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2298107df0 0x7f22980ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.169+0000 7f2297fff700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22981034c0 0x7f22980ff2a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f228800d980 tx=0x7f228800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.170+0000 7f2295ffb700 1 -- 172.21.15.46:0/3482065109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288016070 con 0x7f22981034c0 2024-05-09T00:10:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.170+0000 7f2295ffb700 1 -- 172.21.15.46:0/3482065109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22880044d0 con 0x7f22981034c0 2024-05-09T00:10:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.170+0000 7f229e3eb700 1 -- 172.21.15.46:0/3482065109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22981a9460 con 0x7f22981034c0 2024-05-09T00:10:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.170+0000 7f2295ffb700 1 -- 172.21.15.46:0/3482065109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288004d00 con 0x7f22981034c0 2024-05-09T00:10:29.865 INFO:teuthology.orchestra.run.smithi046.stdout:180388626498 2024-05-09T00:10:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.170+0000 7f229e3eb700 1 -- 172.21.15.46:0/3482065109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22981a99f0 con 0x7f22981034c0 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.171+0000 7f229e3eb700 1 -- 172.21.15.46:0/3482065109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22980512c0 con 0x7f22981034c0 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.172+0000 7f2295ffb700 1 -- 172.21.15.46:0/3482065109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2288004020 con 0x7f22981034c0 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.173+0000 7f2295ffb700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f22840b2900 0x7f22840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.173+0000 7f228ffff700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f22840b2900 0x7f22840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.173+0000 7f2295ffb700 1 -- 172.21.15.46:0/3482065109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f22880e2640 con 0x7f22981034c0 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.174+0000 7f228ffff700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f22840b2900 0x7f22840b4e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2280005e10 tx=0x7f2280005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.177+0000 7f2295ffb700 1 -- 172.21.15.46:0/3482065109 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f228801f020 con 0x7f22981034c0 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.396+0000 7f229e3eb700 1 -- 172.21.15.46:0/3482065109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f229802e5f0 con 0x7f22981034c0 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.397+0000 7f2295ffb700 1 -- 172.21.15.46:0/3482065109 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f22880aa320 con 0x7f22981034c0 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.399+0000 7f229e3eb700 1 -- 172.21.15.46:0/3482065109 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f22840b2900 msgr2=0x7f22840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.399+0000 7f229e3eb700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f22840b2900 0x7f22840b4e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2280005e10 tx=0x7f2280005d80 comp rx=0 tx=0).stop 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.399+0000 7f229e3eb700 1 -- 172.21.15.46:0/3482065109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22981034c0 msgr2=0x7f22980ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.399+0000 7f229e3eb700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22981034c0 0x7f22980ff2a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f228800d980 tx=0x7f228800dd40 comp rx=0 tx=0).stop 2024-05-09T00:10:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.399+0000 7f229e3eb700 1 -- 172.21.15.46:0/3482065109 shutdown_connections 2024-05-09T00:10:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.399+0000 7f229e3eb700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f22840b2900 0x7f22840b4e10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.399+0000 7f229e3eb700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2298108740 0x7f22980ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.399+0000 7f229e3eb700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22981034c0 0x7f22980ff2a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.399+0000 7f229e3eb700 1 --2- 172.21.15.46:0/3482065109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2298107df0 0x7f22980ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.399+0000 7f229e3eb700 1 -- 172.21.15.46:0/3482065109 >> 172.21.15.46:0/3482065109 conn(0x7f22980fedc0 msgr2=0x7f229806b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.399+0000 7f229e3eb700 1 -- 172.21.15.46:0/3482065109 shutdown_connections 2024-05-09T00:10:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:29.399+0000 7f229e3eb700 1 -- 172.21.15.46:0/3482065109 wait complete. 2024-05-09T00:10:29.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:29 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3482065109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T00:10:30.805 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:30 smithi046 ceph-mon[30151]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:10:30.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:30 smithi169 ceph-mon[34777]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:10:30.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:30 smithi142 ceph-mon[34533]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:10:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.624+0000 7f8e91253700 1 -- 172.21.15.46:0/3023661774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c108f10 msgr2=0x7f8e8c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.624+0000 7f8e91253700 1 --2- 172.21.15.46:0/3023661774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c108f10 0x7f8e8c109320 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8e740099c0 tx=0x7f8e74009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.624+0000 7f8e91253700 1 -- 172.21.15.46:0/3023661774 shutdown_connections 2024-05-09T00:10:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.624+0000 7f8e91253700 1 --2- 172.21.15.46:0/3023661774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e8c109860 0x7f8e8c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.624+0000 7f8e91253700 1 --2- 172.21.15.46:0/3023661774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c1045e0 0x7f8e8c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.624+0000 7f8e91253700 1 --2- 172.21.15.46:0/3023661774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c108f10 0x7f8e8c109320 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.624+0000 7f8e91253700 1 -- 172.21.15.46:0/3023661774 >> 172.21.15.46:0/3023661774 conn(0x7f8e8c0ffe70 msgr2=0x7f8e8c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.624+0000 7f8e91253700 1 -- 172.21.15.46:0/3023661774 shutdown_connections 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.624+0000 7f8e91253700 1 -- 172.21.15.46:0/3023661774 wait complete. 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.625+0000 7f8e91253700 1 Processor -- start 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.625+0000 7f8e91253700 1 -- start start 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.625+0000 7f8e91253700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e8c1045e0 0x7f8e8c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.625+0000 7f8e91253700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c108f10 0x7f8e8c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.625+0000 7f8e91253700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c109860 0x7f8e8c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.625+0000 7f8e91253700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c114470 con 0x7f8e8c108f10 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.625+0000 7f8e91253700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1142f0 con 0x7f8e8c109860 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.625+0000 7f8e91253700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1145f0 con 0x7f8e8c1045e0 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e8a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c108f10 0x7f8e8c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e8a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c108f10 0x7f8e8c19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55100/0 (socket says 172.21.15.46:55100) 2024-05-09T00:10:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e8a59c700 1 -- 172.21.15.46:0/2683916737 learned_addr learned my addr 172.21.15.46:0/2683916737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e8ad9d700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e8c1045e0 0x7f8e8c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e8b59e700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c109860 0x7f8e8c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e8a59c700 1 -- 172.21.15.46:0/2683916737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e8c1045e0 msgr2=0x7f8e8c19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e8a59c700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e8c1045e0 0x7f8e8c19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e8a59c700 1 -- 172.21.15.46:0/2683916737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c109860 msgr2=0x7f8e8c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e8a59c700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c109860 0x7f8e8c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e8a59c700 1 -- 172.21.15.46:0/2683916737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c19fd10 con 0x7f8e8c108f10 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e8b59e700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c109860 0x7f8e8c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e8a59c700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c108f10 0x7f8e8c19b3e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8e740098e0 tx=0x7f8e740057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.626+0000 7f8e83fff700 1 -- 172.21.15.46:0/2683916737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74027070 con 0x7f8e8c108f10 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.627+0000 7f8e91253700 1 -- 172.21.15.46:0/2683916737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c19ffa0 con 0x7f8e8c108f10 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.627+0000 7f8e83fff700 1 -- 172.21.15.46:0/2683916737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e74003f20 con 0x7f8e8c108f10 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.627+0000 7f8e83fff700 1 -- 172.21.15.46:0/2683916737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74022d70 con 0x7f8e8c108f10 2024-05-09T00:10:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.627+0000 7f8e91253700 1 -- 172.21.15.46:0/2683916737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c1a7920 con 0x7f8e8c108f10 2024-05-09T00:10:31.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.628+0000 7f8e91253700 1 -- 172.21.15.46:0/2683916737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e8c105a90 con 0x7f8e8c108f10 2024-05-09T00:10:31.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.629+0000 7f8e83fff700 1 -- 172.21.15.46:0/2683916737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8e7402a070 con 0x7f8e8c108f10 2024-05-09T00:10:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.630+0000 7f8e83fff700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8e780b2880 0x7f8e780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.631+0000 7f8e8ad9d700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8e780b2880 0x7f8e780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.631+0000 7f8e83fff700 1 -- 172.21.15.46:0/2683916737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8e740e2ff0 con 0x7f8e8c108f10 2024-05-09T00:10:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.631+0000 7f8e8ad9d700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8e780b2880 0x7f8e780b4d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c007be0 tx=0x7f8e7c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.640+0000 7f8e83fff700 1 -- 172.21.15.46:0/2683916737 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8e74013070 con 0x7f8e8c108f10 2024-05-09T00:10:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.865+0000 7f8e91253700 1 -- 172.21.15.46:0/2683916737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8e8c19c1a0 con 0x7f8e8c108f10 2024-05-09T00:10:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.866+0000 7f8e83fff700 1 -- 172.21.15.46:0/2683916737 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8e740aada0 con 0x7f8e8c108f10 2024-05-09T00:10:31.863 INFO:teuthology.orchestra.run.smithi046.stdout:201863462973 2024-05-09T00:10:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.868+0000 7f8e91253700 1 -- 172.21.15.46:0/2683916737 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8e780b2880 msgr2=0x7f8e780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.868+0000 7f8e91253700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8e780b2880 0x7f8e780b4d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c007be0 tx=0x7f8e7c0073d0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.868+0000 7f8e91253700 1 -- 172.21.15.46:0/2683916737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c108f10 msgr2=0x7f8e8c19b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.868+0000 7f8e91253700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c108f10 0x7f8e8c19b3e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8e740098e0 tx=0x7f8e740057c0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.869+0000 7f8e91253700 1 -- 172.21.15.46:0/2683916737 shutdown_connections 2024-05-09T00:10:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.869+0000 7f8e91253700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8e780b2880 0x7f8e780b4d90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.869+0000 7f8e91253700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e8c109860 0x7f8e8c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.869+0000 7f8e91253700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e8c1045e0 0x7f8e8c19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.869+0000 7f8e91253700 1 --2- 172.21.15.46:0/2683916737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c108f10 0x7f8e8c19b3e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.869+0000 7f8e91253700 1 -- 172.21.15.46:0/2683916737 >> 172.21.15.46:0/2683916737 conn(0x7f8e8c0ffe70 msgr2=0x7f8e8c108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.869+0000 7f8e91253700 1 -- 172.21.15.46:0/2683916737 shutdown_connections 2024-05-09T00:10:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.869+0000 7f8e91253700 1 -- 172.21.15.46:0/2683916737 wait complete. 2024-05-09T00:10:31.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.890+0000 7f3e02c8e700 1 -- 172.21.15.46:0/4276170718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dfc108760 msgr2=0x7f3dfc1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.890+0000 7f3e02c8e700 1 --2- 172.21.15.46:0/4276170718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dfc108760 0x7f3dfc1100c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009fc0 tx=0x7f3df400b330 comp rx=0 tx=0).stop 2024-05-09T00:10:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.890+0000 7f3e02c8e700 1 -- 172.21.15.46:0/4276170718 shutdown_connections 2024-05-09T00:10:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.890+0000 7f3e02c8e700 1 --2- 172.21.15.46:0/4276170718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dfc108760 0x7f3dfc1100c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.890+0000 7f3e02c8e700 1 --2- 172.21.15.46:0/4276170718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dfc1034e0 0x7f3dfc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.890+0000 7f3e02c8e700 1 --2- 172.21.15.46:0/4276170718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dfc107e10 0x7f3dfc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.890+0000 7f3e02c8e700 1 -- 172.21.15.46:0/4276170718 >> 172.21.15.46:0/4276170718 conn(0x7f3dfc0fef30 msgr2=0x7f3dfc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.890+0000 7f3e02c8e700 1 -- 172.21.15.46:0/4276170718 shutdown_connections 2024-05-09T00:10:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.890+0000 7f3e02c8e700 1 -- 172.21.15.46:0/4276170718 wait complete. 2024-05-09T00:10:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.891+0000 7f3e02c8e700 1 Processor -- start 2024-05-09T00:10:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.891+0000 7f3e02c8e700 1 -- start start 2024-05-09T00:10:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.891+0000 7f3e02c8e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dfc1034e0 0x7f3dfc19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.891+0000 7f3e02c8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dfc107e10 0x7f3dfc19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.891+0000 7f3e02c8e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dfc108760 0x7f3dfc19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.891+0000 7f3e02c8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc113390 con 0x7f3dfc107e10 2024-05-09T00:10:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.891+0000 7f3e02c8e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc113210 con 0x7f3dfc1034e0 2024-05-09T00:10:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.891+0000 7f3e02c8e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc113510 con 0x7f3dfc108760 2024-05-09T00:10:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.891+0000 7f3dfbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dfc107e10 0x7f3dfc19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.891+0000 7f3dfbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dfc107e10 0x7f3dfc19c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55114/0 (socket says 172.21.15.46:55114) 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.891+0000 7f3dfbfff700 1 -- 172.21.15.46:0/1720728703 learned_addr learned my addr 172.21.15.46:0/1720728703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.892+0000 7f3e0122b700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dfc108760 0x7f3dfc19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.892+0000 7f3e00a2a700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dfc1034e0 0x7f3dfc19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.892+0000 7f3dfbfff700 1 -- 172.21.15.46:0/1720728703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dfc108760 msgr2=0x7f3dfc19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.892+0000 7f3dfbfff700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dfc108760 0x7f3dfc19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.892+0000 7f3dfbfff700 1 -- 172.21.15.46:0/1720728703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dfc1034e0 msgr2=0x7f3dfc19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.892+0000 7f3dfbfff700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dfc1034e0 0x7f3dfc19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.892+0000 7f3dfbfff700 1 -- 172.21.15.46:0/1720728703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dfc1a6fa0 con 0x7f3dfc107e10 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.892+0000 7f3dfbfff700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dfc107e10 0x7f3dfc19c190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3de800ec70 tx=0x7f3de800ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.892+0000 7f3df9ffb700 1 -- 172.21.15.46:0/1720728703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de800cc90 con 0x7f3dfc107e10 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.892+0000 7f3e02c8e700 1 -- 172.21.15.46:0/1720728703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dfc1a7290 con 0x7f3dfc107e10 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.893+0000 7f3df9ffb700 1 -- 172.21.15.46:0/1720728703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de800ce30 con 0x7f3dfc107e10 2024-05-09T00:10:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.893+0000 7f3df9ffb700 1 -- 172.21.15.46:0/1720728703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8010430 con 0x7f3dfc107e10 2024-05-09T00:10:31.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.893+0000 7f3e02c8e700 1 -- 172.21.15.46:0/1720728703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dfc1a7820 con 0x7f3dfc107e10 2024-05-09T00:10:31.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.894+0000 7f3e02c8e700 1 -- 172.21.15.46:0/1720728703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dfc0512c0 con 0x7f3dfc107e10 2024-05-09T00:10:31.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.896+0000 7f3df9ffb700 1 -- 172.21.15.46:0/1720728703 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3de8003680 con 0x7f3dfc107e10 2024-05-09T00:10:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.897+0000 7f3df9ffb700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3dec0b2900 0x7f3dec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.897+0000 7f3df9ffb700 1 -- 172.21.15.46:0/1720728703 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f3de80e24e0 con 0x7f3dfc107e10 2024-05-09T00:10:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.897+0000 7f3e00a2a700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3dec0b2900 0x7f3dec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.897+0000 7f3e00a2a700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3dec0b2900 0x7f3dec0b4e10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009680 tx=0x7f3df0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:31.901+0000 7f3df9ffb700 1 -- 172.21.15.46:0/1720728703 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3de80aa200 con 0x7f3dfc107e10 2024-05-09T00:10:32.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.085+0000 7f4888d3c700 1 -- 172.21.15.46:0/2560849650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4884105e70 msgr2=0x7f488410e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:32.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.085+0000 7f4888d3c700 1 --2- 172.21.15.46:0/2560849650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4884105e70 0x7f488410e3c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4878009990 tx=0x7f4878009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.085+0000 7f4888d3c700 1 -- 172.21.15.46:0/2560849650 shutdown_connections 2024-05-09T00:10:32.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.085+0000 7f4888d3c700 1 --2- 172.21.15.46:0/2560849650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f488410e900 0x7f4884110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.085+0000 7f4888d3c700 1 --2- 172.21.15.46:0/2560849650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4884105520 0x7f4884105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.085+0000 7f4888d3c700 1 --2- 172.21.15.46:0/2560849650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4884105e70 0x7f488410e3c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.085+0000 7f4888d3c700 1 -- 172.21.15.46:0/2560849650 >> 172.21.15.46:0/2560849650 conn(0x7f48840fef50 msgr2=0x7f4884101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:32.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.085+0000 7f4888d3c700 1 -- 172.21.15.46:0/2560849650 shutdown_connections 2024-05-09T00:10:32.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.085+0000 7f4888d3c700 1 -- 172.21.15.46:0/2560849650 wait complete. 2024-05-09T00:10:32.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.086+0000 7f4888d3c700 1 Processor -- start 2024-05-09T00:10:32.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.086+0000 7f4888d3c700 1 -- start start 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4888d3c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4884105520 0x7f488419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4888d3c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4884105e70 0x7f488419b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4888d3c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488410e900 0x7f488419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4888d3c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4884114500 con 0x7f4884105e70 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4888d3c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4884114380 con 0x7f488410e900 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4888d3c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4884114680 con 0x7f4884105520 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4881d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4884105e70 0x7f488419b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4881d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4884105e70 0x7f488419b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55122/0 (socket says 172.21.15.46:55122) 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4881d9b700 1 -- 172.21.15.46:0/650287171 learned_addr learned my addr 172.21.15.46:0/650287171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f488259c700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4884105520 0x7f488419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4882d9d700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488410e900 0x7f488419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4881d9b700 1 -- 172.21.15.46:0/650287171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4884105520 msgr2=0x7f488419af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4881d9b700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4884105520 0x7f488419af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4881d9b700 1 -- 172.21.15.46:0/650287171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488410e900 msgr2=0x7f488419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4881d9b700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488410e900 0x7f488419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4881d9b700 1 -- 172.21.15.46:0/650287171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488419fe00 con 0x7f4884105e70 2024-05-09T00:10:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.087+0000 7f4882d9d700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488410e900 0x7f488419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.088+0000 7f4881d9b700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4884105e70 0x7f488419b4d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4878006010 tx=0x7f48780057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.088+0000 7f48737fe700 1 -- 172.21.15.46:0/650287171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878027070 con 0x7f4884105e70 2024-05-09T00:10:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.088+0000 7f48737fe700 1 -- 172.21.15.46:0/650287171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4878003fe0 con 0x7f4884105e70 2024-05-09T00:10:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.088+0000 7f48737fe700 1 -- 172.21.15.46:0/650287171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878022ca0 con 0x7f4884105e70 2024-05-09T00:10:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.088+0000 7f4888d3c700 1 -- 172.21.15.46:0/650287171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48841a0090 con 0x7f4884105e70 2024-05-09T00:10:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.088+0000 7f4888d3c700 1 -- 172.21.15.46:0/650287171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48841a0520 con 0x7f4884105e70 2024-05-09T00:10:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.090+0000 7f4888d3c700 1 -- 172.21.15.46:0/650287171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48840512c0 con 0x7f4884105e70 2024-05-09T00:10:32.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.092+0000 7f48737fe700 1 -- 172.21.15.46:0/650287171 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f487802a070 con 0x7f4884105e70 2024-05-09T00:10:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.093+0000 7f48737fe700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f486c0b2900 0x7f486c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.093+0000 7f488259c700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f486c0b2900 0x7f486c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.093+0000 7f48737fe700 1 -- 172.21.15.46:0/650287171 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f48780e3a50 con 0x7f4884105e70 2024-05-09T00:10:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.094+0000 7f488259c700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f486c0b2900 0x7f486c0b4e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4874005950 tx=0x7f487400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.097+0000 7f48737fe700 1 -- 172.21.15.46:0/650287171 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4878031080 con 0x7f4884105e70 2024-05-09T00:10:32.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.127+0000 7f3e02c8e700 1 -- 172.21.15.46:0/1720728703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f3dfc06c630 con 0x7f3dfc107e10 2024-05-09T00:10:32.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.128+0000 7f3df9ffb700 1 -- 172.21.15.46:0/1720728703 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3de80a9b00 con 0x7f3dfc107e10 2024-05-09T00:10:32.124 INFO:teuthology.orchestra.run.smithi046.stdout:227633266744 2024-05-09T00:10:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.131+0000 7f3e02c8e700 1 -- 172.21.15.46:0/1720728703 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3dec0b2900 msgr2=0x7f3dec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.131+0000 7f3e02c8e700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3dec0b2900 0x7f3dec0b4e10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009680 tx=0x7f3df0009260 comp rx=0 tx=0).stop 2024-05-09T00:10:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.131+0000 7f3e02c8e700 1 -- 172.21.15.46:0/1720728703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dfc107e10 msgr2=0x7f3dfc19c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.131+0000 7f3e02c8e700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dfc107e10 0x7f3dfc19c190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3de800ec70 tx=0x7f3de800ef80 comp rx=0 tx=0).stop 2024-05-09T00:10:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.131+0000 7f3e02c8e700 1 -- 172.21.15.46:0/1720728703 shutdown_connections 2024-05-09T00:10:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.131+0000 7f3e02c8e700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3dec0b2900 0x7f3dec0b4e10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.131+0000 7f3e02c8e700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3dfc108760 0x7f3dfc19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.131+0000 7f3e02c8e700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dfc1034e0 0x7f3dfc19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.131+0000 7f3e02c8e700 1 --2- 172.21.15.46:0/1720728703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dfc107e10 0x7f3dfc19c190 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.131+0000 7f3e02c8e700 1 -- 172.21.15.46:0/1720728703 >> 172.21.15.46:0/1720728703 conn(0x7f3dfc0fef30 msgr2=0x7f3dfc107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.131+0000 7f3e02c8e700 1 -- 172.21.15.46:0/1720728703 shutdown_connections 2024-05-09T00:10:32.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.131+0000 7f3e02c8e700 1 -- 172.21.15.46:0/1720728703 wait complete. 2024-05-09T00:10:32.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.318+0000 7f4888d3c700 1 -- 172.21.15.46:0/650287171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f488402e5f0 con 0x7f4884105e70 2024-05-09T00:10:32.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.319+0000 7f48737fe700 1 -- 172.21.15.46:0/650287171 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4878013070 con 0x7f4884105e70 2024-05-09T00:10:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:274877906990 2024-05-09T00:10:32.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.321+0000 7f4888d3c700 1 -- 172.21.15.46:0/650287171 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f486c0b2900 msgr2=0x7f486c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.321+0000 7f4888d3c700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f486c0b2900 0x7f486c0b4e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4874005950 tx=0x7f487400a300 comp rx=0 tx=0).stop 2024-05-09T00:10:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.321+0000 7f4888d3c700 1 -- 172.21.15.46:0/650287171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4884105e70 msgr2=0x7f488419b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.321+0000 7f4888d3c700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4884105e70 0x7f488419b4d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4878006010 tx=0x7f48780057c0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.321+0000 7f4888d3c700 1 -- 172.21.15.46:0/650287171 shutdown_connections 2024-05-09T00:10:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.321+0000 7f4888d3c700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f486c0b2900 0x7f486c0b4e10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.321+0000 7f4888d3c700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f488410e900 0x7f488419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.321+0000 7f4888d3c700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4884105520 0x7f488419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.321+0000 7f4888d3c700 1 --2- 172.21.15.46:0/650287171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4884105e70 0x7f488419b4d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.321+0000 7f4888d3c700 1 -- 172.21.15.46:0/650287171 >> 172.21.15.46:0/650287171 conn(0x7f48840fef50 msgr2=0x7f48840ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.321+0000 7f4888d3c700 1 -- 172.21.15.46:0/650287171 shutdown_connections 2024-05-09T00:10:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:32.321+0000 7f4888d3c700 1 -- 172.21.15.46:0/650287171 wait complete. 2024-05-09T00:10:32.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:32 smithi142 ceph-mon[34533]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:10:32.743 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:32 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2683916737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T00:10:32.743 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:32 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1720728703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T00:10:32.743 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:32 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/650287171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T00:10:32.748 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:32 smithi046 ceph-mon[30151]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:10:32.748 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:32 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2683916737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T00:10:32.748 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:32 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1720728703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T00:10:32.748 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:32 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/650287171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T00:10:32.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:32 smithi169 ceph-mon[34777]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:10:32.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:32 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2683916737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T00:10:32.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:32 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1720728703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T00:10:32.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:32 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/650287171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T00:10:33.426 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906984 got 274877906990 for osd.10 2024-05-09T00:10:33.427 DEBUG:teuthology.parallel:result is None 2024-05-09T00:10:33.442 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626492 got 180388626498 for osd.6 2024-05-09T00:10:33.442 DEBUG:teuthology.parallel:result is None 2024-05-09T00:10:34.263 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462968 got 201863462973 for osd.7 2024-05-09T00:10:34.264 DEBUG:teuthology.parallel:result is None 2024-05-09T00:10:34.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:34 smithi046 ceph-mon[30151]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:34.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:34 smithi169 ceph-mon[34777]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.897+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/1732115690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e54108f00 msgr2=0x7f0e54109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.897+0000 7f0e5b2f5700 1 --2- 172.21.15.46:0/1732115690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e54108f00 0x7f0e54109310 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0e480099c0 tx=0x7f0e48009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.897+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/1732115690 shutdown_connections 2024-05-09T00:10:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.897+0000 7f0e5b2f5700 1 --2- 172.21.15.46:0/1732115690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e54109850 0x7f0e541111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.897+0000 7f0e5b2f5700 1 --2- 172.21.15.46:0/1732115690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e541045d0 0x7f0e541049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.897+0000 7f0e5b2f5700 1 --2- 172.21.15.46:0/1732115690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e54108f00 0x7f0e54109310 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.897+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/1732115690 >> 172.21.15.46:0/1732115690 conn(0x7f0e540ffea0 msgr2=0x7f0e541022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.897+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/1732115690 shutdown_connections 2024-05-09T00:10:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.897+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/1732115690 wait complete. 2024-05-09T00:10:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e5b2f5700 1 Processor -- start 2024-05-09T00:10:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e5b2f5700 1 -- start start 2024-05-09T00:10:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e5b2f5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e541045d0 0x7f0e5419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e5b2f5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e54108f00 0x7f0e5419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e5b2f5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e54109850 0x7f0e5419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e5b2f5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e54114460 con 0x7f0e54109850 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e5b2f5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541142e0 con 0x7f0e541045d0 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e5b2f5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541145e0 con 0x7f0e54108f00 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e59892700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e54109850 0x7f0e5419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e59892700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e54109850 0x7f0e5419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55148/0 (socket says 172.21.15.46:55148) 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e59892700 1 -- 172.21.15.46:0/3671904450 learned_addr learned my addr 172.21.15.46:0/3671904450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e59091700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e541045d0 0x7f0e5419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.898+0000 7f0e58890700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e54108f00 0x7f0e5419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.899+0000 7f0e59892700 1 -- 172.21.15.46:0/3671904450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e54108f00 msgr2=0x7f0e5419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.899+0000 7f0e59892700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e54108f00 0x7f0e5419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.899+0000 7f0e59892700 1 -- 172.21.15.46:0/3671904450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e541045d0 msgr2=0x7f0e5419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.899+0000 7f0e59892700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e541045d0 0x7f0e5419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.899+0000 7f0e59892700 1 -- 172.21.15.46:0/3671904450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5419fca0 con 0x7f0e54109850 2024-05-09T00:10:34.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.899+0000 7f0e59091700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e541045d0 0x7f0e5419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:10:34.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.899+0000 7f0e59892700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e54109850 0x7f0e5419f600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00ef30 tx=0x7f0e4c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:34.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.899+0000 7f0e467fc700 1 -- 172.21.15.46:0/3671904450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0092e0 con 0x7f0e54109850 2024-05-09T00:10:34.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.899+0000 7f0e467fc700 1 -- 172.21.15.46:0/3671904450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e4c004bb0 con 0x7f0e54109850 2024-05-09T00:10:34.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.899+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/3671904450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5419ff30 con 0x7f0e54109850 2024-05-09T00:10:34.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.899+0000 7f0e467fc700 1 -- 172.21.15.46:0/3671904450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c004020 con 0x7f0e54109850 2024-05-09T00:10:34.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.899+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/3671904450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e541a03f0 con 0x7f0e54109850 2024-05-09T00:10:34.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.900+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/3671904450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e54105a80 con 0x7f0e54109850 2024-05-09T00:10:34.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.902+0000 7f0e467fc700 1 -- 172.21.15.46:0/3671904450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0e4c01b030 con 0x7f0e54109850 2024-05-09T00:10:34.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.902+0000 7f0e467fc700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0e400b2880 0x7f0e400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:34.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.902+0000 7f0e59091700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0e400b2880 0x7f0e400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:34.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.903+0000 7f0e467fc700 1 -- 172.21.15.46:0/3671904450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f0e4c0e2630 con 0x7f0e54109850 2024-05-09T00:10:34.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.903+0000 7f0e59091700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0e400b2880 0x7f0e400b4d90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0e50005950 tx=0x7f0e5000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:34.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:34.906+0000 7f0e467fc700 1 -- 172.21.15.46:0/3671904450 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e4c022020 con 0x7f0e54109850 2024-05-09T00:10:34.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:34 smithi142 ceph-mon[34533]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:35.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.126+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/3671904450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0e5402e5f0 con 0x7f0e54109850 2024-05-09T00:10:35.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.132+0000 7f0e467fc700 1 -- 172.21.15.46:0/3671904450 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e4c010040 con 0x7f0e54109850 2024-05-09T00:10:35.129 INFO:teuthology.orchestra.run.smithi046.stdout:77309411416 2024-05-09T00:10:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.122+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2741471683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4f4107e30 msgr2=0x7fb4f4108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.122+0000 7fb4fa1a0700 1 --2- 172.21.15.46:0/2741471683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4f4107e30 0x7fb4f4108240 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0099c0 tx=0x7fb4dc009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.122+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2741471683 shutdown_connections 2024-05-09T00:10:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.122+0000 7fb4fa1a0700 1 --2- 172.21.15.46:0/2741471683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4f4108780 0x7fb4f41100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.122+0000 7fb4fa1a0700 1 --2- 172.21.15.46:0/2741471683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f4103550 0x7fb4f4103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.122+0000 7fb4fa1a0700 1 --2- 172.21.15.46:0/2741471683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4f4107e30 0x7fb4f4108240 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.122+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2741471683 >> 172.21.15.46:0/2741471683 conn(0x7fb4f4078e00 msgr2=0x7fb4f4079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:35.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.122+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2741471683 shutdown_connections 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.122+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2741471683 wait complete. 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4fa1a0700 1 Processor -- start 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4fa1a0700 1 -- start start 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4fa1a0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4f4103550 0x7fb4f419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4fa1a0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f4107e30 0x7fb4f419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4fa1a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4f4108780 0x7fb4f419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4fa1a0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f4113390 con 0x7fb4f4108780 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4fa1a0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f4113210 con 0x7fb4f4103550 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4fa1a0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f4113510 con 0x7fb4f4107e30 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4f4108780 0x7fb4f419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4f4108780 0x7fb4f419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55176/0 (socket says 172.21.15.46:55176) 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4f3fff700 1 -- 172.21.15.46:0/2121059216 learned_addr learned my addr 172.21.15.46:0/2121059216 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4ebfff700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f4107e30 0x7fb4f419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4f37fe700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4f4103550 0x7fb4f419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4f3fff700 1 -- 172.21.15.46:0/2121059216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f4107e30 msgr2=0x7fb4f419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4f3fff700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f4107e30 0x7fb4f419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4f3fff700 1 -- 172.21.15.46:0/2121059216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4f4103550 msgr2=0x7fb4f419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.123+0000 7fb4f3fff700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4f4103550 0x7fb4f419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.124+0000 7fb4f3fff700 1 -- 172.21.15.46:0/2121059216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f419fd00 con 0x7fb4f4108780 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.124+0000 7fb4f37fe700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4f4103550 0x7fb4f419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.124+0000 7fb4f3fff700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4f4108780 0x7fb4f419f660 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00ef30 tx=0x7fb4ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.124+0000 7fb4f17fa700 1 -- 172.21.15.46:0/2121059216 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec0092e0 con 0x7fb4f4108780 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.124+0000 7fb4f17fa700 1 -- 172.21.15.46:0/2121059216 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4ec004bb0 con 0x7fb4f4108780 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.124+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2121059216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f419ff90 con 0x7fb4f4108780 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.124+0000 7fb4f17fa700 1 -- 172.21.15.46:0/2121059216 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec004280 con 0x7fb4f4108780 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.124+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2121059216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f41a04a0 con 0x7fb4f4108780 2024-05-09T00:10:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.125+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2121059216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f40512c0 con 0x7fb4f4108780 2024-05-09T00:10:35.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.127+0000 7fb4f17fa700 1 -- 172.21.15.46:0/2121059216 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb4ec01b030 con 0x7fb4f4108780 2024-05-09T00:10:35.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.128+0000 7fb4f17fa700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb4e00b2900 0x7fb4e00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:35.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.128+0000 7fb4f37fe700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb4e00b2900 0x7fb4e00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:35.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.128+0000 7fb4f17fa700 1 -- 172.21.15.46:0/2121059216 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb4ec0e2640 con 0x7fb4f4108780 2024-05-09T00:10:35.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.128+0000 7fb4f37fe700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb4e00b2900 0x7fb4e00b4e10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4005950 tx=0x7fb4e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:35.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.132+0000 7fb4f17fa700 1 -- 172.21.15.46:0/2121059216 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb4ec022020 con 0x7fb4f4108780 2024-05-09T00:10:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.134+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/3671904450 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0e400b2880 msgr2=0x7f0e400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.134+0000 7f0e5b2f5700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0e400b2880 0x7f0e400b4d90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0e50005950 tx=0x7f0e5000b410 comp rx=0 tx=0).stop 2024-05-09T00:10:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.134+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/3671904450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e54109850 msgr2=0x7f0e5419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.134+0000 7f0e5b2f5700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e54109850 0x7f0e5419f600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00ef30 tx=0x7f0e4c00c3a0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.134+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/3671904450 shutdown_connections 2024-05-09T00:10:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.134+0000 7f0e5b2f5700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0e400b2880 0x7f0e400b4d90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.134+0000 7f0e5b2f5700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e54109850 0x7f0e5419f600 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.134+0000 7f0e5b2f5700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e541045d0 0x7f0e5419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.134+0000 7f0e5b2f5700 1 --2- 172.21.15.46:0/3671904450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e54108f00 0x7f0e5419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.134+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/3671904450 >> 172.21.15.46:0/3671904450 conn(0x7f0e540ffea0 msgr2=0x7f0e5410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:35.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.135+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/3671904450 shutdown_connections 2024-05-09T00:10:35.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.135+0000 7f0e5b2f5700 1 -- 172.21.15.46:0/3671904450 wait complete. 2024-05-09T00:10:35.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.353+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2121059216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb4f41a79a0 con 0x7fb4f4108780 2024-05-09T00:10:35.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.353+0000 7fb4f17fa700 1 -- 172.21.15.46:0/2121059216 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb4ec010040 con 0x7fb4f4108780 2024-05-09T00:10:35.350 INFO:teuthology.orchestra.run.smithi046.stdout:253403070515 2024-05-09T00:10:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.356+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2121059216 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb4e00b2900 msgr2=0x7fb4e00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.356+0000 7fb4fa1a0700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb4e00b2900 0x7fb4e00b4e10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4005950 tx=0x7fb4e40058e0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.356+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2121059216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4f4108780 msgr2=0x7fb4f419f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.356+0000 7fb4fa1a0700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4f4108780 0x7fb4f419f660 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00ef30 tx=0x7fb4ec00c3a0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.356+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2121059216 shutdown_connections 2024-05-09T00:10:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.356+0000 7fb4fa1a0700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb4e00b2900 0x7fb4e00b4e10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.356+0000 7fb4fa1a0700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4f4108780 0x7fb4f419f660 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.356+0000 7fb4fa1a0700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4f4103550 0x7fb4f419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.356+0000 7fb4fa1a0700 1 --2- 172.21.15.46:0/2121059216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f4107e30 0x7fb4f419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.356+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2121059216 >> 172.21.15.46:0/2121059216 conn(0x7fb4f4078e00 msgr2=0x7fb4f41071b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.356+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2121059216 shutdown_connections 2024-05-09T00:10:35.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:35.356+0000 7fb4fa1a0700 1 -- 172.21.15.46:0/2121059216 wait complete. 2024-05-09T00:10:35.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:35 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3671904450' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T00:10:35.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:35 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2121059216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T00:10:35.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:35 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3671904450' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T00:10:35.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:35 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2121059216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T00:10:35.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:35 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3671904450' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T00:10:35.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:35 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2121059216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T00:10:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.049+0000 7f95816d6700 1 -- 172.21.15.46:0/1722415350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c108f00 msgr2=0x7f957c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.049+0000 7f95816d6700 1 --2- 172.21.15.46:0/1722415350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c108f00 0x7f957c109310 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f95700099c0 tx=0x7f9570009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.049+0000 7f95816d6700 1 -- 172.21.15.46:0/1722415350 shutdown_connections 2024-05-09T00:10:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.049+0000 7f95816d6700 1 --2- 172.21.15.46:0/1722415350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c109850 0x7f957c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.049+0000 7f95816d6700 1 --2- 172.21.15.46:0/1722415350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f957c1045d0 0x7f957c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.049+0000 7f95816d6700 1 --2- 172.21.15.46:0/1722415350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c108f00 0x7f957c109310 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.049+0000 7f95816d6700 1 -- 172.21.15.46:0/1722415350 >> 172.21.15.46:0/1722415350 conn(0x7f957c0ffea0 msgr2=0x7f957c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.049+0000 7f95816d6700 1 -- 172.21.15.46:0/1722415350 shutdown_connections 2024-05-09T00:10:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.049+0000 7f95816d6700 1 -- 172.21.15.46:0/1722415350 wait complete. 2024-05-09T00:10:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.050+0000 7f95816d6700 1 Processor -- start 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.050+0000 7f95816d6700 1 -- start start 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f95816d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c1045d0 0x7f957c076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f95816d6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f957c108f00 0x7f957c077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f95816d6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c109850 0x7f957c077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f95816d6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f957c114460 con 0x7f957c1045d0 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f95816d6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1142e0 con 0x7f957c109850 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f95816d6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1145e0 con 0x7f957c108f00 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c1045d0 0x7f957c076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c1045d0 0x7f957c076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55190/0 (socket says 172.21.15.46:55190) 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957affd700 1 -- 172.21.15.46:0/2656594514 learned_addr learned my addr 172.21.15.46:0/2656594514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957b7fe700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c109850 0x7f957c077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957a7fc700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f957c108f00 0x7f957c077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957affd700 1 -- 172.21.15.46:0/2656594514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f957c108f00 msgr2=0x7f957c077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957affd700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f957c108f00 0x7f957c077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957affd700 1 -- 172.21.15.46:0/2656594514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c109850 msgr2=0x7f957c077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957affd700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c109850 0x7f957c077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957affd700 1 -- 172.21.15.46:0/2656594514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c1a91e0 con 0x7f957c1045d0 2024-05-09T00:10:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957b7fe700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c109850 0x7f957c077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957a7fc700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f957c108f00 0x7f957c077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f957affd700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c1045d0 0x7f957c076bd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f956c00c920 tx=0x7f956c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.051+0000 7f9563fff700 1 -- 172.21.15.46:0/2656594514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c007b00 con 0x7f957c1045d0 2024-05-09T00:10:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.052+0000 7f95816d6700 1 -- 172.21.15.46:0/2656594514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957c1a94d0 con 0x7f957c1045d0 2024-05-09T00:10:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.052+0000 7f9563fff700 1 -- 172.21.15.46:0/2656594514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f956c004ce0 con 0x7f957c1045d0 2024-05-09T00:10:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.052+0000 7f9563fff700 1 -- 172.21.15.46:0/2656594514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c005650 con 0x7f957c1045d0 2024-05-09T00:10:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.052+0000 7f95816d6700 1 -- 172.21.15.46:0/2656594514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957c1a99e0 con 0x7f957c1045d0 2024-05-09T00:10:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.053+0000 7f95816d6700 1 -- 172.21.15.46:0/2656594514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957c0512c0 con 0x7f957c1045d0 2024-05-09T00:10:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.054+0000 7f9563fff700 1 -- 172.21.15.46:0/2656594514 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f956c004750 con 0x7f957c1045d0 2024-05-09T00:10:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.055+0000 7f9563fff700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95640b2880 0x7f95640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.055+0000 7f957a7fc700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95640b2880 0x7f95640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.055+0000 7f9563fff700 1 -- 172.21.15.46:0/2656594514 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f956c0e2220 con 0x7f957c1045d0 2024-05-09T00:10:36.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.056+0000 7f957a7fc700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95640b2880 0x7f95640b4d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9570000c00 tx=0x7f957001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.059+0000 7f9563fff700 1 -- 172.21.15.46:0/2656594514 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f956c014030 con 0x7f957c1045d0 2024-05-09T00:10:36.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.278+0000 7f95816d6700 1 -- 172.21.15.46:0/2656594514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f957c078100 con 0x7f957c1045d0 2024-05-09T00:10:36.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.278+0000 7f9563fff700 1 -- 172.21.15.46:0/2656594514 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f956c0a9f40 con 0x7f957c1045d0 2024-05-09T00:10:36.275 INFO:teuthology.orchestra.run.smithi046.stdout:107374182483 2024-05-09T00:10:36.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.280+0000 7f95816d6700 1 -- 172.21.15.46:0/2656594514 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95640b2880 msgr2=0x7f95640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:36.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.280+0000 7f95816d6700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95640b2880 0x7f95640b4d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9570000c00 tx=0x7f957001a040 comp rx=0 tx=0).stop 2024-05-09T00:10:36.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.280+0000 7f95816d6700 1 -- 172.21.15.46:0/2656594514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c1045d0 msgr2=0x7f957c076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:36.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.280+0000 7f95816d6700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c1045d0 0x7f957c076bd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f956c00c920 tx=0x7f956c00cce0 comp rx=0 tx=0).stop 2024-05-09T00:10:36.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.280+0000 7f95816d6700 1 -- 172.21.15.46:0/2656594514 shutdown_connections 2024-05-09T00:10:36.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.280+0000 7f95816d6700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95640b2880 0x7f95640b4d90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:36.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.280+0000 7f95816d6700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f957c109850 0x7f957c077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:36.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.280+0000 7f95816d6700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c1045d0 0x7f957c076bd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:36.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.280+0000 7f95816d6700 1 --2- 172.21.15.46:0/2656594514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f957c108f00 0x7f957c077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:36.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.280+0000 7f95816d6700 1 -- 172.21.15.46:0/2656594514 >> 172.21.15.46:0/2656594514 conn(0x7f957c0ffea0 msgr2=0x7f957c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:36.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.280+0000 7f95816d6700 1 -- 172.21.15.46:0/2656594514 shutdown_connections 2024-05-09T00:10:36.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:36.280+0000 7f95816d6700 1 -- 172.21.15.46:0/2656594514 wait complete. 2024-05-09T00:10:36.474 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411410 got 77309411416 for osd.2 2024-05-09T00:10:36.474 DEBUG:teuthology.parallel:result is None 2024-05-09T00:10:36.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:36 smithi046 ceph-mon[30151]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:36.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:36 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2656594514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T00:10:36.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:36 smithi169 ceph-mon[34777]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:36.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:36 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2656594514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T00:10:36.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:36 smithi142 ceph-mon[34533]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:36.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:36 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2656594514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T00:10:37.316 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266739 got 227633266744 for osd.8 2024-05-09T00:10:37.316 DEBUG:teuthology.parallel:result is None 2024-05-09T00:10:37.747 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070510 got 253403070515 for osd.9 2024-05-09T00:10:37.747 DEBUG:teuthology.parallel:result is None 2024-05-09T00:10:38.196 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182478 got 107374182483 for osd.3 2024-05-09T00:10:38.196 DEBUG:teuthology.parallel:result is None 2024-05-09T00:10:38.196 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T00:10:38.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T00:10:38.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:38 smithi046 ceph-mon[30151]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:38.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:38 smithi169 ceph-mon[34777]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:38.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:38 smithi142 ceph-mon[34533]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:38.986 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:40.620 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:40 smithi046 ceph-mon[30151]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:10:40.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.639+0000 7f053c333700 1 -- 172.21.15.46:0/3238761784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05341045d0 msgr2=0x7f05341049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.639+0000 7f053c333700 1 --2- 172.21.15.46:0/3238761784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05341045d0 0x7f05341049e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f05300099c0 tx=0x7f0530009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.639+0000 7f053c333700 1 -- 172.21.15.46:0/3238761784 shutdown_connections 2024-05-09T00:10:40.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.639+0000 7f053c333700 1 --2- 172.21.15.46:0/3238761784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0534109850 0x7f05341111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.639+0000 7f053c333700 1 --2- 172.21.15.46:0/3238761784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05341045d0 0x7f05341049e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.639+0000 7f053c333700 1 --2- 172.21.15.46:0/3238761784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0534108f00 0x7f0534109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.639+0000 7f053c333700 1 -- 172.21.15.46:0/3238761784 >> 172.21.15.46:0/3238761784 conn(0x7f05340ffea0 msgr2=0x7f05341022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:40.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.639+0000 7f053c333700 1 -- 172.21.15.46:0/3238761784 shutdown_connections 2024-05-09T00:10:40.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.639+0000 7f053c333700 1 -- 172.21.15.46:0/3238761784 wait complete. 2024-05-09T00:10:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f053c333700 1 Processor -- start 2024-05-09T00:10:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f053c333700 1 -- start start 2024-05-09T00:10:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f053c333700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05341045d0 0x7f0534076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f053c333700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0534108f00 0x7f0534077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f053c333700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0534109850 0x7f0534077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f053c333700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0534114480 con 0x7f0534109850 2024-05-09T00:10:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f053c333700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0534114300 con 0x7f0534108f00 2024-05-09T00:10:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f053c333700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0534114600 con 0x7f05341045d0 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f053a8d0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0534109850 0x7f0534077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f053a8d0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0534109850 0x7f0534077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42410/0 (socket says 172.21.15.46:42410) 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f053a8d0700 1 -- 172.21.15.46:0/2196968883 learned_addr learned my addr 172.21.15.46:0/2196968883 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f05398ce700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0534108f00 0x7f0534077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.640+0000 7f053a0cf700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05341045d0 0x7f0534076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.641+0000 7f053a8d0700 1 -- 172.21.15.46:0/2196968883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05341045d0 msgr2=0x7f0534076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.641+0000 7f053a8d0700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05341045d0 0x7f0534076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.641+0000 7f053a8d0700 1 -- 172.21.15.46:0/2196968883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0534108f00 msgr2=0x7f0534077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.641+0000 7f053a8d0700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0534108f00 0x7f0534077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.641+0000 7f053a8d0700 1 -- 172.21.15.46:0/2196968883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05341a9210 con 0x7f0534109850 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.641+0000 7f053a0cf700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05341045d0 0x7f0534076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.641+0000 7f053a8d0700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0534109850 0x7f0534077650 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f052c00ada0 tx=0x7f052c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.641+0000 7f052b7fe700 1 -- 172.21.15.46:0/2196968883 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c017020 con 0x7f0534109850 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.641+0000 7f053c333700 1 -- 172.21.15.46:0/2196968883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05341a9500 con 0x7f0534109850 2024-05-09T00:10:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.641+0000 7f052b7fe700 1 -- 172.21.15.46:0/2196968883 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f052c00ce30 con 0x7f0534109850 2024-05-09T00:10:40.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.641+0000 7f052b7fe700 1 -- 172.21.15.46:0/2196968883 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c005350 con 0x7f0534109850 2024-05-09T00:10:40.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.641+0000 7f053c333700 1 -- 172.21.15.46:0/2196968883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05341a9a40 con 0x7f0534109850 2024-05-09T00:10:40.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.642+0000 7f053c333700 1 -- 172.21.15.46:0/2196968883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05340512c0 con 0x7f0534109850 2024-05-09T00:10:40.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.645+0000 7f052b7fe700 1 -- 172.21.15.46:0/2196968883 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f052c00c980 con 0x7f0534109850 2024-05-09T00:10:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.646+0000 7f052b7fe700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f05200b2880 0x7f05200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.646+0000 7f053a0cf700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f05200b2880 0x7f05200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.646+0000 7f052b7fe700 1 -- 172.21.15.46:0/2196968883 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f052c0e1e80 con 0x7f0534109850 2024-05-09T00:10:40.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.646+0000 7f053a0cf700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f05200b2880 0x7f05200b4d90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0530000c00 tx=0x7f053001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:40.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.650+0000 7f052b7fe700 1 -- 172.21.15.46:0/2196968883 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f052c021020 con 0x7f0534109850 2024-05-09T00:10:40.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:40 smithi169 ceph-mon[34777]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:10:40.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.869+0000 7f053c333700 1 -- 172.21.15.46:0/2196968883 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0534078100 con 0x7f05200b2880 2024-05-09T00:10:40.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.878+0000 7f052b7fe700 1 -- 172.21.15.46:0/2196968883 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259075 (secure 0 0 0) 0x7f0534078100 con 0x7f05200b2880 2024-05-09T00:10:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:10:40.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.881+0000 7f053c333700 1 -- 172.21.15.46:0/2196968883 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f05200b2880 msgr2=0x7f05200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.881+0000 7f053c333700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f05200b2880 0x7f05200b4d90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0530000c00 tx=0x7f053001a040 comp rx=0 tx=0).stop 2024-05-09T00:10:40.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.881+0000 7f053c333700 1 -- 172.21.15.46:0/2196968883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0534109850 msgr2=0x7f0534077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:40.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.881+0000 7f053c333700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0534109850 0x7f0534077650 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f052c00ada0 tx=0x7f052c00c330 comp rx=0 tx=0).stop 2024-05-09T00:10:40.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.882+0000 7f053c333700 1 -- 172.21.15.46:0/2196968883 shutdown_connections 2024-05-09T00:10:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.882+0000 7f053c333700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f05200b2880 0x7f05200b4d90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.882+0000 7f053c333700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0534109850 0x7f0534077650 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.882+0000 7f053c333700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05341045d0 0x7f0534076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.882+0000 7f053c333700 1 --2- 172.21.15.46:0/2196968883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0534108f00 0x7f0534077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.882+0000 7f053c333700 1 -- 172.21.15.46:0/2196968883 >> 172.21.15.46:0/2196968883 conn(0x7f05340ffea0 msgr2=0x7f0534102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.882+0000 7f053c333700 1 -- 172.21.15.46:0/2196968883 shutdown_connections 2024-05-09T00:10:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:40.882+0000 7f053c333700 1 -- 172.21.15.46:0/2196968883 wait complete. 2024-05-09T00:10:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T00:10:40.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:40 smithi142 ceph-mon[34533]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:10:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-05-09T00:10:39.318241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656819,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":792,"num_read_kb":493,"num_write":297,"num_write_kb":2269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":740,"ondisk_log_size":740,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372288,"kb_used_data":10656,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569536,"statfs":{"total":1151940427776,"available":1151559204864,"internally_reserved":0,"allocated":10911744,"data_stored":7066173,"data_compressed":177897,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003290"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324383+0000","last_change":"2024-05-09T00:08:26.060455+0000","last_active":"2024-05-09T00:10:09.324383+0000","last_peered":"2024-05-09T00:10:09.324383+0000","last_clean":"2024-05-09T00:10:09.324383+0000","last_became_active":"2024-05-09T00:08:26.060068+0000","last_became_peered":"2024-05-09T00:08:26.060068+0000","last_unstale":"2024-05-09T00:10:09.324383+0000","last_undegraded":"2024-05-09T00:10:09.324383+0000","last_fullsized":"2024-05-09T00:10:09.324383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:48.354188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.314519+0000","last_change":"2024-05-09T00:08:26.055737+0000","last_active":"2024-05-09T00:10:09.314519+0000","last_peered":"2024-05-09T00:10:09.314519+0000","last_clean":"2024-05-09T00:10:09.314519+0000","last_became_active":"2024-05-09T00:08:26.055354+0000","last_became_peered":"2024-05-09T00:08:26.055354+0000","last_unstale":"2024-05-09T00:10:09.314519+0000","last_undegraded":"2024-05-09T00:10:09.314519+0000","last_fullsized":"2024-05-09T00:10:09.314519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:54.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324383+0000","last_change":"2024-05-09T00:08:26.050776+0000","last_active":"2024-05-09T00:10:09.324383+0000","last_peered":"2024-05-09T00:10:09.324383+0000","last_clean":"2024-05-09T00:10:09.324383+0000","last_became_active":"2024-05-09T00:08:26.050393+0000","last_became_peered":"2024-05-09T00:08:26.050393+0000","last_unstale":"2024-05-09T00:10:09.324383+0000","last_undegraded":"2024-05-09T00:10:09.324383+0000","last_fullsized":"2024-05-09T00:10:09.324383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:30.819219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.307617+0000","last_change":"2024-05-09T00:08:26.065946+0000","last_active":"2024-05-09T00:10:09.307617+0000","last_peered":"2024-05-09T00:10:09.307617+0000","last_clean":"2024-05-09T00:10:09.307617+0000","last_became_active":"2024-05-09T00:08:26.065498+0000","last_became_peered":"2024-05-09T00:08:26.065498+0000","last_unstale":"2024-05-09T00:10:09.307617+0000","last_undegraded":"2024-05-09T00:10:09.307617+0000","last_fullsized":"2024-05-09T00:10:09.307617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:04.444860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.319557+0000","last_change":"2024-05-09T00:08:26.051961+0000","last_active":"2024-05-09T00:10:09.319557+0000","last_peered":"2024-05-09T00:10:09.319557+0000","last_clean":"2024-05-09T00:10:09.319557+0000","last_became_active":"2024-05-09T00:08:26.051548+0000","last_became_peered":"2024-05-09T00:08:26.051548+0000","last_unstale":"2024-05-09T00:10:09.319557+0000","last_undegraded":"2024-05-09T00:10:09.319557+0000","last_fullsized":"2024-05-09T00:10:09.319557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:07.539117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.311285+0000","last_change":"2024-05-09T00:08:26.052012+0000","last_active":"2024-05-09T00:10:09.311285+0000","last_peered":"2024-05-09T00:10:09.311285+0000","last_clean":"2024-05-09T00:10:09.311285+0000","last_became_active":"2024-05-09T00:08:26.050886+0000","last_became_peered":"2024-05-09T00:08:26.050886+0000","last_unstale":"2024-05-09T00:10:09.311285+0000","last_undegraded":"2024-05-09T00:10:09.311285+0000","last_fullsized":"2024-05-09T00:10:09.311285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:14.651888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324098+0000","last_change":"2024-05-09T00:08:26.058418+0000","last_active":"2024-05-09T00:10:09.324098+0000","last_peered":"2024-05-09T00:10:09.324098+0000","last_clean":"2024-05-09T00:10:09.324098+0000","last_became_active":"2024-05-09T00:08:26.058039+0000","last_became_peered":"2024-05-09T00:08:26.058039+0000","last_unstale":"2024-05-09T00:10:09.324098+0000","last_undegraded":"2024-05-09T00:10:09.324098+0000","last_fullsized":"2024-05-09T00:10:09.324098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:13.678123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.322184+0000","last_change":"2024-05-09T00:08:26.057602+0000","last_active":"2024-05-09T00:10:09.322184+0000","last_peered":"2024-05-09T00:10:09.322184+0000","last_clean":"2024-05-09T00:10:09.322184+0000","last_became_active":"2024-05-09T00:08:26.057169+0000","last_became_peered":"2024-05-09T00:08:26.057169+0000","last_unstale":"2024-05-09T00:10:09.322184+0000","last_undegraded":"2024-05-09T00:10:09.322184+0000","last_fullsized":"2024-05-09T00:10:09.322184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:11.452083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326985+0000","last_change":"2024-05-09T00:08:26.067639+0000","last_active":"2024-05-09T00:10:09.326985+0000","last_peered":"2024-05-09T00:10:09.326985+0000","last_clean":"2024-05-09T00:10:09.326985+0000","last_became_active":"2024-05-09T00:08:26.067226+0000","last_became_peered":"2024-05-09T00:08:26.067226+0000","last_unstale":"2024-05-09T00:10:09.326985+0000","last_undegraded":"2024-05-09T00:10:09.326985+0000","last_fullsized":"2024-05-09T00:10:09.326985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:12.366500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.309003+0000","last_change":"2024-05-09T00:08:26.055234+0000","last_active":"2024-05-09T00:10:09.309003+0000","last_peered":"2024-05-09T00:10:09.309003+0000","last_clean":"2024-05-09T00:10:09.309003+0000","last_became_active":"2024-05-09T00:08:26.054851+0000","last_became_peered":"2024-05-09T00:08:26.054851+0000","last_unstale":"2024-05-09T00:10:09.309003+0000","last_undegraded":"2024-05-09T00:10:09.309003+0000","last_fullsized":"2024-05-09T00:10:09.309003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:54:52.121329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.329724+0000","last_change":"2024-05-09T00:08:26.063375+0000","last_active":"2024-05-09T00:10:09.329724+0000","last_peered":"2024-05-09T00:10:09.329724+0000","last_clean":"2024-05-09T00:10:09.329724+0000","last_became_active":"2024-05-09T00:08:26.062962+0000","last_became_peered":"2024-05-09T00:08:26.062962+0000","last_unstale":"2024-05-09T00:10:09.329724+0000","last_undegraded":"2024-05-09T00:10:09.329724+0000","last_fullsized":"2024-05-09T00:10:09.329724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:05.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.305258+0000","last_change":"2024-05-09T00:08:26.062034+0000","last_active":"2024-05-09T00:10:09.305258+0000","last_peered":"2024-05-09T00:10:09.305258+0000","last_clean":"2024-05-09T00:10:09.305258+0000","last_became_active":"2024-05-09T00:08:26.060376+0000","last_became_peered":"2024-05-09T00:08:26.060376+0000","last_unstale":"2024-05-09T00:10:09.305258+0000","last_undegraded":"2024-05-09T00:10:09.305258+0000","last_fullsized":"2024-05-09T00:10:09.305258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:15.830024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327723+0000","last_change":"2024-05-09T00:08:26.053227+0000","last_active":"2024-05-09T00:10:09.327723+0000","last_peered":"2024-05-09T00:10:09.327723+0000","last_clean":"2024-05-09T00:10:09.327723+0000","last_became_active":"2024-05-09T00:08:26.052878+0000","last_became_peered":"2024-05-09T00:08:26.052878+0000","last_unstale":"2024-05-09T00:10:09.327723+0000","last_undegraded":"2024-05-09T00:10:09.327723+0000","last_fullsized":"2024-05-09T00:10:09.327723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:24.385026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.314291+0000","last_change":"2024-05-09T00:08:26.053041+0000","last_active":"2024-05-09T00:10:09.314291+0000","last_peered":"2024-05-09T00:10:09.314291+0000","last_clean":"2024-05-09T00:10:09.314291+0000","last_became_active":"2024-05-09T00:08:26.052677+0000","last_became_peered":"2024-05-09T00:08:26.052677+0000","last_unstale":"2024-05-09T00:10:09.314291+0000","last_undegraded":"2024-05-09T00:10:09.314291+0000","last_fullsized":"2024-05-09T00:10:09.314291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:24.478270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.315792+0000","last_change":"2024-05-09T00:08:26.063391+0000","last_active":"2024-05-09T00:10:09.315792+0000","last_peered":"2024-05-09T00:10:09.315792+0000","last_clean":"2024-05-09T00:10:09.315792+0000","last_became_active":"2024-05-09T00:08:26.063033+0000","last_became_peered":"2024-05-09T00:08:26.063033+0000","last_unstale":"2024-05-09T00:10:09.315792+0000","last_undegraded":"2024-05-09T00:10:09.315792+0000","last_fullsized":"2024-05-09T00:10:09.315792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:41.900453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.322709+0000","last_change":"2024-05-09T00:08:26.065018+0000","last_active":"2024-05-09T00:10:09.322709+0000","last_peered":"2024-05-09T00:10:09.322709+0000","last_clean":"2024-05-09T00:10:09.322709+0000","last_became_active":"2024-05-09T00:08:26.064642+0000","last_became_peered":"2024-05-09T00:08:26.064642+0000","last_unstale":"2024-05-09T00:10:09.322709+0000","last_undegraded":"2024-05-09T00:10:09.322709+0000","last_fullsized":"2024-05-09T00:10:09.322709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:02.403704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324011+0000","last_change":"2024-05-09T00:08:26.049770+0000","last_active":"2024-05-09T00:10:09.324011+0000","last_peered":"2024-05-09T00:10:09.324011+0000","last_clean":"2024-05-09T00:10:09.324011+0000","last_became_active":"2024-05-09T00:08:26.048555+0000","last_became_peered":"2024-05-09T00:08:26.048555+0000","last_unstale":"2024-05-09T00:10:09.324011+0000","last_undegraded":"2024-05-09T00:10:09.324011+0000","last_fullsized":"2024-05-09T00:10:09.324011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:16.009200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.317346+0000","last_change":"2024-05-09T00:08:26.048587+0000","last_active":"2024-05-09T00:10:09.317346+0000","last_peered":"2024-05-09T00:10:09.317346+0000","last_clean":"2024-05-09T00:10:09.317346+0000","last_became_active":"2024-05-09T00:08:26.048148+0000","last_became_peered":"2024-05-09T00:08:26.048148+0000","last_unstale":"2024-05-09T00:10:09.317346+0000","last_undegraded":"2024-05-09T00:10:09.317346+0000","last_fullsized":"2024-05-09T00:10:09.317346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:42.113829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.322112+0000","last_change":"2024-05-09T00:08:26.053607+0000","last_active":"2024-05-09T00:10:09.322112+0000","last_peered":"2024-05-09T00:10:09.322112+0000","last_clean":"2024-05-09T00:10:09.322112+0000","last_became_active":"2024-05-09T00:08:26.053212+0000","last_became_peered":"2024-05-09T00:08:26.053212+0000","last_unstale":"2024-05-09T00:10:09.322112+0000","last_undegraded":"2024-05-09T00:10:09.322112+0000","last_fullsized":"2024-05-09T00:10:09.322112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:32.937049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469352+0000","last_change":"2024-05-09T00:08:23.024481+0000","last_active":"2024-05-09T00:09:10.469352+0000","last_peered":"2024-05-09T00:09:10.469352+0000","last_clean":"2024-05-09T00:09:10.469352+0000","last_became_active":"2024-05-09T00:08:23.024132+0000","last_became_peered":"2024-05-09T00:08:23.024132+0000","last_unstale":"2024-05-09T00:09:10.469352+0000","last_undegraded":"2024-05-09T00:09:10.469352+0000","last_fullsized":"2024-05-09T00:09:10.469352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:11.188505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325280+0000","last_change":"2024-05-09T00:08:26.054772+0000","last_active":"2024-05-09T00:10:09.325280+0000","last_peered":"2024-05-09T00:10:09.325280+0000","last_clean":"2024-05-09T00:10:09.325280+0000","last_became_active":"2024-05-09T00:08:26.054433+0000","last_became_peered":"2024-05-09T00:08:26.054433+0000","last_unstale":"2024-05-09T00:10:09.325280+0000","last_undegraded":"2024-05-09T00:10:09.325280+0000","last_fullsized":"2024-05-09T00:10:09.325280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:44.283240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468356+0000","last_change":"2024-05-09T00:08:23.118778+0000","last_active":"2024-05-09T00:09:10.468356+0000","last_peered":"2024-05-09T00:09:10.468356+0000","last_clean":"2024-05-09T00:09:10.468356+0000","last_became_active":"2024-05-09T00:08:23.118377+0000","last_became_peered":"2024-05-09T00:08:23.118377+0000","last_unstale":"2024-05-09T00:09:10.468356+0000","last_undegraded":"2024-05-09T00:09:10.468356+0000","last_fullsized":"2024-05-09T00:09:10.468356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:51.352442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325452+0000","last_change":"2024-05-09T00:08:26.059953+0000","last_active":"2024-05-09T00:10:09.325452+0000","last_peered":"2024-05-09T00:10:09.325452+0000","last_clean":"2024-05-09T00:10:09.325452+0000","last_became_active":"2024-05-09T00:08:26.059571+0000","last_became_peered":"2024-05-09T00:08:26.059571+0000","last_unstale":"2024-05-09T00:10:09.325452+0000","last_undegraded":"2024-05-09T00:10:09.325452+0000","last_fullsized":"2024-05-09T00:10:09.325452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:48.056798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475240+0000","last_change":"2024-05-09T00:08:23.021244+0000","last_active":"2024-05-09T00:09:10.475240+0000","last_peered":"2024-05-09T00:09:10.475240+0000","last_clean":"2024-05-09T00:09:10.475240+0000","last_became_active":"2024-05-09T00:08:23.020597+0000","last_became_peered":"2024-05-09T00:08:23.020597+0000","last_unstale":"2024-05-09T00:09:10.475240+0000","last_undegraded":"2024-05-09T00:09:10.475240+0000","last_fullsized":"2024-05-09T00:09:10.475240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.654261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327796+0000","last_change":"2024-05-09T00:08:26.059834+0000","last_active":"2024-05-09T00:10:09.327796+0000","last_peered":"2024-05-09T00:10:09.327796+0000","last_clean":"2024-05-09T00:10:09.327796+0000","last_became_active":"2024-05-09T00:08:26.059349+0000","last_became_peered":"2024-05-09T00:08:26.059349+0000","last_unstale":"2024-05-09T00:10:09.327796+0000","last_undegraded":"2024-05-09T00:10:09.327796+0000","last_fullsized":"2024-05-09T00:10:09.327796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:42.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474158+0000","last_change":"2024-05-09T00:08:23.019568+0000","last_active":"2024-05-09T00:09:10.474158+0000","last_peered":"2024-05-09T00:09:10.474158+0000","last_clean":"2024-05-09T00:09:10.474158+0000","last_became_active":"2024-05-09T00:08:23.019149+0000","last_became_peered":"2024-05-09T00:08:23.019149+0000","last_unstale":"2024-05-09T00:09:10.474158+0000","last_undegraded":"2024-05-09T00:09:10.474158+0000","last_fullsized":"2024-05-09T00:09:10.474158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:44.569911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328328+0000","last_change":"2024-05-09T00:08:26.061308+0000","last_active":"2024-05-09T00:10:09.328328+0000","last_peered":"2024-05-09T00:10:09.328328+0000","last_clean":"2024-05-09T00:10:09.328328+0000","last_became_active":"2024-05-09T00:08:26.060973+0000","last_became_peered":"2024-05-09T00:08:26.060973+0000","last_unstale":"2024-05-09T00:10:09.328328+0000","last_undegraded":"2024-05-09T00:10:09.328328+0000","last_fullsized":"2024-05-09T00:10:09.328328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:16.845074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":145,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:35.556462+0000","last_change":"2024-05-09T00:08:23.020886+0000","last_active":"2024-05-09T00:10:35.556462+0000","last_peered":"2024-05-09T00:10:35.556462+0000","last_clean":"2024-05-09T00:10:35.556462+0000","last_became_active":"2024-05-09T00:08:23.020530+0000","last_became_peered":"2024-05-09T00:08:23.020530+0000","last_unstale":"2024-05-09T00:10:35.556462+0000","last_undegraded":"2024-05-09T00:10:35.556462+0000","last_fullsized":"2024-05-09T00:10:35.556462+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:34.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318636+0000","last_change":"2024-05-09T00:08:26.051018+0000","last_active":"2024-05-09T00:10:09.318636+0000","last_peered":"2024-05-09T00:10:09.318636+0000","last_clean":"2024-05-09T00:10:09.318636+0000","last_became_active":"2024-05-09T00:08:26.050495+0000","last_became_peered":"2024-05-09T00:08:26.050495+0000","last_unstale":"2024-05-09T00:10:09.318636+0000","last_undegraded":"2024-05-09T00:10:09.318636+0000","last_fullsized":"2024-05-09T00:10:09.318636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:07.862339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472087+0000","last_change":"2024-05-09T00:08:23.021187+0000","last_active":"2024-05-09T00:09:10.472087+0000","last_peered":"2024-05-09T00:09:10.472087+0000","last_clean":"2024-05-09T00:09:10.472087+0000","last_became_active":"2024-05-09T00:08:23.020770+0000","last_became_peered":"2024-05-09T00:08:23.020770+0000","last_unstale":"2024-05-09T00:09:10.472087+0000","last_undegraded":"2024-05-09T00:09:10.472087+0000","last_fullsized":"2024-05-09T00:09:10.472087+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:08.992770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.316728+0000","last_change":"2024-05-09T00:09:10.797392+0000","last_active":"2024-05-09T00:10:09.316728+0000","last_peered":"2024-05-09T00:10:09.316728+0000","last_clean":"2024-05-09T00:10:09.316728+0000","last_became_active":"2024-05-09T00:09:10.796969+0000","last_became_peered":"2024-05-09T00:09:10.796969+0000","last_unstale":"2024-05-09T00:10:09.316728+0000","last_undegraded":"2024-05-09T00:10:09.316728+0000","last_fullsized":"2024-05-09T00:10:09.316728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:18.852678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548755+0000","last_change":"2024-05-09T00:08:23.012226+0000","last_active":"2024-05-09T00:09:10.548755+0000","last_peered":"2024-05-09T00:09:10.548755+0000","last_clean":"2024-05-09T00:09:10.548755+0000","last_became_active":"2024-05-09T00:08:23.011781+0000","last_became_peered":"2024-05-09T00:08:23.011781+0000","last_unstale":"2024-05-09T00:09:10.548755+0000","last_undegraded":"2024-05-09T00:09:10.548755+0000","last_fullsized":"2024-05-09T00:09:10.548755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:44.452206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.321520+0000","last_change":"2024-05-09T00:08:26.043035+0000","last_active":"2024-05-09T00:10:09.321520+0000","last_peered":"2024-05-09T00:10:09.321520+0000","last_clean":"2024-05-09T00:10:09.321520+0000","last_became_active":"2024-05-09T00:08:26.042686+0000","last_became_peered":"2024-05-09T00:08:26.042686+0000","last_unstale":"2024-05-09T00:10:09.321520+0000","last_undegraded":"2024-05-09T00:10:09.321520+0000","last_fullsized":"2024-05-09T00:10:09.321520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:24.511309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469042+0000","last_change":"2024-05-09T00:08:23.022017+0000","last_active":"2024-05-09T00:09:10.469042+0000","last_peered":"2024-05-09T00:09:10.469042+0000","last_clean":"2024-05-09T00:09:10.469042+0000","last_became_active":"2024-05-09T00:08:23.021540+0000","last_became_peered":"2024-05-09T00:08:23.021540+0000","last_unstale":"2024-05-09T00:09:10.469042+0000","last_undegraded":"2024-05-09T00:09:10.469042+0000","last_fullsized":"2024-05-09T00:09:10.469042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:50.773211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325952+0000","last_change":"2024-05-09T00:08:26.048039+0000","last_active":"2024-05-09T00:10:09.325952+0000","last_peered":"2024-05-09T00:10:09.325952+0000","last_clean":"2024-05-09T00:10:09.325952+0000","last_became_active":"2024-05-09T00:08:26.047695+0000","last_became_peered":"2024-05-09T00:08:26.047695+0000","last_unstale":"2024-05-09T00:10:09.325952+0000","last_undegraded":"2024-05-09T00:10:09.325952+0000","last_fullsized":"2024-05-09T00:10:09.325952+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:27.363847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":144,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:37.558867+0000","last_change":"2024-05-09T00:08:23.019565+0000","last_active":"2024-05-09T00:10:37.558867+0000","last_peered":"2024-05-09T00:10:37.558867+0000","last_clean":"2024-05-09T00:10:37.558867+0000","last_became_active":"2024-05-09T00:08:23.019113+0000","last_became_peered":"2024-05-09T00:08:23.019113+0000","last_unstale":"2024-05-09T00:10:37.558867+0000","last_undegraded":"2024-05-09T00:10:37.558867+0000","last_fullsized":"2024-05-09T00:10:37.558867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:45.578934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323716+0000","last_change":"2024-05-09T00:08:26.058869+0000","last_active":"2024-05-09T00:10:09.323716+0000","last_peered":"2024-05-09T00:10:09.323716+0000","last_clean":"2024-05-09T00:10:09.323716+0000","last_became_active":"2024-05-09T00:08:26.058482+0000","last_became_peered":"2024-05-09T00:08:26.058482+0000","last_unstale":"2024-05-09T00:10:09.323716+0000","last_undegraded":"2024-05-09T00:10:09.323716+0000","last_fullsized":"2024-05-09T00:10:09.323716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:33.252208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472150+0000","last_change":"2024-05-09T00:08:23.024425+0000","last_active":"2024-05-09T00:09:10.472150+0000","last_peered":"2024-05-09T00:09:10.472150+0000","last_clean":"2024-05-09T00:09:10.472150+0000","last_became_active":"2024-05-09T00:08:23.023991+0000","last_became_peered":"2024-05-09T00:08:23.023991+0000","last_unstale":"2024-05-09T00:09:10.472150+0000","last_undegraded":"2024-05-09T00:09:10.472150+0000","last_fullsized":"2024-05-09T00:09:10.472150+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:47.907851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330824+0000","last_change":"2024-05-09T00:09:10.475008+0000","last_active":"2024-05-09T00:10:09.330824+0000","last_peered":"2024-05-09T00:10:09.330824+0000","last_clean":"2024-05-09T00:10:09.330824+0000","last_became_active":"2024-05-09T00:09:10.474663+0000","last_became_peered":"2024-05-09T00:09:10.474663+0000","last_unstale":"2024-05-09T00:10:09.330824+0000","last_undegraded":"2024-05-09T00:10:09.330824+0000","last_fullsized":"2024-05-09T00:10:09.330824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:36.990362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470120+0000","last_change":"2024-05-09T00:08:23.118516+0000","last_active":"2024-05-09T00:09:10.470120+0000","last_peered":"2024-05-09T00:09:10.470120+0000","last_clean":"2024-05-09T00:09:10.470120+0000","last_became_active":"2024-05-09T00:08:23.118147+0000","last_became_peered":"2024-05-09T00:08:23.118147+0000","last_unstale":"2024-05-09T00:09:10.470120+0000","last_undegraded":"2024-05-09T00:09:10.470120+0000","last_fullsized":"2024-05-09T00:09:10.470120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:24.205818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326723+0000","last_change":"2024-05-09T00:08:26.067063+0000","last_active":"2024-05-09T00:10:09.326723+0000","last_peered":"2024-05-09T00:10:09.326723+0000","last_clean":"2024-05-09T00:10:09.326723+0000","last_became_active":"2024-05-09T00:08:26.066440+0000","last_became_peered":"2024-05-09T00:08:26.066440+0000","last_unstale":"2024-05-09T00:10:09.326723+0000","last_undegraded":"2024-05-09T00:10:09.326723+0000","last_fullsized":"2024-05-09T00:10:09.326723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:48.648485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'33","reported_seq":299,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:38.560319+0000","last_change":"2024-05-09T00:08:23.020757+0000","last_active":"2024-05-09T00:10:38.560319+0000","last_peered":"2024-05-09T00:10:38.560319+0000","last_clean":"2024-05-09T00:10:38.560319+0000","last_became_active":"2024-05-09T00:08:23.020355+0000","last_became_peered":"2024-05-09T00:08:23.020355+0000","last_unstale":"2024-05-09T00:10:38.560319+0000","last_undegraded":"2024-05-09T00:10:38.560319+0000","last_fullsized":"2024-05-09T00:10:38.560319+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:00.605691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62075,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":240,"num_read_kb":180,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328107+0000","last_change":"2024-05-09T00:08:26.057402+0000","last_active":"2024-05-09T00:10:09.328107+0000","last_peered":"2024-05-09T00:10:09.328107+0000","last_clean":"2024-05-09T00:10:09.328107+0000","last_became_active":"2024-05-09T00:08:26.056974+0000","last_became_peered":"2024-05-09T00:08:26.056974+0000","last_unstale":"2024-05-09T00:10:09.328107+0000","last_undegraded":"2024-05-09T00:10:09.328107+0000","last_fullsized":"2024-05-09T00:10:09.328107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:47.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473890+0000","last_change":"2024-05-09T00:08:23.020208+0000","last_active":"2024-05-09T00:09:10.473890+0000","last_peered":"2024-05-09T00:09:10.473890+0000","last_clean":"2024-05-09T00:09:10.473890+0000","last_became_active":"2024-05-09T00:08:23.019809+0000","last_became_peered":"2024-05-09T00:08:23.019809+0000","last_unstale":"2024-05-09T00:09:10.473890+0000","last_undegraded":"2024-05-09T00:09:10.473890+0000","last_fullsized":"2024-05-09T00:09:10.473890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:47.900731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330380+0000","last_change":"2024-05-09T00:08:26.054273+0000","last_active":"2024-05-09T00:10:09.330380+0000","last_peered":"2024-05-09T00:10:09.330380+0000","last_clean":"2024-05-09T00:10:09.330380+0000","last_became_active":"2024-05-09T00:08:26.053840+0000","last_became_peered":"2024-05-09T00:08:26.053840+0000","last_unstale":"2024-05-09T00:10:09.330380+0000","last_undegraded":"2024-05-09T00:10:09.330380+0000","last_fullsized":"2024-05-09T00:10:09.330380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:27.128582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469240+0000","last_change":"2024-05-09T00:08:23.021594+0000","last_active":"2024-05-09T00:09:10.469240+0000","last_peered":"2024-05-09T00:09:10.469240+0000","last_clean":"2024-05-09T00:09:10.469240+0000","last_became_active":"2024-05-09T00:08:23.021152+0000","last_became_peered":"2024-05-09T00:08:23.021152+0000","last_unstale":"2024-05-09T00:09:10.469240+0000","last_undegraded":"2024-05-09T00:09:10.469240+0000","last_fullsized":"2024-05-09T00:09:10.469240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:58.505781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330080+0000","last_change":"2024-05-09T00:08:26.064464+0000","last_active":"2024-05-09T00:10:09.330080+0000","last_peered":"2024-05-09T00:10:09.330080+0000","last_clean":"2024-05-09T00:10:09.330080+0000","last_became_active":"2024-05-09T00:08:26.064000+0000","last_became_peered":"2024-05-09T00:08:26.064000+0000","last_unstale":"2024-05-09T00:10:09.330080+0000","last_undegraded":"2024-05-09T00:10:09.330080+0000","last_fullsized":"2024-05-09T00:10:09.330080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:01.455979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":149,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:36.558071+0000","last_change":"2024-05-09T00:08:23.020823+0000","last_active":"2024-05-09T00:10:36.558071+0000","last_peered":"2024-05-09T00:10:36.558071+0000","last_clean":"2024-05-09T00:10:36.558071+0000","last_became_active":"2024-05-09T00:08:23.020412+0000","last_became_peered":"2024-05-09T00:08:23.020412+0000","last_unstale":"2024-05-09T00:10:36.558071+0000","last_undegraded":"2024-05-09T00:10:36.558071+0000","last_fullsized":"2024-05-09T00:10:36.558071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:05.457452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.317309+0000","last_change":"2024-05-09T00:09:10.474540+0000","last_active":"2024-05-09T00:10:09.317309+0000","last_peered":"2024-05-09T00:10:09.317309+0000","last_clean":"2024-05-09T00:10:09.317309+0000","last_became_active":"2024-05-09T00:09:10.474068+0000","last_became_peered":"2024-05-09T00:09:10.474068+0000","last_unstale":"2024-05-09T00:10:09.317309+0000","last_undegraded":"2024-05-09T00:10:09.317309+0000","last_fullsized":"2024-05-09T00:10:09.317309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:37.296140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474161+0000","last_change":"2024-05-09T00:08:23.118857+0000","last_active":"2024-05-09T00:09:10.474161+0000","last_peered":"2024-05-09T00:09:10.474161+0000","last_clean":"2024-05-09T00:09:10.474161+0000","last_became_active":"2024-05-09T00:08:23.118433+0000","last_became_peered":"2024-05-09T00:08:23.118433+0000","last_unstale":"2024-05-09T00:09:10.474161+0000","last_undegraded":"2024-05-09T00:09:10.474161+0000","last_fullsized":"2024-05-09T00:09:10.474161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:10.617987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327020+0000","last_change":"2024-05-09T00:08:26.066911+0000","last_active":"2024-05-09T00:10:09.327020+0000","last_peered":"2024-05-09T00:10:09.327020+0000","last_clean":"2024-05-09T00:10:09.327020+0000","last_became_active":"2024-05-09T00:08:26.066548+0000","last_became_peered":"2024-05-09T00:08:26.066548+0000","last_unstale":"2024-05-09T00:10:09.327020+0000","last_undegraded":"2024-05-09T00:10:09.327020+0000","last_fullsized":"2024-05-09T00:10:09.327020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:32.391390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475068+0000","last_change":"2024-05-09T00:08:23.019829+0000","last_active":"2024-05-09T00:09:10.475068+0000","last_peered":"2024-05-09T00:09:10.475068+0000","last_clean":"2024-05-09T00:09:10.475068+0000","last_became_active":"2024-05-09T00:08:23.019336+0000","last_became_peered":"2024-05-09T00:08:23.019336+0000","last_unstale":"2024-05-09T00:09:10.475068+0000","last_undegraded":"2024-05-09T00:09:10.475068+0000","last_fullsized":"2024-05-09T00:09:10.475068+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:33.802738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318674+0000","last_change":"2024-05-09T00:08:26.055975+0000","last_active":"2024-05-09T00:10:09.318674+0000","last_peered":"2024-05-09T00:10:09.318674+0000","last_clean":"2024-05-09T00:10:09.318674+0000","last_became_active":"2024-05-09T00:08:26.055595+0000","last_became_peered":"2024-05-09T00:08:26.055595+0000","last_unstale":"2024-05-09T00:10:09.318674+0000","last_undegraded":"2024-05-09T00:10:09.318674+0000","last_fullsized":"2024-05-09T00:10:09.318674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:20.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469423+0000","last_change":"2024-05-09T00:08:23.120353+0000","last_active":"2024-05-09T00:09:10.469423+0000","last_peered":"2024-05-09T00:09:10.469423+0000","last_clean":"2024-05-09T00:09:10.469423+0000","last_became_active":"2024-05-09T00:08:23.120004+0000","last_became_peered":"2024-05-09T00:08:23.120004+0000","last_unstale":"2024-05-09T00:09:10.469423+0000","last_undegraded":"2024-05-09T00:09:10.469423+0000","last_fullsized":"2024-05-09T00:09:10.469423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:12.168671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.320556+0000","last_change":"2024-05-09T00:08:26.046554+0000","last_active":"2024-05-09T00:10:09.320556+0000","last_peered":"2024-05-09T00:10:09.320556+0000","last_clean":"2024-05-09T00:10:09.320556+0000","last_became_active":"2024-05-09T00:08:26.045924+0000","last_became_peered":"2024-05-09T00:08:26.045924+0000","last_unstale":"2024-05-09T00:10:09.320556+0000","last_undegraded":"2024-05-09T00:10:09.320556+0000","last_fullsized":"2024-05-09T00:10:09.320556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:01.208980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472462+0000","last_change":"2024-05-09T00:08:23.019778+0000","last_active":"2024-05-09T00:09:10.472462+0000","last_peered":"2024-05-09T00:09:10.472462+0000","last_clean":"2024-05-09T00:09:10.472462+0000","last_became_active":"2024-05-09T00:08:23.019400+0000","last_became_peered":"2024-05-09T00:08:23.019400+0000","last_unstale":"2024-05-09T00:09:10.472462+0000","last_undegraded":"2024-05-09T00:09:10.472462+0000","last_fullsized":"2024-05-09T00:09:10.472462+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:06.883167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.308957+0000","last_change":"2024-05-09T00:09:10.474352+0000","last_active":"2024-05-09T00:10:09.308957+0000","last_peered":"2024-05-09T00:10:09.308957+0000","last_clean":"2024-05-09T00:10:09.308957+0000","last_became_active":"2024-05-09T00:09:10.473906+0000","last_became_peered":"2024-05-09T00:09:10.473906+0000","last_unstale":"2024-05-09T00:10:09.308957+0000","last_undegraded":"2024-05-09T00:10:09.308957+0000","last_fullsized":"2024-05-09T00:10:09.308957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:35:18.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.319595+0000","last_change":"2024-05-09T00:09:10.476484+0000","last_active":"2024-05-09T00:10:09.319595+0000","last_peered":"2024-05-09T00:10:09.319595+0000","last_clean":"2024-05-09T00:10:09.319595+0000","last_became_active":"2024-05-09T00:09:10.476001+0000","last_became_peered":"2024-05-09T00:09:10.476001+0000","last_unstale":"2024-05-09T00:10:09.319595+0000","last_undegraded":"2024-05-09T00:10:09.319595+0000","last_fullsized":"2024-05-09T00:10:09.319595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:32.018749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":147,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:37.559184+0000","last_change":"2024-05-09T00:08:23.118322+0000","last_active":"2024-05-09T00:10:37.559184+0000","last_peered":"2024-05-09T00:10:37.559184+0000","last_clean":"2024-05-09T00:10:37.559184+0000","last_became_active":"2024-05-09T00:08:23.117943+0000","last_became_peered":"2024-05-09T00:08:23.117943+0000","last_unstale":"2024-05-09T00:10:37.559184+0000","last_undegraded":"2024-05-09T00:10:37.559184+0000","last_fullsized":"2024-05-09T00:10:37.559184+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:07.890437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323361+0000","last_change":"2024-05-09T00:08:26.052294+0000","last_active":"2024-05-09T00:10:09.323361+0000","last_peered":"2024-05-09T00:10:09.323361+0000","last_clean":"2024-05-09T00:10:09.323361+0000","last_became_active":"2024-05-09T00:08:26.051926+0000","last_became_peered":"2024-05-09T00:08:26.051926+0000","last_unstale":"2024-05-09T00:10:09.323361+0000","last_undegraded":"2024-05-09T00:10:09.323361+0000","last_fullsized":"2024-05-09T00:10:09.323361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:53.651401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796358+0000","last_change":"2024-05-09T00:08:23.015007+0000","last_active":"2024-05-09T00:09:10.796358+0000","last_peered":"2024-05-09T00:09:10.796358+0000","last_clean":"2024-05-09T00:09:10.796358+0000","last_became_active":"2024-05-09T00:08:23.014545+0000","last_became_peered":"2024-05-09T00:08:23.014545+0000","last_unstale":"2024-05-09T00:09:10.796358+0000","last_undegraded":"2024-05-09T00:09:10.796358+0000","last_fullsized":"2024-05-09T00:09:10.796358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:19.850200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":100,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472390+0000","last_change":"2024-05-09T00:07:00.733233+0000","last_active":"2024-05-09T00:09:10.472390+0000","last_peered":"2024-05-09T00:09:10.472390+0000","last_clean":"2024-05-09T00:09:10.472390+0000","last_became_active":"2024-05-09T00:07:00.727365+0000","last_became_peered":"2024-05-09T00:07:00.727365+0000","last_unstale":"2024-05-09T00:09:10.472390+0000","last_undegraded":"2024-05-09T00:09:10.472390+0000","last_fullsized":"2024-05-09T00:09:10.472390+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_clean_scrub_stamp":"2024-05-09T00:03:33.285220+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:08.597290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326209+0000","last_change":"2024-05-09T00:08:26.064000+0000","last_active":"2024-05-09T00:10:09.326209+0000","last_peered":"2024-05-09T00:10:09.326209+0000","last_clean":"2024-05-09T00:10:09.326209+0000","last_became_active":"2024-05-09T00:08:26.063636+0000","last_became_peered":"2024-05-09T00:08:26.063636+0000","last_unstale":"2024-05-09T00:10:09.326209+0000","last_undegraded":"2024-05-09T00:10:09.326209+0000","last_fullsized":"2024-05-09T00:10:09.326209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:59.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472596+0000","last_change":"2024-05-09T00:08:23.023459+0000","last_active":"2024-05-09T00:09:10.472596+0000","last_peered":"2024-05-09T00:09:10.472596+0000","last_clean":"2024-05-09T00:09:10.472596+0000","last_became_active":"2024-05-09T00:08:23.023089+0000","last_became_peered":"2024-05-09T00:08:23.023089+0000","last_unstale":"2024-05-09T00:09:10.472596+0000","last_undegraded":"2024-05-09T00:09:10.472596+0000","last_fullsized":"2024-05-09T00:09:10.472596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:41.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.479186+0000","last_change":"2024-05-09T00:09:10.478686+0000","last_active":"2024-05-09T00:09:10.479186+0000","last_peered":"2024-05-09T00:09:10.479186+0000","last_clean":"2024-05-09T00:09:10.479186+0000","last_became_active":"2024-05-09T00:09:10.474890+0000","last_became_peered":"2024-05-09T00:09:10.474890+0000","last_unstale":"2024-05-09T00:09:10.479186+0000","last_undegraded":"2024-05-09T00:09:10.479186+0000","last_fullsized":"2024-05-09T00:09:10.479186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:41.041247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327716+0000","last_change":"2024-05-09T00:08:26.050685+0000","last_active":"2024-05-09T00:10:09.327716+0000","last_peered":"2024-05-09T00:10:09.327716+0000","last_clean":"2024-05-09T00:10:09.327716+0000","last_became_active":"2024-05-09T00:08:26.050312+0000","last_became_peered":"2024-05-09T00:08:26.050312+0000","last_unstale":"2024-05-09T00:10:09.327716+0000","last_undegraded":"2024-05-09T00:10:09.327716+0000","last_fullsized":"2024-05-09T00:10:09.327716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:31.623182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'12","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:10.069724+0000","last_change":"2024-05-09T00:08:23.019663+0000","last_active":"2024-05-09T00:10:10.069724+0000","last_peered":"2024-05-09T00:10:10.069724+0000","last_clean":"2024-05-09T00:10:10.069724+0000","last_became_active":"2024-05-09T00:08:23.019301+0000","last_became_peered":"2024-05-09T00:08:23.019301+0000","last_unstale":"2024-05-09T00:10:10.069724+0000","last_undegraded":"2024-05-09T00:10:10.069724+0000","last_fullsized":"2024-05-09T00:10:10.069724+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:08.291227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330686+0000","last_change":"2024-05-09T00:08:26.058825+0000","last_active":"2024-05-09T00:10:09.330686+0000","last_peered":"2024-05-09T00:10:09.330686+0000","last_clean":"2024-05-09T00:10:09.330686+0000","last_became_active":"2024-05-09T00:08:26.058469+0000","last_became_peered":"2024-05-09T00:08:26.058469+0000","last_unstale":"2024-05-09T00:10:09.330686+0000","last_undegraded":"2024-05-09T00:10:09.330686+0000","last_fullsized":"2024-05-09T00:10:09.330686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:08.118819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796594+0000","last_change":"2024-05-09T00:08:23.054366+0000","last_active":"2024-05-09T00:09:10.796594+0000","last_peered":"2024-05-09T00:09:10.796594+0000","last_clean":"2024-05-09T00:09:10.796594+0000","last_became_active":"2024-05-09T00:08:23.053977+0000","last_became_peered":"2024-05-09T00:08:23.053977+0000","last_unstale":"2024-05-09T00:09:10.796594+0000","last_undegraded":"2024-05-09T00:09:10.796594+0000","last_fullsized":"2024-05-09T00:09:10.796594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:37.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.316168+0000","last_change":"2024-05-09T00:08:26.057864+0000","last_active":"2024-05-09T00:10:09.316168+0000","last_peered":"2024-05-09T00:10:09.316168+0000","last_clean":"2024-05-09T00:10:09.316168+0000","last_became_active":"2024-05-09T00:08:26.057436+0000","last_became_peered":"2024-05-09T00:08:26.057436+0000","last_unstale":"2024-05-09T00:10:09.316168+0000","last_undegraded":"2024-05-09T00:10:09.316168+0000","last_fullsized":"2024-05-09T00:10:09.316168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:19.976265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468672+0000","last_change":"2024-05-09T00:08:23.118803+0000","last_active":"2024-05-09T00:09:10.468672+0000","last_peered":"2024-05-09T00:09:10.468672+0000","last_clean":"2024-05-09T00:09:10.468672+0000","last_became_active":"2024-05-09T00:08:23.118432+0000","last_became_peered":"2024-05-09T00:08:23.118432+0000","last_unstale":"2024-05-09T00:09:10.468672+0000","last_undegraded":"2024-05-09T00:09:10.468672+0000","last_fullsized":"2024-05-09T00:09:10.468672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:50.253699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.310200+0000","last_change":"2024-05-09T00:08:26.054281+0000","last_active":"2024-05-09T00:10:09.310200+0000","last_peered":"2024-05-09T00:10:09.310200+0000","last_clean":"2024-05-09T00:10:09.310200+0000","last_became_active":"2024-05-09T00:08:26.053874+0000","last_became_peered":"2024-05-09T00:08:26.053874+0000","last_unstale":"2024-05-09T00:10:09.310200+0000","last_undegraded":"2024-05-09T00:10:09.310200+0000","last_fullsized":"2024-05-09T00:10:09.310200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:11.566337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467767+0000","last_change":"2024-05-09T00:08:23.017905+0000","last_active":"2024-05-09T00:09:10.467767+0000","last_peered":"2024-05-09T00:09:10.467767+0000","last_clean":"2024-05-09T00:09:10.467767+0000","last_became_active":"2024-05-09T00:08:23.017481+0000","last_became_peered":"2024-05-09T00:08:23.017481+0000","last_unstale":"2024-05-09T00:09:10.467767+0000","last_undegraded":"2024-05-09T00:09:10.467767+0000","last_fullsized":"2024-05-09T00:09:10.467767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:40.096415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.315580+0000","last_change":"2024-05-09T00:08:26.053392+0000","last_active":"2024-05-09T00:10:09.315580+0000","last_peered":"2024-05-09T00:10:09.315580+0000","last_clean":"2024-05-09T00:10:09.315580+0000","last_became_active":"2024-05-09T00:08:26.052628+0000","last_became_peered":"2024-05-09T00:08:26.052628+0000","last_unstale":"2024-05-09T00:10:09.315580+0000","last_undegraded":"2024-05-09T00:10:09.315580+0000","last_fullsized":"2024-05-09T00:10:09.315580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:28.013955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.481712+0000","last_change":"2024-05-09T00:09:10.481233+0000","last_active":"2024-05-09T00:09:10.481712+0000","last_peered":"2024-05-09T00:09:10.481712+0000","last_clean":"2024-05-09T00:09:10.481712+0000","last_became_active":"2024-05-09T00:09:10.475321+0000","last_became_peered":"2024-05-09T00:09:10.475321+0000","last_unstale":"2024-05-09T00:09:10.481712+0000","last_undegraded":"2024-05-09T00:09:10.481712+0000","last_fullsized":"2024-05-09T00:09:10.481712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:58.859277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.308221+0000","last_change":"2024-05-09T00:08:26.047812+0000","last_active":"2024-05-09T00:10:09.308221+0000","last_peered":"2024-05-09T00:10:09.308221+0000","last_clean":"2024-05-09T00:10:09.308221+0000","last_became_active":"2024-05-09T00:08:26.047459+0000","last_became_peered":"2024-05-09T00:08:26.047459+0000","last_unstale":"2024-05-09T00:10:09.308221+0000","last_undegraded":"2024-05-09T00:10:09.308221+0000","last_fullsized":"2024-05-09T00:10:09.308221+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.257831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473816+0000","last_change":"2024-05-09T00:08:23.023800+0000","last_active":"2024-05-09T00:09:10.473816+0000","last_peered":"2024-05-09T00:09:10.473816+0000","last_clean":"2024-05-09T00:09:10.473816+0000","last_became_active":"2024-05-09T00:08:23.023387+0000","last_became_peered":"2024-05-09T00:08:23.023387+0000","last_unstale":"2024-05-09T00:09:10.473816+0000","last_undegraded":"2024-05-09T00:09:10.473816+0000","last_fullsized":"2024-05-09T00:09:10.473816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:33.714683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330035+0000","last_change":"2024-05-09T00:08:26.056981+0000","last_active":"2024-05-09T00:10:09.330035+0000","last_peered":"2024-05-09T00:10:09.330035+0000","last_clean":"2024-05-09T00:10:09.330035+0000","last_became_active":"2024-05-09T00:08:26.056605+0000","last_became_peered":"2024-05-09T00:08:26.056605+0000","last_unstale":"2024-05-09T00:10:09.330035+0000","last_undegraded":"2024-05-09T00:10:09.330035+0000","last_fullsized":"2024-05-09T00:10:09.330035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:32.897929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473675+0000","last_change":"2024-05-09T00:08:23.120612+0000","last_active":"2024-05-09T00:09:10.473675+0000","last_peered":"2024-05-09T00:09:10.473675+0000","last_clean":"2024-05-09T00:09:10.473675+0000","last_became_active":"2024-05-09T00:08:23.120157+0000","last_became_peered":"2024-05-09T00:08:23.120157+0000","last_unstale":"2024-05-09T00:09:10.473675+0000","last_undegraded":"2024-05-09T00:09:10.473675+0000","last_fullsized":"2024-05-09T00:09:10.473675+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:08.027214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330144+0000","last_change":"2024-05-09T00:08:26.061594+0000","last_active":"2024-05-09T00:10:09.330144+0000","last_peered":"2024-05-09T00:10:09.330144+0000","last_clean":"2024-05-09T00:10:09.330144+0000","last_became_active":"2024-05-09T00:08:26.061201+0000","last_became_peered":"2024-05-09T00:08:26.061201+0000","last_unstale":"2024-05-09T00:10:09.330144+0000","last_undegraded":"2024-05-09T00:10:09.330144+0000","last_fullsized":"2024-05-09T00:10:09.330144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:50.183648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469471+0000","last_change":"2024-05-09T00:08:23.017693+0000","last_active":"2024-05-09T00:09:10.469471+0000","last_peered":"2024-05-09T00:09:10.469471+0000","last_clean":"2024-05-09T00:09:10.469471+0000","last_became_active":"2024-05-09T00:08:23.017211+0000","last_became_peered":"2024-05-09T00:08:23.017211+0000","last_unstale":"2024-05-09T00:09:10.469471+0000","last_undegraded":"2024-05-09T00:09:10.469471+0000","last_fullsized":"2024-05-09T00:09:10.469471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:53.569932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324623+0000","last_change":"2024-05-09T00:08:26.063922+0000","last_active":"2024-05-09T00:10:09.324623+0000","last_peered":"2024-05-09T00:10:09.324623+0000","last_clean":"2024-05-09T00:10:09.324623+0000","last_became_active":"2024-05-09T00:08:26.063554+0000","last_became_peered":"2024-05-09T00:08:26.063554+0000","last_unstale":"2024-05-09T00:10:09.324623+0000","last_undegraded":"2024-05-09T00:10:09.324623+0000","last_fullsized":"2024-05-09T00:10:09.324623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:01.754688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548651+0000","last_change":"2024-05-09T00:08:23.020367+0000","last_active":"2024-05-09T00:09:10.548651+0000","last_peered":"2024-05-09T00:09:10.548651+0000","last_clean":"2024-05-09T00:09:10.548651+0000","last_became_active":"2024-05-09T00:08:23.019801+0000","last_became_peered":"2024-05-09T00:08:23.019801+0000","last_unstale":"2024-05-09T00:09:10.548651+0000","last_undegraded":"2024-05-09T00:09:10.548651+0000","last_fullsized":"2024-05-09T00:09:10.548651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:14.936021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.329969+0000","last_change":"2024-05-09T00:08:26.063863+0000","last_active":"2024-05-09T00:10:09.329969+0000","last_peered":"2024-05-09T00:10:09.329969+0000","last_clean":"2024-05-09T00:10:09.329969+0000","last_became_active":"2024-05-09T00:08:26.061938+0000","last_became_peered":"2024-05-09T00:08:26.061938+0000","last_unstale":"2024-05-09T00:10:09.329969+0000","last_undegraded":"2024-05-09T00:10:09.329969+0000","last_fullsized":"2024-05-09T00:10:09.329969+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:04.597991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.319694+0000","last_change":"2024-05-09T00:08:26.050837+0000","last_active":"2024-05-09T00:10:09.319694+0000","last_peered":"2024-05-09T00:10:09.319694+0000","last_clean":"2024-05-09T00:10:09.319694+0000","last_became_active":"2024-05-09T00:08:26.050424+0000","last_became_peered":"2024-05-09T00:08:26.050424+0000","last_unstale":"2024-05-09T00:10:09.319694+0000","last_undegraded":"2024-05-09T00:10:09.319694+0000","last_fullsized":"2024-05-09T00:10:09.319694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:45.134811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323117+0000","last_change":"2024-05-09T00:09:10.549989+0000","last_active":"2024-05-09T00:10:09.323117+0000","last_peered":"2024-05-09T00:10:09.323117+0000","last_clean":"2024-05-09T00:10:09.323117+0000","last_became_active":"2024-05-09T00:09:10.549502+0000","last_became_peered":"2024-05-09T00:09:10.549502+0000","last_unstale":"2024-05-09T00:10:09.323117+0000","last_undegraded":"2024-05-09T00:10:09.323117+0000","last_fullsized":"2024-05-09T00:10:09.323117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:10.654452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323905+0000","last_change":"2024-05-09T00:08:26.058012+0000","last_active":"2024-05-09T00:10:09.323905+0000","last_peered":"2024-05-09T00:10:09.323905+0000","last_clean":"2024-05-09T00:10:09.323905+0000","last_became_active":"2024-05-09T00:08:26.057282+0000","last_became_peered":"2024-05-09T00:08:26.057282+0000","last_unstale":"2024-05-09T00:10:09.323905+0000","last_undegraded":"2024-05-09T00:10:09.323905+0000","last_fullsized":"2024-05-09T00:10:09.323905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:53.520593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.320691+0000","last_change":"2024-05-09T00:08:26.063331+0000","last_active":"2024-05-09T00:10:09.320691+0000","last_peered":"2024-05-09T00:10:09.320691+0000","last_clean":"2024-05-09T00:10:09.320691+0000","last_became_active":"2024-05-09T00:08:26.062866+0000","last_became_peered":"2024-05-09T00:08:26.062866+0000","last_unstale":"2024-05-09T00:10:09.320691+0000","last_undegraded":"2024-05-09T00:10:09.320691+0000","last_fullsized":"2024-05-09T00:10:09.320691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:10.387852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326309+0000","last_change":"2024-05-09T00:08:26.063735+0000","last_active":"2024-05-09T00:10:09.326309+0000","last_peered":"2024-05-09T00:10:09.326309+0000","last_clean":"2024-05-09T00:10:09.326309+0000","last_became_active":"2024-05-09T00:08:26.063358+0000","last_became_peered":"2024-05-09T00:08:26.063358+0000","last_unstale":"2024-05-09T00:10:09.326309+0000","last_undegraded":"2024-05-09T00:10:09.326309+0000","last_fullsized":"2024-05-09T00:10:09.326309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:06.832071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318241+0000","last_change":"2024-05-09T00:08:26.052746+0000","last_active":"2024-05-09T00:10:09.318241+0000","last_peered":"2024-05-09T00:10:09.318241+0000","last_clean":"2024-05-09T00:10:09.318241+0000","last_became_active":"2024-05-09T00:08:26.051544+0000","last_became_peered":"2024-05-09T00:08:26.051544+0000","last_unstale":"2024-05-09T00:10:09.318241+0000","last_undegraded":"2024-05-09T00:10:09.318241+0000","last_fullsized":"2024-05-09T00:10:09.318241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:31.963268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323267+0000","last_change":"2024-05-09T00:08:26.058075+0000","last_active":"2024-05-09T00:10:09.323267+0000","last_peered":"2024-05-09T00:10:09.323267+0000","last_clean":"2024-05-09T00:10:09.323267+0000","last_became_active":"2024-05-09T00:08:26.057349+0000","last_became_peered":"2024-05-09T00:08:26.057349+0000","last_unstale":"2024-05-09T00:10:09.323267+0000","last_undegraded":"2024-05-09T00:10:09.323267+0000","last_fullsized":"2024-05-09T00:10:09.323267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:59.322903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324094+0000","last_change":"2024-05-09T00:08:26.056429+0000","last_active":"2024-05-09T00:10:09.324094+0000","last_peered":"2024-05-09T00:10:09.324094+0000","last_clean":"2024-05-09T00:10:09.324094+0000","last_became_active":"2024-05-09T00:08:26.054852+0000","last_became_peered":"2024-05-09T00:08:26.054852+0000","last_unstale":"2024-05-09T00:10:09.324094+0000","last_undegraded":"2024-05-09T00:10:09.324094+0000","last_fullsized":"2024-05-09T00:10:09.324094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:05.337636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327096+0000","last_change":"2024-05-09T00:09:10.549880+0000","last_active":"2024-05-09T00:10:09.327096+0000","last_peered":"2024-05-09T00:10:09.327096+0000","last_clean":"2024-05-09T00:10:09.327096+0000","last_became_active":"2024-05-09T00:09:10.549448+0000","last_became_peered":"2024-05-09T00:09:10.549448+0000","last_unstale":"2024-05-09T00:10:09.327096+0000","last_undegraded":"2024-05-09T00:10:09.327096+0000","last_fullsized":"2024-05-09T00:10:09.327096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:16.990729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324673+0000","last_change":"2024-05-09T00:08:26.054579+0000","last_active":"2024-05-09T00:10:09.324673+0000","last_peered":"2024-05-09T00:10:09.324673+0000","last_clean":"2024-05-09T00:10:09.324673+0000","last_became_active":"2024-05-09T00:08:26.053186+0000","last_became_peered":"2024-05-09T00:08:26.053186+0000","last_unstale":"2024-05-09T00:10:09.324673+0000","last_undegraded":"2024-05-09T00:10:09.324673+0000","last_fullsized":"2024-05-09T00:10:09.324673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:37.561691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324353+0000","last_change":"2024-05-09T00:08:26.063608+0000","last_active":"2024-05-09T00:10:09.324353+0000","last_peered":"2024-05-09T00:10:09.324353+0000","last_clean":"2024-05-09T00:10:09.324353+0000","last_became_active":"2024-05-09T00:08:26.063078+0000","last_became_peered":"2024-05-09T00:08:26.063078+0000","last_unstale":"2024-05-09T00:10:09.324353+0000","last_undegraded":"2024-05-09T00:10:09.324353+0000","last_fullsized":"2024-05-09T00:10:09.324353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:06.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330947+0000","last_change":"2024-05-09T00:08:26.055058+0000","last_active":"2024-05-09T00:10:09.330947+0000","last_peered":"2024-05-09T00:10:09.330947+0000","last_clean":"2024-05-09T00:10:09.330947+0000","last_became_active":"2024-05-09T00:08:26.054695+0000","last_became_peered":"2024-05-09T00:08:26.054695+0000","last_unstale":"2024-05-09T00:10:09.330947+0000","last_undegraded":"2024-05-09T00:10:09.330947+0000","last_fullsized":"2024-05-09T00:10:09.330947+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:58.925990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.312724+0000","last_change":"2024-05-09T00:08:26.055366+0000","last_active":"2024-05-09T00:10:09.312724+0000","last_peered":"2024-05-09T00:10:09.312724+0000","last_clean":"2024-05-09T00:10:09.312724+0000","last_became_active":"2024-05-09T00:08:26.054904+0000","last_became_peered":"2024-05-09T00:08:26.054904+0000","last_unstale":"2024-05-09T00:10:09.312724+0000","last_undegraded":"2024-05-09T00:10:09.312724+0000","last_fullsized":"2024-05-09T00:10:09.312724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:40.754878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285241,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66217,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":716,"num_read_kb":429,"num_write":172,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710761,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":425247,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1079313,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070516,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":880640,"data_stored":425461,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266746,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":425359,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":1017452,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":868352,"data_stored":424850,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":486718,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":888832,"data_stored":426121,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":488068,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":425161,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986255,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":425819,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822729,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":1016604,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:10:41.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T00:10:41.806 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:42.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:42.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:42.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:41 smithi169 ceph-mon[34777]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:10:42.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:41 smithi169 ceph-mon[34777]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:10:42.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:10:42.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:10:42.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:41 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:42.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:42.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:42.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:41 smithi046 ceph-mon[30151]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:10:42.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:41 smithi046 ceph-mon[30151]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:10:42.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:10:42.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:10:42.162 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:41 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:42.190 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:42.190 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:42.190 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:41 smithi142 ceph-mon[34533]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:10:42.190 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:41 smithi142 ceph-mon[34533]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:10:42.190 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:10:42.190 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:10:42.190 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:41 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:10:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.432+0000 7fef6325f700 1 -- 172.21.15.46:0/1805172429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef5c108f20 msgr2=0x7fef5c109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.432+0000 7fef6325f700 1 --2- 172.21.15.46:0/1805172429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef5c108f20 0x7fef5c109330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fef480099c0 tx=0x7fef48009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.432+0000 7fef6325f700 1 -- 172.21.15.46:0/1805172429 shutdown_connections 2024-05-09T00:10:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.432+0000 7fef6325f700 1 --2- 172.21.15.46:0/1805172429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef5c109870 0x7fef5c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.432+0000 7fef6325f700 1 --2- 172.21.15.46:0/1805172429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef5c1045f0 0x7fef5c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.432+0000 7fef6325f700 1 --2- 172.21.15.46:0/1805172429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef5c108f20 0x7fef5c109330 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.432+0000 7fef6325f700 1 -- 172.21.15.46:0/1805172429 >> 172.21.15.46:0/1805172429 conn(0x7fef5c100040 msgr2=0x7fef5c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.432+0000 7fef6325f700 1 -- 172.21.15.46:0/1805172429 shutdown_connections 2024-05-09T00:10:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.432+0000 7fef6325f700 1 -- 172.21.15.46:0/1805172429 wait complete. 2024-05-09T00:10:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.433+0000 7fef6325f700 1 Processor -- start 2024-05-09T00:10:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.433+0000 7fef6325f700 1 -- start start 2024-05-09T00:10:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.433+0000 7fef6325f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef5c1045f0 0x7fef5c19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.433+0000 7fef6325f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef5c108f20 0x7fef5c19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.433+0000 7fef6325f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef5c109870 0x7fef5c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.433+0000 7fef6325f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c1144d0 con 0x7fef5c1045f0 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.433+0000 7fef6325f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c114350 con 0x7fef5c108f20 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.433+0000 7fef6325f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c114650 con 0x7fef5c109870 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.433+0000 7fef60ffb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef5c1045f0 0x7fef5c19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.433+0000 7fef60ffb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef5c1045f0 0x7fef5c19af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42432/0 (socket says 172.21.15.46:42432) 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.433+0000 7fef60ffb700 1 -- 172.21.15.46:0/3217050576 learned_addr learned my addr 172.21.15.46:0/3217050576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef5bfff700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef5c108f20 0x7fef5c19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef617fc700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef5c109870 0x7fef5c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef60ffb700 1 -- 172.21.15.46:0/3217050576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef5c109870 msgr2=0x7fef5c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef60ffb700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef5c109870 0x7fef5c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef60ffb700 1 -- 172.21.15.46:0/3217050576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef5c108f20 msgr2=0x7fef5c19b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef60ffb700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef5c108f20 0x7fef5c19b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:43.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef60ffb700 1 -- 172.21.15.46:0/3217050576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c19fdd0 con 0x7fef5c1045f0 2024-05-09T00:10:43.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef60ffb700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef5c1045f0 0x7fef5c19af00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fef5000d950 tx=0x7fef5000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:43.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef59ffb700 1 -- 172.21.15.46:0/3217050576 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50016070 con 0x7fef5c1045f0 2024-05-09T00:10:43.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef6325f700 1 -- 172.21.15.46:0/3217050576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5c1a00c0 con 0x7fef5c1045f0 2024-05-09T00:10:43.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef59ffb700 1 -- 172.21.15.46:0/3217050576 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef50009df0 con 0x7fef5c1045f0 2024-05-09T00:10:43.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef59ffb700 1 -- 172.21.15.46:0/3217050576 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5000f3c0 con 0x7fef5c1045f0 2024-05-09T00:10:43.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.434+0000 7fef6325f700 1 -- 172.21.15.46:0/3217050576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5c0759f0 con 0x7fef5c1045f0 2024-05-09T00:10:43.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.435+0000 7fef6325f700 1 -- 172.21.15.46:0/3217050576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5c105aa0 con 0x7fef5c1045f0 2024-05-09T00:10:43.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.437+0000 7fef59ffb700 1 -- 172.21.15.46:0/3217050576 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fef50019070 con 0x7fef5c1045f0 2024-05-09T00:10:43.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.437+0000 7fef59ffb700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef4c0b2880 0x7fef4c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:43.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.438+0000 7fef5bfff700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef4c0b2880 0x7fef4c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:43.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.438+0000 7fef59ffb700 1 -- 172.21.15.46:0/3217050576 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fef500e1e30 con 0x7fef5c1045f0 2024-05-09T00:10:43.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.438+0000 7fef5bfff700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef4c0b2880 0x7fef4c0b4d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef48006010 tx=0x7fef48009710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:43.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.442+0000 7fef59ffb700 1 -- 172.21.15.46:0/3217050576 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef500a9b50 con 0x7fef5c1045f0 2024-05-09T00:10:43.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.654+0000 7fef6325f700 1 -- 172.21.15.46:0/3217050576 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef5c19c260 con 0x7fef4c0b2880 2024-05-09T00:10:43.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.661+0000 7fef59ffb700 1 -- 172.21.15.46:0/3217050576 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259075 (secure 0 0 0) 0x7fef48067420 con 0x7fef4c0b2880 2024-05-09T00:10:43.662 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:10:43.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.665+0000 7fef6325f700 1 -- 172.21.15.46:0/3217050576 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef4c0b2880 msgr2=0x7fef4c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:43.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.665+0000 7fef6325f700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef4c0b2880 0x7fef4c0b4d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef48006010 tx=0x7fef48009710 comp rx=0 tx=0).stop 2024-05-09T00:10:43.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.665+0000 7fef6325f700 1 -- 172.21.15.46:0/3217050576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef5c1045f0 msgr2=0x7fef5c19af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:43.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.665+0000 7fef6325f700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef5c1045f0 0x7fef5c19af00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fef5000d950 tx=0x7fef5000dc60 comp rx=0 tx=0).stop 2024-05-09T00:10:43.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.665+0000 7fef6325f700 1 -- 172.21.15.46:0/3217050576 shutdown_connections 2024-05-09T00:10:43.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.665+0000 7fef6325f700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef4c0b2880 0x7fef4c0b4d90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:43.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.665+0000 7fef6325f700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef5c109870 0x7fef5c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:43.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.665+0000 7fef6325f700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef5c1045f0 0x7fef5c19af00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:43.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.665+0000 7fef6325f700 1 --2- 172.21.15.46:0/3217050576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef5c108f20 0x7fef5c19b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:43.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.665+0000 7fef6325f700 1 -- 172.21.15.46:0/3217050576 >> 172.21.15.46:0/3217050576 conn(0x7fef5c100040 msgr2=0x7fef5c10a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:43.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.665+0000 7fef6325f700 1 -- 172.21.15.46:0/3217050576 shutdown_connections 2024-05-09T00:10:43.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:43.665+0000 7fef6325f700 1 -- 172.21.15.46:0/3217050576 wait complete. 2024-05-09T00:10:43.665 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T00:10:44.353 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-05-09T00:10:43.319223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656819,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":802,"num_read_kb":493,"num_write":297,"num_write_kb":2269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":740,"ondisk_log_size":740,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372288,"kb_used_data":10656,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569536,"statfs":{"total":1151940427776,"available":1151559204864,"internally_reserved":0,"allocated":10911744,"data_stored":7066173,"data_compressed":177897,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003150"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324383+0000","last_change":"2024-05-09T00:08:26.060455+0000","last_active":"2024-05-09T00:10:09.324383+0000","last_peered":"2024-05-09T00:10:09.324383+0000","last_clean":"2024-05-09T00:10:09.324383+0000","last_became_active":"2024-05-09T00:08:26.060068+0000","last_became_peered":"2024-05-09T00:08:26.060068+0000","last_unstale":"2024-05-09T00:10:09.324383+0000","last_undegraded":"2024-05-09T00:10:09.324383+0000","last_fullsized":"2024-05-09T00:10:09.324383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:48.354188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.314519+0000","last_change":"2024-05-09T00:08:26.055737+0000","last_active":"2024-05-09T00:10:09.314519+0000","last_peered":"2024-05-09T00:10:09.314519+0000","last_clean":"2024-05-09T00:10:09.314519+0000","last_became_active":"2024-05-09T00:08:26.055354+0000","last_became_peered":"2024-05-09T00:08:26.055354+0000","last_unstale":"2024-05-09T00:10:09.314519+0000","last_undegraded":"2024-05-09T00:10:09.314519+0000","last_fullsized":"2024-05-09T00:10:09.314519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:54.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324383+0000","last_change":"2024-05-09T00:08:26.050776+0000","last_active":"2024-05-09T00:10:09.324383+0000","last_peered":"2024-05-09T00:10:09.324383+0000","last_clean":"2024-05-09T00:10:09.324383+0000","last_became_active":"2024-05-09T00:08:26.050393+0000","last_became_peered":"2024-05-09T00:08:26.050393+0000","last_unstale":"2024-05-09T00:10:09.324383+0000","last_undegraded":"2024-05-09T00:10:09.324383+0000","last_fullsized":"2024-05-09T00:10:09.324383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:30.819219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.307617+0000","last_change":"2024-05-09T00:08:26.065946+0000","last_active":"2024-05-09T00:10:09.307617+0000","last_peered":"2024-05-09T00:10:09.307617+0000","last_clean":"2024-05-09T00:10:09.307617+0000","last_became_active":"2024-05-09T00:08:26.065498+0000","last_became_peered":"2024-05-09T00:08:26.065498+0000","last_unstale":"2024-05-09T00:10:09.307617+0000","last_undegraded":"2024-05-09T00:10:09.307617+0000","last_fullsized":"2024-05-09T00:10:09.307617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:04.444860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.319557+0000","last_change":"2024-05-09T00:08:26.051961+0000","last_active":"2024-05-09T00:10:09.319557+0000","last_peered":"2024-05-09T00:10:09.319557+0000","last_clean":"2024-05-09T00:10:09.319557+0000","last_became_active":"2024-05-09T00:08:26.051548+0000","last_became_peered":"2024-05-09T00:08:26.051548+0000","last_unstale":"2024-05-09T00:10:09.319557+0000","last_undegraded":"2024-05-09T00:10:09.319557+0000","last_fullsized":"2024-05-09T00:10:09.319557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:07.539117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.311285+0000","last_change":"2024-05-09T00:08:26.052012+0000","last_active":"2024-05-09T00:10:09.311285+0000","last_peered":"2024-05-09T00:10:09.311285+0000","last_clean":"2024-05-09T00:10:09.311285+0000","last_became_active":"2024-05-09T00:08:26.050886+0000","last_became_peered":"2024-05-09T00:08:26.050886+0000","last_unstale":"2024-05-09T00:10:09.311285+0000","last_undegraded":"2024-05-09T00:10:09.311285+0000","last_fullsized":"2024-05-09T00:10:09.311285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:14.651888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324098+0000","last_change":"2024-05-09T00:08:26.058418+0000","last_active":"2024-05-09T00:10:09.324098+0000","last_peered":"2024-05-09T00:10:09.324098+0000","last_clean":"2024-05-09T00:10:09.324098+0000","last_became_active":"2024-05-09T00:08:26.058039+0000","last_became_peered":"2024-05-09T00:08:26.058039+0000","last_unstale":"2024-05-09T00:10:09.324098+0000","last_undegraded":"2024-05-09T00:10:09.324098+0000","last_fullsized":"2024-05-09T00:10:09.324098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:13.678123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.322184+0000","last_change":"2024-05-09T00:08:26.057602+0000","last_active":"2024-05-09T00:10:09.322184+0000","last_peered":"2024-05-09T00:10:09.322184+0000","last_clean":"2024-05-09T00:10:09.322184+0000","last_became_active":"2024-05-09T00:08:26.057169+0000","last_became_peered":"2024-05-09T00:08:26.057169+0000","last_unstale":"2024-05-09T00:10:09.322184+0000","last_undegraded":"2024-05-09T00:10:09.322184+0000","last_fullsized":"2024-05-09T00:10:09.322184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:11.452083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326985+0000","last_change":"2024-05-09T00:08:26.067639+0000","last_active":"2024-05-09T00:10:09.326985+0000","last_peered":"2024-05-09T00:10:09.326985+0000","last_clean":"2024-05-09T00:10:09.326985+0000","last_became_active":"2024-05-09T00:08:26.067226+0000","last_became_peered":"2024-05-09T00:08:26.067226+0000","last_unstale":"2024-05-09T00:10:09.326985+0000","last_undegraded":"2024-05-09T00:10:09.326985+0000","last_fullsized":"2024-05-09T00:10:09.326985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:12.366500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.309003+0000","last_change":"2024-05-09T00:08:26.055234+0000","last_active":"2024-05-09T00:10:09.309003+0000","last_peered":"2024-05-09T00:10:09.309003+0000","last_clean":"2024-05-09T00:10:09.309003+0000","last_became_active":"2024-05-09T00:08:26.054851+0000","last_became_peered":"2024-05-09T00:08:26.054851+0000","last_unstale":"2024-05-09T00:10:09.309003+0000","last_undegraded":"2024-05-09T00:10:09.309003+0000","last_fullsized":"2024-05-09T00:10:09.309003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:54:52.121329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.329724+0000","last_change":"2024-05-09T00:08:26.063375+0000","last_active":"2024-05-09T00:10:09.329724+0000","last_peered":"2024-05-09T00:10:09.329724+0000","last_clean":"2024-05-09T00:10:09.329724+0000","last_became_active":"2024-05-09T00:08:26.062962+0000","last_became_peered":"2024-05-09T00:08:26.062962+0000","last_unstale":"2024-05-09T00:10:09.329724+0000","last_undegraded":"2024-05-09T00:10:09.329724+0000","last_fullsized":"2024-05-09T00:10:09.329724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:05.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.305258+0000","last_change":"2024-05-09T00:08:26.062034+0000","last_active":"2024-05-09T00:10:09.305258+0000","last_peered":"2024-05-09T00:10:09.305258+0000","last_clean":"2024-05-09T00:10:09.305258+0000","last_became_active":"2024-05-09T00:08:26.060376+0000","last_became_peered":"2024-05-09T00:08:26.060376+0000","last_unstale":"2024-05-09T00:10:09.305258+0000","last_undegraded":"2024-05-09T00:10:09.305258+0000","last_fullsized":"2024-05-09T00:10:09.305258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:15.830024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327723+0000","last_change":"2024-05-09T00:08:26.053227+0000","last_active":"2024-05-09T00:10:09.327723+0000","last_peered":"2024-05-09T00:10:09.327723+0000","last_clean":"2024-05-09T00:10:09.327723+0000","last_became_active":"2024-05-09T00:08:26.052878+0000","last_became_peered":"2024-05-09T00:08:26.052878+0000","last_unstale":"2024-05-09T00:10:09.327723+0000","last_undegraded":"2024-05-09T00:10:09.327723+0000","last_fullsized":"2024-05-09T00:10:09.327723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:24.385026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.314291+0000","last_change":"2024-05-09T00:08:26.053041+0000","last_active":"2024-05-09T00:10:09.314291+0000","last_peered":"2024-05-09T00:10:09.314291+0000","last_clean":"2024-05-09T00:10:09.314291+0000","last_became_active":"2024-05-09T00:08:26.052677+0000","last_became_peered":"2024-05-09T00:08:26.052677+0000","last_unstale":"2024-05-09T00:10:09.314291+0000","last_undegraded":"2024-05-09T00:10:09.314291+0000","last_fullsized":"2024-05-09T00:10:09.314291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:24.478270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.315792+0000","last_change":"2024-05-09T00:08:26.063391+0000","last_active":"2024-05-09T00:10:09.315792+0000","last_peered":"2024-05-09T00:10:09.315792+0000","last_clean":"2024-05-09T00:10:09.315792+0000","last_became_active":"2024-05-09T00:08:26.063033+0000","last_became_peered":"2024-05-09T00:08:26.063033+0000","last_unstale":"2024-05-09T00:10:09.315792+0000","last_undegraded":"2024-05-09T00:10:09.315792+0000","last_fullsized":"2024-05-09T00:10:09.315792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:41.900453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.322709+0000","last_change":"2024-05-09T00:08:26.065018+0000","last_active":"2024-05-09T00:10:09.322709+0000","last_peered":"2024-05-09T00:10:09.322709+0000","last_clean":"2024-05-09T00:10:09.322709+0000","last_became_active":"2024-05-09T00:08:26.064642+0000","last_became_peered":"2024-05-09T00:08:26.064642+0000","last_unstale":"2024-05-09T00:10:09.322709+0000","last_undegraded":"2024-05-09T00:10:09.322709+0000","last_fullsized":"2024-05-09T00:10:09.322709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:02.403704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324011+0000","last_change":"2024-05-09T00:08:26.049770+0000","last_active":"2024-05-09T00:10:09.324011+0000","last_peered":"2024-05-09T00:10:09.324011+0000","last_clean":"2024-05-09T00:10:09.324011+0000","last_became_active":"2024-05-09T00:08:26.048555+0000","last_became_peered":"2024-05-09T00:08:26.048555+0000","last_unstale":"2024-05-09T00:10:09.324011+0000","last_undegraded":"2024-05-09T00:10:09.324011+0000","last_fullsized":"2024-05-09T00:10:09.324011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:16.009200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.317346+0000","last_change":"2024-05-09T00:08:26.048587+0000","last_active":"2024-05-09T00:10:09.317346+0000","last_peered":"2024-05-09T00:10:09.317346+0000","last_clean":"2024-05-09T00:10:09.317346+0000","last_became_active":"2024-05-09T00:08:26.048148+0000","last_became_peered":"2024-05-09T00:08:26.048148+0000","last_unstale":"2024-05-09T00:10:09.317346+0000","last_undegraded":"2024-05-09T00:10:09.317346+0000","last_fullsized":"2024-05-09T00:10:09.317346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:42.113829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.322240+0000","last_change":"2024-05-09T00:08:26.053607+0000","last_active":"2024-05-09T00:10:09.322240+0000","last_peered":"2024-05-09T00:10:09.322240+0000","last_clean":"2024-05-09T00:10:09.322240+0000","last_became_active":"2024-05-09T00:08:26.053212+0000","last_became_peered":"2024-05-09T00:08:26.053212+0000","last_unstale":"2024-05-09T00:10:09.322240+0000","last_undegraded":"2024-05-09T00:10:09.322240+0000","last_fullsized":"2024-05-09T00:10:09.322240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:32.937049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469352+0000","last_change":"2024-05-09T00:08:23.024481+0000","last_active":"2024-05-09T00:09:10.469352+0000","last_peered":"2024-05-09T00:09:10.469352+0000","last_clean":"2024-05-09T00:09:10.469352+0000","last_became_active":"2024-05-09T00:08:23.024132+0000","last_became_peered":"2024-05-09T00:08:23.024132+0000","last_unstale":"2024-05-09T00:09:10.469352+0000","last_undegraded":"2024-05-09T00:09:10.469352+0000","last_fullsized":"2024-05-09T00:09:10.469352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:11.188505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325280+0000","last_change":"2024-05-09T00:08:26.054772+0000","last_active":"2024-05-09T00:10:09.325280+0000","last_peered":"2024-05-09T00:10:09.325280+0000","last_clean":"2024-05-09T00:10:09.325280+0000","last_became_active":"2024-05-09T00:08:26.054433+0000","last_became_peered":"2024-05-09T00:08:26.054433+0000","last_unstale":"2024-05-09T00:10:09.325280+0000","last_undegraded":"2024-05-09T00:10:09.325280+0000","last_fullsized":"2024-05-09T00:10:09.325280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:44.283240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468356+0000","last_change":"2024-05-09T00:08:23.118778+0000","last_active":"2024-05-09T00:09:10.468356+0000","last_peered":"2024-05-09T00:09:10.468356+0000","last_clean":"2024-05-09T00:09:10.468356+0000","last_became_active":"2024-05-09T00:08:23.118377+0000","last_became_peered":"2024-05-09T00:08:23.118377+0000","last_unstale":"2024-05-09T00:09:10.468356+0000","last_undegraded":"2024-05-09T00:09:10.468356+0000","last_fullsized":"2024-05-09T00:09:10.468356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:51.352442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325452+0000","last_change":"2024-05-09T00:08:26.059953+0000","last_active":"2024-05-09T00:10:09.325452+0000","last_peered":"2024-05-09T00:10:09.325452+0000","last_clean":"2024-05-09T00:10:09.325452+0000","last_became_active":"2024-05-09T00:08:26.059571+0000","last_became_peered":"2024-05-09T00:08:26.059571+0000","last_unstale":"2024-05-09T00:10:09.325452+0000","last_undegraded":"2024-05-09T00:10:09.325452+0000","last_fullsized":"2024-05-09T00:10:09.325452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:48.056798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475240+0000","last_change":"2024-05-09T00:08:23.021244+0000","last_active":"2024-05-09T00:09:10.475240+0000","last_peered":"2024-05-09T00:09:10.475240+0000","last_clean":"2024-05-09T00:09:10.475240+0000","last_became_active":"2024-05-09T00:08:23.020597+0000","last_became_peered":"2024-05-09T00:08:23.020597+0000","last_unstale":"2024-05-09T00:09:10.475240+0000","last_undegraded":"2024-05-09T00:09:10.475240+0000","last_fullsized":"2024-05-09T00:09:10.475240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.654261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327796+0000","last_change":"2024-05-09T00:08:26.059834+0000","last_active":"2024-05-09T00:10:09.327796+0000","last_peered":"2024-05-09T00:10:09.327796+0000","last_clean":"2024-05-09T00:10:09.327796+0000","last_became_active":"2024-05-09T00:08:26.059349+0000","last_became_peered":"2024-05-09T00:08:26.059349+0000","last_unstale":"2024-05-09T00:10:09.327796+0000","last_undegraded":"2024-05-09T00:10:09.327796+0000","last_fullsized":"2024-05-09T00:10:09.327796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:42.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474158+0000","last_change":"2024-05-09T00:08:23.019568+0000","last_active":"2024-05-09T00:09:10.474158+0000","last_peered":"2024-05-09T00:09:10.474158+0000","last_clean":"2024-05-09T00:09:10.474158+0000","last_became_active":"2024-05-09T00:08:23.019149+0000","last_became_peered":"2024-05-09T00:08:23.019149+0000","last_unstale":"2024-05-09T00:09:10.474158+0000","last_undegraded":"2024-05-09T00:09:10.474158+0000","last_fullsized":"2024-05-09T00:09:10.474158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:44.569911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328328+0000","last_change":"2024-05-09T00:08:26.061308+0000","last_active":"2024-05-09T00:10:09.328328+0000","last_peered":"2024-05-09T00:10:09.328328+0000","last_clean":"2024-05-09T00:10:09.328328+0000","last_became_active":"2024-05-09T00:08:26.060973+0000","last_became_peered":"2024-05-09T00:08:26.060973+0000","last_unstale":"2024-05-09T00:10:09.328328+0000","last_undegraded":"2024-05-09T00:10:09.328328+0000","last_fullsized":"2024-05-09T00:10:09.328328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:16.845074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":150,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:40.563643+0000","last_change":"2024-05-09T00:08:23.020886+0000","last_active":"2024-05-09T00:10:40.563643+0000","last_peered":"2024-05-09T00:10:40.563643+0000","last_clean":"2024-05-09T00:10:40.563643+0000","last_became_active":"2024-05-09T00:08:23.020530+0000","last_became_peered":"2024-05-09T00:08:23.020530+0000","last_unstale":"2024-05-09T00:10:40.563643+0000","last_undegraded":"2024-05-09T00:10:40.563643+0000","last_fullsized":"2024-05-09T00:10:40.563643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:34.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318636+0000","last_change":"2024-05-09T00:08:26.051018+0000","last_active":"2024-05-09T00:10:09.318636+0000","last_peered":"2024-05-09T00:10:09.318636+0000","last_clean":"2024-05-09T00:10:09.318636+0000","last_became_active":"2024-05-09T00:08:26.050495+0000","last_became_peered":"2024-05-09T00:08:26.050495+0000","last_unstale":"2024-05-09T00:10:09.318636+0000","last_undegraded":"2024-05-09T00:10:09.318636+0000","last_fullsized":"2024-05-09T00:10:09.318636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:07.862339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472087+0000","last_change":"2024-05-09T00:08:23.021187+0000","last_active":"2024-05-09T00:09:10.472087+0000","last_peered":"2024-05-09T00:09:10.472087+0000","last_clean":"2024-05-09T00:09:10.472087+0000","last_became_active":"2024-05-09T00:08:23.020770+0000","last_became_peered":"2024-05-09T00:08:23.020770+0000","last_unstale":"2024-05-09T00:09:10.472087+0000","last_undegraded":"2024-05-09T00:09:10.472087+0000","last_fullsized":"2024-05-09T00:09:10.472087+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:08.992770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.316728+0000","last_change":"2024-05-09T00:09:10.797392+0000","last_active":"2024-05-09T00:10:09.316728+0000","last_peered":"2024-05-09T00:10:09.316728+0000","last_clean":"2024-05-09T00:10:09.316728+0000","last_became_active":"2024-05-09T00:09:10.796969+0000","last_became_peered":"2024-05-09T00:09:10.796969+0000","last_unstale":"2024-05-09T00:10:09.316728+0000","last_undegraded":"2024-05-09T00:10:09.316728+0000","last_fullsized":"2024-05-09T00:10:09.316728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:18.852678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548755+0000","last_change":"2024-05-09T00:08:23.012226+0000","last_active":"2024-05-09T00:09:10.548755+0000","last_peered":"2024-05-09T00:09:10.548755+0000","last_clean":"2024-05-09T00:09:10.548755+0000","last_became_active":"2024-05-09T00:08:23.011781+0000","last_became_peered":"2024-05-09T00:08:23.011781+0000","last_unstale":"2024-05-09T00:09:10.548755+0000","last_undegraded":"2024-05-09T00:09:10.548755+0000","last_fullsized":"2024-05-09T00:09:10.548755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:44.452206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.321520+0000","last_change":"2024-05-09T00:08:26.043035+0000","last_active":"2024-05-09T00:10:09.321520+0000","last_peered":"2024-05-09T00:10:09.321520+0000","last_clean":"2024-05-09T00:10:09.321520+0000","last_became_active":"2024-05-09T00:08:26.042686+0000","last_became_peered":"2024-05-09T00:08:26.042686+0000","last_unstale":"2024-05-09T00:10:09.321520+0000","last_undegraded":"2024-05-09T00:10:09.321520+0000","last_fullsized":"2024-05-09T00:10:09.321520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:24.511309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469042+0000","last_change":"2024-05-09T00:08:23.022017+0000","last_active":"2024-05-09T00:09:10.469042+0000","last_peered":"2024-05-09T00:09:10.469042+0000","last_clean":"2024-05-09T00:09:10.469042+0000","last_became_active":"2024-05-09T00:08:23.021540+0000","last_became_peered":"2024-05-09T00:08:23.021540+0000","last_unstale":"2024-05-09T00:09:10.469042+0000","last_undegraded":"2024-05-09T00:09:10.469042+0000","last_fullsized":"2024-05-09T00:09:10.469042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:50.773211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325952+0000","last_change":"2024-05-09T00:08:26.048039+0000","last_active":"2024-05-09T00:10:09.325952+0000","last_peered":"2024-05-09T00:10:09.325952+0000","last_clean":"2024-05-09T00:10:09.325952+0000","last_became_active":"2024-05-09T00:08:26.047695+0000","last_became_peered":"2024-05-09T00:08:26.047695+0000","last_unstale":"2024-05-09T00:10:09.325952+0000","last_undegraded":"2024-05-09T00:10:09.325952+0000","last_fullsized":"2024-05-09T00:10:09.325952+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:27.363847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":144,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:37.558867+0000","last_change":"2024-05-09T00:08:23.019565+0000","last_active":"2024-05-09T00:10:37.558867+0000","last_peered":"2024-05-09T00:10:37.558867+0000","last_clean":"2024-05-09T00:10:37.558867+0000","last_became_active":"2024-05-09T00:08:23.019113+0000","last_became_peered":"2024-05-09T00:08:23.019113+0000","last_unstale":"2024-05-09T00:10:37.558867+0000","last_undegraded":"2024-05-09T00:10:37.558867+0000","last_fullsized":"2024-05-09T00:10:37.558867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:45.578934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323716+0000","last_change":"2024-05-09T00:08:26.058869+0000","last_active":"2024-05-09T00:10:09.323716+0000","last_peered":"2024-05-09T00:10:09.323716+0000","last_clean":"2024-05-09T00:10:09.323716+0000","last_became_active":"2024-05-09T00:08:26.058482+0000","last_became_peered":"2024-05-09T00:08:26.058482+0000","last_unstale":"2024-05-09T00:10:09.323716+0000","last_undegraded":"2024-05-09T00:10:09.323716+0000","last_fullsized":"2024-05-09T00:10:09.323716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:33.252208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472150+0000","last_change":"2024-05-09T00:08:23.024425+0000","last_active":"2024-05-09T00:09:10.472150+0000","last_peered":"2024-05-09T00:09:10.472150+0000","last_clean":"2024-05-09T00:09:10.472150+0000","last_became_active":"2024-05-09T00:08:23.023991+0000","last_became_peered":"2024-05-09T00:08:23.023991+0000","last_unstale":"2024-05-09T00:09:10.472150+0000","last_undegraded":"2024-05-09T00:09:10.472150+0000","last_fullsized":"2024-05-09T00:09:10.472150+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:47.907851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330824+0000","last_change":"2024-05-09T00:09:10.475008+0000","last_active":"2024-05-09T00:10:09.330824+0000","last_peered":"2024-05-09T00:10:09.330824+0000","last_clean":"2024-05-09T00:10:09.330824+0000","last_became_active":"2024-05-09T00:09:10.474663+0000","last_became_peered":"2024-05-09T00:09:10.474663+0000","last_unstale":"2024-05-09T00:10:09.330824+0000","last_undegraded":"2024-05-09T00:10:09.330824+0000","last_fullsized":"2024-05-09T00:10:09.330824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:36.990362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470120+0000","last_change":"2024-05-09T00:08:23.118516+0000","last_active":"2024-05-09T00:09:10.470120+0000","last_peered":"2024-05-09T00:09:10.470120+0000","last_clean":"2024-05-09T00:09:10.470120+0000","last_became_active":"2024-05-09T00:08:23.118147+0000","last_became_peered":"2024-05-09T00:08:23.118147+0000","last_unstale":"2024-05-09T00:09:10.470120+0000","last_undegraded":"2024-05-09T00:09:10.470120+0000","last_fullsized":"2024-05-09T00:09:10.470120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:24.205818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326723+0000","last_change":"2024-05-09T00:08:26.067063+0000","last_active":"2024-05-09T00:10:09.326723+0000","last_peered":"2024-05-09T00:10:09.326723+0000","last_clean":"2024-05-09T00:10:09.326723+0000","last_became_active":"2024-05-09T00:08:26.066440+0000","last_became_peered":"2024-05-09T00:08:26.066440+0000","last_unstale":"2024-05-09T00:10:09.326723+0000","last_undegraded":"2024-05-09T00:10:09.326723+0000","last_fullsized":"2024-05-09T00:10:09.326723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:48.648485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'33","reported_seq":299,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:38.560319+0000","last_change":"2024-05-09T00:08:23.020757+0000","last_active":"2024-05-09T00:10:38.560319+0000","last_peered":"2024-05-09T00:10:38.560319+0000","last_clean":"2024-05-09T00:10:38.560319+0000","last_became_active":"2024-05-09T00:08:23.020355+0000","last_became_peered":"2024-05-09T00:08:23.020355+0000","last_unstale":"2024-05-09T00:10:38.560319+0000","last_undegraded":"2024-05-09T00:10:38.560319+0000","last_fullsized":"2024-05-09T00:10:38.560319+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:00.605691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62075,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":240,"num_read_kb":180,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328107+0000","last_change":"2024-05-09T00:08:26.057402+0000","last_active":"2024-05-09T00:10:09.328107+0000","last_peered":"2024-05-09T00:10:09.328107+0000","last_clean":"2024-05-09T00:10:09.328107+0000","last_became_active":"2024-05-09T00:08:26.056974+0000","last_became_peered":"2024-05-09T00:08:26.056974+0000","last_unstale":"2024-05-09T00:10:09.328107+0000","last_undegraded":"2024-05-09T00:10:09.328107+0000","last_fullsized":"2024-05-09T00:10:09.328107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:47.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473953+0000","last_change":"2024-05-09T00:08:23.020208+0000","last_active":"2024-05-09T00:09:10.473953+0000","last_peered":"2024-05-09T00:09:10.473953+0000","last_clean":"2024-05-09T00:09:10.473953+0000","last_became_active":"2024-05-09T00:08:23.019809+0000","last_became_peered":"2024-05-09T00:08:23.019809+0000","last_unstale":"2024-05-09T00:09:10.473953+0000","last_undegraded":"2024-05-09T00:09:10.473953+0000","last_fullsized":"2024-05-09T00:09:10.473953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:47.900731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330380+0000","last_change":"2024-05-09T00:08:26.054273+0000","last_active":"2024-05-09T00:10:09.330380+0000","last_peered":"2024-05-09T00:10:09.330380+0000","last_clean":"2024-05-09T00:10:09.330380+0000","last_became_active":"2024-05-09T00:08:26.053840+0000","last_became_peered":"2024-05-09T00:08:26.053840+0000","last_unstale":"2024-05-09T00:10:09.330380+0000","last_undegraded":"2024-05-09T00:10:09.330380+0000","last_fullsized":"2024-05-09T00:10:09.330380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:27.128582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469240+0000","last_change":"2024-05-09T00:08:23.021594+0000","last_active":"2024-05-09T00:09:10.469240+0000","last_peered":"2024-05-09T00:09:10.469240+0000","last_clean":"2024-05-09T00:09:10.469240+0000","last_became_active":"2024-05-09T00:08:23.021152+0000","last_became_peered":"2024-05-09T00:08:23.021152+0000","last_unstale":"2024-05-09T00:09:10.469240+0000","last_undegraded":"2024-05-09T00:09:10.469240+0000","last_fullsized":"2024-05-09T00:09:10.469240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:58.505781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330080+0000","last_change":"2024-05-09T00:08:26.064464+0000","last_active":"2024-05-09T00:10:09.330080+0000","last_peered":"2024-05-09T00:10:09.330080+0000","last_clean":"2024-05-09T00:10:09.330080+0000","last_became_active":"2024-05-09T00:08:26.064000+0000","last_became_peered":"2024-05-09T00:08:26.064000+0000","last_unstale":"2024-05-09T00:10:09.330080+0000","last_undegraded":"2024-05-09T00:10:09.330080+0000","last_fullsized":"2024-05-09T00:10:09.330080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:01.455979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":155,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:42.566459+0000","last_change":"2024-05-09T00:08:23.020823+0000","last_active":"2024-05-09T00:10:42.566459+0000","last_peered":"2024-05-09T00:10:42.566459+0000","last_clean":"2024-05-09T00:10:42.566459+0000","last_became_active":"2024-05-09T00:08:23.020412+0000","last_became_peered":"2024-05-09T00:08:23.020412+0000","last_unstale":"2024-05-09T00:10:42.566459+0000","last_undegraded":"2024-05-09T00:10:42.566459+0000","last_fullsized":"2024-05-09T00:10:42.566459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:05.457452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.317309+0000","last_change":"2024-05-09T00:09:10.474540+0000","last_active":"2024-05-09T00:10:09.317309+0000","last_peered":"2024-05-09T00:10:09.317309+0000","last_clean":"2024-05-09T00:10:09.317309+0000","last_became_active":"2024-05-09T00:09:10.474068+0000","last_became_peered":"2024-05-09T00:09:10.474068+0000","last_unstale":"2024-05-09T00:10:09.317309+0000","last_undegraded":"2024-05-09T00:10:09.317309+0000","last_fullsized":"2024-05-09T00:10:09.317309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:37.296140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474161+0000","last_change":"2024-05-09T00:08:23.118857+0000","last_active":"2024-05-09T00:09:10.474161+0000","last_peered":"2024-05-09T00:09:10.474161+0000","last_clean":"2024-05-09T00:09:10.474161+0000","last_became_active":"2024-05-09T00:08:23.118433+0000","last_became_peered":"2024-05-09T00:08:23.118433+0000","last_unstale":"2024-05-09T00:09:10.474161+0000","last_undegraded":"2024-05-09T00:09:10.474161+0000","last_fullsized":"2024-05-09T00:09:10.474161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:10.617987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327020+0000","last_change":"2024-05-09T00:08:26.066911+0000","last_active":"2024-05-09T00:10:09.327020+0000","last_peered":"2024-05-09T00:10:09.327020+0000","last_clean":"2024-05-09T00:10:09.327020+0000","last_became_active":"2024-05-09T00:08:26.066548+0000","last_became_peered":"2024-05-09T00:08:26.066548+0000","last_unstale":"2024-05-09T00:10:09.327020+0000","last_undegraded":"2024-05-09T00:10:09.327020+0000","last_fullsized":"2024-05-09T00:10:09.327020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:32.391390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475068+0000","last_change":"2024-05-09T00:08:23.019829+0000","last_active":"2024-05-09T00:09:10.475068+0000","last_peered":"2024-05-09T00:09:10.475068+0000","last_clean":"2024-05-09T00:09:10.475068+0000","last_became_active":"2024-05-09T00:08:23.019336+0000","last_became_peered":"2024-05-09T00:08:23.019336+0000","last_unstale":"2024-05-09T00:09:10.475068+0000","last_undegraded":"2024-05-09T00:09:10.475068+0000","last_fullsized":"2024-05-09T00:09:10.475068+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:33.802738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318674+0000","last_change":"2024-05-09T00:08:26.055975+0000","last_active":"2024-05-09T00:10:09.318674+0000","last_peered":"2024-05-09T00:10:09.318674+0000","last_clean":"2024-05-09T00:10:09.318674+0000","last_became_active":"2024-05-09T00:08:26.055595+0000","last_became_peered":"2024-05-09T00:08:26.055595+0000","last_unstale":"2024-05-09T00:10:09.318674+0000","last_undegraded":"2024-05-09T00:10:09.318674+0000","last_fullsized":"2024-05-09T00:10:09.318674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:20.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469423+0000","last_change":"2024-05-09T00:08:23.120353+0000","last_active":"2024-05-09T00:09:10.469423+0000","last_peered":"2024-05-09T00:09:10.469423+0000","last_clean":"2024-05-09T00:09:10.469423+0000","last_became_active":"2024-05-09T00:08:23.120004+0000","last_became_peered":"2024-05-09T00:08:23.120004+0000","last_unstale":"2024-05-09T00:09:10.469423+0000","last_undegraded":"2024-05-09T00:09:10.469423+0000","last_fullsized":"2024-05-09T00:09:10.469423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:12.168671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.320556+0000","last_change":"2024-05-09T00:08:26.046554+0000","last_active":"2024-05-09T00:10:09.320556+0000","last_peered":"2024-05-09T00:10:09.320556+0000","last_clean":"2024-05-09T00:10:09.320556+0000","last_became_active":"2024-05-09T00:08:26.045924+0000","last_became_peered":"2024-05-09T00:08:26.045924+0000","last_unstale":"2024-05-09T00:10:09.320556+0000","last_undegraded":"2024-05-09T00:10:09.320556+0000","last_fullsized":"2024-05-09T00:10:09.320556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:01.208980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472462+0000","last_change":"2024-05-09T00:08:23.019778+0000","last_active":"2024-05-09T00:09:10.472462+0000","last_peered":"2024-05-09T00:09:10.472462+0000","last_clean":"2024-05-09T00:09:10.472462+0000","last_became_active":"2024-05-09T00:08:23.019400+0000","last_became_peered":"2024-05-09T00:08:23.019400+0000","last_unstale":"2024-05-09T00:09:10.472462+0000","last_undegraded":"2024-05-09T00:09:10.472462+0000","last_fullsized":"2024-05-09T00:09:10.472462+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:06.883167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.308957+0000","last_change":"2024-05-09T00:09:10.474352+0000","last_active":"2024-05-09T00:10:09.308957+0000","last_peered":"2024-05-09T00:10:09.308957+0000","last_clean":"2024-05-09T00:10:09.308957+0000","last_became_active":"2024-05-09T00:09:10.473906+0000","last_became_peered":"2024-05-09T00:09:10.473906+0000","last_unstale":"2024-05-09T00:10:09.308957+0000","last_undegraded":"2024-05-09T00:10:09.308957+0000","last_fullsized":"2024-05-09T00:10:09.308957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:35:18.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.319595+0000","last_change":"2024-05-09T00:09:10.476484+0000","last_active":"2024-05-09T00:10:09.319595+0000","last_peered":"2024-05-09T00:10:09.319595+0000","last_clean":"2024-05-09T00:10:09.319595+0000","last_became_active":"2024-05-09T00:09:10.476001+0000","last_became_peered":"2024-05-09T00:09:10.476001+0000","last_unstale":"2024-05-09T00:10:09.319595+0000","last_undegraded":"2024-05-09T00:10:09.319595+0000","last_fullsized":"2024-05-09T00:10:09.319595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:32.018749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":147,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:37.559184+0000","last_change":"2024-05-09T00:08:23.118322+0000","last_active":"2024-05-09T00:10:37.559184+0000","last_peered":"2024-05-09T00:10:37.559184+0000","last_clean":"2024-05-09T00:10:37.559184+0000","last_became_active":"2024-05-09T00:08:23.117943+0000","last_became_peered":"2024-05-09T00:08:23.117943+0000","last_unstale":"2024-05-09T00:10:37.559184+0000","last_undegraded":"2024-05-09T00:10:37.559184+0000","last_fullsized":"2024-05-09T00:10:37.559184+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:07.890437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323361+0000","last_change":"2024-05-09T00:08:26.052294+0000","last_active":"2024-05-09T00:10:09.323361+0000","last_peered":"2024-05-09T00:10:09.323361+0000","last_clean":"2024-05-09T00:10:09.323361+0000","last_became_active":"2024-05-09T00:08:26.051926+0000","last_became_peered":"2024-05-09T00:08:26.051926+0000","last_unstale":"2024-05-09T00:10:09.323361+0000","last_undegraded":"2024-05-09T00:10:09.323361+0000","last_fullsized":"2024-05-09T00:10:09.323361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:53.651401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796358+0000","last_change":"2024-05-09T00:08:23.015007+0000","last_active":"2024-05-09T00:09:10.796358+0000","last_peered":"2024-05-09T00:09:10.796358+0000","last_clean":"2024-05-09T00:09:10.796358+0000","last_became_active":"2024-05-09T00:08:23.014545+0000","last_became_peered":"2024-05-09T00:08:23.014545+0000","last_unstale":"2024-05-09T00:09:10.796358+0000","last_undegraded":"2024-05-09T00:09:10.796358+0000","last_fullsized":"2024-05-09T00:09:10.796358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:19.850200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":100,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472390+0000","last_change":"2024-05-09T00:07:00.733233+0000","last_active":"2024-05-09T00:09:10.472390+0000","last_peered":"2024-05-09T00:09:10.472390+0000","last_clean":"2024-05-09T00:09:10.472390+0000","last_became_active":"2024-05-09T00:07:00.727365+0000","last_became_peered":"2024-05-09T00:07:00.727365+0000","last_unstale":"2024-05-09T00:09:10.472390+0000","last_undegraded":"2024-05-09T00:09:10.472390+0000","last_fullsized":"2024-05-09T00:09:10.472390+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_clean_scrub_stamp":"2024-05-09T00:03:33.285220+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:08.597290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326209+0000","last_change":"2024-05-09T00:08:26.064000+0000","last_active":"2024-05-09T00:10:09.326209+0000","last_peered":"2024-05-09T00:10:09.326209+0000","last_clean":"2024-05-09T00:10:09.326209+0000","last_became_active":"2024-05-09T00:08:26.063636+0000","last_became_peered":"2024-05-09T00:08:26.063636+0000","last_unstale":"2024-05-09T00:10:09.326209+0000","last_undegraded":"2024-05-09T00:10:09.326209+0000","last_fullsized":"2024-05-09T00:10:09.326209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:59.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472596+0000","last_change":"2024-05-09T00:08:23.023459+0000","last_active":"2024-05-09T00:09:10.472596+0000","last_peered":"2024-05-09T00:09:10.472596+0000","last_clean":"2024-05-09T00:09:10.472596+0000","last_became_active":"2024-05-09T00:08:23.023089+0000","last_became_peered":"2024-05-09T00:08:23.023089+0000","last_unstale":"2024-05-09T00:09:10.472596+0000","last_undegraded":"2024-05-09T00:09:10.472596+0000","last_fullsized":"2024-05-09T00:09:10.472596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:41.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.479186+0000","last_change":"2024-05-09T00:09:10.478686+0000","last_active":"2024-05-09T00:09:10.479186+0000","last_peered":"2024-05-09T00:09:10.479186+0000","last_clean":"2024-05-09T00:09:10.479186+0000","last_became_active":"2024-05-09T00:09:10.474890+0000","last_became_peered":"2024-05-09T00:09:10.474890+0000","last_unstale":"2024-05-09T00:09:10.479186+0000","last_undegraded":"2024-05-09T00:09:10.479186+0000","last_fullsized":"2024-05-09T00:09:10.479186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:41.041247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327716+0000","last_change":"2024-05-09T00:08:26.050685+0000","last_active":"2024-05-09T00:10:09.327716+0000","last_peered":"2024-05-09T00:10:09.327716+0000","last_clean":"2024-05-09T00:10:09.327716+0000","last_became_active":"2024-05-09T00:08:26.050312+0000","last_became_peered":"2024-05-09T00:08:26.050312+0000","last_unstale":"2024-05-09T00:10:09.327716+0000","last_undegraded":"2024-05-09T00:10:09.327716+0000","last_fullsized":"2024-05-09T00:10:09.327716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:31.623182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'12","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:10.069724+0000","last_change":"2024-05-09T00:08:23.019663+0000","last_active":"2024-05-09T00:10:10.069724+0000","last_peered":"2024-05-09T00:10:10.069724+0000","last_clean":"2024-05-09T00:10:10.069724+0000","last_became_active":"2024-05-09T00:08:23.019301+0000","last_became_peered":"2024-05-09T00:08:23.019301+0000","last_unstale":"2024-05-09T00:10:10.069724+0000","last_undegraded":"2024-05-09T00:10:10.069724+0000","last_fullsized":"2024-05-09T00:10:10.069724+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:08.291227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330686+0000","last_change":"2024-05-09T00:08:26.058825+0000","last_active":"2024-05-09T00:10:09.330686+0000","last_peered":"2024-05-09T00:10:09.330686+0000","last_clean":"2024-05-09T00:10:09.330686+0000","last_became_active":"2024-05-09T00:08:26.058469+0000","last_became_peered":"2024-05-09T00:08:26.058469+0000","last_unstale":"2024-05-09T00:10:09.330686+0000","last_undegraded":"2024-05-09T00:10:09.330686+0000","last_fullsized":"2024-05-09T00:10:09.330686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:08.118819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796594+0000","last_change":"2024-05-09T00:08:23.054366+0000","last_active":"2024-05-09T00:09:10.796594+0000","last_peered":"2024-05-09T00:09:10.796594+0000","last_clean":"2024-05-09T00:09:10.796594+0000","last_became_active":"2024-05-09T00:08:23.053977+0000","last_became_peered":"2024-05-09T00:08:23.053977+0000","last_unstale":"2024-05-09T00:09:10.796594+0000","last_undegraded":"2024-05-09T00:09:10.796594+0000","last_fullsized":"2024-05-09T00:09:10.796594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:37.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.316437+0000","last_change":"2024-05-09T00:08:26.057864+0000","last_active":"2024-05-09T00:10:09.316437+0000","last_peered":"2024-05-09T00:10:09.316437+0000","last_clean":"2024-05-09T00:10:09.316437+0000","last_became_active":"2024-05-09T00:08:26.057436+0000","last_became_peered":"2024-05-09T00:08:26.057436+0000","last_unstale":"2024-05-09T00:10:09.316437+0000","last_undegraded":"2024-05-09T00:10:09.316437+0000","last_fullsized":"2024-05-09T00:10:09.316437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:19.976265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468672+0000","last_change":"2024-05-09T00:08:23.118803+0000","last_active":"2024-05-09T00:09:10.468672+0000","last_peered":"2024-05-09T00:09:10.468672+0000","last_clean":"2024-05-09T00:09:10.468672+0000","last_became_active":"2024-05-09T00:08:23.118432+0000","last_became_peered":"2024-05-09T00:08:23.118432+0000","last_unstale":"2024-05-09T00:09:10.468672+0000","last_undegraded":"2024-05-09T00:09:10.468672+0000","last_fullsized":"2024-05-09T00:09:10.468672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:50.253699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.310200+0000","last_change":"2024-05-09T00:08:26.054281+0000","last_active":"2024-05-09T00:10:09.310200+0000","last_peered":"2024-05-09T00:10:09.310200+0000","last_clean":"2024-05-09T00:10:09.310200+0000","last_became_active":"2024-05-09T00:08:26.053874+0000","last_became_peered":"2024-05-09T00:08:26.053874+0000","last_unstale":"2024-05-09T00:10:09.310200+0000","last_undegraded":"2024-05-09T00:10:09.310200+0000","last_fullsized":"2024-05-09T00:10:09.310200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:11.566337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467767+0000","last_change":"2024-05-09T00:08:23.017905+0000","last_active":"2024-05-09T00:09:10.467767+0000","last_peered":"2024-05-09T00:09:10.467767+0000","last_clean":"2024-05-09T00:09:10.467767+0000","last_became_active":"2024-05-09T00:08:23.017481+0000","last_became_peered":"2024-05-09T00:08:23.017481+0000","last_unstale":"2024-05-09T00:09:10.467767+0000","last_undegraded":"2024-05-09T00:09:10.467767+0000","last_fullsized":"2024-05-09T00:09:10.467767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:40.096415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.317514+0000","last_change":"2024-05-09T00:08:26.053392+0000","last_active":"2024-05-09T00:10:09.317514+0000","last_peered":"2024-05-09T00:10:09.317514+0000","last_clean":"2024-05-09T00:10:09.317514+0000","last_became_active":"2024-05-09T00:08:26.052628+0000","last_became_peered":"2024-05-09T00:08:26.052628+0000","last_unstale":"2024-05-09T00:10:09.317514+0000","last_undegraded":"2024-05-09T00:10:09.317514+0000","last_fullsized":"2024-05-09T00:10:09.317514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:28.013955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.481712+0000","last_change":"2024-05-09T00:09:10.481233+0000","last_active":"2024-05-09T00:09:10.481712+0000","last_peered":"2024-05-09T00:09:10.481712+0000","last_clean":"2024-05-09T00:09:10.481712+0000","last_became_active":"2024-05-09T00:09:10.475321+0000","last_became_peered":"2024-05-09T00:09:10.475321+0000","last_unstale":"2024-05-09T00:09:10.481712+0000","last_undegraded":"2024-05-09T00:09:10.481712+0000","last_fullsized":"2024-05-09T00:09:10.481712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:58.859277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.308335+0000","last_change":"2024-05-09T00:08:26.047812+0000","last_active":"2024-05-09T00:10:09.308335+0000","last_peered":"2024-05-09T00:10:09.308335+0000","last_clean":"2024-05-09T00:10:09.308335+0000","last_became_active":"2024-05-09T00:08:26.047459+0000","last_became_peered":"2024-05-09T00:08:26.047459+0000","last_unstale":"2024-05-09T00:10:09.308335+0000","last_undegraded":"2024-05-09T00:10:09.308335+0000","last_fullsized":"2024-05-09T00:10:09.308335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.257831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473816+0000","last_change":"2024-05-09T00:08:23.023800+0000","last_active":"2024-05-09T00:09:10.473816+0000","last_peered":"2024-05-09T00:09:10.473816+0000","last_clean":"2024-05-09T00:09:10.473816+0000","last_became_active":"2024-05-09T00:08:23.023387+0000","last_became_peered":"2024-05-09T00:08:23.023387+0000","last_unstale":"2024-05-09T00:09:10.473816+0000","last_undegraded":"2024-05-09T00:09:10.473816+0000","last_fullsized":"2024-05-09T00:09:10.473816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:33.714683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330035+0000","last_change":"2024-05-09T00:08:26.056981+0000","last_active":"2024-05-09T00:10:09.330035+0000","last_peered":"2024-05-09T00:10:09.330035+0000","last_clean":"2024-05-09T00:10:09.330035+0000","last_became_active":"2024-05-09T00:08:26.056605+0000","last_became_peered":"2024-05-09T00:08:26.056605+0000","last_unstale":"2024-05-09T00:10:09.330035+0000","last_undegraded":"2024-05-09T00:10:09.330035+0000","last_fullsized":"2024-05-09T00:10:09.330035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:32.897929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473675+0000","last_change":"2024-05-09T00:08:23.120612+0000","last_active":"2024-05-09T00:09:10.473675+0000","last_peered":"2024-05-09T00:09:10.473675+0000","last_clean":"2024-05-09T00:09:10.473675+0000","last_became_active":"2024-05-09T00:08:23.120157+0000","last_became_peered":"2024-05-09T00:08:23.120157+0000","last_unstale":"2024-05-09T00:09:10.473675+0000","last_undegraded":"2024-05-09T00:09:10.473675+0000","last_fullsized":"2024-05-09T00:09:10.473675+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:08.027214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330144+0000","last_change":"2024-05-09T00:08:26.061594+0000","last_active":"2024-05-09T00:10:09.330144+0000","last_peered":"2024-05-09T00:10:09.330144+0000","last_clean":"2024-05-09T00:10:09.330144+0000","last_became_active":"2024-05-09T00:08:26.061201+0000","last_became_peered":"2024-05-09T00:08:26.061201+0000","last_unstale":"2024-05-09T00:10:09.330144+0000","last_undegraded":"2024-05-09T00:10:09.330144+0000","last_fullsized":"2024-05-09T00:10:09.330144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:50.183648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470328+0000","last_change":"2024-05-09T00:08:23.017693+0000","last_active":"2024-05-09T00:09:10.470328+0000","last_peered":"2024-05-09T00:09:10.470328+0000","last_clean":"2024-05-09T00:09:10.470328+0000","last_became_active":"2024-05-09T00:08:23.017211+0000","last_became_peered":"2024-05-09T00:08:23.017211+0000","last_unstale":"2024-05-09T00:09:10.470328+0000","last_undegraded":"2024-05-09T00:09:10.470328+0000","last_fullsized":"2024-05-09T00:09:10.470328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:53.569932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324623+0000","last_change":"2024-05-09T00:08:26.063922+0000","last_active":"2024-05-09T00:10:09.324623+0000","last_peered":"2024-05-09T00:10:09.324623+0000","last_clean":"2024-05-09T00:10:09.324623+0000","last_became_active":"2024-05-09T00:08:26.063554+0000","last_became_peered":"2024-05-09T00:08:26.063554+0000","last_unstale":"2024-05-09T00:10:09.324623+0000","last_undegraded":"2024-05-09T00:10:09.324623+0000","last_fullsized":"2024-05-09T00:10:09.324623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:01.754688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548651+0000","last_change":"2024-05-09T00:08:23.020367+0000","last_active":"2024-05-09T00:09:10.548651+0000","last_peered":"2024-05-09T00:09:10.548651+0000","last_clean":"2024-05-09T00:09:10.548651+0000","last_became_active":"2024-05-09T00:08:23.019801+0000","last_became_peered":"2024-05-09T00:08:23.019801+0000","last_unstale":"2024-05-09T00:09:10.548651+0000","last_undegraded":"2024-05-09T00:09:10.548651+0000","last_fullsized":"2024-05-09T00:09:10.548651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:14.936021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.329969+0000","last_change":"2024-05-09T00:08:26.063863+0000","last_active":"2024-05-09T00:10:09.329969+0000","last_peered":"2024-05-09T00:10:09.329969+0000","last_clean":"2024-05-09T00:10:09.329969+0000","last_became_active":"2024-05-09T00:08:26.061938+0000","last_became_peered":"2024-05-09T00:08:26.061938+0000","last_unstale":"2024-05-09T00:10:09.329969+0000","last_undegraded":"2024-05-09T00:10:09.329969+0000","last_fullsized":"2024-05-09T00:10:09.329969+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:04.597991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.319694+0000","last_change":"2024-05-09T00:08:26.050837+0000","last_active":"2024-05-09T00:10:09.319694+0000","last_peered":"2024-05-09T00:10:09.319694+0000","last_clean":"2024-05-09T00:10:09.319694+0000","last_became_active":"2024-05-09T00:08:26.050424+0000","last_became_peered":"2024-05-09T00:08:26.050424+0000","last_unstale":"2024-05-09T00:10:09.319694+0000","last_undegraded":"2024-05-09T00:10:09.319694+0000","last_fullsized":"2024-05-09T00:10:09.319694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:45.134811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323117+0000","last_change":"2024-05-09T00:09:10.549989+0000","last_active":"2024-05-09T00:10:09.323117+0000","last_peered":"2024-05-09T00:10:09.323117+0000","last_clean":"2024-05-09T00:10:09.323117+0000","last_became_active":"2024-05-09T00:09:10.549502+0000","last_became_peered":"2024-05-09T00:09:10.549502+0000","last_unstale":"2024-05-09T00:10:09.323117+0000","last_undegraded":"2024-05-09T00:10:09.323117+0000","last_fullsized":"2024-05-09T00:10:09.323117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:10.654452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323905+0000","last_change":"2024-05-09T00:08:26.058012+0000","last_active":"2024-05-09T00:10:09.323905+0000","last_peered":"2024-05-09T00:10:09.323905+0000","last_clean":"2024-05-09T00:10:09.323905+0000","last_became_active":"2024-05-09T00:08:26.057282+0000","last_became_peered":"2024-05-09T00:08:26.057282+0000","last_unstale":"2024-05-09T00:10:09.323905+0000","last_undegraded":"2024-05-09T00:10:09.323905+0000","last_fullsized":"2024-05-09T00:10:09.323905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:53.520593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.320691+0000","last_change":"2024-05-09T00:08:26.063331+0000","last_active":"2024-05-09T00:10:09.320691+0000","last_peered":"2024-05-09T00:10:09.320691+0000","last_clean":"2024-05-09T00:10:09.320691+0000","last_became_active":"2024-05-09T00:08:26.062866+0000","last_became_peered":"2024-05-09T00:08:26.062866+0000","last_unstale":"2024-05-09T00:10:09.320691+0000","last_undegraded":"2024-05-09T00:10:09.320691+0000","last_fullsized":"2024-05-09T00:10:09.320691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:10.387852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326309+0000","last_change":"2024-05-09T00:08:26.063735+0000","last_active":"2024-05-09T00:10:09.326309+0000","last_peered":"2024-05-09T00:10:09.326309+0000","last_clean":"2024-05-09T00:10:09.326309+0000","last_became_active":"2024-05-09T00:08:26.063358+0000","last_became_peered":"2024-05-09T00:08:26.063358+0000","last_unstale":"2024-05-09T00:10:09.326309+0000","last_undegraded":"2024-05-09T00:10:09.326309+0000","last_fullsized":"2024-05-09T00:10:09.326309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:06.832071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318431+0000","last_change":"2024-05-09T00:08:26.052746+0000","last_active":"2024-05-09T00:10:09.318431+0000","last_peered":"2024-05-09T00:10:09.318431+0000","last_clean":"2024-05-09T00:10:09.318431+0000","last_became_active":"2024-05-09T00:08:26.051544+0000","last_became_peered":"2024-05-09T00:08:26.051544+0000","last_unstale":"2024-05-09T00:10:09.318431+0000","last_undegraded":"2024-05-09T00:10:09.318431+0000","last_fullsized":"2024-05-09T00:10:09.318431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:31.963268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323267+0000","last_change":"2024-05-09T00:08:26.058075+0000","last_active":"2024-05-09T00:10:09.323267+0000","last_peered":"2024-05-09T00:10:09.323267+0000","last_clean":"2024-05-09T00:10:09.323267+0000","last_became_active":"2024-05-09T00:08:26.057349+0000","last_became_peered":"2024-05-09T00:08:26.057349+0000","last_unstale":"2024-05-09T00:10:09.323267+0000","last_undegraded":"2024-05-09T00:10:09.323267+0000","last_fullsized":"2024-05-09T00:10:09.323267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:59.322903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324094+0000","last_change":"2024-05-09T00:08:26.056429+0000","last_active":"2024-05-09T00:10:09.324094+0000","last_peered":"2024-05-09T00:10:09.324094+0000","last_clean":"2024-05-09T00:10:09.324094+0000","last_became_active":"2024-05-09T00:08:26.054852+0000","last_became_peered":"2024-05-09T00:08:26.054852+0000","last_unstale":"2024-05-09T00:10:09.324094+0000","last_undegraded":"2024-05-09T00:10:09.324094+0000","last_fullsized":"2024-05-09T00:10:09.324094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:05.337636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327096+0000","last_change":"2024-05-09T00:09:10.549880+0000","last_active":"2024-05-09T00:10:09.327096+0000","last_peered":"2024-05-09T00:10:09.327096+0000","last_clean":"2024-05-09T00:10:09.327096+0000","last_became_active":"2024-05-09T00:09:10.549448+0000","last_became_peered":"2024-05-09T00:09:10.549448+0000","last_unstale":"2024-05-09T00:10:09.327096+0000","last_undegraded":"2024-05-09T00:10:09.327096+0000","last_fullsized":"2024-05-09T00:10:09.327096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:16.990729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325082+0000","last_change":"2024-05-09T00:08:26.054579+0000","last_active":"2024-05-09T00:10:09.325082+0000","last_peered":"2024-05-09T00:10:09.325082+0000","last_clean":"2024-05-09T00:10:09.325082+0000","last_became_active":"2024-05-09T00:08:26.053186+0000","last_became_peered":"2024-05-09T00:08:26.053186+0000","last_unstale":"2024-05-09T00:10:09.325082+0000","last_undegraded":"2024-05-09T00:10:09.325082+0000","last_fullsized":"2024-05-09T00:10:09.325082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:37.561691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324353+0000","last_change":"2024-05-09T00:08:26.063608+0000","last_active":"2024-05-09T00:10:09.324353+0000","last_peered":"2024-05-09T00:10:09.324353+0000","last_clean":"2024-05-09T00:10:09.324353+0000","last_became_active":"2024-05-09T00:08:26.063078+0000","last_became_peered":"2024-05-09T00:08:26.063078+0000","last_unstale":"2024-05-09T00:10:09.324353+0000","last_undegraded":"2024-05-09T00:10:09.324353+0000","last_fullsized":"2024-05-09T00:10:09.324353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:06.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330947+0000","last_change":"2024-05-09T00:08:26.055058+0000","last_active":"2024-05-09T00:10:09.330947+0000","last_peered":"2024-05-09T00:10:09.330947+0000","last_clean":"2024-05-09T00:10:09.330947+0000","last_became_active":"2024-05-09T00:08:26.054695+0000","last_became_peered":"2024-05-09T00:08:26.054695+0000","last_unstale":"2024-05-09T00:10:09.330947+0000","last_undegraded":"2024-05-09T00:10:09.330947+0000","last_fullsized":"2024-05-09T00:10:09.330947+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:58.925990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.312724+0000","last_change":"2024-05-09T00:08:26.055366+0000","last_active":"2024-05-09T00:10:09.312724+0000","last_peered":"2024-05-09T00:10:09.312724+0000","last_clean":"2024-05-09T00:10:09.312724+0000","last_became_active":"2024-05-09T00:08:26.054904+0000","last_became_peered":"2024-05-09T00:08:26.054904+0000","last_unstale":"2024-05-09T00:10:09.312724+0000","last_undegraded":"2024-05-09T00:10:09.312724+0000","last_fullsized":"2024-05-09T00:10:09.312724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:40.754878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285241,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66217,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":726,"num_read_kb":429,"num_write":172,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710762,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":425247,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1079313,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070517,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":880640,"data_stored":425461,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266746,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":425359,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":1017452,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":868352,"data_stored":424850,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":486718,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":888832,"data_stored":426121,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":488068,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":425161,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986256,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":425819,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822730,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":1016604,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:10:44.357 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T00:10:44.357 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T00:10:44.357 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-09T00:10:44.357 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-09T00:10:44.611 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:44.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:44 smithi142 ceph-mon[34533]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:44.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:44 smithi142 ceph-mon[34533]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:10:44.733 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:44 smithi046 ceph-mon[30151]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:44.733 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:44 smithi046 ceph-mon[30151]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:10:44.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:44 smithi169 ceph-mon[34777]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:44.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:44 smithi169 ceph-mon[34777]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:10:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.332+0000 7ff763b38700 1 -- 172.21.15.46:0/2545431893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff75c1045d0 msgr2=0x7ff75c1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.332+0000 7ff763b38700 1 --2- 172.21.15.46:0/2545431893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff75c1045d0 0x7ff75c1049e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff7580099c0 tx=0x7ff758009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.332+0000 7ff763b38700 1 -- 172.21.15.46:0/2545431893 shutdown_connections 2024-05-09T00:10:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.332+0000 7ff763b38700 1 --2- 172.21.15.46:0/2545431893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff75c109850 0x7ff75c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.332+0000 7ff763b38700 1 --2- 172.21.15.46:0/2545431893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff75c1045d0 0x7ff75c1049e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.332+0000 7ff763b38700 1 --2- 172.21.15.46:0/2545431893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff75c108f00 0x7ff75c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.332+0000 7ff763b38700 1 -- 172.21.15.46:0/2545431893 >> 172.21.15.46:0/2545431893 conn(0x7ff75c0ffea0 msgr2=0x7ff75c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.332+0000 7ff763b38700 1 -- 172.21.15.46:0/2545431893 shutdown_connections 2024-05-09T00:10:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.332+0000 7ff763b38700 1 -- 172.21.15.46:0/2545431893 wait complete. 2024-05-09T00:10:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.333+0000 7ff763b38700 1 Processor -- start 2024-05-09T00:10:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.333+0000 7ff763b38700 1 -- start start 2024-05-09T00:10:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.333+0000 7ff763b38700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff75c1045d0 0x7ff75c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.333+0000 7ff763b38700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff75c108f00 0x7ff75c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.333+0000 7ff763b38700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff75c109850 0x7ff75c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.333+0000 7ff763b38700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c114480 con 0x7ff75c1045d0 2024-05-09T00:10:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.333+0000 7ff763b38700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c114300 con 0x7ff75c108f00 2024-05-09T00:10:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.333+0000 7ff763b38700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c114600 con 0x7ff75c109850 2024-05-09T00:10:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.333+0000 7ff7618d4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff75c1045d0 0x7ff75c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff7618d4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff75c1045d0 0x7ff75c19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48018/0 (socket says 172.21.15.46:48018) 2024-05-09T00:10:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff7618d4700 1 -- 172.21.15.46:0/1264661389 learned_addr learned my addr 172.21.15.46:0/1264661389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff7610d3700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff75c108f00 0x7ff75c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff7620d5700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff75c109850 0x7ff75c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff7618d4700 1 -- 172.21.15.46:0/1264661389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff75c109850 msgr2=0x7ff75c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff7618d4700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff75c109850 0x7ff75c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff7618d4700 1 -- 172.21.15.46:0/1264661389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff75c108f00 msgr2=0x7ff75c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff7618d4700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff75c108f00 0x7ff75c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff7618d4700 1 -- 172.21.15.46:0/1264661389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c19fd40 con 0x7ff75c1045d0 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff7618d4700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff75c1045d0 0x7ff75c19ae70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff75800b580 tx=0x7ff7580047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff752ffd700 1 -- 172.21.15.46:0/1264661389 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758027070 con 0x7ff75c1045d0 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff752ffd700 1 -- 172.21.15.46:0/1264661389 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff758003f20 con 0x7ff75c1045d0 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff763b38700 1 -- 172.21.15.46:0/1264661389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75c19ffd0 con 0x7ff75c1045d0 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff752ffd700 1 -- 172.21.15.46:0/1264661389 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758022d70 con 0x7ff75c1045d0 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.334+0000 7ff763b38700 1 -- 172.21.15.46:0/1264661389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75c1a04b0 con 0x7ff75c1045d0 2024-05-09T00:10:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.335+0000 7ff763b38700 1 -- 172.21.15.46:0/1264661389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75c0512c0 con 0x7ff75c1045d0 2024-05-09T00:10:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.337+0000 7ff752ffd700 1 -- 172.21.15.46:0/1264661389 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff758022500 con 0x7ff75c1045d0 2024-05-09T00:10:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.338+0000 7ff752ffd700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff7480b2880 0x7ff7480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.338+0000 7ff7610d3700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff7480b2880 0x7ff7480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.338+0000 7ff752ffd700 1 -- 172.21.15.46:0/1264661389 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff7580e32e0 con 0x7ff75c1045d0 2024-05-09T00:10:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.338+0000 7ff7610d3700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff7480b2880 0x7ff7480b4d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff75c103520 tx=0x7ff74c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:46.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.342+0000 7ff752ffd700 1 -- 172.21.15.46:0/1264661389 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff758013070 con 0x7ff75c1045d0 2024-05-09T00:10:46.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.606+0000 7ff763b38700 1 -- 172.21.15.46:0/1264661389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff75c02e5f0 con 0x7ff75c1045d0 2024-05-09T00:10:46.603 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:46 smithi046 ceph-mon[30151]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:46.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.606+0000 7ff752ffd700 1 -- 172.21.15.46:0/1264661389 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff7580ab090 con 0x7ff75c1045d0 2024-05-09T00:10:46.604 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:10:46.604 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T00:10:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.610+0000 7ff763b38700 1 -- 172.21.15.46:0/1264661389 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff7480b2880 msgr2=0x7ff7480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.610+0000 7ff763b38700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff7480b2880 0x7ff7480b4d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff75c103520 tx=0x7ff74c00b410 comp rx=0 tx=0).stop 2024-05-09T00:10:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.610+0000 7ff763b38700 1 -- 172.21.15.46:0/1264661389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff75c1045d0 msgr2=0x7ff75c19ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.610+0000 7ff763b38700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff75c1045d0 0x7ff75c19ae70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff75800b580 tx=0x7ff7580047b0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.610+0000 7ff763b38700 1 -- 172.21.15.46:0/1264661389 shutdown_connections 2024-05-09T00:10:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.610+0000 7ff763b38700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff7480b2880 0x7ff7480b4d90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.610+0000 7ff763b38700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff75c109850 0x7ff75c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.610+0000 7ff763b38700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff75c1045d0 0x7ff75c19ae70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.610+0000 7ff763b38700 1 --2- 172.21.15.46:0/1264661389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff75c108f00 0x7ff75c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.610+0000 7ff763b38700 1 -- 172.21.15.46:0/1264661389 >> 172.21.15.46:0/1264661389 conn(0x7ff75c0ffea0 msgr2=0x7ff75c10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.610+0000 7ff763b38700 1 -- 172.21.15.46:0/1264661389 shutdown_connections 2024-05-09T00:10:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:46.610+0000 7ff763b38700 1 -- 172.21.15.46:0/1264661389 wait complete. 2024-05-09T00:10:46.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:46 smithi142 ceph-mon[34533]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:46.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:46 smithi169 ceph-mon[34777]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:47.278 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-09T00:10:47.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:10:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:47.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:47 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1264661389' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T00:10:47.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:47 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1264661389' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T00:10:47.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:47 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1264661389' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T00:10:48.641 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:48 smithi046 ceph-mon[30151]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:48.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:48 smithi142 ceph-mon[34533]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:48.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:48 smithi169 ceph-mon[34777]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.172+0000 7f58c798e700 1 -- 172.21.15.46:0/2448206354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0105e10 msgr2=0x7f58c010e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.172+0000 7f58c798e700 1 --2- 172.21.15.46:0/2448206354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0105e10 0x7f58c010e360 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009990 tx=0x7f58bc009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.172+0000 7f58c798e700 1 -- 172.21.15.46:0/2448206354 shutdown_connections 2024-05-09T00:10:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.172+0000 7f58c798e700 1 --2- 172.21.15.46:0/2448206354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c010e8f0 0x7f58c0110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.172+0000 7f58c798e700 1 --2- 172.21.15.46:0/2448206354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c01054c0 0x7f58c01058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.172+0000 7f58c798e700 1 --2- 172.21.15.46:0/2448206354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0105e10 0x7f58c010e360 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.172+0000 7f58c798e700 1 -- 172.21.15.46:0/2448206354 >> 172.21.15.46:0/2448206354 conn(0x7f58c00fedf0 msgr2=0x7f58c0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.172+0000 7f58c798e700 1 -- 172.21.15.46:0/2448206354 shutdown_connections 2024-05-09T00:10:49.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.172+0000 7f58c798e700 1 -- 172.21.15.46:0/2448206354 wait complete. 2024-05-09T00:10:49.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c798e700 1 Processor -- start 2024-05-09T00:10:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c798e700 1 -- start start 2024-05-09T00:10:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c798e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c01054c0 0x7f58c019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c798e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c0105e10 0x7f58c019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c798e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c010e8f0 0x7f58c019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c798e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58c0114450 con 0x7f58c010e8f0 2024-05-09T00:10:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c798e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f58c01142d0 con 0x7f58c01054c0 2024-05-09T00:10:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c798e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f58c01145d0 con 0x7f58c0105e10 2024-05-09T00:10:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c5f2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c010e8f0 0x7f58c019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c5f2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c010e8f0 0x7f58c019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48036/0 (socket says 172.21.15.46:48036) 2024-05-09T00:10:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c5f2b700 1 -- 172.21.15.46:0/3393062263 learned_addr learned my addr 172.21.15.46:0/3393062263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c572a700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c01054c0 0x7f58c019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.173+0000 7f58c4f29700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c0105e10 0x7f58c019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.174+0000 7f58c5f2b700 1 -- 172.21.15.46:0/3393062263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c0105e10 msgr2=0x7f58c019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.174+0000 7f58c5f2b700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c0105e10 0x7f58c019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.174+0000 7f58c5f2b700 1 -- 172.21.15.46:0/3393062263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c01054c0 msgr2=0x7f58c019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.174+0000 7f58c5f2b700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c01054c0 0x7f58c019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.174+0000 7f58c5f2b700 1 -- 172.21.15.46:0/3393062263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c019fc90 con 0x7f58c010e8f0 2024-05-09T00:10:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.174+0000 7f58c5f2b700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c010e8f0 0x7f58c019f5f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f58b8007ee0 tx=0x7f58b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.174+0000 7f58b27fc700 1 -- 172.21.15.46:0/3393062263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8004bb0 con 0x7f58c010e8f0 2024-05-09T00:10:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.174+0000 7f58b27fc700 1 -- 172.21.15.46:0/3393062263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58b8004d50 con 0x7f58c010e8f0 2024-05-09T00:10:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.174+0000 7f58c798e700 1 -- 172.21.15.46:0/3393062263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c019ff80 con 0x7f58c010e8f0 2024-05-09T00:10:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.174+0000 7f58b27fc700 1 -- 172.21.15.46:0/3393062263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8005920 con 0x7f58c010e8f0 2024-05-09T00:10:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.174+0000 7f58c798e700 1 -- 172.21.15.46:0/3393062263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c01a78a0 con 0x7f58c010e8f0 2024-05-09T00:10:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.175+0000 7f58c798e700 1 -- 172.21.15.46:0/3393062263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c0076270 con 0x7f58c010e8f0 2024-05-09T00:10:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.177+0000 7f58b27fc700 1 -- 172.21.15.46:0/3393062263 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f58b801a020 con 0x7f58c010e8f0 2024-05-09T00:10:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.178+0000 7f58b27fc700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58ac0b2880 0x7f58ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.178+0000 7f58c572a700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58ac0b2880 0x7f58ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.178+0000 7f58b27fc700 1 -- 172.21.15.46:0/3393062263 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f58b80e1830 con 0x7f58c010e8f0 2024-05-09T00:10:49.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.178+0000 7f58c572a700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58ac0b2880 0x7f58ac0b4d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58c006d150 tx=0x7f58b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.183+0000 7f58b27fc700 1 -- 172.21.15.46:0/3393062263 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f58b8010070 con 0x7f58c010e8f0 2024-05-09T00:10:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.446+0000 7f58c798e700 1 -- 172.21.15.46:0/3393062263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58c019c120 con 0x7f58c010e8f0 2024-05-09T00:10:49.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.447+0000 7f58b27fc700 1 -- 172.21.15.46:0/3393062263 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11879 (secure 0 0 0) 0x7f58b80a9550 con 0x7f58c010e8f0 2024-05-09T00:10:49.444 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:10:49.444 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:08:47.423159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24497,24499,24487]},"id":1}]} 2024-05-09T00:10:49.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.449+0000 7f58c798e700 1 -- 172.21.15.46:0/3393062263 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58ac0b2880 msgr2=0x7f58ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.449+0000 7f58c798e700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58ac0b2880 0x7f58ac0b4d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58c006d150 tx=0x7f58b400b410 comp rx=0 tx=0).stop 2024-05-09T00:10:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.449+0000 7f58c798e700 1 -- 172.21.15.46:0/3393062263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c010e8f0 msgr2=0x7f58c019f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.449+0000 7f58c798e700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c010e8f0 0x7f58c019f5f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f58b8007ee0 tx=0x7f58b800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:10:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.449+0000 7f58c798e700 1 -- 172.21.15.46:0/3393062263 shutdown_connections 2024-05-09T00:10:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.449+0000 7f58c798e700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58ac0b2880 0x7f58ac0b4d90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.449+0000 7f58c798e700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c010e8f0 0x7f58c019f5f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.449+0000 7f58c798e700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58c01054c0 0x7f58c019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.449+0000 7f58c798e700 1 --2- 172.21.15.46:0/3393062263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c0105e10 0x7f58c019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.449+0000 7f58c798e700 1 -- 172.21.15.46:0/3393062263 >> 172.21.15.46:0/3393062263 conn(0x7f58c00fedf0 msgr2=0x7f58c0100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.449+0000 7f58c798e700 1 -- 172.21.15.46:0/3393062263 shutdown_connections 2024-05-09T00:10:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:49.449+0000 7f58c798e700 1 -- 172.21.15.46:0/3393062263 wait complete. 2024-05-09T00:10:49.447 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T00:10:50.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:10:50.446 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:50.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:50 smithi046 ceph-mon[30151]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:10:50.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:50 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3393062263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:50.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:50 smithi142 ceph-mon[34533]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:10:50.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:50 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3393062263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:50.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:50 smithi169 ceph-mon[34777]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:10:50.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:50 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3393062263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.130+0000 7fc230d11700 1 -- 172.21.15.46:0/2084804595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c1045f0 msgr2=0x7fc22c104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.130+0000 7fc230d11700 1 --2- 172.21.15.46:0/2084804595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c1045f0 0x7fc22c104a00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0099c0 tx=0x7fc21c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.130+0000 7fc230d11700 1 -- 172.21.15.46:0/2084804595 shutdown_connections 2024-05-09T00:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.130+0000 7fc230d11700 1 --2- 172.21.15.46:0/2084804595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc22c109870 0x7fc22c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.130+0000 7fc230d11700 1 --2- 172.21.15.46:0/2084804595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c1045f0 0x7fc22c104a00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.130+0000 7fc230d11700 1 --2- 172.21.15.46:0/2084804595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc22c108f20 0x7fc22c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.130+0000 7fc230d11700 1 -- 172.21.15.46:0/2084804595 >> 172.21.15.46:0/2084804595 conn(0x7fc22c100040 msgr2=0x7fc22c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.130+0000 7fc230d11700 1 -- 172.21.15.46:0/2084804595 shutdown_connections 2024-05-09T00:10:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.130+0000 7fc230d11700 1 -- 172.21.15.46:0/2084804595 wait complete. 2024-05-09T00:10:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc230d11700 1 Processor -- start 2024-05-09T00:10:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc230d11700 1 -- start start 2024-05-09T00:10:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc230d11700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc22c1045f0 0x7fc22c19af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc230d11700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc22c108f20 0x7fc22c19b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc230d11700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c109870 0x7fc22c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc230d11700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c114560 con 0x7fc22c109870 2024-05-09T00:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc230d11700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c1143e0 con 0x7fc22c108f20 2024-05-09T00:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc230d11700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c1146e0 con 0x7fc22c1045f0 2024-05-09T00:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc22ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c109870 0x7fc22c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc22ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c109870 0x7fc22c19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48044/0 (socket says 172.21.15.46:48044) 2024-05-09T00:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc22ad9d700 1 -- 172.21.15.46:0/2973364898 learned_addr learned my addr 172.21.15.46:0/2973364898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc22a59c700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc22c1045f0 0x7fc22c19af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.131+0000 7fc229d9b700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc22c108f20 0x7fc22c19b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.132+0000 7fc22ad9d700 1 -- 172.21.15.46:0/2973364898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc22c1045f0 msgr2=0x7fc22c19af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.132+0000 7fc22ad9d700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc22c1045f0 0x7fc22c19af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.132+0000 7fc22ad9d700 1 -- 172.21.15.46:0/2973364898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc22c108f20 msgr2=0x7fc22c19b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.132+0000 7fc22ad9d700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc22c108f20 0x7fc22c19b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.132+0000 7fc22ad9d700 1 -- 172.21.15.46:0/2973364898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c19fde0 con 0x7fc22c109870 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.132+0000 7fc229d9b700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc22c108f20 0x7fc22c19b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.132+0000 7fc22ad9d700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c109870 0x7fc22c19f740 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc224007ee0 tx=0x7fc22400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.132+0000 7fc21b7fe700 1 -- 172.21.15.46:0/2973364898 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22400ddd0 con 0x7fc22c109870 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.132+0000 7fc230d11700 1 -- 172.21.15.46:0/2973364898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c1a00d0 con 0x7fc22c109870 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.132+0000 7fc21b7fe700 1 -- 172.21.15.46:0/2973364898 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc224004020 con 0x7fc22c109870 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.132+0000 7fc21b7fe700 1 -- 172.21.15.46:0/2973364898 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224015660 con 0x7fc22c109870 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.132+0000 7fc230d11700 1 -- 172.21.15.46:0/2973364898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c1a79f0 con 0x7fc22c109870 2024-05-09T00:10:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.133+0000 7fc230d11700 1 -- 172.21.15.46:0/2973364898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22c0512c0 con 0x7fc22c109870 2024-05-09T00:10:52.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.136+0000 7fc21b7fe700 1 -- 172.21.15.46:0/2973364898 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc2240041d0 con 0x7fc22c109870 2024-05-09T00:10:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.137+0000 7fc21b7fe700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc2140b2880 0x7fc2140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.137+0000 7fc22a59c700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc2140b2880 0x7fc2140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:52.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.137+0000 7fc21b7fe700 1 -- 172.21.15.46:0/2973364898 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc2240e24c0 con 0x7fc22c109870 2024-05-09T00:10:52.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.137+0000 7fc22a59c700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc2140b2880 0x7fc2140b4d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009990 tx=0x7fc21c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.140+0000 7fc21b7fe700 1 -- 172.21.15.46:0/2973364898 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc224010070 con 0x7fc22c109870 2024-05-09T00:10:52.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.400+0000 7fc230d11700 1 -- 172.21.15.46:0/2973364898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc22c02e5f0 con 0x7fc22c109870 2024-05-09T00:10:52.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.400+0000 7fc21b7fe700 1 -- 172.21.15.46:0/2973364898 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11879 (secure 0 0 0) 0x7fc2240aa1e0 con 0x7fc22c109870 2024-05-09T00:10:52.397 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:10:52.397 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:08:47.423159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24497,24499,24487]},"id":1}]} 2024-05-09T00:10:52.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.403+0000 7fc230d11700 1 -- 172.21.15.46:0/2973364898 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc2140b2880 msgr2=0x7fc2140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:52.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.403+0000 7fc230d11700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc2140b2880 0x7fc2140b4d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009990 tx=0x7fc21c000bc0 comp rx=0 tx=0).stop 2024-05-09T00:10:52.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.403+0000 7fc230d11700 1 -- 172.21.15.46:0/2973364898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c109870 msgr2=0x7fc22c19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.403+0000 7fc230d11700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c109870 0x7fc22c19f740 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc224007ee0 tx=0x7fc22400d3b0 comp rx=0 tx=0).stop 2024-05-09T00:10:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.403+0000 7fc230d11700 1 -- 172.21.15.46:0/2973364898 shutdown_connections 2024-05-09T00:10:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.403+0000 7fc230d11700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc2140b2880 0x7fc2140b4d90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.403+0000 7fc230d11700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc22c109870 0x7fc22c19f740 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.403+0000 7fc230d11700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc22c1045f0 0x7fc22c19af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.403+0000 7fc230d11700 1 --2- 172.21.15.46:0/2973364898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc22c108f20 0x7fc22c19b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.403+0000 7fc230d11700 1 -- 172.21.15.46:0/2973364898 >> 172.21.15.46:0/2973364898 conn(0x7fc22c100040 msgr2=0x7fc22c1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.403+0000 7fc230d11700 1 -- 172.21.15.46:0/2973364898 shutdown_connections 2024-05-09T00:10:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:52.403+0000 7fc230d11700 1 -- 172.21.15.46:0/2973364898 wait complete. 2024-05-09T00:10:52.401 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T00:10:52.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:52 smithi142 ceph-mon[34533]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:10:52.722 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:52 smithi046 ceph-mon[30151]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:10:52.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:52 smithi169 ceph-mon[34777]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:10:53.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:10:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:53.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:53 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2973364898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:53.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:53 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2973364898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:53.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:53 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2973364898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:54.650 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:54 smithi046 ceph-mon[30151]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:54.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:54 smithi142 ceph-mon[34533]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:54.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:54 smithi169 ceph-mon[34777]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:55.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.024+0000 7fc4ff311700 1 -- 172.21.15.46:0/329311102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4f81034c0 msgr2=0x7fc4f81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:55.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.024+0000 7fc4ff311700 1 --2- 172.21.15.46:0/329311102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4f81034c0 0x7fc4f81038d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40099c0 tx=0x7fc4f4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:10:55.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.024+0000 7fc4ff311700 1 -- 172.21.15.46:0/329311102 shutdown_connections 2024-05-09T00:10:55.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.024+0000 7fc4ff311700 1 --2- 172.21.15.46:0/329311102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f8108740 0x7fc4f81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:55.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.024+0000 7fc4ff311700 1 --2- 172.21.15.46:0/329311102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4f81034c0 0x7fc4f81038d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:55.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.024+0000 7fc4ff311700 1 --2- 172.21.15.46:0/329311102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8107df0 0x7fc4f8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:55.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.024+0000 7fc4ff311700 1 -- 172.21.15.46:0/329311102 >> 172.21.15.46:0/329311102 conn(0x7fc4f80fedf0 msgr2=0x7fc4f8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:55.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.025+0000 7fc4ff311700 1 -- 172.21.15.46:0/329311102 shutdown_connections 2024-05-09T00:10:55.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.025+0000 7fc4ff311700 1 -- 172.21.15.46:0/329311102 wait complete. 2024-05-09T00:10:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.025+0000 7fc4ff311700 1 Processor -- start 2024-05-09T00:10:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.025+0000 7fc4ff311700 1 -- start start 2024-05-09T00:10:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4ff311700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4f81034c0 0x7fc4f819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4ff311700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f8107df0 0x7fc4f819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4ff311700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8108740 0x7fc4f819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4ff311700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8113370 con 0x7fc4f81034c0 2024-05-09T00:10:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4ff311700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f81131f0 con 0x7fc4f8108740 2024-05-09T00:10:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4ff311700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f81134f0 con 0x7fc4f8107df0 2024-05-09T00:10:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4fd0ad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4f81034c0 0x7fc4f819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4fd0ad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4f81034c0 0x7fc4f819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48050/0 (socket says 172.21.15.46:48050) 2024-05-09T00:10:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4fd0ad700 1 -- 172.21.15.46:0/3759464216 learned_addr learned my addr 172.21.15.46:0/3759464216 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4fc8ac700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f8107df0 0x7fc4f819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4fd8ae700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8108740 0x7fc4f819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4fd0ad700 1 -- 172.21.15.46:0/3759464216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f8107df0 msgr2=0x7fc4f819c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4fd0ad700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f8107df0 0x7fc4f819c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4fd0ad700 1 -- 172.21.15.46:0/3759464216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8108740 msgr2=0x7fc4f819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4fd0ad700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8108740 0x7fc4f819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4fd0ad700 1 -- 172.21.15.46:0/3759464216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f81a6fa0 con 0x7fc4f81034c0 2024-05-09T00:10:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4fd0ad700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4f81034c0 0x7fc4f819bc30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40098e0 tx=0x7fc4f40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.026+0000 7fc4ee7fc700 1 -- 172.21.15.46:0/3759464216 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4027070 con 0x7fc4f81034c0 2024-05-09T00:10:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.027+0000 7fc4ff311700 1 -- 172.21.15.46:0/3759464216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f81a7230 con 0x7fc4f81034c0 2024-05-09T00:10:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.027+0000 7fc4ee7fc700 1 -- 172.21.15.46:0/3759464216 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f4003f20 con 0x7fc4f81034c0 2024-05-09T00:10:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.027+0000 7fc4ee7fc700 1 -- 172.21.15.46:0/3759464216 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4022ca0 con 0x7fc4f81034c0 2024-05-09T00:10:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.027+0000 7fc4ff311700 1 -- 172.21.15.46:0/3759464216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f81a7680 con 0x7fc4f81034c0 2024-05-09T00:10:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.028+0000 7fc4ff311700 1 -- 172.21.15.46:0/3759464216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f80512c0 con 0x7fc4f81034c0 2024-05-09T00:10:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.029+0000 7fc4ee7fc700 1 -- 172.21.15.46:0/3759464216 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc4f402a070 con 0x7fc4f81034c0 2024-05-09T00:10:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.030+0000 7fc4ee7fc700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc4e40b2880 0x7fc4e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.030+0000 7fc4fc8ac700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc4e40b2880 0x7fc4e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.030+0000 7fc4ee7fc700 1 -- 172.21.15.46:0/3759464216 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc4f40e3860 con 0x7fc4f81034c0 2024-05-09T00:10:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.031+0000 7fc4fc8ac700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc4e40b2880 0x7fc4e40b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8106860 tx=0x7fc4e80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.034+0000 7fc4ee7fc700 1 -- 172.21.15.46:0/3759464216 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc4f4031080 con 0x7fc4f81034c0 2024-05-09T00:10:55.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.248+0000 7fc4ff311700 1 -- 172.21.15.46:0/3759464216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4f802e7b0 con 0x7fc4f81034c0 2024-05-09T00:10:55.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.250+0000 7fc4ee7fc700 1 -- 172.21.15.46:0/3759464216 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22694 (secure 0 0 0) 0x7fc4f4013070 con 0x7fc4f81034c0 2024-05-09T00:10:55.247 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:10:55.247 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","created":"2024-05-09T00:00:14.356524+0000","modified":"2024-05-09T00:09:10.458026+0000","last_up_change":"2024-05-09T00:07:23.332796+0000","last_in_change":"2024-05-09T00:07:03.238539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:03:33.278910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:08:21.014416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:08:24.922968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"740d6388-5c1f-45eb-93b0-7583bfa5a225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6803","nonce":3727342125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6805","nonce":3727342125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6809","nonce":3727342125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6807","nonce":3727342125}]},"public_addr":"172.21.15.46:6803/3727342125","cluster_addr":"172.21.15.46:6805/3727342125","heartbeat_back_addr":"172.21.15.46:6809/3727342125","heartbeat_front_addr":"172.21.15.46:6807/3727342125","state":["exists","up"]},{"osd":1,"uuid":"85fbf3ae-0be8-4504-84d6-66fae0d2169d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6801","nonce":3156042469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6803","nonce":3156042469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6807","nonce":3156042469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6805","nonce":3156042469}]},"public_addr":"172.21.15.142:6801/3156042469","cluster_addr":"172.21.15.142:6803/3156042469","heartbeat_back_addr":"172.21.15.142:6807/3156042469","heartbeat_front_addr":"172.21.15.142:6805/3156042469","state":["exists","up"]},{"osd":2,"uuid":"05c070e7-83ad-4949-9834-4ce51ac0a2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6801","nonce":2680673064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6803","nonce":2680673064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6807","nonce":2680673064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6805","nonce":2680673064}]},"public_addr":"172.21.15.169:6801/2680673064","cluster_addr":"172.21.15.169:6803/2680673064","heartbeat_back_addr":"172.21.15.169:6807/2680673064","heartbeat_front_addr":"172.21.15.169:6805/2680673064","state":["exists","up"]},{"osd":3,"uuid":"09d7bbe1-58b6-4407-bef2-920f179dfb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6811","nonce":1158536261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6813","nonce":1158536261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6817","nonce":1158536261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6815","nonce":1158536261}]},"public_addr":"172.21.15.46:6811/1158536261","cluster_addr":"172.21.15.46:6813/1158536261","heartbeat_back_addr":"172.21.15.46:6817/1158536261","heartbeat_front_addr":"172.21.15.46:6815/1158536261","state":["exists","up"]},{"osd":4,"uuid":"66c9644b-6b0b-4fe7-aa63-b9d87bff7153","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6809","nonce":2299616865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6811","nonce":2299616865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6815","nonce":2299616865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6813","nonce":2299616865}]},"public_addr":"172.21.15.142:6809/2299616865","cluster_addr":"172.21.15.142:6811/2299616865","heartbeat_back_addr":"172.21.15.142:6815/2299616865","heartbeat_front_addr":"172.21.15.142:6813/2299616865","state":["exists","up"]},{"osd":5,"uuid":"96a72e01-1c5f-418e-bb9b-b790d9b3059b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6809","nonce":445044713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6811","nonce":445044713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6815","nonce":445044713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6813","nonce":445044713}]},"public_addr":"172.21.15.169:6809/445044713","cluster_addr":"172.21.15.169:6811/445044713","heartbeat_back_addr":"172.21.15.169:6815/445044713","heartbeat_front_addr":"172.21.15.169:6813/445044713","state":["exists","up"]},{"osd":6,"uuid":"6b459542-0e56-4369-b542-e36c46a18f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6819","nonce":1394035482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6821","nonce":1394035482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6825","nonce":1394035482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6823","nonce":1394035482}]},"public_addr":"172.21.15.46:6819/1394035482","cluster_addr":"172.21.15.46:6821/1394035482","heartbeat_back_addr":"172.21.15.46:6825/1394035482","heartbeat_front_addr":"172.21.15.46:6823/1394035482","state":["exists","up"]},{"osd":7,"uuid":"786b4517-4cbd-4fa3-b5a0-39cccf839893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6817","nonce":596805607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6819","nonce":596805607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6823","nonce":596805607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6821","nonce":596805607}]},"public_addr":"172.21.15.142:6817/596805607","cluster_addr":"172.21.15.142:6819/596805607","heartbeat_back_addr":"172.21.15.142:6823/596805607","heartbeat_front_addr":"172.21.15.142:6821/596805607","state":["exists","up"]},{"osd":8,"uuid":"b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6817","nonce":2650652376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6819","nonce":2650652376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6823","nonce":2650652376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6821","nonce":2650652376}]},"public_addr":"172.21.15.169:6817/2650652376","cluster_addr":"172.21.15.169:6819/2650652376","heartbeat_back_addr":"172.21.15.169:6823/2650652376","heartbeat_front_addr":"172.21.15.169:6821/2650652376","state":["exists","up"]},{"osd":9,"uuid":"32df0dab-5875-4beb-b733-60a915c94a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6827","nonce":374861248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6829","nonce":374861248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6833","nonce":374861248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6831","nonce":374861248}]},"public_addr":"172.21.15.46:6827/374861248","cluster_addr":"172.21.15.46:6829/374861248","heartbeat_back_addr":"172.21.15.46:6833/374861248","heartbeat_front_addr":"172.21.15.46:6831/374861248","state":["exists","up"]},{"osd":10,"uuid":"380a4f47-e528-4258-8836-32eac0e6e630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6825","nonce":1364891982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6827","nonce":1364891982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6831","nonce":1364891982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6829","nonce":1364891982}]},"public_addr":"172.21.15.142:6825/1364891982","cluster_addr":"172.21.15.142:6827/1364891982","heartbeat_back_addr":"172.21.15.142:6831/1364891982","heartbeat_front_addr":"172.21.15.142:6829/1364891982","state":["exists","up"]},{"osd":11,"uuid":"56b04b15-6fcd-497a-aa66-19da9624b656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6825","nonce":4262478976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6827","nonce":4262478976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6831","nonce":4262478976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6829","nonce":4262478976}]},"public_addr":"172.21.15.169:6825/4262478976","cluster_addr":"172.21.15.169:6827/4262478976","heartbeat_back_addr":"172.21.15.169:6831/4262478976","heartbeat_front_addr":"172.21.15.169:6829/4262478976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:02:42.159298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:06.053328+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:29.587137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:54.573782+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:20.276073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:45.320565+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:10.985522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:36.806137+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:02.266736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:29.788773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:55.827639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:07:21.762382+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/1072138393":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2786891364":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/7723180":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/626888490":"2024-05-10T00:01:09.191239+0000","172.21.15.46:6800/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/4236543921":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2168313126":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6800/953818590":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6801/953818590":"2024-05-10T00:00:41.833323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:10:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.252+0000 7fc4ff311700 1 -- 172.21.15.46:0/3759464216 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc4e40b2880 msgr2=0x7fc4e40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.252+0000 7fc4ff311700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc4e40b2880 0x7fc4e40b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8106860 tx=0x7fc4e80073d0 comp rx=0 tx=0).stop 2024-05-09T00:10:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.252+0000 7fc4ff311700 1 -- 172.21.15.46:0/3759464216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4f81034c0 msgr2=0x7fc4f819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.252+0000 7fc4ff311700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4f81034c0 0x7fc4f819bc30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40098e0 tx=0x7fc4f40057c0 comp rx=0 tx=0).stop 2024-05-09T00:10:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.252+0000 7fc4ff311700 1 -- 172.21.15.46:0/3759464216 shutdown_connections 2024-05-09T00:10:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.252+0000 7fc4ff311700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc4e40b2880 0x7fc4e40b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.252+0000 7fc4ff311700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4f8108740 0x7fc4f819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.252+0000 7fc4ff311700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4f81034c0 0x7fc4f819bc30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.252+0000 7fc4ff311700 1 --2- 172.21.15.46:0/3759464216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f8107df0 0x7fc4f819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.252+0000 7fc4ff311700 1 -- 172.21.15.46:0/3759464216 >> 172.21.15.46:0/3759464216 conn(0x7fc4f80fedf0 msgr2=0x7fc4f81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.252+0000 7fc4ff311700 1 -- 172.21.15.46:0/3759464216 shutdown_connections 2024-05-09T00:10:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:55.252+0000 7fc4ff311700 1 -- 172.21.15.46:0/3759464216 wait complete. 2024-05-09T00:10:55.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:55 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3759464216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:10:55.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:55 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3759464216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:10:55.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:55 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3759464216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:10:55.896 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:10:56.149 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:10:56.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:56 smithi046 ceph-mon[30151]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:56.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:56 smithi142 ceph-mon[34533]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:56.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:56 smithi169 ceph-mon[34777]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:10:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.823+0000 7fd45e3c1700 1 -- 172.21.15.46:0/3058474821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4581054f0 msgr2=0x7fd458105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.823+0000 7fd45e3c1700 1 --2- 172.21.15.46:0/3058474821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4581054f0 0x7fd458105900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd448009990 tx=0x7fd448009ca0 comp rx=0 tx=0).stop 2024-05-09T00:10:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.823+0000 7fd45e3c1700 1 -- 172.21.15.46:0/3058474821 shutdown_connections 2024-05-09T00:10:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.823+0000 7fd45e3c1700 1 --2- 172.21.15.46:0/3058474821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd45810e920 0x7fd458110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.823+0000 7fd45e3c1700 1 --2- 172.21.15.46:0/3058474821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4581054f0 0x7fd458105900 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.823+0000 7fd45e3c1700 1 --2- 172.21.15.46:0/3058474821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd458105e40 0x7fd45810e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.823+0000 7fd45e3c1700 1 -- 172.21.15.46:0/3058474821 >> 172.21.15.46:0/3058474821 conn(0x7fd458078e00 msgr2=0x7fd458079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.823+0000 7fd45e3c1700 1 -- 172.21.15.46:0/3058474821 shutdown_connections 2024-05-09T00:10:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.823+0000 7fd45e3c1700 1 -- 172.21.15.46:0/3058474821 wait complete. 2024-05-09T00:10:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd45e3c1700 1 Processor -- start 2024-05-09T00:10:57.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd45e3c1700 1 -- start start 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd45e3c1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4581054f0 0x7fd45819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd45e3c1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd458105e40 0x7fd45819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd45e3c1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd45810e920 0x7fd45819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd45e3c1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd458114470 con 0x7fd4581054f0 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd45e3c1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4581142f0 con 0x7fd458105e40 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd45e3c1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4581145f0 con 0x7fd45810e920 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd457fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4581054f0 0x7fd45819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd457fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4581054f0 0x7fd45819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58878/0 (socket says 172.21.15.46:58878) 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd457fff700 1 -- 172.21.15.46:0/2653093715 learned_addr learned my addr 172.21.15.46:0/2653093715 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd44ffff700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd458105e40 0x7fd45819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.824+0000 7fd45c95e700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd45810e920 0x7fd45819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.825+0000 7fd457fff700 1 -- 172.21.15.46:0/2653093715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd45810e920 msgr2=0x7fd45819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.825+0000 7fd457fff700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd45810e920 0x7fd45819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.825+0000 7fd457fff700 1 -- 172.21.15.46:0/2653093715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd458105e40 msgr2=0x7fd45819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:57.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.825+0000 7fd457fff700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd458105e40 0x7fd45819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.825+0000 7fd457fff700 1 -- 172.21.15.46:0/2653093715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45819fc40 con 0x7fd4581054f0 2024-05-09T00:10:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.825+0000 7fd457fff700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4581054f0 0x7fd45819ae20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd448000c00 tx=0x7fd4480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.825+0000 7fd455ffb700 1 -- 172.21.15.46:0/2653093715 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448027070 con 0x7fd4581054f0 2024-05-09T00:10:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.825+0000 7fd455ffb700 1 -- 172.21.15.46:0/2653093715 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd448003fa0 con 0x7fd4581054f0 2024-05-09T00:10:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.825+0000 7fd455ffb700 1 -- 172.21.15.46:0/2653093715 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448022db0 con 0x7fd4581054f0 2024-05-09T00:10:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.825+0000 7fd45e3c1700 1 -- 172.21.15.46:0/2653093715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45819fed0 con 0x7fd4581054f0 2024-05-09T00:10:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.825+0000 7fd45e3c1700 1 -- 172.21.15.46:0/2653093715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4581a0360 con 0x7fd4581054f0 2024-05-09T00:10:57.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.826+0000 7fd45e3c1700 1 -- 172.21.15.46:0/2653093715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4580512c0 con 0x7fd4581054f0 2024-05-09T00:10:57.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.829+0000 7fd455ffb700 1 -- 172.21.15.46:0/2653093715 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd44802a070 con 0x7fd4581054f0 2024-05-09T00:10:57.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.830+0000 7fd455ffb700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4440b2880 0x7fd4440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:10:57.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.830+0000 7fd44ffff700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4440b2880 0x7fd4440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:10:57.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.830+0000 7fd455ffb700 1 -- 172.21.15.46:0/2653093715 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd4480e3440 con 0x7fd4581054f0 2024-05-09T00:10:57.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.830+0000 7fd44ffff700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4440b2880 0x7fd4440b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd440005950 tx=0x7fd4400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:10:57.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:57.835+0000 7fd455ffb700 1 -- 172.21.15.46:0/2653093715 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd448031080 con 0x7fd4581054f0 2024-05-09T00:10:58.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.100+0000 7fd45e3c1700 1 -- 172.21.15.46:0/2653093715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd45802e5f0 con 0x7fd4581054f0 2024-05-09T00:10:58.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.101+0000 7fd455ffb700 1 -- 172.21.15.46:0/2653093715 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11879 (secure 0 0 0) 0x7fd448013070 con 0x7fd4581054f0 2024-05-09T00:10:58.098 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:10:58.098 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:08:47.423159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24497,24499,24487]},"id":1}]} 2024-05-09T00:10:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.103+0000 7fd45e3c1700 1 -- 172.21.15.46:0/2653093715 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4440b2880 msgr2=0x7fd4440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.103+0000 7fd45e3c1700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4440b2880 0x7fd4440b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd440005950 tx=0x7fd4400058e0 comp rx=0 tx=0).stop 2024-05-09T00:10:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.103+0000 7fd45e3c1700 1 -- 172.21.15.46:0/2653093715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4581054f0 msgr2=0x7fd45819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:10:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.103+0000 7fd45e3c1700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4581054f0 0x7fd45819ae20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd448000c00 tx=0x7fd4480047b0 comp rx=0 tx=0).stop 2024-05-09T00:10:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.103+0000 7fd45e3c1700 1 -- 172.21.15.46:0/2653093715 shutdown_connections 2024-05-09T00:10:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.103+0000 7fd45e3c1700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4440b2880 0x7fd4440b4d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.103+0000 7fd45e3c1700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd45810e920 0x7fd45819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.103+0000 7fd45e3c1700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4581054f0 0x7fd45819ae20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.103+0000 7fd45e3c1700 1 --2- 172.21.15.46:0/2653093715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd458105e40 0x7fd45819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:10:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.103+0000 7fd45e3c1700 1 -- 172.21.15.46:0/2653093715 >> 172.21.15.46:0/2653093715 conn(0x7fd458078e00 msgr2=0x7fd458100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:10:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.104+0000 7fd45e3c1700 1 -- 172.21.15.46:0/2653093715 shutdown_connections 2024-05-09T00:10:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:10:58.104+0000 7fd45e3c1700 1 -- 172.21.15.46:0/2653093715 wait complete. 2024-05-09T00:10:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T00:10:58.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:58 smithi142 ceph-mon[34533]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:58.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:10:58 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2653093715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:58.758 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:58 smithi046 ceph-mon[30151]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:58.758 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:10:58 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2653093715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:58.793 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T00:08:28.009815+0000', 'modified': '2024-05-09T00:08:47.423159+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24499, 'mds_1': 24497, 'mds_2': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/2701851507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2701851507}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 2701851507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6837/3779152362', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 3779152362}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 3779152362}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6835/1842121683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 1842121683}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 1842121683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.169:6837/1024289476', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1024289476}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1024289476}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.142:6839/3645430666', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3645430666}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3645430666}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.169:6839/1664161265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 1664161265}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 1664161265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24499, 'qdb_cluster': [24497, 24499, 24487]} 2024-05-09T00:10:58.793 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-09T00:10:58.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T00:10:58.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:58 smithi169 ceph-mon[34777]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:10:58.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:10:58 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2653093715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:10:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:00.460 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:00 smithi046 ceph-mon[30151]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:00.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:00 smithi142 ceph-mon[34533]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.697+0000 7f9f99f27700 1 -- 172.21.15.46:0/3954693973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f94076e40 msgr2=0x7f9f94077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.697+0000 7f9f99f27700 1 --2- 172.21.15.46:0/3954693973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f94076e40 0x7f9f94077250 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009990 tx=0x7f9f84009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.697+0000 7f9f99f27700 1 -- 172.21.15.46:0/3954693973 shutdown_connections 2024-05-09T00:11:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.697+0000 7f9f99f27700 1 --2- 172.21.15.46:0/3954693973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f94110b80 0x7f9f94112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.697+0000 7f9f99f27700 1 --2- 172.21.15.46:0/3954693973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f94076e40 0x7f9f94077250 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.697+0000 7f9f99f27700 1 --2- 172.21.15.46:0/3954693973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f94077790 0x7f9f94110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.697+0000 7f9f99f27700 1 -- 172.21.15.46:0/3954693973 >> 172.21.15.46:0/3954693973 conn(0x7f9f940ffea0 msgr2=0x7f9f941022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.697+0000 7f9f99f27700 1 -- 172.21.15.46:0/3954693973 shutdown_connections 2024-05-09T00:11:00.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.697+0000 7f9f99f27700 1 -- 172.21.15.46:0/3954693973 wait complete. 2024-05-09T00:11:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f99f27700 1 Processor -- start 2024-05-09T00:11:00.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f99f27700 1 -- start start 2024-05-09T00:11:00.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f99f27700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f94076e40 0x7f9f9419d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f99f27700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f94077790 0x7f9f9419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f99f27700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f94110b80 0x7f9f941a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f99f27700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f941166f0 con 0x7f9f94077790 2024-05-09T00:11:00.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f99f27700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f94116570 con 0x7f9f94076e40 2024-05-09T00:11:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f99f27700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f94116870 con 0x7f9f94110b80 2024-05-09T00:11:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f92ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f94077790 0x7f9f9419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f92ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f94077790 0x7f9f9419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58900/0 (socket says 172.21.15.46:58900) 2024-05-09T00:11:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f92ffd700 1 -- 172.21.15.46:0/2448583180 learned_addr learned my addr 172.21.15.46:0/2448583180 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f937fe700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f94076e40 0x7f9f9419d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.698+0000 7f9f93fff700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f94110b80 0x7f9f941a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f92ffd700 1 -- 172.21.15.46:0/2448583180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f94110b80 msgr2=0x7f9f941a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f92ffd700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f94110b80 0x7f9f941a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f92ffd700 1 -- 172.21.15.46:0/2448583180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f94076e40 msgr2=0x7f9f9419d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f92ffd700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f94076e40 0x7f9f9419d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f92ffd700 1 -- 172.21.15.46:0/2448583180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f941a1f50 con 0x7f9f94077790 2024-05-09T00:11:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f93fff700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f94110b80 0x7f9f941a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f937fe700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f94076e40 0x7f9f9419d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:00.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f92ffd700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f94077790 0x7f9f9419d620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800b530 tx=0x7f9f8800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:00.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f90ff9700 1 -- 172.21.15.46:0/2448583180 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88015020 con 0x7f9f94077790 2024-05-09T00:11:00.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f99f27700 1 -- 172.21.15.46:0/2448583180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f941a2240 con 0x7f9f94077790 2024-05-09T00:11:00.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f90ff9700 1 -- 172.21.15.46:0/2448583180 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f88011ce0 con 0x7f9f94077790 2024-05-09T00:11:00.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f90ff9700 1 -- 172.21.15.46:0/2448583180 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88010e40 con 0x7f9f94077790 2024-05-09T00:11:00.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.699+0000 7f9f99f27700 1 -- 172.21.15.46:0/2448583180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f941a9b60 con 0x7f9f94077790 2024-05-09T00:11:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.702+0000 7f9f90ff9700 1 -- 172.21.15.46:0/2448583180 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9f88018020 con 0x7f9f94077790 2024-05-09T00:11:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.703+0000 7f9f90ff9700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9f7c0b2880 0x7f9f7c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.703+0000 7f9f937fe700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9f7c0b2880 0x7f9f7c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.703+0000 7f9f90ff9700 1 -- 172.21.15.46:0/2448583180 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9f880e1b00 con 0x7f9f94077790 2024-05-09T00:11:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.703+0000 7f9f937fe700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9f7c0b2880 0x7f9f7c0b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400b5c0 tx=0x7f9f8401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9fa893db80 1 Processor -- start 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9fa893db80 1 -- start start 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9fa893db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f959155cb0 0x55f95925e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9fa893db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 0x55f95925e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9fa893db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9592583b0 0x55f9592587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9fa893db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f95917b050 con 0x55f9590db450 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9fa893db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f959120950 con 0x55f9592583b0 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9fa893db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f959177010 con 0x55f959155cb0 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9f92ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 0x55f95925e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9f92ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 0x55f95925e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58914/0 (socket says 172.21.15.46:58914) 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9f937fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f959155cb0 0x55f95925e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9f92ffd700 1 -- 172.21.15.46:0/834277545 learned_addr learned my addr 172.21.15.46:0/834277545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9f93fff700 1 --2- 172.21.15.46:0/834277545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9592583b0 0x55f9592587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9f92ffd700 1 -- 172.21.15.46:0/834277545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f959155cb0 msgr2=0x55f95925e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9f92ffd700 1 --2- 172.21.15.46:0/834277545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f959155cb0 0x55f95925e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9f92ffd700 1 -- 172.21.15.46:0/834277545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9592583b0 msgr2=0x55f9592587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9f92ffd700 1 --2- 172.21.15.46:0/834277545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9592583b0 0x55f9592587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9f92ffd700 1 -- 172.21.15.46:0/834277545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9591114b0 con 0x55f9590db450 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.704+0000 7f9f93fff700 1 --2- 172.21.15.46:0/834277545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9592583b0 0x55f9592587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.705+0000 7f9f92ffd700 1 --2- 172.21.15.46:0/834277545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 0x55f95925e730 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9f880a8de0 tx=0x7f9f880a8920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac21b07d7f0ffa88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.705+0000 7f9f79ffb700 1 -- 172.21.15.46:0/834277545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f880a97e0 con 0x55f9590db450 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.705+0000 7f9f79ffb700 1 -- 172.21.15.46:0/834277545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f880a90e0 con 0x55f9590db450 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.705+0000 7f9f79ffb700 1 -- 172.21.15.46:0/834277545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f880b1d10 con 0x55f9590db450 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.706+0000 7f9fa893db80 1 -- 172.21.15.46:0/834277545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 msgr2=0x55f95925e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.706+0000 7f9fa893db80 1 --2- 172.21.15.46:0/834277545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 0x55f95925e730 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9f880a8de0 tx=0x7f9f880a8920 comp rx=0 tx=0).stop 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.706+0000 7f9fa893db80 1 -- 172.21.15.46:0/834277545 shutdown_connections 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.706+0000 7f9fa893db80 1 --2- 172.21.15.46:0/834277545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9592583b0 0x55f9592587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.706+0000 7f9fa893db80 1 --2- 172.21.15.46:0/834277545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f959155cb0 0x55f95925e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.706+0000 7f9fa893db80 1 --2- 172.21.15.46:0/834277545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 0x55f95925e730 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.706+0000 7f9fa893db80 1 -- 172.21.15.46:0/834277545 >> 172.21.15.46:0/834277545 conn(0x55f9590def20 msgr2=0x55f9592531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.706+0000 7f9fa893db80 1 -- 172.21.15.46:0/834277545 shutdown_connections 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.706+0000 7f9fa893db80 1 -- 172.21.15.46:0/834277545 wait complete. 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.706+0000 7f9fa893db80 1 -- start start 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.706+0000 7f9fa893db80 1 Processor -- start 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9fa893db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9590db450 0x55f959261f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9fa893db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f959155cb0 0x55f9592624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9fa893db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9592583b0 0x55f959262a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9fa893db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f9592590b0 con 0x55f9592583b0 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9fa893db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f959115ad0 con 0x55f9590db450 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9fa893db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f95926e1b0 con 0x55f959155cb0 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9fa893db80 10 client.0 _ll_register_callbacks cb 0x55f958fec670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9592583b0 0x55f959262a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9592583b0 0x55f959262a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58924/0 (socket says 172.21.15.46:58924) 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f93fff700 1 -- 172.21.15.46:0/1457273155 learned_addr learned my addr 172.21.15.46:0/1457273155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f92ffd700 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f959155cb0 0x55f9592624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f937fe700 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9590db450 0x55f959261f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f93fff700 1 -- 172.21.15.46:0/1457273155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f959155cb0 msgr2=0x55f9592624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f93fff700 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f959155cb0 0x55f9592624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f93fff700 1 -- 172.21.15.46:0/1457273155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9590db450 msgr2=0x55f959261f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f93fff700 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9590db450 0x55f959261f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f93fff700 1 -- 172.21.15.46:0/1457273155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f958ff6440 con 0x55f9592583b0 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f937fe700 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9590db450 0x55f959261f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f92ffd700 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f959155cb0 0x55f9592624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f93fff700 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9592583b0 0x55f959262a00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c007b70 tx=0x7f9f8c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedbeb1334dfbf1a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.707+0000 7f9f72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9f72ffd700 1 -- 172.21.15.46:0/1457273155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c019020 con 0x55f9592583b0 2024-05-09T00:11:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9f72ffd700 1 -- 172.21.15.46:0/1457273155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9f8c0108c0 con 0x55f9592583b0 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9fa893db80 1 -- 172.21.15.46:0/1457273155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f880a90e0 con 0x55f9592583b0 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9f72ffd700 1 -- 172.21.15.46:0/1457273155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f8c010b00 con 0x55f9592583b0 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9fa893db80 10 client.14961 fetch_fsmap learned FSMap version 25 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9fa893db80 1 -- 172.21.15.46:0/1457273155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f95926fa70 con 0x55f9592583b0 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9f72ffd700 1 -- 172.21.15.46:0/1457273155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f880a90e0 con 0x55f9592583b0 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9f72ffd700 1 -- 172.21.15.46:0/1457273155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9f8c015400 con 0x55f9592583b0 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9fa893db80 10 client.14961 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9fa893db80 10 client.14961 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9fa893db80 20 client.14961 populate_metadata read hostname 'smithi046' 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9fa893db80 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f9592720d0 0x55f959273540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9fa893db80 4 client.14961 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.708+0000 7f9fa893db80 1 -- 172.21.15.46:0/1457273155 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f959188a30 con 0x55f9592720d0 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.709+0000 7f9f937fe700 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f9592720d0 0x55f959273540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.709+0000 7f9f937fe700 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f9592720d0 0x55f959273540 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bacb76ac9c8da1ce server_cookie=4b28ed542b47e9f9 in_seq=0 out_seq=0 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.709+0000 7f9f72ffd700 10 client.14961 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9f72ffd700 1 -- 172.21.15.46:0/1457273155 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f959188a30 con 0x55f9592720d0 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9f72ffd700 10 client.14961 handle_command_reply: tid=0 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9fa893db80 1 client.14961 shutdown 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9fa893db80 1 -- 172.21.15.46:0/1457273155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9592583b0 msgr2=0x55f959262a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9fa893db80 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9592583b0 0x55f959262a00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c007b70 tx=0x7f9f8c007f30 comp rx=0 tx=0).stop 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9fa893db80 1 -- 172.21.15.46:0/1457273155 shutdown_connections 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9fa893db80 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f9592720d0 0x55f959273540 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9fa893db80 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9592583b0 0x55f959262a00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9fa893db80 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9590db450 0x55f959261f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9fa893db80 1 --2- 172.21.15.46:0/1457273155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f959155cb0 0x55f9592624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9fa893db80 1 -- 172.21.15.46:0/1457273155 >> 172.21.15.46:0/1457273155 conn(0x55f9590def20 msgr2=0x55f95924cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9f72ffd700 0 client.14961 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:11:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9fa893db80 1 -- 172.21.15.46:0/1457273155 shutdown_connections 2024-05-09T00:11:00.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9fa893db80 1 -- 172.21.15.46:0/1457273155 wait complete. 2024-05-09T00:11:00.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.710+0000 7f9fa893db80 20 client.14961 trim_cache size 0 max 16384 2024-05-09T00:11:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.731+0000 7f9fa893db80 1 Processor -- start 2024-05-09T00:11:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.731+0000 7f9fa893db80 1 -- start start 2024-05-09T00:11:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9fa893db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 0x55f959262840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9fa893db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f959155cb0 0x55f95926e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9fa893db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9592583b0 0x55f959260fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9fa893db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f959271880 con 0x55f9590db450 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9fa893db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f959271f50 con 0x55f959155cb0 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9fa893db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f959272620 con 0x55f9592583b0 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9f937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 0x55f959262840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9f937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 0x55f959262840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58938/0 (socket says 172.21.15.46:58938) 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9f937fe700 1 -- 172.21.15.46:0/39281319 learned_addr learned my addr 172.21.15.46:0/39281319 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9f93fff700 1 --2- 172.21.15.46:0/39281319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9592583b0 0x55f959260fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9f937fe700 1 -- 172.21.15.46:0/39281319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9592583b0 msgr2=0x55f959260fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9f937fe700 1 --2- 172.21.15.46:0/39281319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9592583b0 0x55f959260fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9f937fe700 1 -- 172.21.15.46:0/39281319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f959155cb0 msgr2=0x55f95926e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9f92ffd700 1 --2- 172.21.15.46:0/39281319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f959155cb0 0x55f95926e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9f937fe700 1 --2- 172.21.15.46:0/39281319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f959155cb0 0x55f95926e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.732+0000 7f9f937fe700 1 -- 172.21.15.46:0/39281319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f959264820 con 0x55f9590db450 2024-05-09T00:11:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.733+0000 7f9f937fe700 1 --2- 172.21.15.46:0/39281319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 0x55f959262840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f84000c00 tx=0x7f9f8401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff13f70773321673 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.733+0000 7f9f797fa700 1 -- 172.21.15.46:0/39281319 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84038070 con 0x55f9590db450 2024-05-09T00:11:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.733+0000 7f9f797fa700 1 -- 172.21.15.46:0/39281319 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f840055c0 con 0x55f9590db450 2024-05-09T00:11:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.733+0000 7f9f797fa700 1 -- 172.21.15.46:0/39281319 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8400fc90 con 0x55f9590db450 2024-05-09T00:11:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.734+0000 7f9fa893db80 1 -- 172.21.15.46:0/39281319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 msgr2=0x55f959262840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.734+0000 7f9fa893db80 1 --2- 172.21.15.46:0/39281319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 0x55f959262840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f84000c00 tx=0x7f9f8401f000 comp rx=0 tx=0).stop 2024-05-09T00:11:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.734+0000 7f9fa893db80 1 -- 172.21.15.46:0/39281319 shutdown_connections 2024-05-09T00:11:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.734+0000 7f9fa893db80 1 --2- 172.21.15.46:0/39281319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9592583b0 0x55f959260fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.734+0000 7f9fa893db80 1 --2- 172.21.15.46:0/39281319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9590db450 0x55f959262840 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.734+0000 7f9fa893db80 1 --2- 172.21.15.46:0/39281319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f959155cb0 0x55f95926e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.734+0000 7f9fa893db80 1 -- 172.21.15.46:0/39281319 >> 172.21.15.46:0/39281319 conn(0x55f9590def20 msgr2=0x55f959260890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.734+0000 7f9fa893db80 1 -- 172.21.15.46:0/39281319 shutdown_connections 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.734+0000 7f9fa893db80 1 -- 172.21.15.46:0/39281319 wait complete. 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9fa893db80 1 -- start start 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9fa893db80 1 Processor -- start 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9fa893db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9590db450 0x55f95927c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9fa893db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f959155cb0 0x55f95927cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9fa893db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9592583b0 0x55f959277970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9fa893db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f9592733c0 con 0x55f959155cb0 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9fa893db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f959272cf0 con 0x55f9590db450 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9fa893db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f959273a90 con 0x55f9592583b0 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9fa893db80 10 client.0 _ll_register_callbacks cb 0x55f958fec670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9f92ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f959155cb0 0x55f95927cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9f92ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f959155cb0 0x55f95927cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58940/0 (socket says 172.21.15.46:58940) 2024-05-09T00:11:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9f92ffd700 1 -- 172.21.15.46:0/2026441936 learned_addr learned my addr 172.21.15.46:0/2026441936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.735+0000 7f9f937fe700 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9590db450 0x55f95927c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.736+0000 7f9f93fff700 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9592583b0 0x55f959277970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.736+0000 7f9f937fe700 1 -- 172.21.15.46:0/2026441936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9592583b0 msgr2=0x55f959277970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.736+0000 7f9f937fe700 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9592583b0 0x55f959277970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.736+0000 7f9f937fe700 1 -- 172.21.15.46:0/2026441936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f959155cb0 msgr2=0x55f95927cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.736+0000 7f9f937fe700 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f959155cb0 0x55f95927cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.736+0000 7f9f937fe700 1 -- 172.21.15.46:0/2026441936 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f959277f30 con 0x55f9590db450 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.736+0000 7f9f92ffd700 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f959155cb0 0x55f95927cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.736+0000 7f9f937fe700 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9590db450 0x55f95927c9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55f959133200 tx=0x7f9f84009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76221b1c19bceb83 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.736+0000 7f9f72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.736+0000 7f9f72ffd700 1 -- 172.21.15.46:0/2026441936 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84038040 con 0x55f9590db450 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.736+0000 7f9fa893db80 1 -- 172.21.15.46:0/2026441936 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f840055c0 con 0x55f9590db450 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.736+0000 7f9f72ffd700 1 -- 172.21.15.46:0/2026441936 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9f84005310 con 0x55f9590db450 2024-05-09T00:11:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.737+0000 7f9f72ffd700 1 -- 172.21.15.46:0/2026441936 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f84003900 con 0x55f9590db450 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.737+0000 7f9fa893db80 10 client.24767 fetch_fsmap learned FSMap version 25 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.737+0000 7f9fa893db80 1 -- 172.21.15.46:0/2026441936 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9592ddd40 con 0x55f9590db450 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.737+0000 7f9f72ffd700 1 -- 172.21.15.46:0/2026441936 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f840055c0 con 0x55f9590db450 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.737+0000 7f9f72ffd700 1 -- 172.21.15.46:0/2026441936 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9f84032430 con 0x55f9590db450 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.737+0000 7f9fa893db80 10 client.24767 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.737+0000 7f9fa893db80 10 client.24767 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.737+0000 7f9fa893db80 20 client.24767 populate_metadata read hostname 'smithi046' 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.737+0000 7f9fa893db80 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f9592dfcd0 0x55f9592e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.737+0000 7f9fa893db80 4 client.24767 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.737+0000 7f9fa893db80 1 -- 172.21.15.46:0/2026441936 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "status"}) -- 0x55f959182ad0 con 0x55f9592dfcd0 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.737+0000 7f9f92ffd700 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f9592dfcd0 0x55f9592e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9f72ffd700 10 client.24767 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:11:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9f92ffd700 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f9592dfcd0 0x55f9592e1140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d59c2926a83959b server_cookie=566b81e5fe8d9dba in_seq=0 out_seq=0 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9f72ffd700 1 -- 172.21.15.46:0/2026441936 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f959182ad0 con 0x55f9592dfcd0 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9f72ffd700 10 client.24767 handle_command_reply: tid=0 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9fa893db80 1 client.24767 shutdown 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9fa893db80 1 -- 172.21.15.46:0/2026441936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9590db450 msgr2=0x55f95927c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9fa893db80 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9590db450 0x55f95927c9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55f959133200 tx=0x7f9f84009290 comp rx=0 tx=0).stop 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9fa893db80 1 -- 172.21.15.46:0/2026441936 shutdown_connections 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9fa893db80 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f9592dfcd0 0x55f9592e1140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9fa893db80 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f9592583b0 0x55f959277970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9fa893db80 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9590db450 0x55f95927c9c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9fa893db80 1 --2- 172.21.15.46:0/2026441936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f959155cb0 0x55f95927cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9fa893db80 1 -- 172.21.15.46:0/2026441936 >> 172.21.15.46:0/2026441936 conn(0x55f9590def20 msgr2=0x55f95924cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.738+0000 7f9f72ffd700 0 client.24767 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9fa893db80 1 -- 172.21.15.46:0/2026441936 shutdown_connections 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9fa893db80 1 -- 172.21.15.46:0/2026441936 wait complete. 2024-05-09T00:11:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9fa893db80 20 client.24767 trim_cache size 0 max 16384 2024-05-09T00:11:00.740 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:11:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "eec67860-0d96-11ef-bc97-c7b262605968", 2024-05-09T00:11:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-05-09T00:11:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24487, 2024-05-09T00:11:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-09T00:11:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-09T00:11:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-09T00:11:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 139.37035959100001, 2024-05-09T00:11:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 25, 2024-05-09T00:11:00.741 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-05-09T00:11:00.741 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:11:00.741 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 184.551883258 2024-05-09T00:11:00.741 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:11:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9f99f27700 1 -- 172.21.15.46:0/2448583180 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9f7c0b2880 msgr2=0x7f9f7c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9f99f27700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9f7c0b2880 0x7f9f7c0b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400b5c0 tx=0x7f9f8401a040 comp rx=0 tx=0).stop 2024-05-09T00:11:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9f99f27700 1 -- 172.21.15.46:0/2448583180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f94077790 msgr2=0x7f9f9419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9f99f27700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f94077790 0x7f9f9419d620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800b530 tx=0x7f9f8800b840 comp rx=0 tx=0).stop 2024-05-09T00:11:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9f99f27700 1 -- 172.21.15.46:0/2448583180 shutdown_connections 2024-05-09T00:11:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9f99f27700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9f7c0b2880 0x7f9f7c0b4d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9f99f27700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f94110b80 0x7f9f941a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9f99f27700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f94076e40 0x7f9f9419d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9f99f27700 1 --2- 172.21.15.46:0/2448583180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f94077790 0x7f9f9419d620 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9f99f27700 1 -- 172.21.15.46:0/2448583180 >> 172.21.15.46:0/2448583180 conn(0x7f9f940ffea0 msgr2=0x7f9f941004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9f99f27700 1 -- 172.21.15.46:0/2448583180 shutdown_connections 2024-05-09T00:11:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:00.739+0000 7f9f99f27700 1 -- 172.21.15.46:0/2448583180 wait complete. 2024-05-09T00:11:00.852 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:00 smithi169 ceph-mon[34777]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:01.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T00:11:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:02.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:02 smithi142 ceph-mon[34533]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:02.739 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:02 smithi046 ceph-mon[30151]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:02.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:02 smithi169 ceph-mon[34777]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:03.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.526+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2082407281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5281034c0 msgr2=0x7fc5281038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:03.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.526+0000 7fc52e2b1700 1 --2- 172.21.15.46:0/2082407281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5281034c0 0x7fc5281038d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc5180099c0 tx=0x7fc518009cd0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.526+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2082407281 shutdown_connections 2024-05-09T00:11:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.526+0000 7fc52e2b1700 1 --2- 172.21.15.46:0/2082407281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc528108740 0x7fc5281100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.526+0000 7fc52e2b1700 1 --2- 172.21.15.46:0/2082407281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5281034c0 0x7fc5281038d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.526+0000 7fc52e2b1700 1 --2- 172.21.15.46:0/2082407281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc528107df0 0x7fc528108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.526+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2082407281 >> 172.21.15.46:0/2082407281 conn(0x7fc5280fedc0 msgr2=0x7fc5281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.526+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2082407281 shutdown_connections 2024-05-09T00:11:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.526+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2082407281 wait complete. 2024-05-09T00:11:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc52e2b1700 1 Processor -- start 2024-05-09T00:11:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc52e2b1700 1 -- start start 2024-05-09T00:11:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc52e2b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5281034c0 0x7fc52819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc52e2b1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc528107df0 0x7fc52819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc52e2b1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc528108740 0x7fc52819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc52e2b1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5281133f0 con 0x7fc5281034c0 2024-05-09T00:11:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc52e2b1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc528113270 con 0x7fc528108740 2024-05-09T00:11:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc52e2b1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc528113570 con 0x7fc528107df0 2024-05-09T00:11:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc527fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5281034c0 0x7fc52819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc527fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5281034c0 0x7fc52819ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58952/0 (socket says 172.21.15.46:58952) 2024-05-09T00:11:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc527fff700 1 -- 172.21.15.46:0/2078025784 learned_addr learned my addr 172.21.15.46:0/2078025784 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc52c84e700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc528108740 0x7fc52819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.527+0000 7fc5277fe700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc528107df0 0x7fc52819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.528+0000 7fc527fff700 1 -- 172.21.15.46:0/2078025784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc528107df0 msgr2=0x7fc52819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.528+0000 7fc527fff700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc528107df0 0x7fc52819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.528+0000 7fc527fff700 1 -- 172.21.15.46:0/2078025784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc528108740 msgr2=0x7fc52819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.528+0000 7fc527fff700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc528108740 0x7fc52819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.528+0000 7fc527fff700 1 -- 172.21.15.46:0/2078025784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52819fd20 con 0x7fc5281034c0 2024-05-09T00:11:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.528+0000 7fc527fff700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5281034c0 0x7fc52819ae50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5180098e0 tx=0x7fc5180057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.528+0000 7fc5257fa700 1 -- 172.21.15.46:0/2078025784 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518027070 con 0x7fc5281034c0 2024-05-09T00:11:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.528+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2078025784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52819ffb0 con 0x7fc5281034c0 2024-05-09T00:11:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.528+0000 7fc5257fa700 1 -- 172.21.15.46:0/2078025784 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc518003f20 con 0x7fc5281034c0 2024-05-09T00:11:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.528+0000 7fc5257fa700 1 -- 172.21.15.46:0/2078025784 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518022d70 con 0x7fc5281034c0 2024-05-09T00:11:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.528+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2078025784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5281a7930 con 0x7fc5281034c0 2024-05-09T00:11:03.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.531+0000 7fc5257fa700 1 -- 172.21.15.46:0/2078025784 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc51802a070 con 0x7fc5281034c0 2024-05-09T00:11:03.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.532+0000 7fc53ccc7b80 1 Processor -- start 2024-05-09T00:11:03.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.532+0000 7fc5257fa700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc5100b2880 0x7fc5100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.532+0000 7fc53ccc7b80 1 -- start start 2024-05-09T00:11:03.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.532+0000 7fc52c84e700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc5100b2880 0x7fc5100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.532+0000 7fc5257fa700 1 -- 172.21.15.46:0/2078025784 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc5180e30b0 con 0x7fc5281034c0 2024-05-09T00:11:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.532+0000 7fc53ccc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 0x561f284af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc53ccc7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f2832c450 0x561f284af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc53ccc7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc53ccc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f283cc050 con 0x561f283a6cb0 2024-05-09T00:11:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc53ccc7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561f28371950 con 0x561f2832c450 2024-05-09T00:11:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc53ccc7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561f283c8010 con 0x561f284a93b0 2024-05-09T00:11:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc527fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 0x561f284af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc527fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 0x561f284af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58958/0 (socket says 172.21.15.46:58958) 2024-05-09T00:11:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc527fff700 1 -- 172.21.15.46:0/3830235897 learned_addr learned my addr 172.21.15.46:0/3830235897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc52c84e700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc5100b2880 0x7fc5100b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc520009fb0 tx=0x7fc52000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc52c84e700 1 --2- 172.21.15.46:0/3830235897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc5277fe700 1 --2- 172.21.15.46:0/3830235897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f2832c450 0x561f284af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc527fff700 1 -- 172.21.15.46:0/3830235897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 msgr2=0x561f284a97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc527fff700 1 --2- 172.21.15.46:0/3830235897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284a97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc527fff700 1 -- 172.21.15.46:0/3830235897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f2832c450 msgr2=0x561f284af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc527fff700 1 --2- 172.21.15.46:0/3830235897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f2832c450 0x561f284af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc527fff700 1 -- 172.21.15.46:0/3830235897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f283624b0 con 0x561f283a6cb0 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc52c84e700 1 --2- 172.21.15.46:0/3830235897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284a97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc527fff700 1 --2- 172.21.15.46:0/3830235897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 0x561f284af1f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc5180aa420 tx=0x7fc518022470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1aa86d8be4f1b39 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc50e7fc700 1 -- 172.21.15.46:0/3830235897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5180aead0 con 0x561f283a6cb0 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc50e7fc700 1 -- 172.21.15.46:0/3830235897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5180b34f0 con 0x561f283a6cb0 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.533+0000 7fc50e7fc700 1 -- 172.21.15.46:0/3830235897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5180b32a0 con 0x561f283a6cb0 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.534+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/3830235897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 msgr2=0x561f284af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.534+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/3830235897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 0x561f284af1f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc5180aa420 tx=0x7fc518022470 comp rx=0 tx=0).stop 2024-05-09T00:11:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.534+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/3830235897 shutdown_connections 2024-05-09T00:11:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.534+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/3830235897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284a97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.534+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/3830235897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 0x561f284af1f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.534+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/3830235897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f2832c450 0x561f284af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.534+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/3830235897 >> 172.21.15.46:0/3830235897 conn(0x561f2832ff20 msgr2=0x561f284a41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.534+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/3830235897 shutdown_connections 2024-05-09T00:11:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.534+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/3830235897 wait complete. 2024-05-09T00:11:03.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.535+0000 7fc53ccc7b80 1 -- start start 2024-05-09T00:11:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.535+0000 7fc53ccc7b80 1 Processor -- start 2024-05-09T00:11:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.535+0000 7fc53ccc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 0x561f284b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.535+0000 7fc53ccc7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f283a6cb0 0x561f284b9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.535+0000 7fc53ccc7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f284a93b0 0x561f284b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.535+0000 7fc53ccc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f283cc050 con 0x561f2832c450 2024-05-09T00:11:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.535+0000 7fc53ccc7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561f28366ad0 con 0x561f284a93b0 2024-05-09T00:11:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.535+0000 7fc53ccc7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561f284a9f60 con 0x561f283a6cb0 2024-05-09T00:11:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc53ccc7b80 10 client.0 _ll_register_callbacks cb 0x561f2823d670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc527fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 0x561f284b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc527fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 0x561f284b8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58966/0 (socket says 172.21.15.46:58966) 2024-05-09T00:11:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc527fff700 1 -- 172.21.15.46:0/1602945896 learned_addr learned my addr 172.21.15.46:0/1602945896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc5277fe700 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f283a6cb0 0x561f284b9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc52c84e700 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f284a93b0 0x561f284b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc527fff700 1 -- 172.21.15.46:0/1602945896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f283a6cb0 msgr2=0x561f284b9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc527fff700 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f283a6cb0 0x561f284b9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc527fff700 1 -- 172.21.15.46:0/1602945896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f284a93b0 msgr2=0x561f284b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc527fff700 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f284a93b0 0x561f284b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc527fff700 1 -- 172.21.15.46:0/1602945896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f28247440 con 0x561f2832c450 2024-05-09T00:11:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc52c84e700 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f284a93b0 0x561f284b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc527fff700 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 0x561f284b8b50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc5180aa660 tx=0x7fc518009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11aaa88f6f967bc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc5077fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:11:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc5077fe700 1 -- 172.21.15.46:0/1602945896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5180c3070 con 0x561f2832c450 2024-05-09T00:11:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc5077fe700 1 -- 172.21.15.46:0/1602945896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc5180b3270 con 0x561f2832c450 2024-05-09T00:11:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/1602945896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5180b34f0 con 0x561f2832c450 2024-05-09T00:11:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc5077fe700 1 -- 172.21.15.46:0/1602945896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc5180aead0 con 0x561f2832c450 2024-05-09T00:11:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc53ccc7b80 10 client.14979 fetch_fsmap learned FSMap version 25 2024-05-09T00:11:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.536+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/1602945896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f284c0770 con 0x561f2832c450 2024-05-09T00:11:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.537+0000 7fc5077fe700 1 -- 172.21.15.46:0/1602945896 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5180b34f0 con 0x561f2832c450 2024-05-09T00:11:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.537+0000 7fc5077fe700 1 -- 172.21.15.46:0/1602945896 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc5180bfae0 con 0x561f2832c450 2024-05-09T00:11:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.537+0000 7fc53ccc7b80 10 client.14979 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T00:11:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.537+0000 7fc53ccc7b80 10 client.14979 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T00:11:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.537+0000 7fc53ccc7b80 20 client.14979 populate_metadata read hostname 'smithi046' 2024-05-09T00:11:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.537+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x561f284c29e0 0x561f284c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.537+0000 7fc53ccc7b80 4 client.14979 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.537+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/1602945896 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f283d9a30 con 0x561f284c29e0 2024-05-09T00:11:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.537+0000 7fc5277fe700 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x561f284c29e0 0x561f284c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.538+0000 7fc5077fe700 10 client.14979 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:11:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.538+0000 7fc5277fe700 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x561f284c29e0 0x561f284c3e50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f889c4e657ab25f server_cookie=b59a9e651dc23faa in_seq=0 out_seq=0 2024-05-09T00:11:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc5077fe700 1 -- 172.21.15.46:0/1602945896 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561f283d9a30 con 0x561f284c29e0 2024-05-09T00:11:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc5077fe700 10 client.14979 handle_command_reply: tid=0 2024-05-09T00:11:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc53ccc7b80 1 client.14979 shutdown 2024-05-09T00:11:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/1602945896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 msgr2=0x561f284b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 0x561f284b8b50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc5180aa660 tx=0x7fc518009cd0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/1602945896 shutdown_connections 2024-05-09T00:11:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x561f284c29e0 0x561f284c3e50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f284a93b0 0x561f284b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 0x561f284b8b50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc5077fe700 0 client.14979 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:11:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/1602945896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f283a6cb0 0x561f284b9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/1602945896 >> 172.21.15.46:0/1602945896 conn(0x561f2832ff20 msgr2=0x561f2849dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/1602945896 shutdown_connections 2024-05-09T00:11:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/1602945896 wait complete. 2024-05-09T00:11:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.539+0000 7fc53ccc7b80 20 client.14979 trim_cache size 0 max 16384 2024-05-09T00:11:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc53ccc7b80 1 Processor -- start 2024-05-09T00:11:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc53ccc7b80 1 -- start start 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc53ccc7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f2832c450 0x561f284b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc53ccc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 0x561f284b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc53ccc7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc53ccc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f284c2190 con 0x561f283a6cb0 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc53ccc7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561f284c2860 con 0x561f2832c450 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc53ccc7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561f284c2f30 con 0x561f284a93b0 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc5277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 0x561f284b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc5277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 0x561f284b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58972/0 (socket says 172.21.15.46:58972) 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc5277fe700 1 -- 172.21.15.46:0/7996493 learned_addr learned my addr 172.21.15.46:0/7996493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc52c84e700 1 --2- 172.21.15.46:0/7996493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc5277fe700 1 -- 172.21.15.46:0/7996493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 msgr2=0x561f284ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc527fff700 1 --2- 172.21.15.46:0/7996493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f2832c450 0x561f284b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc5277fe700 1 --2- 172.21.15.46:0/7996493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc5277fe700 1 -- 172.21.15.46:0/7996493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f2832c450 msgr2=0x561f284b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc5277fe700 1 --2- 172.21.15.46:0/7996493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f2832c450 0x561f284b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.561+0000 7fc5277fe700 1 -- 172.21.15.46:0/7996493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f284c1fc0 con 0x561f283a6cb0 2024-05-09T00:11:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.562+0000 7fc5277fe700 1 --2- 172.21.15.46:0/7996493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 0x561f284b2f10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc51c005950 tx=0x7fc51c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123d06da8028af9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.562+0000 7fc50dffb700 1 -- 172.21.15.46:0/7996493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c021030 con 0x561f283a6cb0 2024-05-09T00:11:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.562+0000 7fc50dffb700 1 -- 172.21.15.46:0/7996493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc51c0044d0 con 0x561f283a6cb0 2024-05-09T00:11:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.562+0000 7fc50dffb700 1 -- 172.21.15.46:0/7996493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c003680 con 0x561f283a6cb0 2024-05-09T00:11:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.562+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/7996493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 msgr2=0x561f284b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.563+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/7996493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 0x561f284b2f10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc51c005950 tx=0x7fc51c011720 comp rx=0 tx=0).stop 2024-05-09T00:11:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.563+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/7996493 shutdown_connections 2024-05-09T00:11:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.563+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/7996493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.563+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/7996493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f2832c450 0x561f284b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.563+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/7996493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f283a6cb0 0x561f284b2f10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.563+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/7996493 >> 172.21.15.46:0/7996493 conn(0x561f2832ff20 msgr2=0x561f284b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.563+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/7996493 shutdown_connections 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.563+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/7996493 wait complete. 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.563+0000 7fc53ccc7b80 1 -- start start 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.563+0000 7fc53ccc7b80 1 Processor -- start 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc53ccc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 0x561f28526140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc53ccc7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f283a6cb0 0x561f28526680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc53ccc7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284bb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc53ccc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f284c3cd0 con 0x561f2832c450 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc53ccc7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561f284c3600 con 0x561f283a6cb0 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc53ccc7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561f284c43a0 con 0x561f284a93b0 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc53ccc7b80 10 client.0 _ll_register_callbacks cb 0x561f2823d670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc527fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 0x561f28526140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc527fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 0x561f28526140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58988/0 (socket says 172.21.15.46:58988) 2024-05-09T00:11:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc527fff700 1 -- 172.21.15.46:0/2203005605 learned_addr learned my addr 172.21.15.46:0/2203005605 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc5277fe700 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f283a6cb0 0x561f28526680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc52c84e700 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284bb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc527fff700 1 -- 172.21.15.46:0/2203005605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 msgr2=0x561f284bb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc527fff700 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284bb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc527fff700 1 -- 172.21.15.46:0/2203005605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f283a6cb0 msgr2=0x561f28526680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc527fff700 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f283a6cb0 0x561f28526680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc527fff700 1 -- 172.21.15.46:0/2203005605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f284bbae0 con 0x561f2832c450 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc527fff700 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 0x561f28526140 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc5180a67a0 tx=0x7fc5180227e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ba3f406a3713cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.564+0000 7fc5077fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc5077fe700 1 -- 172.21.15.46:0/2203005605 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5180c3070 con 0x561f2832c450 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc5077fe700 1 -- 172.21.15.46:0/2203005605 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc5180a6340 con 0x561f2832c450 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/2203005605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc51c0044d0 con 0x561f2832c450 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc5077fe700 1 -- 172.21.15.46:0/2203005605 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc518084e30 con 0x561f2832c450 2024-05-09T00:11:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc53ccc7b80 10 client.14985 fetch_fsmap learned FSMap version 25 2024-05-09T00:11:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/2203005605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f284bbae0 con 0x561f2832c450 2024-05-09T00:11:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc5077fe700 1 -- 172.21.15.46:0/2203005605 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc51c0044d0 con 0x561f2832c450 2024-05-09T00:11:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc5077fe700 1 -- 172.21.15.46:0/2203005605 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc5180bfae0 con 0x561f2832c450 2024-05-09T00:11:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc53ccc7b80 10 client.14985 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T00:11:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc53ccc7b80 10 client.14985 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T00:11:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc53ccc7b80 20 client.14985 populate_metadata read hostname 'smithi046' 2024-05-09T00:11:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x561f28530990 0x561f28531e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc53ccc7b80 4 client.14985 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:11:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/2203005605 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "status"}) -- 0x561f283d3ad0 con 0x561f28530990 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.565+0000 7fc5277fe700 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x561f28530990 0x561f28531e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc5277fe700 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x561f28530990 0x561f28531e00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4c67283adf99e1 server_cookie=78c662176d8c256c in_seq=0 out_seq=0 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc5077fe700 10 client.14985 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc5077fe700 1 -- 172.21.15.46:0/2203005605 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561f283d3ad0 con 0x561f28530990 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc5077fe700 10 client.14985 handle_command_reply: tid=0 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc53ccc7b80 1 client.14985 shutdown 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/2203005605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 msgr2=0x561f28526140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 0x561f28526140 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc5180a67a0 tx=0x7fc5180227e0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/2203005605 shutdown_connections 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x561f28530990 0x561f28531e00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f284a93b0 0x561f284bb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f2832c450 0x561f28526140 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc53ccc7b80 1 --2- 172.21.15.46:0/2203005605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561f283a6cb0 0x561f28526680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc5077fe700 0 client.14985 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:11:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.566+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/2203005605 >> 172.21.15.46:0/2203005605 conn(0x561f2832ff20 msgr2=0x561f284ae040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/2203005605 shutdown_connections 2024-05-09T00:11:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc53ccc7b80 1 -- 172.21.15.46:0/2203005605 wait complete. 2024-05-09T00:11:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc53ccc7b80 20 client.14985 trim_cache size 0 max 16384 2024-05-09T00:11:03.570 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:11:03.570 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "eec67860-0d96-11ef-bc97-c7b262605968", 2024-05-09T00:11:03.570 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-05-09T00:11:03.570 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24497, 2024-05-09T00:11:03.570 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-09T00:11:03.570 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-09T00:11:03.571 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-09T00:11:03.571 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 143.21202891799999, 2024-05-09T00:11:03.571 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 25, 2024-05-09T00:11:03.571 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-05-09T00:11:03.571 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:11:03.571 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 170.61134016099999 2024-05-09T00:11:03.571 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:11:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2078025784 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc5100b2880 msgr2=0x7fc5100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc52e2b1700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc5100b2880 0x7fc5100b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc520009fb0 tx=0x7fc52000d040 comp rx=0 tx=0).stop 2024-05-09T00:11:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2078025784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5281034c0 msgr2=0x7fc52819ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc52e2b1700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5281034c0 0x7fc52819ae50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5180098e0 tx=0x7fc5180057c0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2078025784 shutdown_connections 2024-05-09T00:11:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc52e2b1700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc5100b2880 0x7fc5100b4d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc52e2b1700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc528108740 0x7fc52819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc52e2b1700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5281034c0 0x7fc52819ae50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc52e2b1700 1 --2- 172.21.15.46:0/2078025784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc528107df0 0x7fc52819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.567+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2078025784 >> 172.21.15.46:0/2078025784 conn(0x7fc5280fedc0 msgr2=0x7fc52806b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:03.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.568+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2078025784 shutdown_connections 2024-05-09T00:11:03.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:03.568+0000 7fc52e2b1700 1 -- 172.21.15.46:0/2078025784 wait complete. 2024-05-09T00:11:04.403 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T00:11:04.660 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:04.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:04 smithi142 ceph-mon[34533]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:04.783 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:04 smithi046 ceph-mon[30151]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:04.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:04 smithi169 ceph-mon[34777]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:06.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.368+0000 7fe14cebe700 1 -- 172.21.15.46:0/1783200494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe148108740 msgr2=0x7fe1481100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.368+0000 7fe14cebe700 1 --2- 172.21.15.46:0/1783200494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe148108740 0x7fe1481100a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe1400099c0 tx=0x7fe140009cd0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.369+0000 7fe14cebe700 1 -- 172.21.15.46:0/1783200494 shutdown_connections 2024-05-09T00:11:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.369+0000 7fe14cebe700 1 --2- 172.21.15.46:0/1783200494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe148108740 0x7fe1481100a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.369+0000 7fe14cebe700 1 --2- 172.21.15.46:0/1783200494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1481034c0 0x7fe1481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.369+0000 7fe14cebe700 1 --2- 172.21.15.46:0/1783200494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148107df0 0x7fe148108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.369+0000 7fe14cebe700 1 -- 172.21.15.46:0/1783200494 >> 172.21.15.46:0/1783200494 conn(0x7fe1480fedc0 msgr2=0x7fe1481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.369+0000 7fe14cebe700 1 -- 172.21.15.46:0/1783200494 shutdown_connections 2024-05-09T00:11:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.369+0000 7fe14cebe700 1 -- 172.21.15.46:0/1783200494 wait complete. 2024-05-09T00:11:06.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.369+0000 7fe14cebe700 1 Processor -- start 2024-05-09T00:11:06.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14cebe700 1 -- start start 2024-05-09T00:11:06.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14cebe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1481034c0 0x7fe1480ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14cebe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe148107df0 0x7fe1480ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14cebe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148108740 0x7fe1480ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14cebe700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe148113370 con 0x7fe1481034c0 2024-05-09T00:11:06.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14cebe700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe1481131f0 con 0x7fe148108740 2024-05-09T00:11:06.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14cebe700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe1481134f0 con 0x7fe148107df0 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1481034c0 0x7fe1480ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1481034c0 0x7fe1480ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51640/0 (socket says 172.21.15.46:51640) 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14659c700 1 -- 172.21.15.46:0/587509617 learned_addr learned my addr 172.21.15.46:0/587509617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe145d9b700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe148107df0 0x7fe1480ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe146d9d700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148108740 0x7fe1480ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14659c700 1 -- 172.21.15.46:0/587509617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe148107df0 msgr2=0x7fe1480ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14659c700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe148107df0 0x7fe1480ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14659c700 1 -- 172.21.15.46:0/587509617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148108740 msgr2=0x7fe1480ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14659c700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148108740 0x7fe1480ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe14659c700 1 -- 172.21.15.46:0/587509617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1481a91b0 con 0x7fe1481034c0 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.370+0000 7fe146d9d700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148108740 0x7fe1480ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.371+0000 7fe14659c700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1481034c0 0x7fe1480ff2a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe13800ada0 tx=0x7fe13800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.371+0000 7fe1377fe700 1 -- 172.21.15.46:0/587509617 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138017020 con 0x7fe1481034c0 2024-05-09T00:11:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.371+0000 7fe14cebe700 1 -- 172.21.15.46:0/587509617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1481a94a0 con 0x7fe1481034c0 2024-05-09T00:11:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.371+0000 7fe14cebe700 1 -- 172.21.15.46:0/587509617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1481a9a30 con 0x7fe1481034c0 2024-05-09T00:11:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.371+0000 7fe1377fe700 1 -- 172.21.15.46:0/587509617 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe13800ce30 con 0x7fe1481034c0 2024-05-09T00:11:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.371+0000 7fe1377fe700 1 -- 172.21.15.46:0/587509617 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138005420 con 0x7fe1481034c0 2024-05-09T00:11:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.374+0000 7fe1377fe700 1 -- 172.21.15.46:0/587509617 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe1380055c0 con 0x7fe1481034c0 2024-05-09T00:11:06.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.375+0000 7fe1377fe700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe1300b2880 0x7fe1300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.375+0000 7fe15b8d4b80 1 Processor -- start 2024-05-09T00:11:06.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.375+0000 7fe145d9b700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe1300b2880 0x7fe1300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.375+0000 7fe1377fe700 1 -- 172.21.15.46:0/587509617 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe1380e1a00 con 0x7fe1481034c0 2024-05-09T00:11:06.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.375+0000 7fe15b8d4b80 1 -- start start 2024-05-09T00:11:06.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.375+0000 7fe145d9b700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe1300b2880 0x7fe1300b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe148109970 tx=0x7fe13c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:06.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.375+0000 7fe15b8d4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b681842cb0 0x55b68194b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.375+0000 7fe15b8d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 0x55b68194b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.375+0000 7fe15b8d4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6819453b0 0x55b6819457c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.375+0000 7fe15b8d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b681868050 con 0x55b6817c8450 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.375+0000 7fe15b8d4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b68180d950 con 0x55b681842cb0 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe15b8d4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b681864010 con 0x55b6819453b0 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe145d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 0x55b68194b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe145d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 0x55b68194b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51646/0 (socket says 172.21.15.46:51646) 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe145d9b700 1 -- 172.21.15.46:0/1973277095 learned_addr learned my addr 172.21.15.46:0/1973277095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe14659c700 1 --2- 172.21.15.46:0/1973277095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b681842cb0 0x55b68194b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe146d9d700 1 --2- 172.21.15.46:0/1973277095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6819453b0 0x55b6819457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe145d9b700 1 -- 172.21.15.46:0/1973277095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6819453b0 msgr2=0x55b6819457c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe145d9b700 1 --2- 172.21.15.46:0/1973277095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6819453b0 0x55b6819457c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe145d9b700 1 -- 172.21.15.46:0/1973277095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b681842cb0 msgr2=0x55b68194b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe145d9b700 1 --2- 172.21.15.46:0/1973277095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b681842cb0 0x55b68194b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe145d9b700 1 -- 172.21.15.46:0/1973277095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6817fe4b0 con 0x55b6817c8450 2024-05-09T00:11:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe146d9d700 1 --2- 172.21.15.46:0/1973277095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6819453b0 0x55b6819457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe145d9b700 1 --2- 172.21.15.46:0/1973277095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 0x55b68194b730 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe13c010dc0 tx=0x7fe13c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5f7f235baa4a8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe134ff9700 1 -- 172.21.15.46:0/1973277095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c011bb0 con 0x55b6817c8450 2024-05-09T00:11:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe134ff9700 1 -- 172.21.15.46:0/1973277095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe13c011d50 con 0x55b6817c8450 2024-05-09T00:11:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.376+0000 7fe134ff9700 1 -- 172.21.15.46:0/1973277095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c01c6e0 con 0x55b6817c8450 2024-05-09T00:11:06.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.377+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/1973277095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 msgr2=0x55b68194b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:06.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.377+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/1973277095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 0x55b68194b730 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe13c010dc0 tx=0x7fe13c013000 comp rx=0 tx=0).stop 2024-05-09T00:11:06.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.377+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/1973277095 shutdown_connections 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.377+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/1973277095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6819453b0 0x55b6819457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.377+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/1973277095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b681842cb0 0x55b68194b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.377+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/1973277095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 0x55b68194b730 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.377+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/1973277095 >> 172.21.15.46:0/1973277095 conn(0x55b6817cbf20 msgr2=0x55b6819401d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.378+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/1973277095 shutdown_connections 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.378+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/1973277095 wait complete. 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.378+0000 7fe15b8d4b80 1 -- start start 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.378+0000 7fe15b8d4b80 1 Processor -- start 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.378+0000 7fe15b8d4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6817c8450 0x55b68194ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.378+0000 7fe15b8d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b681842cb0 0x55b68194f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.378+0000 7fe15b8d4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6819453b0 0x55b68194f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.378+0000 7fe15b8d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b681946110 con 0x55b681842cb0 2024-05-09T00:11:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.378+0000 7fe15b8d4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b681802ad0 con 0x55b6819453b0 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.378+0000 7fe15b8d4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b681951fd0 con 0x55b6817c8450 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.378+0000 7fe15b8d4b80 10 client.0 _ll_register_callbacks cb 0x55b6816d9670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.378+0000 7fe145d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b681842cb0 0x55b68194f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe145d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b681842cb0 0x55b68194f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51648/0 (socket says 172.21.15.46:51648) 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe145d9b700 1 -- 172.21.15.46:0/2347455774 learned_addr learned my addr 172.21.15.46:0/2347455774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe14659c700 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6817c8450 0x55b68194ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe146d9d700 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6819453b0 0x55b68194f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe145d9b700 1 -- 172.21.15.46:0/2347455774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6817c8450 msgr2=0x55b68194ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe145d9b700 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6817c8450 0x55b68194ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe145d9b700 1 -- 172.21.15.46:0/2347455774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6819453b0 msgr2=0x55b68194f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe145d9b700 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6819453b0 0x55b68194f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe145d9b700 1 -- 172.21.15.46:0/2347455774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6816e3440 con 0x55b681842cb0 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe145d9b700 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b681842cb0 0x55b68194f480 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00ba00 tx=0x7fe13c011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6267298d3c43342 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe12dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe12dffb700 1 -- 172.21.15.46:0/2347455774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c036060 con 0x55b681842cb0 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe12dffb700 1 -- 172.21.15.46:0/2347455774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe13c01e420 con 0x55b681842cb0 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2347455774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe13c011d50 con 0x55b681842cb0 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.379+0000 7fe12dffb700 1 -- 172.21.15.46:0/2347455774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe13c01e620 con 0x55b681842cb0 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.380+0000 7fe15b8d4b80 10 client.14997 fetch_fsmap learned FSMap version 25 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.380+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2347455774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b68195c910 con 0x55b681842cb0 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.380+0000 7fe12dffb700 1 -- 172.21.15.46:0/2347455774 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe13c011d50 con 0x55b681842cb0 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.380+0000 7fe12dffb700 1 -- 172.21.15.46:0/2347455774 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fe13c03b070 con 0x55b681842cb0 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.380+0000 7fe15b8d4b80 10 client.14997 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.380+0000 7fe15b8d4b80 10 client.14997 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.380+0000 7fe15b8d4b80 20 client.14997 populate_metadata read hostname 'smithi046' 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.380+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b68195eaf0 0x55b68195ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.380+0000 7fe15b8d4b80 4 client.14997 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.380+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2347455774 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b681875a30 con 0x55b68195eaf0 2024-05-09T00:11:06.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.380+0000 7fe14659c700 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b68195eaf0 0x55b68195ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.381+0000 7fe14659c700 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b68195eaf0 0x55b68195ff60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ce8bae612eb7071 server_cookie=2ce2a0a771b8d842 in_seq=0 out_seq=0 2024-05-09T00:11:06.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.381+0000 7fe12dffb700 10 client.14997 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:11:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe12dffb700 1 -- 172.21.15.46:0/2347455774 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b681875a30 con 0x55b68195eaf0 2024-05-09T00:11:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe12dffb700 10 client.14997 handle_command_reply: tid=0 2024-05-09T00:11:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe15b8d4b80 1 client.14997 shutdown 2024-05-09T00:11:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2347455774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b681842cb0 msgr2=0x55b68194f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b681842cb0 0x55b68194f480 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00ba00 tx=0x7fe13c011bb0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2347455774 shutdown_connections 2024-05-09T00:11:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b68195eaf0 0x55b68195ff60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6819453b0 0x55b68194f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6817c8450 0x55b68194ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe12dffb700 0 client.14997 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:11:06.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2347455774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b681842cb0 0x55b68194f480 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2347455774 >> 172.21.15.46:0/2347455774 conn(0x55b6817cbf20 msgr2=0x55b681939fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:06.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2347455774 shutdown_connections 2024-05-09T00:11:06.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2347455774 wait complete. 2024-05-09T00:11:06.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.382+0000 7fe15b8d4b80 20 client.14997 trim_cache size 0 max 16384 2024-05-09T00:11:06.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe15b8d4b80 1 Processor -- start 2024-05-09T00:11:06.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe15b8d4b80 1 -- start start 2024-05-09T00:11:06.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe15b8d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 0x55b68194df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe15b8d4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b681842cb0 0x55b68193a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe15b8d4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6819453b0 0x55b68194a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe15b8d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b68195e2a0 con 0x55b6817c8450 2024-05-09T00:11:06.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe15b8d4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b68195e970 con 0x55b681842cb0 2024-05-09T00:11:06.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe15b8d4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b68195f040 con 0x55b6819453b0 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe14659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 0x55b68194df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe14659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 0x55b68194df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51658/0 (socket says 172.21.15.46:51658) 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe14659c700 1 -- 172.21.15.46:0/2863783439 learned_addr learned my addr 172.21.15.46:0/2863783439 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe145d9b700 1 --2- 172.21.15.46:0/2863783439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b681842cb0 0x55b68193a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe146d9d700 1 --2- 172.21.15.46:0/2863783439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6819453b0 0x55b68194a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe14659c700 1 -- 172.21.15.46:0/2863783439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6819453b0 msgr2=0x55b68194a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe14659c700 1 --2- 172.21.15.46:0/2863783439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6819453b0 0x55b68194a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe14659c700 1 -- 172.21.15.46:0/2863783439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b681842cb0 msgr2=0x55b68193a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe14659c700 1 --2- 172.21.15.46:0/2863783439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b681842cb0 0x55b68193a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe14659c700 1 -- 172.21.15.46:0/2863783439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b681951d00 con 0x55b6817c8450 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe145d9b700 1 --2- 172.21.15.46:0/2863783439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b681842cb0 0x55b68193a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.404+0000 7fe146d9d700 1 --2- 172.21.15.46:0/2863783439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6819453b0 0x55b68194a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.405+0000 7fe14659c700 1 --2- 172.21.15.46:0/2863783439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 0x55b68194df00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe1380a50f0 tx=0x7fe1380df390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec47c08f044a9ad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.405+0000 7fe12ffff700 1 -- 172.21.15.46:0/2863783439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1380df5d0 con 0x55b6817c8450 2024-05-09T00:11:06.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.405+0000 7fe12ffff700 1 -- 172.21.15.46:0/2863783439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1380a5300 con 0x55b6817c8450 2024-05-09T00:11:06.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.405+0000 7fe12ffff700 1 -- 172.21.15.46:0/2863783439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1380b19b0 con 0x55b6817c8450 2024-05-09T00:11:06.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.405+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2863783439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 msgr2=0x55b68194df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:06.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.405+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2863783439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 0x55b68194df00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe1380a50f0 tx=0x7fe1380df390 comp rx=0 tx=0).stop 2024-05-09T00:11:06.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.405+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2863783439 shutdown_connections 2024-05-09T00:11:06.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.405+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2863783439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b6819453b0 0x55b68194a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.405+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2863783439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6817c8450 0x55b68194df00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.405+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2863783439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b681842cb0 0x55b68193a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.405+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2863783439 >> 172.21.15.46:0/2863783439 conn(0x55b6817cbf20 msgr2=0x55b6819443a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:06.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2863783439 shutdown_connections 2024-05-09T00:11:06.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2863783439 wait complete. 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe15b8d4b80 1 -- start start 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe15b8d4b80 1 Processor -- start 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe15b8d4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6817c8450 0x55b68194c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe15b8d4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b681842cb0 0x55b68194c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe15b8d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6819453b0 0x55b681967b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe15b8d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b68195fde0 con 0x55b6819453b0 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe15b8d4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b68195f710 con 0x55b6817c8450 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe15b8d4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b6819604b0 con 0x55b681842cb0 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe15b8d4b80 10 client.0 _ll_register_callbacks cb 0x55b6816d9670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe146d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6819453b0 0x55b681967b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe146d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6819453b0 0x55b681967b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51668/0 (socket says 172.21.15.46:51668) 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.406+0000 7fe146d9d700 1 -- 172.21.15.46:0/2461334104 learned_addr learned my addr 172.21.15.46:0/2461334104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe14659c700 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6817c8450 0x55b68194c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe145d9b700 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b681842cb0 0x55b68194c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe146d9d700 1 -- 172.21.15.46:0/2461334104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b681842cb0 msgr2=0x55b68194c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe146d9d700 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b681842cb0 0x55b68194c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe146d9d700 1 -- 172.21.15.46:0/2461334104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6817c8450 msgr2=0x55b68194c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe146d9d700 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6817c8450 0x55b68194c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe146d9d700 1 -- 172.21.15.46:0/2461334104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68194cfc0 con 0x55b6819453b0 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe146d9d700 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6819453b0 0x55b681967b20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe140003f00 tx=0x7fe140003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6b9c7cb232b728 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe12dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe12dffb700 1 -- 172.21.15.46:0/2461334104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1400052b0 con 0x55b6819453b0 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe12dffb700 1 -- 172.21.15.46:0/2461334104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe140005450 con 0x55b6819453b0 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2461334104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1380a5300 con 0x55b6819453b0 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe12dffb700 1 -- 172.21.15.46:0/2461334104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe140005650 con 0x55b6819453b0 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe15b8d4b80 10 client.15003 fetch_fsmap learned FSMap version 25 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.407+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2461334104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b6819ca490 con 0x55b6819453b0 2024-05-09T00:11:06.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.408+0000 7fe12dffb700 1 -- 172.21.15.46:0/2461334104 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1380a5300 con 0x55b6819453b0 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.408+0000 7fe12dffb700 1 -- 172.21.15.46:0/2461334104 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fe14000f910 con 0x55b6819453b0 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.408+0000 7fe15b8d4b80 10 client.15003 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.408+0000 7fe15b8d4b80 10 client.15003 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.408+0000 7fe15b8d4b80 20 client.15003 populate_metadata read hostname 'smithi046' 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.408+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b6819cc990 0x55b6819cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.408+0000 7fe15b8d4b80 4 client.15003 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.408+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2461334104 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "status"}) -- 0x55b68186fad0 con 0x55b6819cc990 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.408+0000 7fe14659c700 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b6819cc990 0x55b6819cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.408+0000 7fe12dffb700 10 client.15003 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.408+0000 7fe14659c700 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b6819cc990 0x55b6819cde00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6744425a47b0ee2f server_cookie=66185d827ae68031 in_seq=0 out_seq=0 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe12dffb700 1 -- 172.21.15.46:0/2461334104 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b68186fad0 con 0x55b6819cc990 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe12dffb700 10 client.15003 handle_command_reply: tid=0 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe15b8d4b80 1 client.15003 shutdown 2024-05-09T00:11:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2461334104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6819453b0 msgr2=0x55b681967b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6819453b0 0x55b681967b20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe140003f00 tx=0x7fe140003f30 comp rx=0 tx=0).stop 2024-05-09T00:11:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2461334104 shutdown_connections 2024-05-09T00:11:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b6819cc990 0x55b6819cde00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe12dffb700 0 client.15003 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:11:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b6819453b0 0x55b681967b20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b6817c8450 0x55b68194c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe15b8d4b80 1 --2- 172.21.15.46:0/2461334104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b681842cb0 0x55b68194c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2461334104 >> 172.21.15.46:0/2461334104 conn(0x55b6817cbf20 msgr2=0x55b68193a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2461334104 shutdown_connections 2024-05-09T00:11:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe15b8d4b80 1 -- 172.21.15.46:0/2461334104 wait complete. 2024-05-09T00:11:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.409+0000 7fe15b8d4b80 20 client.15003 trim_cache size 0 max 16384 2024-05-09T00:11:06.414 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:11:06.414 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "eec67860-0d96-11ef-bc97-c7b262605968", 2024-05-09T00:11:06.414 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-05-09T00:11:06.414 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24499, 2024-05-09T00:11:06.414 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-09T00:11:06.414 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-09T00:11:06.414 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-09T00:11:06.414 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 158.36003810700001, 2024-05-09T00:11:06.414 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 25, 2024-05-09T00:11:06.414 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-05-09T00:11:06.414 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:11:06.414 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 184.46657190299999 2024-05-09T00:11:06.415 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:11:06.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.410+0000 7fe14cebe700 1 -- 172.21.15.46:0/587509617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe1300b2880 msgr2=0x7fe1300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:06.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.410+0000 7fe14cebe700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe1300b2880 0x7fe1300b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe148109970 tx=0x7fe13c00b410 comp rx=0 tx=0).stop 2024-05-09T00:11:06.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.410+0000 7fe14cebe700 1 -- 172.21.15.46:0/587509617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1481034c0 msgr2=0x7fe1480ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:06.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.410+0000 7fe14cebe700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1481034c0 0x7fe1480ff2a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe13800ada0 tx=0x7fe13800c330 comp rx=0 tx=0).stop 2024-05-09T00:11:06.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.410+0000 7fe14cebe700 1 -- 172.21.15.46:0/587509617 shutdown_connections 2024-05-09T00:11:06.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.410+0000 7fe14cebe700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe1300b2880 0x7fe1300b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.410+0000 7fe14cebe700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148108740 0x7fe1480ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.410+0000 7fe14cebe700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1481034c0 0x7fe1480ff2a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.410+0000 7fe14cebe700 1 --2- 172.21.15.46:0/587509617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe148107df0 0x7fe1480ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.410+0000 7fe14cebe700 1 -- 172.21.15.46:0/587509617 >> 172.21.15.46:0/587509617 conn(0x7fe1480fedc0 msgr2=0x7fe14806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.410+0000 7fe14cebe700 1 -- 172.21.15.46:0/587509617 shutdown_connections 2024-05-09T00:11:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:06.410+0000 7fe14cebe700 1 -- 172.21.15.46:0/587509617 wait complete. 2024-05-09T00:11:06.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:06 smithi142 ceph-mon[34533]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:06.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:06 smithi046 ceph-mon[30151]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:06.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:06 smithi169 ceph-mon[34777]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:07.224 INFO:tasks.cephadm:Setup complete, yielding 2024-05-09T00:11:07.225 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T00:11:07.235 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T00:11:07.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-09T00:11:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:08.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:08 smithi142 ceph-mon[34533]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:08.728 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:08 smithi046 ceph-mon[30151]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:08.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:08 smithi169 ceph-mon[34777]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.412+0000 7f86603f5700 1 -- 172.21.15.46:0/1926130721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658104380 msgr2=0x7f8658104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.412+0000 7f86603f5700 1 --2- 172.21.15.46:0/1926130721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658104380 0x7f8658104790 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8654009990 tx=0x7f8654009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.412+0000 7f86603f5700 1 -- 172.21.15.46:0/1926130721 shutdown_connections 2024-05-09T00:11:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.412+0000 7f86603f5700 1 --2- 172.21.15.46:0/1926130721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8658104d20 0x7f865810b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.412+0000 7f86603f5700 1 --2- 172.21.15.46:0/1926130721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8658103250 0x7f8658103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.412+0000 7f86603f5700 1 --2- 172.21.15.46:0/1926130721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658104380 0x7f8658104790 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.412+0000 7f86603f5700 1 -- 172.21.15.46:0/1926130721 >> 172.21.15.46:0/1926130721 conn(0x7f86580fe890 msgr2=0x7f8658100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.412+0000 7f86603f5700 1 -- 172.21.15.46:0/1926130721 shutdown_connections 2024-05-09T00:11:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.412+0000 7f86603f5700 1 -- 172.21.15.46:0/1926130721 wait complete. 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f86603f5700 1 Processor -- start 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f86603f5700 1 -- start start 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f86603f5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8658103250 0x7f86581997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f86603f5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658104380 0x7f8658199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f86603f5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8658104d20 0x7f86581a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f86603f5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f865810ecb0 con 0x7f8658104380 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f86603f5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f865810eb30 con 0x7f8658104d20 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f86603f5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f865810ee30 con 0x7f8658103250 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f865d990700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658104380 0x7f8658199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f865d990700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658104380 0x7f8658199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51688/0 (socket says 172.21.15.46:51688) 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f865d990700 1 -- 172.21.15.46:0/1592830289 learned_addr learned my addr 172.21.15.46:0/1592830289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f865e191700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8658103250 0x7f86581997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f865e992700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8658104d20 0x7f86581a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f865d990700 1 -- 172.21.15.46:0/1592830289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8658103250 msgr2=0x7f86581997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f865d990700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8658103250 0x7f86581997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f865d990700 1 -- 172.21.15.46:0/1592830289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8658104d20 msgr2=0x7f86581a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f865d990700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8658104d20 0x7f86581a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.413+0000 7f865d990700 1 -- 172.21.15.46:0/1592830289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86581a1350 con 0x7f8658104380 2024-05-09T00:11:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.414+0000 7f865d990700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658104380 0x7f8658199d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8654006010 tx=0x7f86540047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.414+0000 7f864b7fe700 1 -- 172.21.15.46:0/1592830289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654027070 con 0x7f8658104380 2024-05-09T00:11:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.414+0000 7f86603f5700 1 -- 172.21.15.46:0/1592830289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86581a15e0 con 0x7f8658104380 2024-05-09T00:11:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.414+0000 7f864b7fe700 1 -- 172.21.15.46:0/1592830289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8654004000 con 0x7f8658104380 2024-05-09T00:11:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.414+0000 7f864b7fe700 1 -- 172.21.15.46:0/1592830289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654022d30 con 0x7f8658104380 2024-05-09T00:11:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.414+0000 7f86603f5700 1 -- 172.21.15.46:0/1592830289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86581a1a90 con 0x7f8658104380 2024-05-09T00:11:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.415+0000 7f86603f5700 1 -- 172.21.15.46:0/1592830289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86580512c0 con 0x7f8658104380 2024-05-09T00:11:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.417+0000 7f864b7fe700 1 -- 172.21.15.46:0/1592830289 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f865402a070 con 0x7f8658104380 2024-05-09T00:11:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.417+0000 7f864b7fe700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f86440b2880 0x7f86440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:09.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.417+0000 7f865e191700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f86440b2880 0x7f86440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:09.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.417+0000 7f864b7fe700 1 -- 172.21.15.46:0/1592830289 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f86540e32e0 con 0x7f8658104380 2024-05-09T00:11:09.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.418+0000 7f865e191700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f86440b2880 0x7f86440b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8658104280 tx=0x7f864c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.421+0000 7f864b7fe700 1 -- 172.21.15.46:0/1592830289 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8654013070 con 0x7f8658104380 2024-05-09T00:11:09.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.668+0000 7f86603f5700 1 -- 172.21.15.46:0/1592830289 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f8658063c00 con 0x7f86440b2880 2024-05-09T00:11:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.669+0000 7f864b7fe700 1 -- 172.21.15.46:0/1592830289 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f8658063c00 con 0x7f86440b2880 2024-05-09T00:11:09.665 INFO:teuthology.orchestra.run.smithi046.stdout:Backend: cephadm 2024-05-09T00:11:09.666 INFO:teuthology.orchestra.run.smithi046.stdout:Available: Yes 2024-05-09T00:11:09.666 INFO:teuthology.orchestra.run.smithi046.stdout:Paused: No 2024-05-09T00:11:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.671+0000 7f86603f5700 1 -- 172.21.15.46:0/1592830289 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f86440b2880 msgr2=0x7f86440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.671+0000 7f86603f5700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f86440b2880 0x7f86440b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8658104280 tx=0x7f864c00a380 comp rx=0 tx=0).stop 2024-05-09T00:11:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.671+0000 7f86603f5700 1 -- 172.21.15.46:0/1592830289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658104380 msgr2=0x7f8658199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.671+0000 7f86603f5700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658104380 0x7f8658199d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8654006010 tx=0x7f86540047b0 comp rx=0 tx=0).stop 2024-05-09T00:11:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.671+0000 7f86603f5700 1 -- 172.21.15.46:0/1592830289 shutdown_connections 2024-05-09T00:11:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.671+0000 7f86603f5700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f86440b2880 0x7f86440b4d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.671+0000 7f86603f5700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8658104d20 0x7f86581a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.671+0000 7f86603f5700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8658103250 0x7f86581997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.671+0000 7f86603f5700 1 --2- 172.21.15.46:0/1592830289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658104380 0x7f8658199d30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.671+0000 7f86603f5700 1 -- 172.21.15.46:0/1592830289 >> 172.21.15.46:0/1592830289 conn(0x7f86580fe890 msgr2=0x7f86581075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:09.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.671+0000 7f86603f5700 1 -- 172.21.15.46:0/1592830289 shutdown_connections 2024-05-09T00:11:09.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:09.671+0000 7f86603f5700 1 -- 172.21.15.46:0/1592830289 wait complete. 2024-05-09T00:11:10.418 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-09T00:11:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:10.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:10 smithi142 ceph-mon[34533]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:10.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:10 smithi142 ceph-mon[34533]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:10.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:10 smithi046 ceph-mon[30151]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:10.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:10 smithi046 ceph-mon[30151]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:10.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:10 smithi169 ceph-mon[34777]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:10.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:10 smithi169 ceph-mon[34777]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.402+0000 7fd7639a7700 1 -- 172.21.15.46:0/1049488436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd75c103230 msgr2=0x7fd75c103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.402+0000 7fd7639a7700 1 --2- 172.21.15.46:0/1049488436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd75c103230 0x7fd75c103640 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd758009990 tx=0x7fd758009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.402+0000 7fd7639a7700 1 -- 172.21.15.46:0/1049488436 shutdown_connections 2024-05-09T00:11:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.402+0000 7fd7639a7700 1 --2- 172.21.15.46:0/1049488436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c104d00 0x7fd75c10b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.402+0000 7fd7639a7700 1 --2- 172.21.15.46:0/1049488436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd75c103230 0x7fd75c103640 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.402+0000 7fd7639a7700 1 --2- 172.21.15.46:0/1049488436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd75c104360 0x7fd75c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.402+0000 7fd7639a7700 1 -- 172.21.15.46:0/1049488436 >> 172.21.15.46:0/1049488436 conn(0x7fd75c0fe870 msgr2=0x7fd75c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.403+0000 7fd7639a7700 1 -- 172.21.15.46:0/1049488436 shutdown_connections 2024-05-09T00:11:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.403+0000 7fd7639a7700 1 -- 172.21.15.46:0/1049488436 wait complete. 2024-05-09T00:11:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.403+0000 7fd7639a7700 1 Processor -- start 2024-05-09T00:11:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.403+0000 7fd7639a7700 1 -- start start 2024-05-09T00:11:12.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd7639a7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c103230 0x7fd75c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:12.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd7639a7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd75c104360 0x7fd75c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:12.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd7639a7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd75c104d00 0x7fd75c1a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:12.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd7639a7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10ec80 con 0x7fd75c104360 2024-05-09T00:11:12.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd7639a7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10eb00 con 0x7fd75c103230 2024-05-09T00:11:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd7639a7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10ee00 con 0x7fd75c104d00 2024-05-09T00:11:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd760f42700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd75c104360 0x7fd75c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd760f42700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd75c104360 0x7fd75c19d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51706/0 (socket says 172.21.15.46:51706) 2024-05-09T00:11:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd760f42700 1 -- 172.21.15.46:0/623005574 learned_addr learned my addr 172.21.15.46:0/623005574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd761f44700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd75c104d00 0x7fd75c1a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd761743700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c103230 0x7fd75c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd760f42700 1 -- 172.21.15.46:0/623005574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd75c104d00 msgr2=0x7fd75c1a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd760f42700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd75c104d00 0x7fd75c1a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd760f42700 1 -- 172.21.15.46:0/623005574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c103230 msgr2=0x7fd75c19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd760f42700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c103230 0x7fd75c19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd760f42700 1 -- 172.21.15.46:0/623005574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c1a4590 con 0x7fd75c104360 2024-05-09T00:11:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd761f44700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd75c104d00 0x7fd75c1a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd761743700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c103230 0x7fd75c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.404+0000 7fd760f42700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd75c104360 0x7fd75c19d000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00c9b0 tx=0x7fd74c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.405+0000 7fd7527fc700 1 -- 172.21.15.46:0/623005574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c007b00 con 0x7fd75c104360 2024-05-09T00:11:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.405+0000 7fd7527fc700 1 -- 172.21.15.46:0/623005574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74c007ca0 con 0x7fd75c104360 2024-05-09T00:11:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.405+0000 7fd7639a7700 1 -- 172.21.15.46:0/623005574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c1a4880 con 0x7fd75c104360 2024-05-09T00:11:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.405+0000 7fd7527fc700 1 -- 172.21.15.46:0/623005574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c018730 con 0x7fd75c104360 2024-05-09T00:11:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.405+0000 7fd7639a7700 1 -- 172.21.15.46:0/623005574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75c1a4e10 con 0x7fd75c104360 2024-05-09T00:11:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.406+0000 7fd7639a7700 1 -- 172.21.15.46:0/623005574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd75c0512c0 con 0x7fd75c104360 2024-05-09T00:11:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.407+0000 7fd7527fc700 1 -- 172.21.15.46:0/623005574 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd74c00f450 con 0x7fd75c104360 2024-05-09T00:11:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.408+0000 7fd7527fc700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7480b2900 0x7fd7480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.408+0000 7fd761743700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7480b2900 0x7fd7480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.408+0000 7fd7527fc700 1 -- 172.21.15.46:0/623005574 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd74c0e2560 con 0x7fd75c104360 2024-05-09T00:11:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.409+0000 7fd761743700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7480b2900 0x7fd7480b4e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b5c0 tx=0x7fd758005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.412+0000 7fd7527fc700 1 -- 172.21.15.46:0/623005574 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd74c014030 con 0x7fd75c104360 2024-05-09T00:11:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.635+0000 7fd7639a7700 1 -- 172.21.15.46:0/623005574 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd75c1a5040 con 0x7fd7480b2900 2024-05-09T00:11:12.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.649+0000 7fd7527fc700 1 -- 172.21.15.46:0/623005574 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fd75c1a5040 con 0x7fd7480b2900 2024-05-09T00:11:12.649 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:12 smithi046 ceph-mon[30151]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T00:11:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:mds.a smithi046 running (3m) 31s ago 3m 84.5M - 19.0.0-3594-gc9da4049 db794d706f49 f835197cfca6 2024-05-09T00:11:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:mds.b smithi142 running (3m) 45s ago 2m 89.6M - 19.0.0-3594-gc9da4049 db794d706f49 09a12a4d824b 2024-05-09T00:11:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:mds.c smithi169 running (3m) 45s ago 3m 87.0M - 19.0.0-3594-gc9da4049 db794d706f49 e161ca7048f4 2024-05-09T00:11:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:mds.d smithi046 running (3m) 31s ago 3m 81.2M - 19.0.0-3594-gc9da4049 db794d706f49 516f9b745697 2024-05-09T00:11:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:mds.e smithi142 running (3m) 45s ago 3m 88.5M - 19.0.0-3594-gc9da4049 db794d706f49 8c35d1b6fb6e 2024-05-09T00:11:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:mds.f smithi169 running (3m) 45s ago 3m 90.7M - 19.0.0-3594-gc9da4049 db794d706f49 c8111ddbd597 2024-05-09T00:11:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:mds.g smithi046 running (3m) 31s ago 3m 82.1M - 19.0.0-3594-gc9da4049 db794d706f49 552d4e2443f6 2024-05-09T00:11:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:mds.h smithi142 running (3m) 45s ago 3m 87.1M - 19.0.0-3594-gc9da4049 db794d706f49 f07c30c5b6cc 2024-05-09T00:11:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:mds.i smithi169 running (3m) 45s ago 3m 97.4M - 19.0.0-3594-gc9da4049 db794d706f49 83bac6934c60 2024-05-09T00:11:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:mds.j smithi046 running (3m) 31s ago 3m 84.7M - 19.0.0-3594-gc9da4049 db794d706f49 4b11d193045d 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:mds.k smithi142 running (3m) 45s ago 3m 82.5M - 19.0.0-3594-gc9da4049 db794d706f49 2b3a54540edf 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:mds.l smithi169 running (3m) 45s ago 3m 81.2M - 19.0.0-3594-gc9da4049 db794d706f49 b0c6cb191e9f 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.x smithi046 *:9283,8765 running (10m) 31s ago 10m 597M - 19.0.0-3594-gc9da4049 db794d706f49 0ab481730155 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.y smithi142 *:8443,8765 running (8m) 45s ago 8m 535M - 19.0.0-3594-gc9da4049 db794d706f49 9dcca95c5554 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.z smithi169 *:8443,8765 running (8m) 45s ago 8m 535M - 19.0.0-3594-gc9da4049 db794d706f49 d6dd060bda59 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:mon.a smithi046 running (10m) 31s ago 10m 122M 2048M 19.0.0-3594-gc9da4049 db794d706f49 74e04927330a 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:mon.b smithi142 running (9m) 45s ago 9m 108M 2048M 19.0.0-3594-gc9da4049 db794d706f49 4de2356ef3de 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:mon.c smithi169 running (9m) 45s ago 9m 109M 2048M 19.0.0-3594-gc9da4049 db794d706f49 c6d470b5b04d 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (8m) 31s ago 8m 125M 4273M 19.0.0-3594-gc9da4049 db794d706f49 ce03115b2dc9 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi142 running (8m) 45s ago 8m 123M 4272M 19.0.0-3594-gc9da4049 db794d706f49 4a41407b4d74 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi169 running (7m) 45s ago 7m 126M 4272M 19.0.0-3594-gc9da4049 db794d706f49 3697d9ad88be 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi046 running (7m) 31s ago 7m 112M 4273M 19.0.0-3594-gc9da4049 db794d706f49 6e5f45b29ff0 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi142 running (6m) 45s ago 6m 110M 4272M 19.0.0-3594-gc9da4049 db794d706f49 b557784b3cba 2024-05-09T00:11:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi169 running (6m) 45s ago 6m 112M 4272M 19.0.0-3594-gc9da4049 db794d706f49 dd8522f6cdb0 2024-05-09T00:11:12.652 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 smithi046 running (6m) 31s ago 6m 120M 4273M 19.0.0-3594-gc9da4049 db794d706f49 11ad0e87abef 2024-05-09T00:11:12.652 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 smithi142 running (5m) 45s ago 5m 114M 4272M 19.0.0-3594-gc9da4049 db794d706f49 16b0dc942852 2024-05-09T00:11:12.652 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 smithi169 running (5m) 45s ago 5m 114M 4272M 19.0.0-3594-gc9da4049 db794d706f49 67e23aea7726 2024-05-09T00:11:12.652 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 smithi046 running (4m) 31s ago 4m 121M 4273M 19.0.0-3594-gc9da4049 db794d706f49 b37f53f83511 2024-05-09T00:11:12.652 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 smithi142 running (4m) 45s ago 4m 125M 4272M 19.0.0-3594-gc9da4049 db794d706f49 ede833d5601b 2024-05-09T00:11:12.652 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 smithi169 running (3m) 45s ago 3m 110M 4272M 19.0.0-3594-gc9da4049 db794d706f49 52249d1e3782 2024-05-09T00:11:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.651+0000 7fd7639a7700 1 -- 172.21.15.46:0/623005574 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7480b2900 msgr2=0x7fd7480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.651+0000 7fd7639a7700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7480b2900 0x7fd7480b4e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b5c0 tx=0x7fd758005a90 comp rx=0 tx=0).stop 2024-05-09T00:11:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.651+0000 7fd7639a7700 1 -- 172.21.15.46:0/623005574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd75c104360 msgr2=0x7fd75c19d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.651+0000 7fd7639a7700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd75c104360 0x7fd75c19d000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00c9b0 tx=0x7fd74c00cd70 comp rx=0 tx=0).stop 2024-05-09T00:11:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.651+0000 7fd7639a7700 1 -- 172.21.15.46:0/623005574 shutdown_connections 2024-05-09T00:11:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.651+0000 7fd7639a7700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7480b2900 0x7fd7480b4e10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.651+0000 7fd7639a7700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd75c104d00 0x7fd75c1a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.651+0000 7fd7639a7700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c103230 0x7fd75c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.651+0000 7fd7639a7700 1 --2- 172.21.15.46:0/623005574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd75c104360 0x7fd75c19d000 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.651+0000 7fd7639a7700 1 -- 172.21.15.46:0/623005574 >> 172.21.15.46:0/623005574 conn(0x7fd75c0fe870 msgr2=0x7fd75c100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.651+0000 7fd7639a7700 1 -- 172.21.15.46:0/623005574 shutdown_connections 2024-05-09T00:11:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:12.651+0000 7fd7639a7700 1 -- 172.21.15.46:0/623005574 wait complete. 2024-05-09T00:11:12.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:12 smithi142 ceph-mon[34533]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:12.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:12 smithi169 ceph-mon[34777]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:13.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-09T00:11:13.598 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:13.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:13 smithi142 ceph-mon[34533]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:13.720 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:13 smithi046 ceph-mon[30151]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:13.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:13 smithi169 ceph-mon[34777]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:14.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:14 smithi142 ceph-mon[34533]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:14.804 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:14 smithi046 ceph-mon[30151]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:14.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:14 smithi169 ceph-mon[34777]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.364+0000 7feb8ad5a700 1 -- 172.21.15.46:0/992815726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb840766b0 msgr2=0x7feb84106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.364+0000 7feb8ad5a700 1 --2- 172.21.15.46:0/992815726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb840766b0 0x7feb84106ec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feb78009990 tx=0x7feb78009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.365+0000 7feb8ad5a700 1 -- 172.21.15.46:0/992815726 shutdown_connections 2024-05-09T00:11:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.365+0000 7feb8ad5a700 1 --2- 172.21.15.46:0/992815726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb8410a000 0x7feb8410c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.365+0000 7feb8ad5a700 1 --2- 172.21.15.46:0/992815726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb840766b0 0x7feb84106ec0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.365+0000 7feb8ad5a700 1 --2- 172.21.15.46:0/992815726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb84107490 0x7feb84109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.365+0000 7feb8ad5a700 1 -- 172.21.15.46:0/992815726 >> 172.21.15.46:0/992815726 conn(0x7feb840fe890 msgr2=0x7feb84100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.365+0000 7feb8ad5a700 1 -- 172.21.15.46:0/992815726 shutdown_connections 2024-05-09T00:11:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.365+0000 7feb8ad5a700 1 -- 172.21.15.46:0/992815726 wait complete. 2024-05-09T00:11:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.365+0000 7feb8ad5a700 1 Processor -- start 2024-05-09T00:11:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.365+0000 7feb8ad5a700 1 -- start start 2024-05-09T00:11:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb8ad5a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb840766b0 0x7feb8419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb8ad5a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb84107490 0x7feb8419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb8ad5a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb8410a000 0x7feb841a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb8ad5a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb8410fde0 con 0x7feb8410a000 2024-05-09T00:11:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb8ad5a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb8410fc60 con 0x7feb840766b0 2024-05-09T00:11:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb8ad5a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feb8410ff60 con 0x7feb84107490 2024-05-09T00:11:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb892f7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb8410a000 0x7feb841a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb892f7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb8410a000 0x7feb841a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51720/0 (socket says 172.21.15.46:51720) 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb892f7700 1 -- 172.21.15.46:0/1651244223 learned_addr learned my addr 172.21.15.46:0/1651244223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb88af6700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb840766b0 0x7feb8419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb83fff700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb84107490 0x7feb8419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb892f7700 1 -- 172.21.15.46:0/1651244223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb84107490 msgr2=0x7feb8419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb892f7700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb84107490 0x7feb8419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb892f7700 1 -- 172.21.15.46:0/1651244223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb840766b0 msgr2=0x7feb8419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb892f7700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb840766b0 0x7feb8419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb892f7700 1 -- 172.21.15.46:0/1651244223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb841a24c0 con 0x7feb8410a000 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.366+0000 7feb892f7700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb8410a000 0x7feb841a1e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00ada0 tx=0x7feb7c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.367+0000 7feb81ffb700 1 -- 172.21.15.46:0/1651244223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c017020 con 0x7feb8410a000 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.367+0000 7feb8ad5a700 1 -- 172.21.15.46:0/1651244223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb841a27b0 con 0x7feb8410a000 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.367+0000 7feb81ffb700 1 -- 172.21.15.46:0/1651244223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb7c00cd80 con 0x7feb8410a000 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.367+0000 7feb81ffb700 1 -- 172.21.15.46:0/1651244223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c005260 con 0x7feb8410a000 2024-05-09T00:11:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.367+0000 7feb8ad5a700 1 -- 172.21.15.46:0/1651244223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb841a2cf0 con 0x7feb8410a000 2024-05-09T00:11:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.368+0000 7feb8ad5a700 1 -- 172.21.15.46:0/1651244223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb840512c0 con 0x7feb8410a000 2024-05-09T00:11:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.370+0000 7feb81ffb700 1 -- 172.21.15.46:0/1651244223 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7feb7c00c8d0 con 0x7feb8410a000 2024-05-09T00:11:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.370+0000 7feb81ffb700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7feb740b2880 0x7feb740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.370+0000 7feb88af6700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7feb740b2880 0x7feb740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.370+0000 7feb81ffb700 1 -- 172.21.15.46:0/1651244223 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7feb7c0e1970 con 0x7feb8410a000 2024-05-09T00:11:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.371+0000 7feb88af6700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7feb740b2880 0x7feb740b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb7800b5c0 tx=0x7feb7801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.375+0000 7feb81ffb700 1 -- 172.21.15.46:0/1651244223 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feb7c021020 con 0x7feb8410a000 2024-05-09T00:11:15.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.599+0000 7feb8ad5a700 1 -- 172.21.15.46:0/1651244223 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7feb84063c00 con 0x7feb740b2880 2024-05-09T00:11:15.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.605+0000 7feb81ffb700 1 -- 172.21.15.46:0/1651244223 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7feb84063c00 con 0x7feb740b2880 2024-05-09T00:11:15.602 INFO:teuthology.orchestra.run.smithi046.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-09T00:11:15.602 INFO:teuthology.orchestra.run.smithi046.stdout:mds.all 12/12 48s ago 3m smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-05-09T00:11:15.602 INFO:teuthology.orchestra.run.smithi046.stdout:mgr 3/3 48s ago 9m smithi046=x;smithi142=y;smithi169=z;count:3 2024-05-09T00:11:15.602 INFO:teuthology.orchestra.run.smithi046.stdout:mon 3/3 48s ago 9m smithi046:172.21.15.46=a;smithi142:172.21.15.142=b;smithi169:172.21.15.169=c;count:3 2024-05-09T00:11:15.602 INFO:teuthology.orchestra.run.smithi046.stdout:osd 12 48s ago - 2024-05-09T00:11:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.608+0000 7feb8ad5a700 1 -- 172.21.15.46:0/1651244223 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7feb740b2880 msgr2=0x7feb740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.608+0000 7feb8ad5a700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7feb740b2880 0x7feb740b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb7800b5c0 tx=0x7feb7801a040 comp rx=0 tx=0).stop 2024-05-09T00:11:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.608+0000 7feb8ad5a700 1 -- 172.21.15.46:0/1651244223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb8410a000 msgr2=0x7feb841a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.608+0000 7feb8ad5a700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb8410a000 0x7feb841a1e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00ada0 tx=0x7feb7c00c330 comp rx=0 tx=0).stop 2024-05-09T00:11:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.608+0000 7feb8ad5a700 1 -- 172.21.15.46:0/1651244223 shutdown_connections 2024-05-09T00:11:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.608+0000 7feb8ad5a700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7feb740b2880 0x7feb740b4d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.608+0000 7feb8ad5a700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb8410a000 0x7feb841a1e40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.608+0000 7feb8ad5a700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb840766b0 0x7feb8419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.608+0000 7feb8ad5a700 1 --2- 172.21.15.46:0/1651244223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb84107490 0x7feb8419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.608+0000 7feb8ad5a700 1 -- 172.21.15.46:0/1651244223 >> 172.21.15.46:0/1651244223 conn(0x7feb840fe890 msgr2=0x7feb84105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.608+0000 7feb8ad5a700 1 -- 172.21.15.46:0/1651244223 shutdown_connections 2024-05-09T00:11:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:15.608+0000 7feb8ad5a700 1 -- 172.21.15.46:0/1651244223 wait complete. 2024-05-09T00:11:16.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-09T00:11:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:16.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:16 smithi142 ceph-mon[34533]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:16.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:16 smithi142 ceph-mon[34533]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:16.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:16 smithi046 ceph-mon[30151]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:16.806 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:16 smithi046 ceph-mon[30151]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:16.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:16 smithi169 ceph-mon[34777]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:16.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:16 smithi169 ceph-mon[34777]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.444+0000 7fc1b97f6700 1 -- 172.21.15.46:0/1286368137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4103250 msgr2=0x7fc1b4103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.444+0000 7fc1b97f6700 1 --2- 172.21.15.46:0/1286368137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4103250 0x7fc1b4103660 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009990 tx=0x7fc1a4009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.444+0000 7fc1b97f6700 1 -- 172.21.15.46:0/1286368137 shutdown_connections 2024-05-09T00:11:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.444+0000 7fc1b97f6700 1 --2- 172.21.15.46:0/1286368137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4104d20 0x7fc1b410b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.444+0000 7fc1b97f6700 1 --2- 172.21.15.46:0/1286368137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4103250 0x7fc1b4103660 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.444+0000 7fc1b97f6700 1 --2- 172.21.15.46:0/1286368137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b4104380 0x7fc1b4104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.444+0000 7fc1b97f6700 1 -- 172.21.15.46:0/1286368137 >> 172.21.15.46:0/1286368137 conn(0x7fc1b40fe890 msgr2=0x7fc1b4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.445+0000 7fc1b97f6700 1 -- 172.21.15.46:0/1286368137 shutdown_connections 2024-05-09T00:11:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.445+0000 7fc1b97f6700 1 -- 172.21.15.46:0/1286368137 wait complete. 2024-05-09T00:11:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.445+0000 7fc1b97f6700 1 Processor -- start 2024-05-09T00:11:18.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.445+0000 7fc1b97f6700 1 -- start start 2024-05-09T00:11:18.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b97f6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4103250 0x7fc1b4076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b97f6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4104380 0x7fc1b419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b97f6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b4104d20 0x7fc1b41a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b97f6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b410ec90 con 0x7fc1b4104380 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b97f6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b410eb10 con 0x7fc1b4103250 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b97f6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b410ee10 con 0x7fc1b4104d20 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b27fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4104380 0x7fc1b419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b27fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4104380 0x7fc1b419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45750/0 (socket says 172.21.15.46:45750) 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b27fc700 1 -- 172.21.15.46:0/660675714 learned_addr learned my addr 172.21.15.46:0/660675714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b2ffd700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4103250 0x7fc1b4076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b37fe700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b4104d20 0x7fc1b41a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b27fc700 1 -- 172.21.15.46:0/660675714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b4104d20 msgr2=0x7fc1b41a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b27fc700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b4104d20 0x7fc1b41a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b27fc700 1 -- 172.21.15.46:0/660675714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4103250 msgr2=0x7fc1b4076450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b27fc700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4103250 0x7fc1b4076450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b27fc700 1 -- 172.21.15.46:0/660675714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b41a24f0 con 0x7fc1b4104380 2024-05-09T00:11:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b37fe700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b4104d20 0x7fc1b41a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b2ffd700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4103250 0x7fc1b4076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc1b27fc700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4104380 0x7fc1b419f750 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800e9a0 tx=0x7fc1a800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.446+0000 7fc19bfff700 1 -- 172.21.15.46:0/660675714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a800cbd0 con 0x7fc1b4104380 2024-05-09T00:11:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.447+0000 7fc19bfff700 1 -- 172.21.15.46:0/660675714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a800cd70 con 0x7fc1b4104380 2024-05-09T00:11:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.447+0000 7fc1b97f6700 1 -- 172.21.15.46:0/660675714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b41a27e0 con 0x7fc1b4104380 2024-05-09T00:11:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.447+0000 7fc19bfff700 1 -- 172.21.15.46:0/660675714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8018680 con 0x7fc1b4104380 2024-05-09T00:11:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.447+0000 7fc1b97f6700 1 -- 172.21.15.46:0/660675714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b41a2d20 con 0x7fc1b4104380 2024-05-09T00:11:18.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.448+0000 7fc1b97f6700 1 -- 172.21.15.46:0/660675714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b40512c0 con 0x7fc1b4104380 2024-05-09T00:11:18.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.449+0000 7fc19bfff700 1 -- 172.21.15.46:0/660675714 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc1a8010970 con 0x7fc1b4104380 2024-05-09T00:11:18.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.450+0000 7fc19bfff700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc19c0b2880 0x7fc19c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:18.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.450+0000 7fc1b2ffd700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc19c0b2880 0x7fc19c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:18.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.450+0000 7fc19bfff700 1 -- 172.21.15.46:0/660675714 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc1a80e2a30 con 0x7fc1b4104380 2024-05-09T00:11:18.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.451+0000 7fc1b2ffd700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc19c0b2880 0x7fc19c0b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400b760 tx=0x7fc1a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.454+0000 7fc19bfff700 1 -- 172.21.15.46:0/660675714 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1a80aa750 con 0x7fc1b4104380 2024-05-09T00:11:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.673+0000 7fc1b97f6700 1 -- 172.21.15.46:0/660675714 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc1b4063c00 con 0x7fc19c0b2880 2024-05-09T00:11:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.675+0000 7fc19bfff700 1 -- 172.21.15.46:0/660675714 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc1b4063c00 con 0x7fc19c0b2880 2024-05-09T00:11:18.673 INFO:teuthology.orchestra.run.smithi046.stdout:HOST ADDR LABELS STATUS 2024-05-09T00:11:18.673 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 172.21.15.46 2024-05-09T00:11:18.673 INFO:teuthology.orchestra.run.smithi046.stdout:smithi142 172.21.15.142 2024-05-09T00:11:18.673 INFO:teuthology.orchestra.run.smithi046.stdout:smithi169 172.21.15.169 2024-05-09T00:11:18.673 INFO:teuthology.orchestra.run.smithi046.stdout:3 hosts in cluster 2024-05-09T00:11:18.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.677+0000 7fc1b97f6700 1 -- 172.21.15.46:0/660675714 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc19c0b2880 msgr2=0x7fc19c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.677+0000 7fc1b97f6700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc19c0b2880 0x7fc19c0b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400b760 tx=0x7fc1a4005fb0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.677+0000 7fc1b97f6700 1 -- 172.21.15.46:0/660675714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4104380 msgr2=0x7fc1b419f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.677+0000 7fc1b97f6700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4104380 0x7fc1b419f750 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800e9a0 tx=0x7fc1a800ed60 comp rx=0 tx=0).stop 2024-05-09T00:11:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.677+0000 7fc1b97f6700 1 -- 172.21.15.46:0/660675714 shutdown_connections 2024-05-09T00:11:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.677+0000 7fc1b97f6700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc19c0b2880 0x7fc19c0b4d90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.677+0000 7fc1b97f6700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b4104d20 0x7fc1b41a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.677+0000 7fc1b97f6700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4103250 0x7fc1b4076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.677+0000 7fc1b97f6700 1 --2- 172.21.15.46:0/660675714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4104380 0x7fc1b419f750 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.677+0000 7fc1b97f6700 1 -- 172.21.15.46:0/660675714 >> 172.21.15.46:0/660675714 conn(0x7fc1b40fe890 msgr2=0x7fc1b41075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.678+0000 7fc1b97f6700 1 -- 172.21.15.46:0/660675714 shutdown_connections 2024-05-09T00:11:18.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:18.678+0000 7fc1b97f6700 1 -- 172.21.15.46:0/660675714 wait complete. 2024-05-09T00:11:18.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:18 smithi046 ceph-mon[30151]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:18.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:18 smithi142 ceph-mon[34533]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:18.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:18 smithi169 ceph-mon[34777]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:19.402 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-09T00:11:19.656 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:19.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:19 smithi046 ceph-mon[30151]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:19.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:19 smithi169 ceph-mon[34777]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:19.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:19 smithi142 ceph-mon[34533]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:20.739 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:20 smithi046 ceph-mon[30151]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:20.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:20 smithi169 ceph-mon[34777]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:20.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:20 smithi142 ceph-mon[34533]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.312+0000 7fea41bed700 1 -- 172.21.15.46:0/2211201600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea3c06c290 msgr2=0x7fea3c106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.312+0000 7fea41bed700 1 --2- 172.21.15.46:0/2211201600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea3c06c290 0x7fea3c106ef0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00b0a0 tx=0x7fea2c00b3b0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.312+0000 7fea41bed700 1 -- 172.21.15.46:0/2211201600 shutdown_connections 2024-05-09T00:11:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.312+0000 7fea41bed700 1 --2- 172.21.15.46:0/2211201600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea3c109ff0 0x7fea3c10c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.312+0000 7fea41bed700 1 --2- 172.21.15.46:0/2211201600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea3c06c290 0x7fea3c106ef0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.312+0000 7fea41bed700 1 --2- 172.21.15.46:0/2211201600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea3c1074c0 0x7fea3c109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.312+0000 7fea41bed700 1 -- 172.21.15.46:0/2211201600 >> 172.21.15.46:0/2211201600 conn(0x7fea3c0fe880 msgr2=0x7fea3c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.312+0000 7fea41bed700 1 -- 172.21.15.46:0/2211201600 shutdown_connections 2024-05-09T00:11:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.312+0000 7fea41bed700 1 -- 172.21.15.46:0/2211201600 wait complete. 2024-05-09T00:11:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.313+0000 7fea41bed700 1 Processor -- start 2024-05-09T00:11:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.313+0000 7fea41bed700 1 -- start start 2024-05-09T00:11:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea41bed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea3c06c290 0x7fea3c104740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea41bed700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea3c1074c0 0x7fea3c102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea41bed700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea3c109ff0 0x7fea3c103330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea41bed700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c10fdc0 con 0x7fea3c06c290 2024-05-09T00:11:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea41bed700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c10fc40 con 0x7fea3c109ff0 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea41bed700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c10ff40 con 0x7fea3c1074c0 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea3c06c290 0x7fea3c104740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea3c06c290 0x7fea3c104740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45764/0 (socket says 172.21.15.46:45764) 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3b7fe700 1 -- 172.21.15.46:0/3973101938 learned_addr learned my addr 172.21.15.46:0/3973101938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3affd700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea3c1074c0 0x7fea3c102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3b7fe700 1 -- 172.21.15.46:0/3973101938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea3c1074c0 msgr2=0x7fea3c102df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3b7fe700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea3c1074c0 0x7fea3c102df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3b7fe700 1 -- 172.21.15.46:0/3973101938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea3c109ff0 msgr2=0x7fea3c103330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3bfff700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea3c109ff0 0x7fea3c103330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3b7fe700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea3c109ff0 0x7fea3c103330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3b7fe700 1 -- 172.21.15.46:0/3973101938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3c103a00 con 0x7fea3c06c290 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3affd700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea3c1074c0 0x7fea3c102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3bfff700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea3c109ff0 0x7fea3c103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea3b7fe700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea3c06c290 0x7fea3c104740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0062a0 tx=0x7fea2c004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea38ff9700 1 -- 172.21.15.46:0/3973101938 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c021070 con 0x7fea3c06c290 2024-05-09T00:11:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.314+0000 7fea41bed700 1 -- 172.21.15.46:0/3973101938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3c1a4a10 con 0x7fea3c06c290 2024-05-09T00:11:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.315+0000 7fea38ff9700 1 -- 172.21.15.46:0/3973101938 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2c004020 con 0x7fea3c06c290 2024-05-09T00:11:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.315+0000 7fea38ff9700 1 -- 172.21.15.46:0/3973101938 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c01d6b0 con 0x7fea3c06c290 2024-05-09T00:11:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.315+0000 7fea41bed700 1 -- 172.21.15.46:0/3973101938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3c1a4e00 con 0x7fea3c06c290 2024-05-09T00:11:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.316+0000 7fea41bed700 1 -- 172.21.15.46:0/3973101938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3c110580 con 0x7fea3c06c290 2024-05-09T00:11:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.317+0000 7fea38ff9700 1 -- 172.21.15.46:0/3973101938 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fea2c003a00 con 0x7fea3c06c290 2024-05-09T00:11:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.318+0000 7fea38ff9700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fea280b2880 0x7fea280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.319+0000 7fea3affd700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fea280b2880 0x7fea280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.319+0000 7fea38ff9700 1 -- 172.21.15.46:0/3973101938 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fea2c00f020 con 0x7fea3c06c290 2024-05-09T00:11:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.319+0000 7fea3affd700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fea280b2880 0x7fea280b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea24005fd0 tx=0x7fea24005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:21.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.322+0000 7fea38ff9700 1 -- 172.21.15.46:0/3973101938 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea2c02a020 con 0x7fea3c06c290 2024-05-09T00:11:21.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.545+0000 7fea41bed700 1 -- 172.21.15.46:0/3973101938 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fea3c106b40 con 0x7fea280b2880 2024-05-09T00:11:21.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.547+0000 7fea38ff9700 1 -- 172.21.15.46:0/3973101938 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fea3c106b40 con 0x7fea280b2880 2024-05-09T00:11:21.545 INFO:teuthology.orchestra.run.smithi046.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-09T00:11:21.545 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7512009A480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-09T00:11:21.545 INFO:teuthology.orchestra.run.smithi046.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-09T00:11:21.545 INFO:teuthology.orchestra.run.smithi046.stdout:smithi169 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300LN400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-09T00:11:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.549+0000 7fea41bed700 1 -- 172.21.15.46:0/3973101938 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fea280b2880 msgr2=0x7fea280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.549+0000 7fea41bed700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fea280b2880 0x7fea280b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea24005fd0 tx=0x7fea24005e40 comp rx=0 tx=0).stop 2024-05-09T00:11:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.549+0000 7fea41bed700 1 -- 172.21.15.46:0/3973101938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea3c06c290 msgr2=0x7fea3c104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.549+0000 7fea41bed700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea3c06c290 0x7fea3c104740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0062a0 tx=0x7fea2c004ef0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.549+0000 7fea41bed700 1 -- 172.21.15.46:0/3973101938 shutdown_connections 2024-05-09T00:11:21.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.549+0000 7fea41bed700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fea280b2880 0x7fea280b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.549+0000 7fea41bed700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea3c109ff0 0x7fea3c103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.549+0000 7fea41bed700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea3c06c290 0x7fea3c104740 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.549+0000 7fea41bed700 1 --2- 172.21.15.46:0/3973101938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea3c1074c0 0x7fea3c102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:21.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.549+0000 7fea41bed700 1 -- 172.21.15.46:0/3973101938 >> 172.21.15.46:0/3973101938 conn(0x7fea3c0fe880 msgr2=0x7fea3c10c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:21.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.550+0000 7fea41bed700 1 -- 172.21.15.46:0/3973101938 shutdown_connections 2024-05-09T00:11:21.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:21.550+0000 7fea41bed700 1 -- 172.21.15.46:0/3973101938 wait complete. 2024-05-09T00:11:22.323 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T00:11:22.334 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T00:11:22.334 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T00:11:22.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:22 smithi046 ceph-mon[30151]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:22.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:22 smithi046 ceph-mon[30151]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:22.592 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:22.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:22 smithi169 ceph-mon[34777]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:22.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:22 smithi169 ceph-mon[34777]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:22.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:22 smithi142 ceph-mon[34533]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:22.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:22 smithi142 ceph-mon[34533]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T00:11:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.376+0000 7f6f26b05700 1 -- 172.21.15.46:0/3253740246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f20109ff0 msgr2=0x7f6f2010c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.376+0000 7f6f26b05700 1 --2- 172.21.15.46:0/3253740246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f20109ff0 0x7f6f2010c500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009990 tx=0x7f6f18009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.376+0000 7f6f26b05700 1 -- 172.21.15.46:0/3253740246 shutdown_connections 2024-05-09T00:11:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.376+0000 7f6f26b05700 1 --2- 172.21.15.46:0/3253740246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f20109ff0 0x7f6f2010c500 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.376+0000 7f6f26b05700 1 --2- 172.21.15.46:0/3253740246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f2006c2d0 0x7f6f20106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.376+0000 7f6f26b05700 1 --2- 172.21.15.46:0/3253740246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f20107480 0x7f6f20109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.376+0000 7f6f26b05700 1 -- 172.21.15.46:0/3253740246 >> 172.21.15.46:0/3253740246 conn(0x7f6f200fe880 msgr2=0x7f6f20100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.376+0000 7f6f26b05700 1 -- 172.21.15.46:0/3253740246 shutdown_connections 2024-05-09T00:11:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.376+0000 7f6f26b05700 1 -- 172.21.15.46:0/3253740246 wait complete. 2024-05-09T00:11:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.377+0000 7f6f26b05700 1 Processor -- start 2024-05-09T00:11:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.377+0000 7f6f26b05700 1 -- start start 2024-05-09T00:11:24.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f26b05700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f2006c2d0 0x7f6f2019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:24.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f26b05700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f20107480 0x7f6f2019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:24.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f26b05700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f20109ff0 0x7f6f201a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:24.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f26b05700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2010fd80 con 0x7f6f20109ff0 2024-05-09T00:11:24.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f26b05700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2010fc00 con 0x7f6f2006c2d0 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f26b05700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2010ff00 con 0x7f6f20107480 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f250a2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f20109ff0 0x7f6f201a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f250a2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f20109ff0 0x7f6f201a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45790/0 (socket says 172.21.15.46:45790) 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f250a2700 1 -- 172.21.15.46:0/2972428204 learned_addr learned my addr 172.21.15.46:0/2972428204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f248a1700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f2006c2d0 0x7f6f2019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f1ffff700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f20107480 0x7f6f2019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f250a2700 1 -- 172.21.15.46:0/2972428204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f20107480 msgr2=0x7f6f2019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f250a2700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f20107480 0x7f6f2019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f250a2700 1 -- 172.21.15.46:0/2972428204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f2006c2d0 msgr2=0x7f6f2019a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f250a2700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f2006c2d0 0x7f6f2019a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f250a2700 1 -- 172.21.15.46:0/2972428204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f201a2340 con 0x7f6f20109ff0 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f1ffff700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f20107480 0x7f6f2019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.378+0000 7f6f250a2700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f20109ff0 0x7f6f201a1d50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f180038a0 tx=0x7f6f18009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:24.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.379+0000 7f6f1dffb700 1 -- 172.21.15.46:0/2972428204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18027070 con 0x7f6f20109ff0 2024-05-09T00:11:24.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.379+0000 7f6f26b05700 1 -- 172.21.15.46:0/2972428204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f201a25d0 con 0x7f6f20109ff0 2024-05-09T00:11:24.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.379+0000 7f6f1dffb700 1 -- 172.21.15.46:0/2972428204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f180040c0 con 0x7f6f20109ff0 2024-05-09T00:11:24.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.379+0000 7f6f1dffb700 1 -- 172.21.15.46:0/2972428204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18022e40 con 0x7f6f20109ff0 2024-05-09T00:11:24.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.379+0000 7f6f26b05700 1 -- 172.21.15.46:0/2972428204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f201a2a20 con 0x7f6f20109ff0 2024-05-09T00:11:24.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.380+0000 7f6f26b05700 1 -- 172.21.15.46:0/2972428204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f200512c0 con 0x7f6f20109ff0 2024-05-09T00:11:24.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.382+0000 7f6f1dffb700 1 -- 172.21.15.46:0/2972428204 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6f1802a070 con 0x7f6f20109ff0 2024-05-09T00:11:24.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.383+0000 7f6f1dffb700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6f0c0b2880 0x7f6f0c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:24.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.383+0000 7f6f248a1700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6f0c0b2880 0x7f6f0c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:24.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.383+0000 7f6f1dffb700 1 -- 172.21.15.46:0/2972428204 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6f180e3540 con 0x7f6f20109ff0 2024-05-09T00:11:24.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.384+0000 7f6f248a1700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6f0c0b2880 0x7f6f0c0b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000e270 tx=0x7f6f10009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:24.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.387+0000 7f6f1dffb700 1 -- 172.21.15.46:0/2972428204 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f18013070 con 0x7f6f20109ff0 2024-05-09T00:11:24.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.646+0000 7f6f26b05700 1 -- 172.21.15.46:0/2972428204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6f200643d0 con 0x7f6f20109ff0 2024-05-09T00:11:24.645 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:24 smithi046 ceph-mon[30151]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:24.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.647+0000 7f6f1dffb700 1 -- 172.21.15.46:0/2972428204 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3055 (secure 0 0 0) 0x7f6f180ab2b0 con 0x7f6f20109ff0 2024-05-09T00:11:24.645 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-05-09T00:11:24.645 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T00:11:24.645 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 25 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-05-09T00:08:28.009815+0000 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-05-09T00:08:47.423159+0000 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 300 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-05-09T00:11:24.646 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 3 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1,2 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=24499,1=24497,2=24487} 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data disabled 2024-05-09T00:11:24.647 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 0 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 24499 members: 24497,24499,24487 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.i{0:24499} state up:active seq 8 addr [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.f{0:24481} state up:standby-replay seq 1 addr [v2:172.21.15.169:6834/1842121683,v1:172.21.15.169:6835/1842121683] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b{1:24497} state up:active seq 8 addr [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.h{1:24470} state up:standby-replay seq 1 addr [v2:172.21.15.142:6836/3779152362,v1:172.21.15.142:6837/3779152362] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c{2:24487} state up:active seq 13 addr [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.e{2:24446} state up:standby-replay seq 1 addr [v2:172.21.15.142:6834/2701851507,v1:172.21.15.142:6835/2701851507] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.j{-1:14517} state up:standby seq 1 addr [v2:172.21.15.46:6834/1791470384,v1:172.21.15.46:6835/1791470384] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:11:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d{-1:14529} state up:standby seq 1 addr [v2:172.21.15.46:6836/4102968476,v1:172.21.15.46:6837/4102968476] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:11:24.649 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.g{-1:14574} state up:standby seq 1 addr [v2:172.21.15.46:6838/612832273,v1:172.21.15.46:6839/612832273] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:11:24.649 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a{-1:14586} state up:standby seq 1 addr [v2:172.21.15.46:6840/1387176303,v1:172.21.15.46:6841/1387176303] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:11:24.649 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.k{-1:24425} state up:standby seq 1 addr [v2:172.21.15.142:6832/512022880,v1:172.21.15.142:6833/512022880] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:11:24.649 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.l{-1:24442} state up:standby seq 1 addr [v2:172.21.15.169:6832/2870369857,v1:172.21.15.169:6833/2870369857] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T00:11:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.650+0000 7f6f26b05700 1 -- 172.21.15.46:0/2972428204 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6f0c0b2880 msgr2=0x7f6f0c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.650+0000 7f6f26b05700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6f0c0b2880 0x7f6f0c0b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000e270 tx=0x7f6f10009040 comp rx=0 tx=0).stop 2024-05-09T00:11:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.650+0000 7f6f26b05700 1 -- 172.21.15.46:0/2972428204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f20109ff0 msgr2=0x7f6f201a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.650+0000 7f6f26b05700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f20109ff0 0x7f6f201a1d50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f180038a0 tx=0x7f6f18009510 comp rx=0 tx=0).stop 2024-05-09T00:11:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.651+0000 7f6f26b05700 1 -- 172.21.15.46:0/2972428204 shutdown_connections 2024-05-09T00:11:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.651+0000 7f6f26b05700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6f0c0b2880 0x7f6f0c0b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.651+0000 7f6f26b05700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f20109ff0 0x7f6f201a1d50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.651+0000 7f6f26b05700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f2006c2d0 0x7f6f2019a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.651+0000 7f6f26b05700 1 --2- 172.21.15.46:0/2972428204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f20107480 0x7f6f2019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.651+0000 7f6f26b05700 1 -- 172.21.15.46:0/2972428204 >> 172.21.15.46:0/2972428204 conn(0x7f6f200fe880 msgr2=0x7f6f20105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.651+0000 7f6f26b05700 1 -- 172.21.15.46:0/2972428204 shutdown_connections 2024-05-09T00:11:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:24.651+0000 7f6f26b05700 1 -- 172.21.15.46:0/2972428204 wait complete. 2024-05-09T00:11:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T00:11:24.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:24 smithi169 ceph-mon[34777]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:24.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:24 smithi142 ceph-mon[34533]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:25.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-09T00:11:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:25.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:25 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2972428204' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T00:11:25.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:25 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2972428204' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T00:11:25.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:25 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2972428204' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T00:11:26.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:26 smithi046 ceph-mon[30151]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:26.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:26 smithi169 ceph-mon[34777]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:26.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:26 smithi142 ceph-mon[34533]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:27.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.455+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/3285097110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d941077b0 msgr2=0x7f6d94109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:27.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.455+0000 7f6d9a9eb700 1 --2- 172.21.15.46:0/3285097110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d941077b0 0x7f6d94109bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009990 tx=0x7f6d7c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:27.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.455+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/3285097110 shutdown_connections 2024-05-09T00:11:27.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.455+0000 7f6d9a9eb700 1 --2- 172.21.15.46:0/3285097110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9410a110 0x7f6d9410c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:27.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.455+0000 7f6d9a9eb700 1 --2- 172.21.15.46:0/3285097110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d94104e50 0x7f6d94107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.455+0000 7f6d9a9eb700 1 --2- 172.21.15.46:0/3285097110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d941077b0 0x7f6d94109bd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.455+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/3285097110 >> 172.21.15.46:0/3285097110 conn(0x7f6d940fe890 msgr2=0x7f6d94100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.455+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/3285097110 shutdown_connections 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.455+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/3285097110 wait complete. 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d9a9eb700 1 Processor -- start 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d9a9eb700 1 -- start start 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d9a9eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d94104e50 0x7f6d9419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d9a9eb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d941077b0 0x7f6d9419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d9a9eb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9410a110 0x7f6d941a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d9a9eb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9410fdb0 con 0x7f6d94104e50 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d9a9eb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9410fc30 con 0x7f6d941077b0 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d9a9eb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9410ff30 con 0x7f6d9410a110 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d94104e50 0x7f6d9419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d94104e50 0x7f6d9419a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39450/0 (socket says 172.21.15.46:39450) 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d93fff700 1 -- 172.21.15.46:0/1572627761 learned_addr learned my addr 172.21.15.46:0/1572627761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d8bfff700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d941077b0 0x7f6d9419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.456+0000 7f6d98f88700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9410a110 0x7f6d941a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.457+0000 7f6d93fff700 1 -- 172.21.15.46:0/1572627761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9410a110 msgr2=0x7f6d941a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.457+0000 7f6d93fff700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9410a110 0x7f6d941a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.457+0000 7f6d93fff700 1 -- 172.21.15.46:0/1572627761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d941077b0 msgr2=0x7f6d9419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.457+0000 7f6d93fff700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d941077b0 0x7f6d9419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.457+0000 7f6d93fff700 1 -- 172.21.15.46:0/1572627761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d941a2480 con 0x7f6d94104e50 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.457+0000 7f6d93fff700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d94104e50 0x7f6d9419a920 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400d980 tx=0x7f6d8400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.457+0000 7f6d91ffb700 1 -- 172.21.15.46:0/1572627761 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84017070 con 0x7f6d94104e50 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.457+0000 7f6d91ffb700 1 -- 172.21.15.46:0/1572627761 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d84009de0 con 0x7f6d94104e50 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.457+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/1572627761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d941a2770 con 0x7f6d94104e50 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.457+0000 7f6d91ffb700 1 -- 172.21.15.46:0/1572627761 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8400f3c0 con 0x7f6d94104e50 2024-05-09T00:11:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.457+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/1572627761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d941a2cb0 con 0x7f6d94104e50 2024-05-09T00:11:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.458+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/1572627761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d940512c0 con 0x7f6d94104e50 2024-05-09T00:11:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.460+0000 7f6d91ffb700 1 -- 172.21.15.46:0/1572627761 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6d84009930 con 0x7f6d94104e50 2024-05-09T00:11:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.461+0000 7f6d91ffb700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d800b2880 0x7f6d800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.461+0000 7f6d8bfff700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d800b2880 0x7f6d800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.461+0000 7f6d91ffb700 1 -- 172.21.15.46:0/1572627761 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6d840e20f0 con 0x7f6d94104e50 2024-05-09T00:11:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.462+0000 7f6d8bfff700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d800b2880 0x7f6d800b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00b5c0 tx=0x7f6d7c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:27.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.465+0000 7f6d91ffb700 1 -- 172.21.15.46:0/1572627761 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d840a9dd0 con 0x7f6d94104e50 2024-05-09T00:11:27.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.681+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/1572627761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f6d9402e5f0 con 0x7f6d94104e50 2024-05-09T00:11:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.682+0000 7f6d91ffb700 1 -- 172.21.15.46:0/1572627761 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v80) ==== 56+0+5304 (secure 0 0 0) 0x7f6d840a96d0 con 0x7f6d94104e50 2024-05-09T00:11:27.679 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 80 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:fsid eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-05-09T00:00:14.356524+0000 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-05-09T00:09:10.458026+0000 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:crush_version 26 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:full_ratio 0.95 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:backfillfull_ratio 0.9 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:nearfull_ratio 0.85 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:require_min_compat_client luminous 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:min_compat_client luminous 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:require_osd_release squid 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:stretch_mode_enabled false 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-09T00:11:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:max_osd 12 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6802/3727342125,v1:172.21.15.46:6803/3727342125] [v2:172.21.15.46:6804/3727342125,v1:172.21.15.46:6805/3727342125] exists,up 740d6388-5c1f-45eb-93b0-7583bfa5a225 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 up in weight 1 up_from 13 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/3156042469,v1:172.21.15.142:6801/3156042469] [v2:172.21.15.142:6802/3156042469,v1:172.21.15.142:6803/3156042469] exists,up 85fbf3ae-0be8-4504-84d6-66fae0d2169d 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 up in weight 1 up_from 18 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6800/2680673064,v1:172.21.15.169:6801/2680673064] [v2:172.21.15.169:6802/2680673064,v1:172.21.15.169:6803/2680673064] exists,up 05c070e7-83ad-4949-9834-4ce51ac0a2f1 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 up in weight 1 up_from 25 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6810/1158536261,v1:172.21.15.46:6811/1158536261] [v2:172.21.15.46:6812/1158536261,v1:172.21.15.46:6813/1158536261] exists,up 09d7bbe1-58b6-4407-bef2-920f179dfb9e 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 up in weight 1 up_from 31 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6808/2299616865,v1:172.21.15.142:6809/2299616865] [v2:172.21.15.142:6810/2299616865,v1:172.21.15.142:6811/2299616865] exists,up 66c9644b-6b0b-4fe7-aa63-b9d87bff7153 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 up in weight 1 up_from 36 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6808/445044713,v1:172.21.15.169:6809/445044713] [v2:172.21.15.169:6810/445044713,v1:172.21.15.169:6811/445044713] exists,up 96a72e01-1c5f-418e-bb9b-b790d9b3059b 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 up in weight 1 up_from 42 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6818/1394035482,v1:172.21.15.46:6819/1394035482] [v2:172.21.15.46:6820/1394035482,v1:172.21.15.46:6821/1394035482] exists,up 6b459542-0e56-4369-b542-e36c46a18f05 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 up in weight 1 up_from 47 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/596805607,v1:172.21.15.142:6817/596805607] [v2:172.21.15.142:6818/596805607,v1:172.21.15.142:6819/596805607] exists,up 786b4517-4cbd-4fa3-b5a0-39cccf839893 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 up in weight 1 up_from 53 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6816/2650652376,v1:172.21.15.169:6817/2650652376] [v2:172.21.15.169:6818/2650652376,v1:172.21.15.169:6819/2650652376] exists,up b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 up in weight 1 up_from 59 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6826/374861248,v1:172.21.15.46:6827/374861248] [v2:172.21.15.46:6828/374861248,v1:172.21.15.46:6829/374861248] exists,up 32df0dab-5875-4beb-b733-60a915c94a9b 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 up in weight 1 up_from 64 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/1364891982,v1:172.21.15.142:6825/1364891982] [v2:172.21.15.142:6826/1364891982,v1:172.21.15.142:6827/1364891982] exists,up 380a4f47-e528-4258-8836-32eac0e6e630 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 up in weight 1 up_from 70 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6824/4262478976,v1:172.21.15.169:6825/4262478976] [v2:172.21.15.169:6826/4262478976,v1:172.21.15.169:6827/4262478976] exists,up 56b04b15-6fcd-497a-aa66-19da9624b656 2024-05-09T00:11:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.5 [8,11] 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.a [2,11] 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.0 [4,8] 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.3 [4,9] 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.10 [5,9] 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.17 [4,7] 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.1b [5,9] 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.21 [4,6] 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.28 [3,6] 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/1620504773 expires 2024-05-10T00:01:09.191239+0000 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/1072138393 expires 2024-05-10T00:01:09.191239+0000 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/2786891364 expires 2024-05-10T00:00:41.833323+0000 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/7723180 expires 2024-05-10T00:00:41.833323+0000 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/626888490 expires 2024-05-10T00:01:09.191239+0000 2024-05-09T00:11:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/1620504773 expires 2024-05-10T00:01:09.191239+0000 2024-05-09T00:11:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/4236543921 expires 2024-05-10T00:01:09.191239+0000 2024-05-09T00:11:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/2168313126 expires 2024-05-10T00:00:41.833323+0000 2024-05-09T00:11:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/953818590 expires 2024-05-10T00:00:41.833323+0000 2024-05-09T00:11:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/953818590 expires 2024-05-10T00:00:41.833323+0000 2024-05-09T00:11:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.684+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/1572627761 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d800b2880 msgr2=0x7f6d800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.684+0000 7f6d9a9eb700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d800b2880 0x7f6d800b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00b5c0 tx=0x7f6d7c01a040 comp rx=0 tx=0).stop 2024-05-09T00:11:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.684+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/1572627761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d94104e50 msgr2=0x7f6d9419a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.684+0000 7f6d9a9eb700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d94104e50 0x7f6d9419a920 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400d980 tx=0x7f6d8400dc90 comp rx=0 tx=0).stop 2024-05-09T00:11:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.684+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/1572627761 shutdown_connections 2024-05-09T00:11:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.684+0000 7f6d9a9eb700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d800b2880 0x7f6d800b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.684+0000 7f6d9a9eb700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9410a110 0x7f6d941a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.684+0000 7f6d9a9eb700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d94104e50 0x7f6d9419a920 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.684+0000 7f6d9a9eb700 1 --2- 172.21.15.46:0/1572627761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d941077b0 0x7f6d9419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.684+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/1572627761 >> 172.21.15.46:0/1572627761 conn(0x7f6d940fe890 msgr2=0x7f6d941059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.684+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/1572627761 shutdown_connections 2024-05-09T00:11:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:27.684+0000 7f6d9a9eb700 1 -- 172.21.15.46:0/1572627761 wait complete. 2024-05-09T00:11:28.369 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-09T00:11:28.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:11:28.662 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:28.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:28 smithi046 ceph-mon[30151]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:28.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:28 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1572627761' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T00:11:28.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:28 smithi169 ceph-mon[34777]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:28.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:28 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1572627761' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T00:11:28.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:28 smithi142 ceph-mon[34533]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:28.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:28 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1572627761' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T00:11:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.488+0000 7f8b54d62700 1 -- 172.21.15.46:0/1731933729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b50107df0 msgr2=0x7f8b50108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.488+0000 7f8b54d62700 1 --2- 172.21.15.46:0/1731933729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b50107df0 0x7f8b50108200 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8b440099c0 tx=0x7f8b44009cd0 comp rx=0 tx=0).stop 2024-05-09T00:11:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.488+0000 7f8b54d62700 1 -- 172.21.15.46:0/1731933729 shutdown_connections 2024-05-09T00:11:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.488+0000 7f8b54d62700 1 --2- 172.21.15.46:0/1731933729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b50108740 0x7f8b501100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.488+0000 7f8b54d62700 1 --2- 172.21.15.46:0/1731933729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b501034c0 0x7f8b501038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.488+0000 7f8b54d62700 1 --2- 172.21.15.46:0/1731933729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b50107df0 0x7f8b50108200 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:30.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.488+0000 7f8b54d62700 1 -- 172.21.15.46:0/1731933729 >> 172.21.15.46:0/1731933729 conn(0x7f8b500fedc0 msgr2=0x7f8b501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:30.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.488+0000 7f8b54d62700 1 -- 172.21.15.46:0/1731933729 shutdown_connections 2024-05-09T00:11:30.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.488+0000 7f8b54d62700 1 -- 172.21.15.46:0/1731933729 wait complete. 2024-05-09T00:11:30.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.489+0000 7f8b54d62700 1 Processor -- start 2024-05-09T00:11:30.488 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:30 smithi046 ceph-mon[30151]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.489+0000 7f8b54d62700 1 -- start start 2024-05-09T00:11:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b54d62700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b501034c0 0x7f8b5019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b54d62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b50107df0 0x7f8b5019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b54d62700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b50108740 0x7f8b5019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b54d62700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8b501133c0 con 0x7f8b50107df0 2024-05-09T00:11:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b54d62700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50113240 con 0x7f8b501034c0 2024-05-09T00:11:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b54d62700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50113540 con 0x7f8b50108740 2024-05-09T00:11:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b4dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b50107df0 0x7f8b5019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b4dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b50107df0 0x7f8b5019b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39464/0 (socket says 172.21.15.46:39464) 2024-05-09T00:11:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b4dd9b700 1 -- 172.21.15.46:0/3743098041 learned_addr learned my addr 172.21.15.46:0/3743098041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b4e59c700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b501034c0 0x7f8b5019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b4ed9d700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b50108740 0x7f8b5019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b4dd9b700 1 -- 172.21.15.46:0/3743098041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b50108740 msgr2=0x7f8b5019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b4dd9b700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b50108740 0x7f8b5019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b4dd9b700 1 -- 172.21.15.46:0/3743098041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b501034c0 msgr2=0x7f8b5019aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b4dd9b700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b501034c0 0x7f8b5019aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.490+0000 7f8b4dd9b700 1 -- 172.21.15.46:0/3743098041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5019fd70 con 0x7f8b50107df0 2024-05-09T00:11:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.491+0000 7f8b4dd9b700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b50107df0 0x7f8b5019b440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8b44006010 tx=0x7f8b44003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.491+0000 7f8b3f7fe700 1 -- 172.21.15.46:0/3743098041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44003cb0 con 0x7f8b50107df0 2024-05-09T00:11:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.491+0000 7f8b54d62700 1 -- 172.21.15.46:0/3743098041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b501a0000 con 0x7f8b50107df0 2024-05-09T00:11:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.491+0000 7f8b3f7fe700 1 -- 172.21.15.46:0/3743098041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b44004830 con 0x7f8b50107df0 2024-05-09T00:11:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.491+0000 7f8b3f7fe700 1 -- 172.21.15.46:0/3743098041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b440054c0 con 0x7f8b50107df0 2024-05-09T00:11:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.491+0000 7f8b54d62700 1 -- 172.21.15.46:0/3743098041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b501a0480 con 0x7f8b50107df0 2024-05-09T00:11:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.492+0000 7f8b54d62700 1 -- 172.21.15.46:0/3743098041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b500512c0 con 0x7f8b50107df0 2024-05-09T00:11:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.494+0000 7f8b3f7fe700 1 -- 172.21.15.46:0/3743098041 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8b440049e0 con 0x7f8b50107df0 2024-05-09T00:11:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.495+0000 7f8b3f7fe700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8b380b2880 0x7f8b380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.495+0000 7f8b4e59c700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8b380b2880 0x7f8b380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.495+0000 7f8b3f7fe700 1 -- 172.21.15.46:0/3743098041 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8b440e3ea0 con 0x7f8b50107df0 2024-05-09T00:11:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.495+0000 7f8b4e59c700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8b380b2880 0x7f8b380b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8b40006fd0 tx=0x7f8b40008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:30.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.499+0000 7f8b3f7fe700 1 -- 172.21.15.46:0/3743098041 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8b44030070 con 0x7f8b50107df0 2024-05-09T00:11:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.758+0000 7f8b54d62700 1 -- 172.21.15.46:0/3743098041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b5002e5f0 con 0x7f8b50107df0 2024-05-09T00:11:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.759+0000 7f8b3f7fe700 1 -- 172.21.15.46:0/3743098041 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11879 (secure 0 0 0) 0x7f8b44013070 con 0x7f8b50107df0 2024-05-09T00:11:30.757 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:30.757 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:08:47.423159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24497,24499,24487]},"id":1}]} 2024-05-09T00:11:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.761+0000 7f8b54d62700 1 -- 172.21.15.46:0/3743098041 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8b380b2880 msgr2=0x7f8b380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.761+0000 7f8b54d62700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8b380b2880 0x7f8b380b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8b40006fd0 tx=0x7f8b40008040 comp rx=0 tx=0).stop 2024-05-09T00:11:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.761+0000 7f8b54d62700 1 -- 172.21.15.46:0/3743098041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b50107df0 msgr2=0x7f8b5019b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.761+0000 7f8b54d62700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b50107df0 0x7f8b5019b440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8b44006010 tx=0x7f8b44003730 comp rx=0 tx=0).stop 2024-05-09T00:11:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.761+0000 7f8b54d62700 1 -- 172.21.15.46:0/3743098041 shutdown_connections 2024-05-09T00:11:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.761+0000 7f8b54d62700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8b380b2880 0x7f8b380b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.761+0000 7f8b54d62700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b50108740 0x7f8b5019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.761+0000 7f8b54d62700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b501034c0 0x7f8b5019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.761+0000 7f8b54d62700 1 --2- 172.21.15.46:0/3743098041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b50107df0 0x7f8b5019b440 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.761+0000 7f8b54d62700 1 -- 172.21.15.46:0/3743098041 >> 172.21.15.46:0/3743098041 conn(0x7f8b500fedc0 msgr2=0x7f8b5006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.761+0000 7f8b54d62700 1 -- 172.21.15.46:0/3743098041 shutdown_connections 2024-05-09T00:11:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:30.761+0000 7f8b54d62700 1 -- 172.21.15.46:0/3743098041 wait complete. 2024-05-09T00:11:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T00:11:30.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:30 smithi169 ceph-mon[34777]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:30.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:30 smithi142 ceph-mon[34533]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:31.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:11:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:31.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:31 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3743098041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:31.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:31 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3743098041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:31.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:31 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3743098041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:32.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:32 smithi046 ceph-mon[30151]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:32.841 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:32 smithi142 ceph-mon[34533]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:32.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:32 smithi169 ceph-mon[34777]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.444+0000 7f7e8d04d700 1 -- 172.21.15.46:0/616353111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e88108f10 msgr2=0x7f7e88109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.444+0000 7f7e8d04d700 1 --2- 172.21.15.46:0/616353111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e88108f10 0x7f7e88109320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0099c0 tx=0x7f7e7c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:11:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.444+0000 7f7e8d04d700 1 -- 172.21.15.46:0/616353111 shutdown_connections 2024-05-09T00:11:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.444+0000 7f7e8d04d700 1 --2- 172.21.15.46:0/616353111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88109860 0x7f7e881111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.444+0000 7f7e8d04d700 1 --2- 172.21.15.46:0/616353111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e881045e0 0x7f7e881049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:33.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.444+0000 7f7e8d04d700 1 --2- 172.21.15.46:0/616353111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e88108f10 0x7f7e88109320 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:33.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.444+0000 7f7e8d04d700 1 -- 172.21.15.46:0/616353111 >> 172.21.15.46:0/616353111 conn(0x7f7e880ffed0 msgr2=0x7f7e88102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:33.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.444+0000 7f7e8d04d700 1 -- 172.21.15.46:0/616353111 shutdown_connections 2024-05-09T00:11:33.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.444+0000 7f7e8d04d700 1 -- 172.21.15.46:0/616353111 wait complete. 2024-05-09T00:11:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.445+0000 7f7e8d04d700 1 Processor -- start 2024-05-09T00:11:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.445+0000 7f7e8d04d700 1 -- start start 2024-05-09T00:11:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.445+0000 7f7e8d04d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e881045e0 0x7f7e88076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.445+0000 7f7e8d04d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88108f10 0x7f7e88077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.445+0000 7f7e8d04d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e88109860 0x7f7e88077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.445+0000 7f7e8d04d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e881144e0 con 0x7f7e881045e0 2024-05-09T00:11:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.445+0000 7f7e8d04d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88114360 con 0x7f7e88109860 2024-05-09T00:11:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.445+0000 7f7e8d04d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88114660 con 0x7f7e88108f10 2024-05-09T00:11:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e86d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e881045e0 0x7f7e88076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e86d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e881045e0 0x7f7e88076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39478/0 (socket says 172.21.15.46:39478) 2024-05-09T00:11:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e86d9d700 1 -- 172.21.15.46:0/1049035955 learned_addr learned my addr 172.21.15.46:0/1049035955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e8759e700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e88109860 0x7f7e88077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e8659c700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88108f10 0x7f7e88077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e86d9d700 1 -- 172.21.15.46:0/1049035955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88108f10 msgr2=0x7f7e88077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e86d9d700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88108f10 0x7f7e88077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e86d9d700 1 -- 172.21.15.46:0/1049035955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e88109860 msgr2=0x7f7e88077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e86d9d700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e88109860 0x7f7e88077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e86d9d700 1 -- 172.21.15.46:0/1049035955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e881a9180 con 0x7f7e881045e0 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e8659c700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88108f10 0x7f7e88077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e86d9d700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e881045e0 0x7f7e88076bd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800c9b0 tx=0x7f7e7800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e6ffff700 1 -- 172.21.15.46:0/1049035955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78007b00 con 0x7f7e881045e0 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e8d04d700 1 -- 172.21.15.46:0/1049035955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e881a9470 con 0x7f7e881045e0 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e6ffff700 1 -- 172.21.15.46:0/1049035955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e78007ca0 con 0x7f7e881045e0 2024-05-09T00:11:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e6ffff700 1 -- 172.21.15.46:0/1049035955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78018730 con 0x7f7e881045e0 2024-05-09T00:11:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.446+0000 7f7e8d04d700 1 -- 172.21.15.46:0/1049035955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e881a99b0 con 0x7f7e881045e0 2024-05-09T00:11:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.447+0000 7f7e8d04d700 1 -- 172.21.15.46:0/1049035955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e880512c0 con 0x7f7e881045e0 2024-05-09T00:11:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.449+0000 7f7e6ffff700 1 -- 172.21.15.46:0/1049035955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7e7800f450 con 0x7f7e881045e0 2024-05-09T00:11:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.450+0000 7f7e6ffff700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e700b2880 0x7f7e700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.450+0000 7f7e8659c700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e700b2880 0x7f7e700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.450+0000 7f7e6ffff700 1 -- 172.21.15.46:0/1049035955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7e780e25a0 con 0x7f7e881045e0 2024-05-09T00:11:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.450+0000 7f7e8659c700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e700b2880 0x7f7e700b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c006010 tx=0x7f7e7c005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.453+0000 7f7e6ffff700 1 -- 172.21.15.46:0/1049035955 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e78014030 con 0x7f7e881045e0 2024-05-09T00:11:33.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.717+0000 7f7e8d04d700 1 -- 172.21.15.46:0/1049035955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e88078100 con 0x7f7e881045e0 2024-05-09T00:11:33.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.718+0000 7f7e6ffff700 1 -- 172.21.15.46:0/1049035955 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11879 (secure 0 0 0) 0x7f7e780aa350 con 0x7f7e881045e0 2024-05-09T00:11:33.715 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:33.716 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:08:47.423159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24497,24499,24487]},"id":1}]} 2024-05-09T00:11:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.721+0000 7f7e8d04d700 1 -- 172.21.15.46:0/1049035955 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e700b2880 msgr2=0x7f7e700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.721+0000 7f7e8d04d700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e700b2880 0x7f7e700b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c006010 tx=0x7f7e7c005e80 comp rx=0 tx=0).stop 2024-05-09T00:11:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.721+0000 7f7e8d04d700 1 -- 172.21.15.46:0/1049035955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e881045e0 msgr2=0x7f7e88076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.721+0000 7f7e8d04d700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e881045e0 0x7f7e88076bd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800c9b0 tx=0x7f7e7800cd70 comp rx=0 tx=0).stop 2024-05-09T00:11:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.721+0000 7f7e8d04d700 1 -- 172.21.15.46:0/1049035955 shutdown_connections 2024-05-09T00:11:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.721+0000 7f7e8d04d700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e700b2880 0x7f7e700b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.721+0000 7f7e8d04d700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e88109860 0x7f7e88077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.721+0000 7f7e8d04d700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e881045e0 0x7f7e88076bd0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.721+0000 7f7e8d04d700 1 --2- 172.21.15.46:0/1049035955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88108f10 0x7f7e88077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.721+0000 7f7e8d04d700 1 -- 172.21.15.46:0/1049035955 >> 172.21.15.46:0/1049035955 conn(0x7f7e880ffed0 msgr2=0x7f7e8810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.721+0000 7f7e8d04d700 1 -- 172.21.15.46:0/1049035955 shutdown_connections 2024-05-09T00:11:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:33.721+0000 7f7e8d04d700 1 -- 172.21.15.46:0/1049035955 wait complete. 2024-05-09T00:11:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T00:11:34.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:11:34.703 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:34.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:34 smithi046 ceph-mon[30151]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:34.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:34 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1049035955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:34.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:34 smithi169 ceph-mon[34777]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:34.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:34 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1049035955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:34.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:34 smithi142 ceph-mon[34533]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:34.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:34 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1049035955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.488+0000 7f551d131700 1 -- 172.21.15.46:0/1954474066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55181045d0 msgr2=0x7f55181049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.488+0000 7f551d131700 1 --2- 172.21.15.46:0/1954474066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55181045d0 0x7f55181049e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f55080099c0 tx=0x7f5508009cd0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.489+0000 7f551d131700 1 -- 172.21.15.46:0/1954474066 shutdown_connections 2024-05-09T00:11:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.489+0000 7f551d131700 1 --2- 172.21.15.46:0/1954474066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5518109850 0x7f55181111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.489+0000 7f551d131700 1 --2- 172.21.15.46:0/1954474066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55181045d0 0x7f55181049e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.489+0000 7f551d131700 1 --2- 172.21.15.46:0/1954474066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5518108f00 0x7f5518109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.489+0000 7f551d131700 1 -- 172.21.15.46:0/1954474066 >> 172.21.15.46:0/1954474066 conn(0x7f55180ffe80 msgr2=0x7f55181022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.489+0000 7f551d131700 1 -- 172.21.15.46:0/1954474066 shutdown_connections 2024-05-09T00:11:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.489+0000 7f551d131700 1 -- 172.21.15.46:0/1954474066 wait complete. 2024-05-09T00:11:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.489+0000 7f551d131700 1 Processor -- start 2024-05-09T00:11:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f551d131700 1 -- start start 2024-05-09T00:11:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f551d131700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55181045d0 0x7f551819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f551d131700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5518108f00 0x7f551819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:36.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f551d131700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518109850 0x7f551819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:36.489 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:36 smithi046 ceph-mon[30151]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:36.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f551d131700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f55181144d0 con 0x7f5518109850 2024-05-09T00:11:36.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f551d131700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5518114350 con 0x7f55181045d0 2024-05-09T00:11:36.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f551d131700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5518114650 con 0x7f5518108f00 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f551759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518109850 0x7f551819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f551659c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5518108f00 0x7f551819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f551759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518109850 0x7f551819f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40656/0 (socket says 172.21.15.46:40656) 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f551659c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5518108f00 0x7f551819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:59950/0 (socket says 172.21.15.46:59950) 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f551759e700 1 -- 172.21.15.46:0/3925889387 learned_addr learned my addr 172.21.15.46:0/3925889387 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.490+0000 7f5516d9d700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55181045d0 0x7f551819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f5516d9d700 1 -- 172.21.15.46:0/3925889387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5518108f00 msgr2=0x7f551819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f5516d9d700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5518108f00 0x7f551819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f5516d9d700 1 -- 172.21.15.46:0/3925889387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518109850 msgr2=0x7f551819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f5516d9d700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518109850 0x7f551819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f5516d9d700 1 -- 172.21.15.46:0/3925889387 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551819fcf0 con 0x7f55181045d0 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f551759e700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518109850 0x7f551819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f551659c700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5518108f00 0x7f551819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f5516d9d700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55181045d0 0x7f551819ae20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f55080098e0 tx=0x7f55080045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f54fffff700 1 -- 172.21.15.46:0/3925889387 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508009e60 con 0x7f55181045d0 2024-05-09T00:11:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f54fffff700 1 -- 172.21.15.46:0/3925889387 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5508005030 con 0x7f55181045d0 2024-05-09T00:11:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f551d131700 1 -- 172.21.15.46:0/3925889387 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551819ff80 con 0x7f55181045d0 2024-05-09T00:11:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f54fffff700 1 -- 172.21.15.46:0/3925889387 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55080049f0 con 0x7f55181045d0 2024-05-09T00:11:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.491+0000 7f551d131700 1 -- 172.21.15.46:0/3925889387 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55181a03b0 con 0x7f55181045d0 2024-05-09T00:11:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.493+0000 7f551d131700 1 -- 172.21.15.46:0/3925889387 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55180512c0 con 0x7f55181045d0 2024-05-09T00:11:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.494+0000 7f54fffff700 1 -- 172.21.15.46:0/3925889387 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5508028060 con 0x7f55181045d0 2024-05-09T00:11:36.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.494+0000 7f54fffff700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f55000b2880 0x7f55000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:36.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.495+0000 7f551659c700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f55000b2880 0x7f55000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:36.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.495+0000 7f54fffff700 1 -- 172.21.15.46:0/3925889387 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f55080e27e0 con 0x7f55181045d0 2024-05-09T00:11:36.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.495+0000 7f551659c700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f55000b2880 0x7f55000b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5518107d10 tx=0x7f550c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.500+0000 7f54fffff700 1 -- 172.21.15.46:0/3925889387 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5508013070 con 0x7f55181045d0 2024-05-09T00:11:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.715+0000 7f551d131700 1 -- 172.21.15.46:0/3925889387 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f551802e5f0 con 0x7f55181045d0 2024-05-09T00:11:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.717+0000 7f54fffff700 1 -- 172.21.15.46:0/3925889387 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22694 (secure 0 0 0) 0x7f55080aa590 con 0x7f55181045d0 2024-05-09T00:11:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:36.716 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","created":"2024-05-09T00:00:14.356524+0000","modified":"2024-05-09T00:09:10.458026+0000","last_up_change":"2024-05-09T00:07:23.332796+0000","last_in_change":"2024-05-09T00:07:03.238539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:03:33.278910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:08:21.014416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:08:24.922968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"740d6388-5c1f-45eb-93b0-7583bfa5a225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6803","nonce":3727342125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6805","nonce":3727342125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6809","nonce":3727342125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6807","nonce":3727342125}]},"public_addr":"172.21.15.46:6803/3727342125","cluster_addr":"172.21.15.46:6805/3727342125","heartbeat_back_addr":"172.21.15.46:6809/3727342125","heartbeat_front_addr":"172.21.15.46:6807/3727342125","state":["exists","up"]},{"osd":1,"uuid":"85fbf3ae-0be8-4504-84d6-66fae0d2169d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6801","nonce":3156042469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6803","nonce":3156042469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6807","nonce":3156042469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6805","nonce":3156042469}]},"public_addr":"172.21.15.142:6801/3156042469","cluster_addr":"172.21.15.142:6803/3156042469","heartbeat_back_addr":"172.21.15.142:6807/3156042469","heartbeat_front_addr":"172.21.15.142:6805/3156042469","state":["exists","up"]},{"osd":2,"uuid":"05c070e7-83ad-4949-9834-4ce51ac0a2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6801","nonce":2680673064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6803","nonce":2680673064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6807","nonce":2680673064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6805","nonce":2680673064}]},"public_addr":"172.21.15.169:6801/2680673064","cluster_addr":"172.21.15.169:6803/2680673064","heartbeat_back_addr":"172.21.15.169:6807/2680673064","heartbeat_front_addr":"172.21.15.169:6805/2680673064","state":["exists","up"]},{"osd":3,"uuid":"09d7bbe1-58b6-4407-bef2-920f179dfb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6811","nonce":1158536261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6813","nonce":1158536261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6817","nonce":1158536261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6815","nonce":1158536261}]},"public_addr":"172.21.15.46:6811/1158536261","cluster_addr":"172.21.15.46:6813/1158536261","heartbeat_back_addr":"172.21.15.46:6817/1158536261","heartbeat_front_addr":"172.21.15.46:6815/1158536261","state":["exists","up"]},{"osd":4,"uuid":"66c9644b-6b0b-4fe7-aa63-b9d87bff7153","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6809","nonce":2299616865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6811","nonce":2299616865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6815","nonce":2299616865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6813","nonce":2299616865}]},"public_addr":"172.21.15.142:6809/2299616865","cluster_addr":"172.21.15.142:6811/2299616865","heartbeat_back_addr":"172.21.15.142:6815/2299616865","heartbeat_front_addr":"172.21.15.142:6813/2299616865","state":["exists","up"]},{"osd":5,"uuid":"96a72e01-1c5f-418e-bb9b-b790d9b3059b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6809","nonce":445044713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6811","nonce":445044713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6815","nonce":445044713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6813","nonce":445044713}]},"public_addr":"172.21.15.169:6809/445044713","cluster_addr":"172.21.15.169:6811/445044713","heartbeat_back_addr":"172.21.15.169:6815/445044713","heartbeat_front_addr":"172.21.15.169:6813/445044713","state":["exists","up"]},{"osd":6,"uuid":"6b459542-0e56-4369-b542-e36c46a18f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6819","nonce":1394035482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6821","nonce":1394035482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6825","nonce":1394035482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6823","nonce":1394035482}]},"public_addr":"172.21.15.46:6819/1394035482","cluster_addr":"172.21.15.46:6821/1394035482","heartbeat_back_addr":"172.21.15.46:6825/1394035482","heartbeat_front_addr":"172.21.15.46:6823/1394035482","state":["exists","up"]},{"osd":7,"uuid":"786b4517-4cbd-4fa3-b5a0-39cccf839893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6817","nonce":596805607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6819","nonce":596805607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6823","nonce":596805607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6821","nonce":596805607}]},"public_addr":"172.21.15.142:6817/596805607","cluster_addr":"172.21.15.142:6819/596805607","heartbeat_back_addr":"172.21.15.142:6823/596805607","heartbeat_front_addr":"172.21.15.142:6821/596805607","state":["exists","up"]},{"osd":8,"uuid":"b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6817","nonce":2650652376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6819","nonce":2650652376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6823","nonce":2650652376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6821","nonce":2650652376}]},"public_addr":"172.21.15.169:6817/2650652376","cluster_addr":"172.21.15.169:6819/2650652376","heartbeat_back_addr":"172.21.15.169:6823/2650652376","heartbeat_front_addr":"172.21.15.169:6821/2650652376","state":["exists","up"]},{"osd":9,"uuid":"32df0dab-5875-4beb-b733-60a915c94a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6827","nonce":374861248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6829","nonce":374861248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6833","nonce":374861248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6831","nonce":374861248}]},"public_addr":"172.21.15.46:6827/374861248","cluster_addr":"172.21.15.46:6829/374861248","heartbeat_back_addr":"172.21.15.46:6833/374861248","heartbeat_front_addr":"172.21.15.46:6831/374861248","state":["exists","up"]},{"osd":10,"uuid":"380a4f47-e528-4258-8836-32eac0e6e630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6825","nonce":1364891982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6827","nonce":1364891982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6831","nonce":1364891982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6829","nonce":1364891982}]},"public_addr":"172.21.15.142:6825/1364891982","cluster_addr":"172.21.15.142:6827/1364891982","heartbeat_back_addr":"172.21.15.142:6831/1364891982","heartbeat_front_addr":"172.21.15.142:6829/1364891982","state":["exists","up"]},{"osd":11,"uuid":"56b04b15-6fcd-497a-aa66-19da9624b656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6825","nonce":4262478976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6827","nonce":4262478976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6831","nonce":4262478976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6829","nonce":4262478976}]},"public_addr":"172.21.15.169:6825/4262478976","cluster_addr":"172.21.15.169:6827/4262478976","heartbeat_back_addr":"172.21.15.169:6831/4262478976","heartbeat_front_addr":"172.21.15.169:6829/4262478976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:02:42.159298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:06.053328+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:29.587137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:54.573782+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:20.276073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:45.320565+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:10.985522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:36.806137+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:02.266736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:29.788773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:55.827639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:07:21.762382+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/1072138393":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2786891364":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/7723180":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/626888490":"2024-05-10T00:01:09.191239+0000","172.21.15.46:6800/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/4236543921":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2168313126":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6800/953818590":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6801/953818590":"2024-05-10T00:00:41.833323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:11:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.720+0000 7f551d131700 1 -- 172.21.15.46:0/3925889387 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f55000b2880 msgr2=0x7f55000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.720+0000 7f551d131700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f55000b2880 0x7f55000b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5518107d10 tx=0x7f550c009260 comp rx=0 tx=0).stop 2024-05-09T00:11:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.720+0000 7f551d131700 1 -- 172.21.15.46:0/3925889387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55181045d0 msgr2=0x7f551819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.720+0000 7f551d131700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55181045d0 0x7f551819ae20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f55080098e0 tx=0x7f55080045e0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.720+0000 7f551d131700 1 -- 172.21.15.46:0/3925889387 shutdown_connections 2024-05-09T00:11:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.720+0000 7f551d131700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f55000b2880 0x7f55000b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.720+0000 7f551d131700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518109850 0x7f551819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.720+0000 7f551d131700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55181045d0 0x7f551819ae20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.720+0000 7f551d131700 1 --2- 172.21.15.46:0/3925889387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5518108f00 0x7f551819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.720+0000 7f551d131700 1 -- 172.21.15.46:0/3925889387 >> 172.21.15.46:0/3925889387 conn(0x7f55180ffe80 msgr2=0x7f55181022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.720+0000 7f551d131700 1 -- 172.21.15.46:0/3925889387 shutdown_connections 2024-05-09T00:11:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:36.720+0000 7f551d131700 1 -- 172.21.15.46:0/3925889387 wait complete. 2024-05-09T00:11:36.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:36 smithi169 ceph-mon[34777]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:36.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:36 smithi142 ceph-mon[34533]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:37.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T00:08:28.009815+0000', 'modified': '2024-05-09T00:08:47.423159+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24499, 'mds_1': 24497, 'mds_2': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/2701851507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2701851507}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 2701851507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6837/3779152362', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 3779152362}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 3779152362}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6835/1842121683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 1842121683}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 1842121683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.169:6837/1024289476', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1024289476}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1024289476}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.142:6839/3645430666', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3645430666}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3645430666}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.169:6839/1664161265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 1664161265}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 1664161265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24499, 'qdb_cluster': [24497, 24499, 24487]} 2024-05-09T00:11:37.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-09T00:11:37.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-09T00:11:37.730 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:37.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:37 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3925889387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:11:37.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:37 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3925889387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:11:37.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:37 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3925889387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:11:38.754 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:38 smithi169 ceph-mon[34777]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:38.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:38 smithi046 ceph-mon[30151]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:38.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:38 smithi142 ceph-mon[34533]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:39.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.460+0000 7f3c1708f700 1 -- 172.21.15.46:0/532740594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c10108740 msgr2=0x7f3c101100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:39.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.460+0000 7f3c1708f700 1 --2- 172.21.15.46:0/532740594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c10108740 0x7f3c101100a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800b0d0 tx=0x7f3c0800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.460+0000 7f3c1708f700 1 -- 172.21.15.46:0/532740594 shutdown_connections 2024-05-09T00:11:39.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.460+0000 7f3c1708f700 1 --2- 172.21.15.46:0/532740594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c10108740 0x7f3c101100a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.460+0000 7f3c1708f700 1 --2- 172.21.15.46:0/532740594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c101034c0 0x7f3c101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.460+0000 7f3c1708f700 1 --2- 172.21.15.46:0/532740594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c10107df0 0x7f3c10108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.460+0000 7f3c1708f700 1 -- 172.21.15.46:0/532740594 >> 172.21.15.46:0/532740594 conn(0x7f3c100fedc0 msgr2=0x7f3c101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:39.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.460+0000 7f3c1708f700 1 -- 172.21.15.46:0/532740594 shutdown_connections 2024-05-09T00:11:39.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.460+0000 7f3c1708f700 1 -- 172.21.15.46:0/532740594 wait complete. 2024-05-09T00:11:39.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.460+0000 7f3c1708f700 1 Processor -- start 2024-05-09T00:11:39.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c1708f700 1 -- start start 2024-05-09T00:11:39.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c1708f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c101034c0 0x7f3c1019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c1708f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c10107df0 0x7f3c1019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c1708f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c10108740 0x7f3c1019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c1708f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10113370 con 0x7f3c10107df0 2024-05-09T00:11:39.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c1708f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101131f0 con 0x7f3c10108740 2024-05-09T00:11:39.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c1708f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101134f0 con 0x7f3c101034c0 2024-05-09T00:11:39.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c0ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c10107df0 0x7f3c1019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c0ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c10107df0 0x7f3c1019c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40670/0 (socket says 172.21.15.46:40670) 2024-05-09T00:11:39.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c0ffff700 1 -- 172.21.15.46:0/3574556844 learned_addr learned my addr 172.21.15.46:0/3574556844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:39.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c1562c700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c10108740 0x7f3c1019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c101034c0 0x7f3c1019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c0ffff700 1 -- 172.21.15.46:0/3574556844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c101034c0 msgr2=0x7f3c1019bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:39.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c0ffff700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c101034c0 0x7f3c1019bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c0ffff700 1 -- 172.21.15.46:0/3574556844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c10108740 msgr2=0x7f3c1019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:39.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c0ffff700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c10108740 0x7f3c1019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c0ffff700 1 -- 172.21.15.46:0/3574556844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c101a6f60 con 0x7f3c10107df0 2024-05-09T00:11:39.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.461+0000 7f3c1562c700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c10108740 0x7f3c1019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:39.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.462+0000 7f3c0ffff700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c10107df0 0x7f3c1019c150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800e970 tx=0x7f3bf800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:39.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.462+0000 7f3c0dffb700 1 -- 172.21.15.46:0/3574556844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8004a30 con 0x7f3c10107df0 2024-05-09T00:11:39.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.462+0000 7f3c0dffb700 1 -- 172.21.15.46:0/3574556844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bf800cc90 con 0x7f3c10107df0 2024-05-09T00:11:39.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.462+0000 7f3c1708f700 1 -- 172.21.15.46:0/3574556844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c101a7250 con 0x7f3c10107df0 2024-05-09T00:11:39.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.462+0000 7f3c0dffb700 1 -- 172.21.15.46:0/3574556844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8005610 con 0x7f3c10107df0 2024-05-09T00:11:39.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.462+0000 7f3c1708f700 1 -- 172.21.15.46:0/3574556844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c101a7790 con 0x7f3c10107df0 2024-05-09T00:11:39.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.465+0000 7f3c0dffb700 1 -- 172.21.15.46:0/3574556844 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3bf800c7e0 con 0x7f3c10107df0 2024-05-09T00:11:39.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c0dffb700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3bfc0b2880 0x7f3bfc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c25aa5b80 1 Processor -- start 2024-05-09T00:11:39.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c25aa5b80 1 -- start start 2024-05-09T00:11:39.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3bfc0b2880 0x7f3bfc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c0dffb700 1 -- 172.21.15.46:0/3574556844 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f3bf80e2350 con 0x7f3c10107df0 2024-05-09T00:11:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c25aa5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b2499cb0 0x5604b25a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c25aa5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b241f450 0x5604b25a2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c25aa5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b259c3b0 0x5604b259c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c25aa5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604b24bf050 con 0x5604b2499cb0 2024-05-09T00:11:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c25aa5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5604b2464950 con 0x5604b241f450 2024-05-09T00:11:39.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c25aa5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5604b24bb010 con 0x5604b259c3b0 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c14e2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b2499cb0 0x5604b25a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c14e2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b2499cb0 0x5604b25a21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40680/0 (socket says 172.21.15.46:40680) 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c14e2b700 1 -- 172.21.15.46:0/2917216451 learned_addr learned my addr 172.21.15.46:0/2917216451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c0ffff700 1 --2- 172.21.15.46:0/2917216451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b241f450 0x5604b25a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.466+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3bfc0b2880 0x7f3bfc0b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3c00005950 tx=0x7f3c0000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.467+0000 7f3c1562c700 1 --2- 172.21.15.46:0/2917216451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b259c3b0 0x5604b259c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.467+0000 7f3c14e2b700 1 -- 172.21.15.46:0/2917216451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b259c3b0 msgr2=0x5604b259c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.467+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/2917216451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b259c3b0 0x5604b259c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.467+0000 7f3c14e2b700 1 -- 172.21.15.46:0/2917216451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b241f450 msgr2=0x5604b25a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.467+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/2917216451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b241f450 0x5604b25a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.467+0000 7f3c14e2b700 1 -- 172.21.15.46:0/2917216451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b24554b0 con 0x5604b2499cb0 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.467+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/2917216451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b2499cb0 0x5604b25a21f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000fdf0 tx=0x7f3c00012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4262d141c59d07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.467+0000 7f3c06ffd700 1 -- 172.21.15.46:0/2917216451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00016070 con 0x5604b2499cb0 2024-05-09T00:11:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.467+0000 7f3c06ffd700 1 -- 172.21.15.46:0/2917216451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c00010d70 con 0x5604b2499cb0 2024-05-09T00:11:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.467+0000 7f3c06ffd700 1 -- 172.21.15.46:0/2917216451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0001ce00 con 0x5604b2499cb0 2024-05-09T00:11:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.468+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/2917216451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b2499cb0 msgr2=0x5604b25a21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.468+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/2917216451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b2499cb0 0x5604b25a21f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000fdf0 tx=0x7f3c00012000 comp rx=0 tx=0).stop 2024-05-09T00:11:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.468+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/2917216451 shutdown_connections 2024-05-09T00:11:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.468+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/2917216451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b259c3b0 0x5604b259c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.468+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/2917216451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b2499cb0 0x5604b25a21f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.468+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/2917216451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b241f450 0x5604b25a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.468+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/2917216451 >> 172.21.15.46:0/2917216451 conn(0x5604b2422f20 msgr2=0x5604b25971d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.468+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/2917216451 shutdown_connections 2024-05-09T00:11:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.468+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/2917216451 wait complete. 2024-05-09T00:11:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.469+0000 7f3c25aa5b80 1 -- start start 2024-05-09T00:11:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.469+0000 7f3c25aa5b80 1 Processor -- start 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.469+0000 7f3c25aa5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b241f450 0x5604b25a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.469+0000 7f3c25aa5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 0x5604b25a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.469+0000 7f3c25aa5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b259c3b0 0x5604b25a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.469+0000 7f3c25aa5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604b259d110 con 0x5604b241f450 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.469+0000 7f3c25aa5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5604b2459ad0 con 0x5604b2499cb0 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.469+0000 7f3c25aa5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5604b25a8fd0 con 0x5604b259c3b0 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.469+0000 7f3c25aa5b80 10 client.0 _ll_register_callbacks cb 0x5604b2330670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.469+0000 7f3c14e2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b241f450 0x5604b25a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.469+0000 7f3c14e2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b241f450 0x5604b25a5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40690/0 (socket says 172.21.15.46:40690) 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.469+0000 7f3c14e2b700 1 -- 172.21.15.46:0/1749024807 learned_addr learned my addr 172.21.15.46:0/1749024807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3c0ffff700 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 0x5604b25a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3c1562c700 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b259c3b0 0x5604b25a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3c14e2b700 1 -- 172.21.15.46:0/1749024807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b259c3b0 msgr2=0x5604b25a6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b259c3b0 0x5604b25a6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3c14e2b700 1 -- 172.21.15.46:0/1749024807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 msgr2=0x5604b25a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 0x5604b25a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3c14e2b700 1 -- 172.21.15.46:0/1749024807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b233a440 con 0x5604b241f450 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b241f450 0x5604b25a5fa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3c0001c980 tx=0x7f3c00010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f67efbea3db706 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3beffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3beffff700 1 -- 172.21.15.46:0/1749024807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00032060 con 0x5604b241f450 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3beffff700 1 -- 172.21.15.46:0/1749024807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f3c0001c630 con 0x5604b241f450 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1749024807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c00010d70 con 0x5604b241f450 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3beffff700 1 -- 172.21.15.46:0/1749024807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3c00016040 con 0x5604b241f450 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3c25aa5b80 10 client.15075 fetch_fsmap learned FSMap version 25 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.470+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1749024807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5604b25b3920 con 0x5604b241f450 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.471+0000 7f3beffff700 1 -- 172.21.15.46:0/1749024807 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c00010d70 con 0x5604b241f450 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.471+0000 7f3beffff700 1 -- 172.21.15.46:0/1749024807 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3c00037070 con 0x5604b241f450 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.471+0000 7f3c25aa5b80 10 client.15075 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.471+0000 7f3c25aa5b80 10 client.15075 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T00:11:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.471+0000 7f3c25aa5b80 20 client.15075 populate_metadata read hostname 'smithi046' 2024-05-09T00:11:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.471+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5604b25b5ac0 0x5604b25b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.471+0000 7f3c25aa5b80 4 client.15075 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.471+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1749024807 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604b24cca30 con 0x5604b25b5ac0 2024-05-09T00:11:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.471+0000 7f3c0ffff700 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5604b25b5ac0 0x5604b25b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.472+0000 7f3beffff700 10 client.15075 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:11:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.472+0000 7f3c0ffff700 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5604b25b5ac0 0x5604b25b4d20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74a02ca0ad24c7c4 server_cookie=9a8da1782693c069 in_seq=0 out_seq=0 2024-05-09T00:11:39.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.472+0000 7f3beffff700 1 -- 172.21.15.46:0/1749024807 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5604b24cca30 con 0x5604b25b5ac0 2024-05-09T00:11:39.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.472+0000 7f3beffff700 10 client.15075 handle_command_reply: tid=0 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.472+0000 7f3c25aa5b80 1 client.15075 shutdown 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.473+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1749024807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b241f450 msgr2=0x5604b25a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.473+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b241f450 0x5604b25a5fa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3c0001c980 tx=0x7f3c00010b70 comp rx=0 tx=0).stop 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.473+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1749024807 shutdown_connections 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.473+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5604b25b5ac0 0x5604b25b4d20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.473+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b259c3b0 0x5604b25a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.473+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b241f450 0x5604b25a5fa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.473+0000 7f3beffff700 0 client.15075 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.473+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/1749024807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 0x5604b25a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.473+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1749024807 >> 172.21.15.46:0/1749024807 conn(0x5604b2422f20 msgr2=0x5604b2590fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.473+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1749024807 shutdown_connections 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.473+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1749024807 wait complete. 2024-05-09T00:11:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.473+0000 7f3c25aa5b80 20 client.15075 trim_cache size 0 max 16384 2024-05-09T00:11:39.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.495+0000 7f3c25aa5b80 1 Processor -- start 2024-05-09T00:11:39.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c25aa5b80 1 -- start start 2024-05-09T00:11:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c25aa5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b241f450 0x5604b25a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c25aa5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 0x5604b25919d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c25aa5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 0x5604b25a1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c25aa5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604b25b5270 con 0x5604b259c3b0 2024-05-09T00:11:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c25aa5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5604b25b5940 con 0x5604b2499cb0 2024-05-09T00:11:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c25aa5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5604b25b6010 con 0x5604b241f450 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c1562c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 0x5604b25a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c1562c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 0x5604b25a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40700/0 (socket says 172.21.15.46:40700) 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c1562c700 1 -- 172.21.15.46:0/2715036928 learned_addr learned my addr 172.21.15.46:0/2715036928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/2715036928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b241f450 0x5604b25a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c0ffff700 1 --2- 172.21.15.46:0/2715036928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 0x5604b25919d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c1562c700 1 -- 172.21.15.46:0/2715036928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b241f450 msgr2=0x5604b25a4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c1562c700 1 --2- 172.21.15.46:0/2715036928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b241f450 0x5604b25a4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c1562c700 1 -- 172.21.15.46:0/2715036928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 msgr2=0x5604b25919d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c1562c700 1 --2- 172.21.15.46:0/2715036928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 0x5604b25919d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.496+0000 7f3c1562c700 1 -- 172.21.15.46:0/2715036928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b259d360 con 0x5604b259c3b0 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.497+0000 7f3c1562c700 1 --2- 172.21.15.46:0/2715036928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 0x5604b25a1720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800b0d0 tx=0x7f3c08009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f10fa87cb21964 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.497+0000 7f3c067fc700 1 -- 172.21.15.46:0/2715036928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0802b070 con 0x5604b259c3b0 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.497+0000 7f3c067fc700 1 -- 172.21.15.46:0/2715036928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c08003c00 con 0x5604b259c3b0 2024-05-09T00:11:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.497+0000 7f3c067fc700 1 -- 172.21.15.46:0/2715036928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c080052b0 con 0x5604b259c3b0 2024-05-09T00:11:39.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.498+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/2715036928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 msgr2=0x5604b25a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:39.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.498+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/2715036928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 0x5604b25a1720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800b0d0 tx=0x7f3c08009520 comp rx=0 tx=0).stop 2024-05-09T00:11:39.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.498+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/2715036928 shutdown_connections 2024-05-09T00:11:39.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.498+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/2715036928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 0x5604b25a1720 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.498+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/2715036928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b241f450 0x5604b25a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.498+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/2715036928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 0x5604b25919d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.498+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/2715036928 >> 172.21.15.46:0/2715036928 conn(0x5604b2422f20 msgr2=0x5604b259b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:39.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.498+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/2715036928 shutdown_connections 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.498+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/2715036928 wait complete. 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.499+0000 7f3c25aa5b80 1 -- start start 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.499+0000 7f3c25aa5b80 1 Processor -- start 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.499+0000 7f3c25aa5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b241f450 0x5604b25ba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.499+0000 7f3c25aa5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 0x5604b25bab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.499+0000 7f3c25aa5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 0x5604b25beba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.499+0000 7f3c25aa5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604b25b6db0 con 0x5604b259c3b0 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.499+0000 7f3c25aa5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5604b25b66e0 con 0x5604b2499cb0 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.499+0000 7f3c25aa5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5604b25b7480 con 0x5604b241f450 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.499+0000 7f3c25aa5b80 10 client.0 _ll_register_callbacks cb 0x5604b2330670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c1562c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 0x5604b25beba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c1562c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 0x5604b25beba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40706/0 (socket says 172.21.15.46:40706) 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c1562c700 1 -- 172.21.15.46:0/1312271500 learned_addr learned my addr 172.21.15.46:0/1312271500 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c0ffff700 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 0x5604b25bab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b241f450 0x5604b25ba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c1562c700 1 -- 172.21.15.46:0/1312271500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b241f450 msgr2=0x5604b25ba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c1562c700 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b241f450 0x5604b25ba5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c1562c700 1 -- 172.21.15.46:0/1312271500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 msgr2=0x5604b25bab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c1562c700 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 0x5604b25bab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c1562c700 1 -- 172.21.15.46:0/1312271500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b2593980 con 0x5604b259c3b0 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c1562c700 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 0x5604b25beba0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c08029040 tx=0x7f3c08006140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a61bf1645410df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3beffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3beffff700 1 -- 172.21.15.46:0/1312271500 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0802b040 con 0x5604b259c3b0 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3beffff700 1 -- 172.21.15.46:0/1312271500 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f3c080050c0 con 0x5604b259c3b0 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1312271500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c08003c00 con 0x5604b259c3b0 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3beffff700 1 -- 172.21.15.46:0/1312271500 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3c080052c0 con 0x5604b259c3b0 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c25aa5b80 10 client.15081 fetch_fsmap learned FSMap version 25 2024-05-09T00:11:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.500+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1312271500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5604b2621410 con 0x5604b259c3b0 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.501+0000 7f3beffff700 1 -- 172.21.15.46:0/1312271500 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c08003c00 con 0x5604b259c3b0 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.501+0000 7f3beffff700 1 -- 172.21.15.46:0/1312271500 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3c08025030 con 0x5604b259c3b0 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.501+0000 7f3c25aa5b80 10 client.15081 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.501+0000 7f3c25aa5b80 10 client.15081 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.501+0000 7f3c25aa5b80 20 client.15081 populate_metadata read hostname 'smithi046' 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.501+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5604b26232d0 0x5604b2624740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.501+0000 7f3c25aa5b80 4 client.15081 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.501+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1312271500 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "status"}) -- 0x5604b24c6ad0 con 0x5604b26232d0 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.501+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5604b26232d0 0x5604b2624740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c14e2b700 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5604b26232d0 0x5604b2624740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1821f020d7b0a6f2 server_cookie=cef9326458d56358 in_seq=0 out_seq=0 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3beffff700 10 client.15081 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3beffff700 1 -- 172.21.15.46:0/1312271500 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5604b24c6ad0 con 0x5604b26232d0 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3beffff700 10 client.15081 handle_command_reply: tid=0 2024-05-09T00:11:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c25aa5b80 1 client.15081 shutdown 2024-05-09T00:11:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1312271500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 msgr2=0x5604b25beba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 0x5604b25beba0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c08029040 tx=0x7f3c08006140 comp rx=0 tx=0).stop 2024-05-09T00:11:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1312271500 shutdown_connections 2024-05-09T00:11:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5604b26232d0 0x5604b2624740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b259c3b0 0x5604b25beba0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5604b241f450 0x5604b25ba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3beffff700 0 client.15081 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:11:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c25aa5b80 1 --2- 172.21.15.46:0/1312271500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604b2499cb0 0x5604b25bab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1312271500 >> 172.21.15.46:0/1312271500 conn(0x5604b2422f20 msgr2=0x5604b25a1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1312271500 shutdown_connections 2024-05-09T00:11:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c25aa5b80 1 -- 172.21.15.46:0/1312271500 wait complete. 2024-05-09T00:11:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.502+0000 7f3c25aa5b80 20 client.15081 trim_cache size 0 max 16384 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "eec67860-0d96-11ef-bc97-c7b262605968", 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24487, 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 178.135060173, 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 25, 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 223.31658341599999 2024-05-09T00:11:39.505 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:11:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.503+0000 7f3c1708f700 1 -- 172.21.15.46:0/3574556844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3bfc0b2880 msgr2=0x7f3bfc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.503+0000 7f3c1708f700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3bfc0b2880 0x7f3bfc0b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3c00005950 tx=0x7f3c0000a300 comp rx=0 tx=0).stop 2024-05-09T00:11:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.503+0000 7f3c1708f700 1 -- 172.21.15.46:0/3574556844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c10107df0 msgr2=0x7f3c1019c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.503+0000 7f3c1708f700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c10107df0 0x7f3c1019c150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800e970 tx=0x7f3bf800ec80 comp rx=0 tx=0).stop 2024-05-09T00:11:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.503+0000 7f3c1708f700 1 -- 172.21.15.46:0/3574556844 shutdown_connections 2024-05-09T00:11:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.503+0000 7f3c1708f700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3bfc0b2880 0x7f3bfc0b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.503+0000 7f3c1708f700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c10108740 0x7f3c1019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.503+0000 7f3c1708f700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c101034c0 0x7f3c1019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.503+0000 7f3c1708f700 1 --2- 172.21.15.46:0/3574556844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c10107df0 0x7f3c1019c150 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.503+0000 7f3c1708f700 1 -- 172.21.15.46:0/3574556844 >> 172.21.15.46:0/3574556844 conn(0x7f3c100fedc0 msgr2=0x7f3c10107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.503+0000 7f3c1708f700 1 -- 172.21.15.46:0/3574556844 shutdown_connections 2024-05-09T00:11:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:39.503+0000 7f3c1708f700 1 -- 172.21.15.46:0/3574556844 wait complete. 2024-05-09T00:11:40.215 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T00:11:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:40.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:40 smithi046 ceph-mon[30151]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:40.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:40 smithi169 ceph-mon[34777]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:40.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:40 smithi142 ceph-mon[34533]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.200+0000 7f613f7f8700 1 -- 172.21.15.46:0/985837821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6138105460 msgr2=0x7f6138105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.200+0000 7f613f7f8700 1 --2- 172.21.15.46:0/985837821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6138105460 0x7f6138105870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6134009990 tx=0x7f6134009ca0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.200+0000 7f613f7f8700 1 -- 172.21.15.46:0/985837821 shutdown_connections 2024-05-09T00:11:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.200+0000 7f613f7f8700 1 --2- 172.21.15.46:0/985837821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f613810e8d0 0x7f6138110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.200+0000 7f613f7f8700 1 --2- 172.21.15.46:0/985837821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6138105460 0x7f6138105870 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.200+0000 7f613f7f8700 1 --2- 172.21.15.46:0/985837821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6138105db0 0x7f613810e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.200+0000 7f613f7f8700 1 -- 172.21.15.46:0/985837821 >> 172.21.15.46:0/985837821 conn(0x7f61380fedc0 msgr2=0x7f61381011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.200+0000 7f613f7f8700 1 -- 172.21.15.46:0/985837821 shutdown_connections 2024-05-09T00:11:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.200+0000 7f613f7f8700 1 -- 172.21.15.46:0/985837821 wait complete. 2024-05-09T00:11:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.201+0000 7f613f7f8700 1 Processor -- start 2024-05-09T00:11:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.201+0000 7f613f7f8700 1 -- start start 2024-05-09T00:11:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.201+0000 7f613f7f8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6138105460 0x7f613819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.201+0000 7f613f7f8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6138105db0 0x7f613819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.201+0000 7f613f7f8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613810e8d0 0x7f613819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.201+0000 7f613f7f8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6138114420 con 0x7f613810e8d0 2024-05-09T00:11:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.201+0000 7f613f7f8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f61381142a0 con 0x7f6138105db0 2024-05-09T00:11:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.201+0000 7f613f7f8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f61381145a0 con 0x7f6138105460 2024-05-09T00:11:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.201+0000 7f613dd95700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613810e8d0 0x7f613819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.201+0000 7f613d594700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6138105460 0x7f613819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.201+0000 7f613cd93700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6138105db0 0x7f613819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.201+0000 7f613dd95700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613810e8d0 0x7f613819f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40720/0 (socket says 172.21.15.46:40720) 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f613dd95700 1 -- 172.21.15.46:0/1697113343 learned_addr learned my addr 172.21.15.46:0/1697113343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f613d594700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6138105460 0x7f613819ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:60014/0 (socket says 172.21.15.46:60014) 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f613cd93700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6138105db0 0x7f613819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:46328/0 (socket says 172.21.15.46:46328) 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f613cd93700 1 -- 172.21.15.46:0/1697113343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6138105460 msgr2=0x7f613819ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f613cd93700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6138105460 0x7f613819ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f613cd93700 1 -- 172.21.15.46:0/1697113343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613810e8d0 msgr2=0x7f613819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f613cd93700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613810e8d0 0x7f613819f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f613cd93700 1 -- 172.21.15.46:0/1697113343 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613819fc60 con 0x7f6138105db0 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f613dd95700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613810e8d0 0x7f613819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f613d594700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6138105460 0x7f613819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f613cd93700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6138105db0 0x7f613819b330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f612800e9f0 tx=0x7f612800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f612e7fc700 1 -- 172.21.15.46:0/1697113343 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612800c9c0 con 0x7f6138105db0 2024-05-09T00:11:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f612e7fc700 1 -- 172.21.15.46:0/1697113343 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f612800cb60 con 0x7f6138105db0 2024-05-09T00:11:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f612e7fc700 1 -- 172.21.15.46:0/1697113343 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128018680 con 0x7f6138105db0 2024-05-09T00:11:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.202+0000 7f613f7f8700 1 -- 172.21.15.46:0/1697113343 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613819ff50 con 0x7f6138105db0 2024-05-09T00:11:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.203+0000 7f613f7f8700 1 -- 172.21.15.46:0/1697113343 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6138103080 con 0x7f6138105db0 2024-05-09T00:11:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.205+0000 7f612e7fc700 1 -- 172.21.15.46:0/1697113343 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6128018820 con 0x7f6138105db0 2024-05-09T00:11:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.206+0000 7f612e7fc700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61240b2880 0x7f61240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.206+0000 7f613d594700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61240b2880 0x7f61240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.206+0000 7f612e7fc700 1 -- 172.21.15.46:0/1697113343 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f61280e2400 con 0x7f6138105db0 2024-05-09T00:11:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.206+0000 7f614e20eb80 1 Processor -- start 2024-05-09T00:11:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.206+0000 7f614e20eb80 1 -- start start 2024-05-09T00:11:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f613d594700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61240b2880 0x7f61240b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f613400b2e0 tx=0x7f6134005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f614e20eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c00a110 0x560f5c0a34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f614e20eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 0x560f5c0a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f614e20eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c09d650 0x560f5c09da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f614e20eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560f5bfbffa0 con 0x560f5bf20450 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f614e20eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560f5bf5e7a0 con 0x560f5c00a110 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f614e20eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560f5bfbbf60 con 0x560f5c09d650 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f613cd93700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 0x560f5c0a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f613cd93700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 0x560f5c0a39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40736/0 (socket says 172.21.15.46:40736) 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f613cd93700 1 -- 172.21.15.46:0/2842500972 learned_addr learned my addr 172.21.15.46:0/2842500972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f613d594700 1 --2- 172.21.15.46:0/2842500972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c00a110 0x560f5c0a34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f613dd95700 1 --2- 172.21.15.46:0/2842500972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c09d650 0x560f5c09da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f613cd93700 1 -- 172.21.15.46:0/2842500972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c09d650 msgr2=0x560f5c09da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f613cd93700 1 --2- 172.21.15.46:0/2842500972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c09d650 0x560f5c09da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f613cd93700 1 -- 172.21.15.46:0/2842500972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c00a110 msgr2=0x560f5c0a34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f613cd93700 1 --2- 172.21.15.46:0/2842500972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c00a110 0x560f5c0a34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.207+0000 7f613cd93700 1 -- 172.21.15.46:0/2842500972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f5bf56320 con 0x560f5bf20450 2024-05-09T00:11:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.208+0000 7f613cd93700 1 --2- 172.21.15.46:0/2842500972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 0x560f5c0a39e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f612800c5b0 tx=0x7f61280a97f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe105fcf51c6b739 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.208+0000 7f61237fe700 1 -- 172.21.15.46:0/2842500972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280aae60 con 0x560f5bf20450 2024-05-09T00:11:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.208+0000 7f61237fe700 1 -- 172.21.15.46:0/2842500972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f612800cd10 con 0x560f5bf20450 2024-05-09T00:11:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.208+0000 7f61237fe700 1 -- 172.21.15.46:0/2842500972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128018b10 con 0x560f5bf20450 2024-05-09T00:11:42.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.209+0000 7f614e20eb80 1 -- 172.21.15.46:0/2842500972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 msgr2=0x560f5c0a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:42.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.209+0000 7f614e20eb80 1 --2- 172.21.15.46:0/2842500972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 0x560f5c0a39e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f612800c5b0 tx=0x7f61280a97f0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.209+0000 7f614e20eb80 1 -- 172.21.15.46:0/2842500972 shutdown_connections 2024-05-09T00:11:42.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.209+0000 7f614e20eb80 1 --2- 172.21.15.46:0/2842500972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c09d650 0x560f5c09da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.209+0000 7f614e20eb80 1 --2- 172.21.15.46:0/2842500972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c00a110 0x560f5c0a34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.209+0000 7f614e20eb80 1 --2- 172.21.15.46:0/2842500972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 0x560f5c0a39e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.209+0000 7f614e20eb80 1 -- 172.21.15.46:0/2842500972 >> 172.21.15.46:0/2842500972 conn(0x560f5bf80650 msgr2=0x560f5c0983b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.209+0000 7f614e20eb80 1 -- 172.21.15.46:0/2842500972 shutdown_connections 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.209+0000 7f614e20eb80 1 -- 172.21.15.46:0/2842500972 wait complete. 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.209+0000 7f614e20eb80 1 -- start start 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.209+0000 7f614e20eb80 1 Processor -- start 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f614e20eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 0x560f5c0a7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f614e20eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c00a110 0x560f5c0a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f614e20eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c09d650 0x560f5c0a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f614e20eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560f5be3bb50 con 0x560f5bf20450 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f614e20eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560f5bf7a100 con 0x560f5c09d650 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f614e20eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560f5c0aa180 con 0x560f5c00a110 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f613d594700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 0x560f5c0a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f614e20eb80 10 client.0 _ll_register_callbacks cb 0x560f5be31670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f613d594700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 0x560f5c0a7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40752/0 (socket says 172.21.15.46:40752) 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f613cd93700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c00a110 0x560f5c0a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f613d594700 1 -- 172.21.15.46:0/715984293 learned_addr learned my addr 172.21.15.46:0/715984293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f613dd95700 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c09d650 0x560f5c0a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f613cd93700 1 -- 172.21.15.46:0/715984293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c09d650 msgr2=0x560f5c0a7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f613cd93700 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c09d650 0x560f5c0a7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f613cd93700 1 -- 172.21.15.46:0/715984293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 msgr2=0x560f5c0a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f613cd93700 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 0x560f5c0a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f613cd93700 1 -- 172.21.15.46:0/715984293 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f5be3b440 con 0x560f5c00a110 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f613d594700 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 0x560f5c0a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.210+0000 7f613dd95700 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c09d650 0x560f5c0a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f613cd93700 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c00a110 0x560f5c0a77b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f61280a9ba0 tx=0x7f61280cc280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f105705cf415db3f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f6120ff9700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f6120ff9700 1 -- 172.21.15.46:0/715984293 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280c2070 con 0x560f5c00a110 2024-05-09T00:11:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f614e20eb80 1 -- 172.21.15.46:0/715984293 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f612800cd10 con 0x560f5c00a110 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f6120ff9700 1 -- 172.21.15.46:0/715984293 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f61280a9070 con 0x560f5c00a110 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f6120ff9700 1 -- 172.21.15.46:0/715984293 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f61280aae60 con 0x560f5c00a110 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f614e20eb80 10 client.24892 fetch_fsmap learned FSMap version 25 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f614e20eb80 1 -- 172.21.15.46:0/715984293 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560f5c0b4c00 con 0x560f5c00a110 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f6120ff9700 1 -- 172.21.15.46:0/715984293 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f612800cd10 con 0x560f5c00a110 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f6120ff9700 1 -- 172.21.15.46:0/715984293 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f612806b030 con 0x560f5c00a110 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f614e20eb80 10 client.24892 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f614e20eb80 10 client.24892 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.211+0000 7f614e20eb80 20 client.24892 populate_metadata read hostname 'smithi046' 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.212+0000 7f614e20eb80 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x560f5c0b7340 0x560f5c0b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.212+0000 7f614e20eb80 4 client.24892 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.212+0000 7f614e20eb80 1 -- 172.21.15.46:0/715984293 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f5bfcd980 con 0x560f5c0b7340 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.212+0000 7f613d594700 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x560f5c0b7340 0x560f5c0b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.212+0000 7f6120ff9700 10 client.24892 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:11:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.212+0000 7f613d594700 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x560f5c0b7340 0x560f5c0b87b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=275162ac03053559 server_cookie=65eab852922d3022 in_seq=0 out_seq=0 2024-05-09T00:11:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f6120ff9700 1 -- 172.21.15.46:0/715984293 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560f5bfcd980 con 0x560f5c0b7340 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f6120ff9700 10 client.24892 handle_command_reply: tid=0 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f614e20eb80 1 client.24892 shutdown 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f614e20eb80 1 -- 172.21.15.46:0/715984293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c00a110 msgr2=0x560f5c0a77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f614e20eb80 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c00a110 0x560f5c0a77b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f61280a9ba0 tx=0x7f61280cc280 comp rx=0 tx=0).stop 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f614e20eb80 1 -- 172.21.15.46:0/715984293 shutdown_connections 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f614e20eb80 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x560f5c0b7340 0x560f5c0b87b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f614e20eb80 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c09d650 0x560f5c0a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f614e20eb80 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf20450 0x560f5c0a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f614e20eb80 1 --2- 172.21.15.46:0/715984293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c00a110 0x560f5c0a77b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f614e20eb80 1 -- 172.21.15.46:0/715984293 >> 172.21.15.46:0/715984293 conn(0x560f5bf80650 msgr2=0x560f5c098170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f6120ff9700 0 client.24892 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.213+0000 7f614e20eb80 1 -- 172.21.15.46:0/715984293 shutdown_connections 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.214+0000 7f614e20eb80 1 -- 172.21.15.46:0/715984293 wait complete. 2024-05-09T00:11:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.214+0000 7f614e20eb80 20 client.24892 trim_cache size 0 max 16384 2024-05-09T00:11:42.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.235+0000 7f614e20eb80 1 Processor -- start 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.235+0000 7f614e20eb80 1 -- start start 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.235+0000 7f614e20eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 0x560f5c0b3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.235+0000 7f614e20eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5bf20450 0x560f5c0a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.235+0000 7f614e20eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c00a110 0x560f5c0a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.235+0000 7f614e20eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560f5c0b6af0 con 0x560f5bf80650 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.235+0000 7f614e20eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560f5c0b71c0 con 0x560f5bf20450 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.235+0000 7f614e20eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560f5c0b7890 con 0x560f5c00a110 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.235+0000 7f613d594700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 0x560f5c0b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.235+0000 7f613d594700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 0x560f5c0b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40762/0 (socket says 172.21.15.46:40762) 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.235+0000 7f613d594700 1 -- 172.21.15.46:0/2043707618 learned_addr learned my addr 172.21.15.46:0/2043707618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.236+0000 7f613cd93700 1 --2- 172.21.15.46:0/2043707618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5bf20450 0x560f5c0a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.236+0000 7f613dd95700 1 --2- 172.21.15.46:0/2043707618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c00a110 0x560f5c0a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.236+0000 7f613d594700 1 -- 172.21.15.46:0/2043707618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c00a110 msgr2=0x560f5c0a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:42.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.236+0000 7f613d594700 1 --2- 172.21.15.46:0/2043707618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c00a110 0x560f5c0a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.236+0000 7f613d594700 1 -- 172.21.15.46:0/2043707618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5bf20450 msgr2=0x560f5c0a77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:42.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.236+0000 7f613d594700 1 --2- 172.21.15.46:0/2043707618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5bf20450 0x560f5c0a77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.236+0000 7f613d594700 1 -- 172.21.15.46:0/2043707618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f5c0a3000 con 0x560f5bf80650 2024-05-09T00:11:42.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.236+0000 7f613cd93700 1 --2- 172.21.15.46:0/2043707618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5bf20450 0x560f5c0a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:42.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.236+0000 7f613d594700 1 --2- 172.21.15.46:0/2043707618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 0x560f5c0b3060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6134000c00 tx=0x7f6134021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4651bff42a3449 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:42.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.236+0000 7f6122ffd700 1 -- 172.21.15.46:0/2043707618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613403a070 con 0x560f5bf80650 2024-05-09T00:11:42.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.236+0000 7f6122ffd700 1 -- 172.21.15.46:0/2043707618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6134035810 con 0x560f5bf80650 2024-05-09T00:11:42.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.236+0000 7f6122ffd700 1 -- 172.21.15.46:0/2043707618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134025920 con 0x560f5bf80650 2024-05-09T00:11:42.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.237+0000 7f614e20eb80 1 -- 172.21.15.46:0/2043707618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 msgr2=0x560f5c0b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:42.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.237+0000 7f614e20eb80 1 --2- 172.21.15.46:0/2043707618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 0x560f5c0b3060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6134000c00 tx=0x7f6134021000 comp rx=0 tx=0).stop 2024-05-09T00:11:42.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.237+0000 7f614e20eb80 1 -- 172.21.15.46:0/2043707618 shutdown_connections 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.237+0000 7f614e20eb80 1 --2- 172.21.15.46:0/2043707618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5c00a110 0x560f5c0a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.237+0000 7f614e20eb80 1 --2- 172.21.15.46:0/2043707618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 0x560f5c0b3060 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.237+0000 7f614e20eb80 1 --2- 172.21.15.46:0/2043707618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5bf20450 0x560f5c0a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.237+0000 7f614e20eb80 1 -- 172.21.15.46:0/2043707618 >> 172.21.15.46:0/2043707618 conn(0x560f5c09ddc0 msgr2=0x560f5c0a2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.237+0000 7f614e20eb80 1 -- 172.21.15.46:0/2043707618 shutdown_connections 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.237+0000 7f614e20eb80 1 -- 172.21.15.46:0/2043707618 wait complete. 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.238+0000 7f614e20eb80 1 -- start start 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.238+0000 7f614e20eb80 1 Processor -- start 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.238+0000 7f614e20eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5bf20450 0x560f5c0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.238+0000 7f614e20eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 0x560f5c0b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.238+0000 7f614e20eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c00a110 0x560f5c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.238+0000 7f614e20eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560f5c0b8630 con 0x560f5bf80650 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.238+0000 7f614e20eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560f5c0b7f60 con 0x560f5c00a110 2024-05-09T00:11:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.238+0000 7f614e20eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560f5c0b8d00 con 0x560f5bf20450 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.238+0000 7f614e20eb80 10 client.0 _ll_register_callbacks cb 0x560f5be31670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f613cd93700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 0x560f5c0b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f613d594700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5bf20450 0x560f5c0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f613cd93700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 0x560f5c0b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40772/0 (socket says 172.21.15.46:40772) 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f613cd93700 1 -- 172.21.15.46:0/1894288096 learned_addr learned my addr 172.21.15.46:0/1894288096 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f613dd95700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c00a110 0x560f5c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f613d594700 1 -- 172.21.15.46:0/1894288096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c00a110 msgr2=0x560f5c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f613d594700 1 --2- 172.21.15.46:0/1894288096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c00a110 0x560f5c0b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f613d594700 1 -- 172.21.15.46:0/1894288096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 msgr2=0x560f5c0b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f613d594700 1 --2- 172.21.15.46:0/1894288096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 0x560f5c0b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f613d594700 1 -- 172.21.15.46:0/1894288096 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f5c0bfea0 con 0x560f5bf20450 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f613cd93700 1 --2- 172.21.15.46:0/1894288096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 0x560f5c0b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f613d594700 1 --2- 172.21.15.46:0/1894288096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5bf20450 0x560f5c0b26d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6134009ce0 tx=0x7f6134025420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b5e6faf3c747b8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f6120ff9700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f6120ff9700 1 -- 172.21.15.46:0/1894288096 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134043060 con 0x560f5bf20450 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f6120ff9700 1 -- 172.21.15.46:0/1894288096 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6134035360 con 0x560f5bf20450 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.239+0000 7f614e20eb80 1 -- 172.21.15.46:0/1894288096 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6134035810 con 0x560f5bf20450 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.240+0000 7f6120ff9700 1 -- 172.21.15.46:0/1894288096 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f613403a040 con 0x560f5bf20450 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.240+0000 7f614e20eb80 10 client.24898 fetch_fsmap learned FSMap version 25 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.240+0000 7f614e20eb80 1 -- 172.21.15.46:0/1894288096 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560f5c1228e0 con 0x560f5bf20450 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.240+0000 7f6120ff9700 1 -- 172.21.15.46:0/1894288096 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6134035810 con 0x560f5bf20450 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.240+0000 7f6120ff9700 1 -- 172.21.15.46:0/1894288096 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f6134022070 con 0x560f5bf20450 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.240+0000 7f614e20eb80 10 client.24898 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.240+0000 7f614e20eb80 10 client.24898 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.240+0000 7f614e20eb80 20 client.24898 populate_metadata read hostname 'smithi046' 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.240+0000 7f614e20eb80 1 --2- 172.21.15.46:0/1894288096 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x560f5c09d650 0x560f5c125c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.240+0000 7f614e20eb80 4 client.24898 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.240+0000 7f614e20eb80 1 -- 172.21.15.46:0/1894288096 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "status"}) -- 0x560f5bfc7a20 con 0x560f5c09d650 2024-05-09T00:11:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.240+0000 7f613cd93700 1 --2- 172.21.15.46:0/1894288096 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x560f5c09d650 0x560f5c125c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f6120ff9700 10 client.24898 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:11:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f613cd93700 1 --2- 172.21.15.46:0/1894288096 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x560f5c09d650 0x560f5c125c00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95f0a4cc7be6adf5 server_cookie=4994a57ed5a8e423 in_seq=0 out_seq=0 2024-05-09T00:11:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f6120ff9700 1 -- 172.21.15.46:0/1894288096 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560f5bfc7a20 con 0x560f5c09d650 2024-05-09T00:11:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f6120ff9700 10 client.24898 handle_command_reply: tid=0 2024-05-09T00:11:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f614e20eb80 1 client.24898 shutdown 2024-05-09T00:11:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f614e20eb80 1 -- 172.21.15.46:0/1894288096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5bf20450 msgr2=0x560f5c0b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f614e20eb80 1 --2- 172.21.15.46:0/1894288096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5bf20450 0x560f5c0b26d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6134009ce0 tx=0x7f6134025420 comp rx=0 tx=0).stop 2024-05-09T00:11:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f614e20eb80 1 -- 172.21.15.46:0/1894288096 shutdown_connections 2024-05-09T00:11:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f614e20eb80 1 --2- 172.21.15.46:0/1894288096 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x560f5c09d650 0x560f5c125c00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f614e20eb80 1 --2- 172.21.15.46:0/1894288096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f5c00a110 0x560f5c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f614e20eb80 1 --2- 172.21.15.46:0/1894288096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f5bf20450 0x560f5c0b26d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f6120ff9700 0 client.24898 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:11:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f614e20eb80 1 --2- 172.21.15.46:0/1894288096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f5bf80650 0x560f5c0b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f614e20eb80 1 -- 172.21.15.46:0/1894288096 >> 172.21.15.46:0/1894288096 conn(0x560f5c09ddc0 msgr2=0x560f5c0a5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f614e20eb80 1 -- 172.21.15.46:0/1894288096 shutdown_connections 2024-05-09T00:11:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.241+0000 7f614e20eb80 1 -- 172.21.15.46:0/1894288096 wait complete. 2024-05-09T00:11:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.242+0000 7f614e20eb80 20 client.24898 trim_cache size 0 max 16384 2024-05-09T00:11:42.244 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:11:42.244 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "eec67860-0d96-11ef-bc97-c7b262605968", 2024-05-09T00:11:42.244 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-05-09T00:11:42.244 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24497, 2024-05-09T00:11:42.244 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-09T00:11:42.244 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-09T00:11:42.245 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-09T00:11:42.245 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 181.88786802499999, 2024-05-09T00:11:42.245 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 25, 2024-05-09T00:11:42.245 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-05-09T00:11:42.245 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:11:42.245 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 209.28717996500001 2024-05-09T00:11:42.245 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:11:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.242+0000 7f613f7f8700 1 -- 172.21.15.46:0/1697113343 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61240b2880 msgr2=0x7f61240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.242+0000 7f613f7f8700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61240b2880 0x7f61240b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f613400b2e0 tx=0x7f6134005960 comp rx=0 tx=0).stop 2024-05-09T00:11:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.242+0000 7f613f7f8700 1 -- 172.21.15.46:0/1697113343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6138105db0 msgr2=0x7f613819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.242+0000 7f613f7f8700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6138105db0 0x7f613819b330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f612800e9f0 tx=0x7f612800ed00 comp rx=0 tx=0).stop 2024-05-09T00:11:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.242+0000 7f613f7f8700 1 -- 172.21.15.46:0/1697113343 shutdown_connections 2024-05-09T00:11:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.242+0000 7f613f7f8700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61240b2880 0x7f61240b4d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.242+0000 7f613f7f8700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613810e8d0 0x7f613819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.242+0000 7f613f7f8700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6138105460 0x7f613819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.242+0000 7f613f7f8700 1 --2- 172.21.15.46:0/1697113343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6138105db0 0x7f613819b330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.242+0000 7f613f7f8700 1 -- 172.21.15.46:0/1697113343 >> 172.21.15.46:0/1697113343 conn(0x7f61380fedc0 msgr2=0x7f61380ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.242+0000 7f613f7f8700 1 -- 172.21.15.46:0/1697113343 shutdown_connections 2024-05-09T00:11:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:42.243+0000 7f613f7f8700 1 -- 172.21.15.46:0/1697113343 wait complete. 2024-05-09T00:11:42.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:42 smithi046 ceph-mon[30151]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:42.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:42 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:11:42.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:42 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:42.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:42 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:42.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:42 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:42.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:42 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:42.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:42 smithi169 ceph-mon[34777]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:42.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:42 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:11:42.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:42 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:42.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:42 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:42.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:42 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:42.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:42 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:42.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:42 smithi142 ceph-mon[34533]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:42.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:42 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:11:42.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:42 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:42.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:42 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:42.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:42 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:42.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:42 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:42.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T00:11:43.240 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:44.353 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:44.353 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:44.353 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:44 smithi169 ceph-mon[34777]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:44.386 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:44.386 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:44.386 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:44 smithi046 ceph-mon[30151]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:44.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:44.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:44.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:44 smithi142 ceph-mon[34533]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.927+0000 7f4bc6613700 1 -- 172.21.15.46:0/3614636379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc0108f20 msgr2=0x7f4bc0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.927+0000 7f4bc6613700 1 --2- 172.21.15.46:0/3614636379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc0108f20 0x7f4bc0109330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40099c0 tx=0x7f4bb4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.927+0000 7f4bc6613700 1 -- 172.21.15.46:0/3614636379 shutdown_connections 2024-05-09T00:11:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.927+0000 7f4bc6613700 1 --2- 172.21.15.46:0/3614636379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bc0109870 0x7f4bc01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.927+0000 7f4bc6613700 1 --2- 172.21.15.46:0/3614636379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc01045f0 0x7f4bc0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.927+0000 7f4bc6613700 1 --2- 172.21.15.46:0/3614636379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc0108f20 0x7f4bc0109330 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.927+0000 7f4bc6613700 1 -- 172.21.15.46:0/3614636379 >> 172.21.15.46:0/3614636379 conn(0x7f4bc0100040 msgr2=0x7f4bc0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.927+0000 7f4bc6613700 1 -- 172.21.15.46:0/3614636379 shutdown_connections 2024-05-09T00:11:44.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.927+0000 7f4bc6613700 1 -- 172.21.15.46:0/3614636379 wait complete. 2024-05-09T00:11:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.927+0000 7f4bc6613700 1 Processor -- start 2024-05-09T00:11:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc6613700 1 -- start start 2024-05-09T00:11:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc6613700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bc01045f0 0x7f4bc019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc6613700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc0108f20 0x7f4bc019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc6613700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc0109870 0x7f4bc019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc6613700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc0114480 con 0x7f4bc0109870 2024-05-09T00:11:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc6613700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc0114300 con 0x7f4bc0108f20 2024-05-09T00:11:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc6613700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc0114600 con 0x7f4bc01045f0 2024-05-09T00:11:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc4bb0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc0109870 0x7f4bc019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc4bb0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc0109870 0x7f4bc019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40792/0 (socket says 172.21.15.46:40792) 2024-05-09T00:11:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/3661286478 learned_addr learned my addr 172.21.15.46:0/3661286478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bbffff700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bc01045f0 0x7f4bc019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bbf7fe700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc0108f20 0x7f4bc019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/3661286478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bc01045f0 msgr2=0x7f4bc019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bc01045f0 0x7f4bc019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.928+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/3661286478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc0108f20 msgr2=0x7f4bc019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.929+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc0108f20 0x7f4bc019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.929+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/3661286478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc019fc50 con 0x7f4bc0109870 2024-05-09T00:11:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.929+0000 7f4bbf7fe700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc0108f20 0x7f4bc019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.929+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc0109870 0x7f4bc019f660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8007ee0 tx=0x7f4bb800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.929+0000 7f4bbd7fa700 1 -- 172.21.15.46:0/3661286478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800dbd0 con 0x7f4bc0109870 2024-05-09T00:11:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.929+0000 7f4bbd7fa700 1 -- 172.21.15.46:0/3661286478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bb800dd70 con 0x7f4bc0109870 2024-05-09T00:11:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.929+0000 7f4bc6613700 1 -- 172.21.15.46:0/3661286478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc019ff40 con 0x7f4bc0109870 2024-05-09T00:11:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.929+0000 7f4bbd7fa700 1 -- 172.21.15.46:0/3661286478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8005990 con 0x7f4bc0109870 2024-05-09T00:11:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.929+0000 7f4bc6613700 1 -- 172.21.15.46:0/3661286478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc01a0400 con 0x7f4bc0109870 2024-05-09T00:11:44.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.932+0000 7f4bbd7fa700 1 -- 172.21.15.46:0/3661286478 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4bb801a020 con 0x7f4bc0109870 2024-05-09T00:11:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.933+0000 7f4bbd7fa700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4ba80b2880 0x7f4ba80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.933+0000 7f4bbffff700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4ba80b2880 0x7f4ba80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.933+0000 7f4bbd7fa700 1 -- 172.21.15.46:0/3661286478 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4bb80e1950 con 0x7f4bc0109870 2024-05-09T00:11:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.933+0000 7f4bd5029b80 1 Processor -- start 2024-05-09T00:11:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.933+0000 7f4bd5029b80 1 -- start start 2024-05-09T00:11:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.933+0000 7f4bbffff700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4ba80b2880 0x7f4ba80b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000ac50 tx=0x7f4bb000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.933+0000 7f4bd5029b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8cb6a110 0x557c8cc01d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.933+0000 7f4bd5029b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8ca80450 0x557c8cc022a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bd5029b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bd5029b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557c8cb1ffa0 con 0x557c8cc063b0 2024-05-09T00:11:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bd5029b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557c8cabe7a0 con 0x557c8cb6a110 2024-05-09T00:11:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bd5029b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557c8cb1bf60 con 0x557c8ca80450 2024-05-09T00:11:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bc4bb0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bc4bb0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40794/0 (socket says 172.21.15.46:40794) 2024-05-09T00:11:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/1059639498 learned_addr learned my addr 172.21.15.46:0/1059639498 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bbf7fe700 1 --2- 172.21.15.46:0/1059639498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8ca80450 0x557c8cc022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bbffff700 1 --2- 172.21.15.46:0/1059639498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8cb6a110 0x557c8cc01d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/1059639498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8ca80450 msgr2=0x557c8cc022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/1059639498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8ca80450 0x557c8cc022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/1059639498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8cb6a110 msgr2=0x557c8cc01d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/1059639498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8cb6a110 0x557c8cc01d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/1059639498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c8cab6320 con 0x557c8cc063b0 2024-05-09T00:11:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bbf7fe700 1 --2- 172.21.15.46:0/1059639498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8ca80450 0x557c8cc022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/1059639498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc067c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80a61e0 tx=0x7f4bb80a8940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db916374de13fdf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4ba67fc700 1 -- 172.21.15.46:0/1059639498 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80a8f70 con 0x557c8cc063b0 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4ba67fc700 1 -- 172.21.15.46:0/1059639498 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bb80ad320 con 0x557c8cc063b0 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.934+0000 7f4ba67fc700 1 -- 172.21.15.46:0/1059639498 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80b1940 con 0x557c8cc063b0 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.935+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1059639498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 msgr2=0x557c8cc067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.935+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/1059639498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc067c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80a61e0 tx=0x7f4bb80a8940 comp rx=0 tx=0).stop 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.935+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1059639498 shutdown_connections 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.935+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/1059639498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc067c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.935+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/1059639498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8cb6a110 0x557c8cc01d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.935+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/1059639498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8ca80450 0x557c8cc022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.935+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1059639498 >> 172.21.15.46:0/1059639498 conn(0x557c8cae0650 msgr2=0x557c8cbf83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.935+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1059639498 shutdown_connections 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.935+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1059639498 wait complete. 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bd5029b80 1 -- start start 2024-05-09T00:11:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bd5029b80 1 Processor -- start 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bd5029b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8ca80450 0x557c8cc0b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bd5029b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8cb6a110 0x557c8cc0bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bd5029b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc0c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bd5029b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557c8c99bb50 con 0x557c8cc063b0 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bd5029b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557c8cada100 con 0x557c8cb6a110 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bd5029b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557c8cc07160 con 0x557c8ca80450 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bd5029b80 10 client.0 _ll_register_callbacks cb 0x557c8c991670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bc4bb0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc0c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bc4bb0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc0c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40810/0 (socket says 172.21.15.46:40810) 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/3546641 learned_addr learned my addr 172.21.15.46:0/3546641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.936+0000 7f4bbf7fe700 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8cb6a110 0x557c8cc0bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4bbffff700 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8ca80450 0x557c8cc0b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/3546641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8ca80450 msgr2=0x557c8cc0b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8ca80450 0x557c8cc0b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/3546641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8cb6a110 msgr2=0x557c8cc0bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8cb6a110 0x557c8cc0bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/3546641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c8c99b440 con 0x557c8cc063b0 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc0c000 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8004f50 tx=0x7f4bb80a9400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1728b306859d752 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4b9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4b9f7fe700 1 -- 172.21.15.46:0/3546641 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb806c070 con 0x557c8cc063b0 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4b9f7fe700 1 -- 172.21.15.46:0/3546641 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4bb80b1b50 con 0x557c8cc063b0 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4bd5029b80 1 -- 172.21.15.46:0/3546641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb80ad320 con 0x557c8cc063b0 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4b9f7fe700 1 -- 172.21.15.46:0/3546641 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4bb80a81d0 con 0x557c8cc063b0 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4bd5029b80 10 client.15111 fetch_fsmap learned FSMap version 25 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.937+0000 7f4bd5029b80 1 -- 172.21.15.46:0/3546641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c8cc18ea0 con 0x557c8cc063b0 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.938+0000 7f4b9f7fe700 1 -- 172.21.15.46:0/3546641 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bb80ad320 con 0x557c8cc063b0 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.938+0000 7f4b9f7fe700 1 -- 172.21.15.46:0/3546641 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4bb80ac030 con 0x557c8cc063b0 2024-05-09T00:11:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.938+0000 7f4bd5029b80 10 client.15111 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T00:11:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.938+0000 7f4bd5029b80 10 client.15111 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T00:11:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.938+0000 7f4bd5029b80 20 client.15111 populate_metadata read hostname 'smithi046' 2024-05-09T00:11:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.938+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557c8cc1afb0 0x557c8cc1c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.938+0000 7f4bd5029b80 4 client.15111 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:11:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.938+0000 7f4bd5029b80 1 -- 172.21.15.46:0/3546641 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c8cb2d980 con 0x557c8cc1afb0 2024-05-09T00:11:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.938+0000 7f4bbffff700 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557c8cc1afb0 0x557c8cc1c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.938+0000 7f4bbffff700 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557c8cc1afb0 0x557c8cc1c420 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c167757fdc66e1 server_cookie=f0b4688d65aa6a71 in_seq=0 out_seq=0 2024-05-09T00:11:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.938+0000 7f4b9f7fe700 10 client.15111 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:11:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.939+0000 7f4b9f7fe700 1 -- 172.21.15.46:0/3546641 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557c8cb2d980 con 0x557c8cc1afb0 2024-05-09T00:11:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.939+0000 7f4b9f7fe700 10 client.15111 handle_command_reply: tid=0 2024-05-09T00:11:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.939+0000 7f4bd5029b80 1 client.15111 shutdown 2024-05-09T00:11:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.940+0000 7f4bd5029b80 1 -- 172.21.15.46:0/3546641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 msgr2=0x557c8cc0c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.940+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc0c000 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8004f50 tx=0x7f4bb80a9400 comp rx=0 tx=0).stop 2024-05-09T00:11:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.940+0000 7f4bd5029b80 1 -- 172.21.15.46:0/3546641 shutdown_connections 2024-05-09T00:11:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.940+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557c8cc1afb0 0x557c8cc1c420 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.940+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc0c000 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.940+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8ca80450 0x557c8cc0b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.940+0000 7f4b9f7fe700 0 client.15111 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:11:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.940+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/3546641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8cb6a110 0x557c8cc0bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.940+0000 7f4bd5029b80 1 -- 172.21.15.46:0/3546641 >> 172.21.15.46:0/3546641 conn(0x557c8cae0650 msgr2=0x557c8cbf8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.940+0000 7f4bd5029b80 1 -- 172.21.15.46:0/3546641 shutdown_connections 2024-05-09T00:11:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.940+0000 7f4bd5029b80 1 -- 172.21.15.46:0/3546641 wait complete. 2024-05-09T00:11:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.940+0000 7f4bd5029b80 20 client.15111 trim_cache size 0 max 16384 2024-05-09T00:11:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.961+0000 7f4bd5029b80 1 Processor -- start 2024-05-09T00:11:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.961+0000 7f4bd5029b80 1 -- start start 2024-05-09T00:11:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.961+0000 7f4bd5029b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8ca80450 0x557c8cc17740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.961+0000 7f4bd5029b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8cb6a110 0x557c8cc09150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.961+0000 7f4bd5029b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc0be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.961+0000 7f4bd5029b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557c8cc1a760 con 0x557c8cc063b0 2024-05-09T00:11:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.961+0000 7f4bd5029b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557c8cc1ae30 con 0x557c8ca80450 2024-05-09T00:11:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.961+0000 7f4bd5029b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557c8cc1b500 con 0x557c8cb6a110 2024-05-09T00:11:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4bc4bb0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc0be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4bc4bb0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc0be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40816/0 (socket says 172.21.15.46:40816) 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/80505954 learned_addr learned my addr 172.21.15.46:0/80505954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4bbf7fe700 1 --2- 172.21.15.46:0/80505954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8cb6a110 0x557c8cc09150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4bbffff700 1 --2- 172.21.15.46:0/80505954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8ca80450 0x557c8cc17740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/80505954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8cb6a110 msgr2=0x557c8cc09150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/80505954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8cb6a110 0x557c8cc09150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/80505954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8ca80450 msgr2=0x557c8cc17740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/80505954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8ca80450 0x557c8cc17740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4bc4bb0700 1 -- 172.21.15.46:0/80505954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c8cbfa130 con 0x557c8cc063b0 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/80505954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc0be40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80a8f00 tx=0x7f4bb800d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c204e2b295fb00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4ba5ffb700 1 -- 172.21.15.46:0/80505954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb806c070 con 0x557c8cc063b0 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4ba5ffb700 1 -- 172.21.15.46:0/80505954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bb80b1dd0 con 0x557c8cc063b0 2024-05-09T00:11:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.962+0000 7f4ba5ffb700 1 -- 172.21.15.46:0/80505954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80a48e0 con 0x557c8cc063b0 2024-05-09T00:11:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.963+0000 7f4bd5029b80 1 -- 172.21.15.46:0/80505954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 msgr2=0x557c8cc0be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.963+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/80505954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc0be40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80a8f00 tx=0x7f4bb800d8e0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.963+0000 7f4bd5029b80 1 -- 172.21.15.46:0/80505954 shutdown_connections 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.963+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/80505954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cc063b0 0x557c8cc0be40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.963+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/80505954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8ca80450 0x557c8cc17740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.963+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/80505954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8cb6a110 0x557c8cc09150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.963+0000 7f4bd5029b80 1 -- 172.21.15.46:0/80505954 >> 172.21.15.46:0/80505954 conn(0x557c8cae0650 msgr2=0x557c8cc01d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.963+0000 7f4bd5029b80 1 -- 172.21.15.46:0/80505954 shutdown_connections 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.963+0000 7f4bd5029b80 1 -- 172.21.15.46:0/80505954 wait complete. 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bd5029b80 1 -- start start 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bd5029b80 1 Processor -- start 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bd5029b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8ca80450 0x557c8cc258a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bd5029b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cb6a110 0x557c8cc25de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bd5029b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8cc063b0 0x557c8cc1e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bd5029b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557c8cc1c2a0 con 0x557c8cb6a110 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bd5029b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557c8cc1bbd0 con 0x557c8ca80450 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bd5029b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557c8cc1c970 con 0x557c8cc063b0 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bd5029b80 10 client.0 _ll_register_callbacks cb 0x557c8c991670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bbffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8ca80450 0x557c8cc258a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bc4bb0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8cc063b0 0x557c8cc1e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bbffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8ca80450 0x557c8cc258a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:46402/0 (socket says 172.21.15.46:46402) 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.964+0000 7f4bbffff700 1 -- 172.21.15.46:0/1144097506 learned_addr learned my addr 172.21.15.46:0/1144097506 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4bbf7fe700 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cb6a110 0x557c8cc25de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4bbffff700 1 -- 172.21.15.46:0/1144097506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8cc063b0 msgr2=0x557c8cc1e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4bbffff700 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8cc063b0 0x557c8cc1e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4bbffff700 1 -- 172.21.15.46:0/1144097506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cb6a110 msgr2=0x557c8cc25de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4bbffff700 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cb6a110 0x557c8cc25de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4bbffff700 1 -- 172.21.15.46:0/1144097506 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c8cc1efc0 con 0x557c8ca80450 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4bbf7fe700 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cb6a110 0x557c8cc25de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4bc4bb0700 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8cc063b0 0x557c8cc1e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4bbffff700 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8ca80450 0x557c8cc258a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000fa30 tx=0x7f4bb00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac4aa6e600feae67 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4b9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4b9f7fe700 1 -- 172.21.15.46:0/1144097506 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb002b060 con 0x557c8ca80450 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1144097506 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb80b1dd0 con 0x557c8ca80450 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.965+0000 7f4b9f7fe700 1 -- 172.21.15.46:0/1144097506 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4bb00264d0 con 0x557c8ca80450 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.966+0000 7f4b9f7fe700 1 -- 172.21.15.46:0/1144097506 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4bb0016070 con 0x557c8ca80450 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.966+0000 7f4bd5029b80 10 client.24887 fetch_fsmap learned FSMap version 25 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.966+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1144097506 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c8cc86e60 con 0x557c8ca80450 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.966+0000 7f4b9f7fe700 1 -- 172.21.15.46:0/1144097506 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bb80b1dd0 con 0x557c8ca80450 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.966+0000 7f4b9f7fe700 1 -- 172.21.15.46:0/1144097506 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4bb0030070 con 0x557c8ca80450 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.966+0000 7f4bd5029b80 10 client.24887 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T00:11:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.966+0000 7f4bd5029b80 10 client.24887 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T00:11:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.966+0000 7f4bd5029b80 20 client.24887 populate_metadata read hostname 'smithi046' 2024-05-09T00:11:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.966+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557c8cc89320 0x557c8cc8a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.966+0000 7f4bd5029b80 4 client.24887 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T00:11:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.966+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1144097506 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "status"}) -- 0x557c8cb27a20 con 0x557c8cc89320 2024-05-09T00:11:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.967+0000 7f4bbf7fe700 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557c8cc89320 0x557c8cc8a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.967+0000 7f4bbf7fe700 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557c8cc89320 0x557c8cc8a790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e7c5415cf29e26 server_cookie=82be8557495061cd in_seq=0 out_seq=0 2024-05-09T00:11:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.967+0000 7f4b9f7fe700 10 client.24887 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:11:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.967+0000 7f4b9f7fe700 1 -- 172.21.15.46:0/1144097506 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557c8cb27a20 con 0x557c8cc89320 2024-05-09T00:11:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.967+0000 7f4b9f7fe700 10 client.24887 handle_command_reply: tid=0 2024-05-09T00:11:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.967+0000 7f4bd5029b80 1 client.24887 shutdown 2024-05-09T00:11:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.967+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1144097506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8ca80450 msgr2=0x557c8cc258a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.967+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8ca80450 0x557c8cc258a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000fa30 tx=0x7f4bb00047c0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1144097506 shutdown_connections 2024-05-09T00:11:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557c8cc89320 0x557c8cc8a790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557c8cc063b0 0x557c8cc1e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c8ca80450 0x557c8cc258a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bd5029b80 1 --2- 172.21.15.46:0/1144097506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c8cb6a110 0x557c8cc25de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4b9f7fe700 0 client.24887 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:11:44.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1144097506 >> 172.21.15.46:0/1144097506 conn(0x557c8cae0650 msgr2=0x557c8cc05570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:44.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1144097506 shutdown_connections 2024-05-09T00:11:44.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bd5029b80 1 -- 172.21.15.46:0/1144097506 wait complete. 2024-05-09T00:11:44.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bd5029b80 20 client.24887 trim_cache size 0 max 16384 2024-05-09T00:11:44.970 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:11:44.970 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "eec67860-0d96-11ef-bc97-c7b262605968", 2024-05-09T00:11:44.970 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-05-09T00:11:44.970 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24499, 2024-05-09T00:11:44.970 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-09T00:11:44.971 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-09T00:11:44.971 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-09T00:11:44.971 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 196.91962075999999, 2024-05-09T00:11:44.971 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 25, 2024-05-09T00:11:44.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-05-09T00:11:44.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-05-09T00:11:44.971 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 223.02615517500001 2024-05-09T00:11:44.971 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:11:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bc6613700 1 -- 172.21.15.46:0/3661286478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4ba80b2880 msgr2=0x7f4ba80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bc6613700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4ba80b2880 0x7f4ba80b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000ac50 tx=0x7f4bb000a380 comp rx=0 tx=0).stop 2024-05-09T00:11:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bc6613700 1 -- 172.21.15.46:0/3661286478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc0109870 msgr2=0x7f4bc019f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.968+0000 7f4bc6613700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc0109870 0x7f4bc019f660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8007ee0 tx=0x7f4bb800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.969+0000 7f4bc6613700 1 -- 172.21.15.46:0/3661286478 shutdown_connections 2024-05-09T00:11:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.969+0000 7f4bc6613700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4ba80b2880 0x7f4ba80b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.969+0000 7f4bc6613700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc0109870 0x7f4bc019f660 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.969+0000 7f4bc6613700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bc01045f0 0x7f4bc019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.969+0000 7f4bc6613700 1 --2- 172.21.15.46:0/3661286478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bc0108f20 0x7f4bc019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.969+0000 7f4bc6613700 1 -- 172.21.15.46:0/3661286478 >> 172.21.15.46:0/3661286478 conn(0x7f4bc0100040 msgr2=0x7f4bc010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.969+0000 7f4bc6613700 1 -- 172.21.15.46:0/3661286478 shutdown_connections 2024-05-09T00:11:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:44.969+0000 7f4bc6613700 1 -- 172.21.15.46:0/3661286478 wait complete. 2024-05-09T00:11:45.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:45 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:11:45.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:45 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:11:45.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:45 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:45.353 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:45 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:11:45.353 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:45 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:11:45.353 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:45 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:45.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:45 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:11:45.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:45 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:11:45.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:45 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:11:45.625 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T00:11:45.636 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T00:11:45.636 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T00:11:45.637 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T00:11:45.637 INFO:teuthology.run_tasks:Running task kclient... 2024-05-09T00:11:45.689 INFO:tasks.kclient:Mounting kernel clients... 2024-05-09T00:11:45.689 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-09T00:11:45.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:11:45.689 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-05-09T00:11:45.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:11:45.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-05-09T00:11:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-05-09T00:11:45.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:11:45.781 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-09T00:11:45.781 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T00:11:45.781 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T00:11:45.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-09T00:11:46.075 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:46.407 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:46 smithi046 ceph-mon[30151]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:46.440 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:46 smithi142 ceph-mon[34533]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:46.603 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:46 smithi169 ceph-mon[34777]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:11:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.773+0000 7f49e6e32700 1 -- 172.21.15.46:0/4270163823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e0107df0 msgr2=0x7f49e0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.773+0000 7f49e6e32700 1 --2- 172.21.15.46:0/4270163823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e0107df0 0x7f49e0108200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f49c80099c0 tx=0x7f49c8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:11:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.773+0000 7f49e6e32700 1 -- 172.21.15.46:0/4270163823 shutdown_connections 2024-05-09T00:11:47.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.773+0000 7f49e6e32700 1 --2- 172.21.15.46:0/4270163823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e0108740 0x7f49e01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:47.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.773+0000 7f49e6e32700 1 --2- 172.21.15.46:0/4270163823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e01034c0 0x7f49e01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:47.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.773+0000 7f49e6e32700 1 --2- 172.21.15.46:0/4270163823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e0107df0 0x7f49e0108200 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:47.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.773+0000 7f49e6e32700 1 -- 172.21.15.46:0/4270163823 >> 172.21.15.46:0/4270163823 conn(0x7f49e00fedc0 msgr2=0x7f49e01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:47.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.773+0000 7f49e6e32700 1 -- 172.21.15.46:0/4270163823 shutdown_connections 2024-05-09T00:11:47.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.773+0000 7f49e6e32700 1 -- 172.21.15.46:0/4270163823 wait complete. 2024-05-09T00:11:47.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49e6e32700 1 Processor -- start 2024-05-09T00:11:47.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49e6e32700 1 -- start start 2024-05-09T00:11:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49e6e32700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e01034c0 0x7f49e00ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49e6e32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e0107df0 0x7f49e00ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49e6e32700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e0108740 0x7f49e00ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49e6e32700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49e01133a0 con 0x7f49e0107df0 2024-05-09T00:11:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49e6e32700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f49e0113220 con 0x7f49e01034c0 2024-05-09T00:11:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49e6e32700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f49e0113520 con 0x7f49e0108740 2024-05-09T00:11:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49dffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e0107df0 0x7f49e00ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49dffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e0107df0 0x7f49e00ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41778/0 (socket says 172.21.15.46:41778) 2024-05-09T00:11:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49dffff700 1 -- 172.21.15.46:0/1420522788 learned_addr learned my addr 172.21.15.46:0/1420522788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49e4bce700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e01034c0 0x7f49e00ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.774+0000 7f49e53cf700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e0108740 0x7f49e00ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.775+0000 7f49dffff700 1 -- 172.21.15.46:0/1420522788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e0108740 msgr2=0x7f49e00ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.775+0000 7f49dffff700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e0108740 0x7f49e00ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.775+0000 7f49dffff700 1 -- 172.21.15.46:0/1420522788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e01034c0 msgr2=0x7f49e00ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.775+0000 7f49dffff700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e01034c0 0x7f49e00ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.775+0000 7f49dffff700 1 -- 172.21.15.46:0/1420522788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e01a91e0 con 0x7f49e0107df0 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.775+0000 7f49e53cf700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e0108740 0x7f49e00ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.775+0000 7f49dffff700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e0107df0 0x7f49e00ff7e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f49c8006010 tx=0x7f49c80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.775+0000 7f49ddffb700 1 -- 172.21.15.46:0/1420522788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8026070 con 0x7f49e0107df0 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.775+0000 7f49ddffb700 1 -- 172.21.15.46:0/1420522788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49c8003f20 con 0x7f49e0107df0 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.775+0000 7f49e6e32700 1 -- 172.21.15.46:0/1420522788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e01a9470 con 0x7f49e0107df0 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.775+0000 7f49ddffb700 1 -- 172.21.15.46:0/1420522788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c800fcc0 con 0x7f49e0107df0 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.775+0000 7f49e6e32700 1 -- 172.21.15.46:0/1420522788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e01a98c0 con 0x7f49e0107df0 2024-05-09T00:11:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.776+0000 7f49e6e32700 1 -- 172.21.15.46:0/1420522788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e00512c0 con 0x7f49e0107df0 2024-05-09T00:11:47.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.778+0000 7f49ddffb700 1 -- 172.21.15.46:0/1420522788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f49c8029070 con 0x7f49e0107df0 2024-05-09T00:11:47.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.779+0000 7f49ddffb700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f49cc0b2880 0x7f49cc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:47.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.779+0000 7f49e4bce700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f49cc0b2880 0x7f49cc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:47.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.779+0000 7f49ddffb700 1 -- 172.21.15.46:0/1420522788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f49c80e2de0 con 0x7f49e0107df0 2024-05-09T00:11:47.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.779+0000 7f49e4bce700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f49cc0b2880 0x7f49cc0b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f49d0005950 tx=0x7f49d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:47.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:47.782+0000 7f49ddffb700 1 -- 172.21.15.46:0/1420522788 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49c805edf0 con 0x7f49e0107df0 2024-05-09T00:11:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.045+0000 7f49e6e32700 1 -- 172.21.15.46:0/1420522788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f49e002e7b0 con 0x7f49e0107df0 2024-05-09T00:11:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.045+0000 7f49ddffb700 1 -- 172.21.15.46:0/1420522788 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f49c8013070 con 0x7f49e0107df0 2024-05-09T00:11:48.043 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T00:11:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.047+0000 7f49e6e32700 1 -- 172.21.15.46:0/1420522788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f49cc0b2880 msgr2=0x7f49cc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.047+0000 7f49e6e32700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f49cc0b2880 0x7f49cc0b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f49d0005950 tx=0x7f49d000b410 comp rx=0 tx=0).stop 2024-05-09T00:11:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.047+0000 7f49e6e32700 1 -- 172.21.15.46:0/1420522788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e0107df0 msgr2=0x7f49e00ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.047+0000 7f49e6e32700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e0107df0 0x7f49e00ff7e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f49c8006010 tx=0x7f49c80047b0 comp rx=0 tx=0).stop 2024-05-09T00:11:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.047+0000 7f49e6e32700 1 -- 172.21.15.46:0/1420522788 shutdown_connections 2024-05-09T00:11:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.047+0000 7f49e6e32700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f49cc0b2880 0x7f49cc0b4d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.047+0000 7f49e6e32700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49e0108740 0x7f49e00ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.047+0000 7f49e6e32700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49e01034c0 0x7f49e00ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.047+0000 7f49e6e32700 1 --2- 172.21.15.46:0/1420522788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e0107df0 0x7f49e00ff7e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.047+0000 7f49e6e32700 1 -- 172.21.15.46:0/1420522788 >> 172.21.15.46:0/1420522788 conn(0x7f49e00fedc0 msgr2=0x7f49e006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.048+0000 7f49e6e32700 1 -- 172.21.15.46:0/1420522788 shutdown_connections 2024-05-09T00:11:48.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48.048+0000 7f49e6e32700 1 -- 172.21.15.46:0/1420522788 wait complete. 2024-05-09T00:11:48.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:48 smithi142 ceph-mon[34533]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:48.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:48 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1420522788' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T00:11:48.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:48 smithi046 ceph-mon[30151]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:48.703 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:48 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1420522788' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T00:11:48.733 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T00:11:48.733 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T00:11:48.734 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-05-09T00:11:48.734 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T00:11:48.734 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T00:11:48.734 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T00:11:48.734 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T00:11:48.734 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T00:11:48.734 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T00:11:48.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:11:48.734 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-05-09T00:11:48.752 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T00:11:48.752 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T00:11:48.752 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T00:11:48.752 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-05-09T00:11:48.752 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-05-09T00:11:48.752 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-05-09T00:11:48.752 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T00:11:48.752 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-05-09T00:11:48.752 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42044sec preferred_lft 42044sec 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-05-09T00:11:48.753 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-05-09T00:11:48.754 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T00:11:48.754 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T00:11:48.754 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-09T00:11:48.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-05-09T00:11:48.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-05-09T00:11:48.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-05-09T00:11:48.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T00:11:48.754 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-09T00:11:48.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T00:11:48.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:48 smithi169 ceph-mon[34777]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:11:48.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:48 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1420522788' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T00:11:48.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:11:48.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:11:48.964 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T00:11:48.991 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-05-09T00:11:48.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:11:48.993 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-05-09T00:11:49.067 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-05-09T00:11:49.067 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T00:11:49.067 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T00:11:49.067 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T00:11:49.067 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T00:11:49.068 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T00:11:49.068 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-09T00:11:49.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T00:11:49.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T00:11:49.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T00:11:49.068 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-09T00:11:49.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T00:11:49.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:11:49.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:11:49.341 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-05-09T00:11:49.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:11:49.358 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-05-09T00:11:49.415 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T00:11:49.415 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-09T00:11:49.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T00:11:49.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T00:11:49.416 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-09T00:11:49.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T00:11:49.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:11:49.536 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T00:11:49.536 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T00:11:49.537 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-09T00:11:49.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T00:11:49.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T00:11:49.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T00:11:49.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T00:11:49.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T00:11:49.537 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-09T00:11:49.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T00:11:49.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:11:49.664 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T00:11:49.664 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-09T00:11:49.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-05-09T00:11:49.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T00:11:49.664 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-09T00:11:49.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T00:11:49.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:11:49.747 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T00:11:49.747 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T00:11:49.763 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T00:11:49.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T00:11:49.763 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T00:11:49.818 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-09T00:11:49.819 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-09T00:11:49.819 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/d5361cba-819b-4ad7-a343-3956db6678cb 2024-05-09T00:11:49.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:11:49.819 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/d5361cba-819b-4ad7-a343-3956db6678cb /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-09T00:11:49.928 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-09T00:11:49.928 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-09T00:11:49.928 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-05-09T00:11:50.036 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-09T00:11:50.036 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-09T00:11:50.036 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-05-09T00:11:50.036 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.142:3300,172.21.15.169:3300" 2024-05-09T00:11:50.036 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:3300,172.21.15.142:3300,172.21.15.169:3300:/volumes/qa/sv_1/d5361cba-819b-4ad7-a343-3956db6678cb 2024-05-09T00:11:50.036 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=eec67860-0d96-11ef-bc97-c7b262605968" will pass to kernel 2024-05-09T00:11:50.037 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-09T00:11:50.037 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-05-09T00:11:50.037 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T00:11:50.037 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T00:11:50.037 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T00:11:50.039 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T00:11:50.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:11:50.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T00:11:50.070 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T00:11:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T00:11:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:11:50.170 INFO:teuthology.orchestra.run.smithi046.stdout:client24925 2024-05-09T00:11:50.170 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T00:11:50.170 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T00:11:50.189 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T00:11:50.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T00:11:50.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:11:50.286 INFO:teuthology.orchestra.run.smithi046.stdout:client24925 2024-05-09T00:11:50.287 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T00:11:50.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T00:11:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:11:50.346 INFO:teuthology.orchestra.run.smithi046.stdout:client24925 2024-05-09T00:11:50.346 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eec67860-0d96-11ef-bc97-c7b262605968.client24925/status') 2024-05-09T00:11:50.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T00:11:50.408 INFO:teuthology.orchestra.run.smithi046.stdout:instance: client.24925 (3)172.21.15.46:0/998743298 2024-05-09T00:11:50.408 INFO:teuthology.orchestra.run.smithi046.stdout:blocklisted: false 2024-05-09T00:11:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records in 2024-05-09T00:11:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records out 2024-05-09T00:11:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:70 bytes copied, 2.568e-05 s, 2.7 MB/s 2024-05-09T00:11:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T00:11:50.416 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T00:11:50.428 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T00:11:50.428 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T00:11:50.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-09T00:11:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.547+0000 7f1445cd8640 1 -- 172.21.15.46:0/1312894876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440103d80 msgr2=0x7f1440104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.547+0000 7f1445cd8640 1 --2- 172.21.15.46:0/1312894876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440103d80 0x7f1440104190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1434009900 tx=0x7f143402a3f0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.547+0000 7f1445cd8640 1 -- 172.21.15.46:0/1312894876 shutdown_connections 2024-05-09T00:11:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.547+0000 7f1445cd8640 1 --2- 172.21.15.46:0/1312894876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1440108a80 0x7f144010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.547+0000 7f1445cd8640 1 --2- 172.21.15.46:0/1312894876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440103d80 0x7f1440104190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.547+0000 7f1445cd8640 1 --2- 172.21.15.46:0/1312894876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1440102b40 0x7f1440103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.547+0000 7f1445cd8640 1 -- 172.21.15.46:0/1312894876 >> 172.21.15.46:0/1312894876 conn(0x7f14400fdd30 msgr2=0x7f14401001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.547+0000 7f1445cd8640 1 -- 172.21.15.46:0/1312894876 shutdown_connections 2024-05-09T00:11:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.547+0000 7f1445cd8640 1 -- 172.21.15.46:0/1312894876 wait complete. 2024-05-09T00:11:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.547+0000 7f1445cd8640 1 Processor -- start 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f1445cd8640 1 -- start start 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f1445cd8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440102b40 0x7f144010d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f1445cd8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1440103d80 0x7f144010bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f1445cd8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1440108a80 0x7f144010bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f1445cd8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f144010f7d0 con 0x7f1440102b40 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f1445cd8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f144010f650 con 0x7f1440103d80 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f1445cd8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f144010f950 con 0x7f1440108a80 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440102b40 0x7f144010d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440102b40 0x7f144010d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41796/0 (socket says 172.21.15.46:41796) 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143f7fe640 1 -- 172.21.15.46:0/1216217128 learned_addr learned my addr 172.21.15.46:0/1216217128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143ffff640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1440108a80 0x7f144010bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143f7fe640 1 -- 172.21.15.46:0/1216217128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1440108a80 msgr2=0x7f144010bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143effd640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1440103d80 0x7f144010bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143f7fe640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1440108a80 0x7f144010bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143f7fe640 1 -- 172.21.15.46:0/1216217128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1440103d80 msgr2=0x7f144010bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143f7fe640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1440103d80 0x7f144010bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143f7fe640 1 -- 172.21.15.46:0/1216217128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144010c690 con 0x7f1440102b40 2024-05-09T00:11:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143ffff640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1440108a80 0x7f144010bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143effd640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1440103d80 0x7f144010bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:11:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.548+0000 7f143f7fe640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440102b40 0x7f144010d410 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f143000c970 tx=0x7f143000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.549+0000 7f143cff9640 1 -- 172.21.15.46:0/1216217128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430007c70 con 0x7f1440102b40 2024-05-09T00:11:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.549+0000 7f1445cd8640 1 -- 172.21.15.46:0/1216217128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14401a4390 con 0x7f1440102b40 2024-05-09T00:11:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.549+0000 7f143cff9640 1 -- 172.21.15.46:0/1216217128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1430007e10 con 0x7f1440102b40 2024-05-09T00:11:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.549+0000 7f143cff9640 1 -- 172.21.15.46:0/1216217128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430018760 con 0x7f1440102b40 2024-05-09T00:11:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.549+0000 7f1445cd8640 1 -- 172.21.15.46:0/1216217128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14401a47f0 con 0x7f1440102b40 2024-05-09T00:11:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.550+0000 7f1445cd8640 1 -- 172.21.15.46:0/1216217128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1440107890 con 0x7f1440102b40 2024-05-09T00:11:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.551+0000 7f143cff9640 1 -- 172.21.15.46:0/1216217128 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f143000fab0 con 0x7f1440102b40 2024-05-09T00:11:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.552+0000 7f143cff9640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f14280b27e0 0x7f14280b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.552+0000 7f143effd640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f14280b27e0 0x7f14280b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.552+0000 7f143cff9640 1 -- 172.21.15.46:0/1216217128 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f14300e2480 con 0x7f1440102b40 2024-05-09T00:11:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.553+0000 7f143effd640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f14280b27e0 0x7f14280b4cf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f143402a910 tx=0x7f1434005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.556+0000 7f143cff9640 1 -- 172.21.15.46:0/1216217128 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1430014030 con 0x7f1440102b40 2024-05-09T00:11:50.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:50 smithi142 ceph-mon[34533]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:50.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:50.792+0000 7f1445cd8640 1 -- 172.21.15.46:0/1216217128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f144002dc30 con 0x7f1440102b40 2024-05-09T00:11:50.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:50 smithi046 ceph-mon[30151]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:50.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:50 smithi169 ceph-mon[34777]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:11:51.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.398+0000 7f143cff9640 1 -- 172.21.15.46:0/1216217128 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f14300aa1f0 con 0x7f1440102b40 2024-05-09T00:11:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.402+0000 7f1445cd8640 1 -- 172.21.15.46:0/1216217128 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f14280b27e0 msgr2=0x7f14280b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.402+0000 7f1445cd8640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f14280b27e0 0x7f14280b4cf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f143402a910 tx=0x7f1434005fd0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.402+0000 7f1445cd8640 1 -- 172.21.15.46:0/1216217128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440102b40 msgr2=0x7f144010d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.402+0000 7f1445cd8640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440102b40 0x7f144010d410 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f143000c970 tx=0x7f143000ce30 comp rx=0 tx=0).stop 2024-05-09T00:11:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.402+0000 7f1445cd8640 1 -- 172.21.15.46:0/1216217128 shutdown_connections 2024-05-09T00:11:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.402+0000 7f1445cd8640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f14280b27e0 0x7f14280b4cf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.402+0000 7f1445cd8640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1440108a80 0x7f144010bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.402+0000 7f1445cd8640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1440103d80 0x7f144010bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.402+0000 7f1445cd8640 1 --2- 172.21.15.46:0/1216217128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1440102b40 0x7f144010d410 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.402+0000 7f1445cd8640 1 -- 172.21.15.46:0/1216217128 >> 172.21.15.46:0/1216217128 conn(0x7f14400fdd30 msgr2=0x7f1440104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.402+0000 7f1445cd8640 1 -- 172.21.15.46:0/1216217128 shutdown_connections 2024-05-09T00:11:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.402+0000 7f1445cd8640 1 -- 172.21.15.46:0/1216217128 wait complete. 2024-05-09T00:11:51.414 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-09T00:11:51.441 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-09T00:11:51.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T00:11:51.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.596+0000 7f3b5978d640 1 -- 172.21.15.46:0/141899751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b54108a70 msgr2=0x7f3b5410ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:51.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.596+0000 7f3b5978d640 1 --2- 172.21.15.46:0/141899751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b54108a70 0x7f3b5410ae90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009ca0 tx=0x7f3b4c02a8c0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.596+0000 7f3b5978d640 1 -- 172.21.15.46:0/141899751 shutdown_connections 2024-05-09T00:11:51.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.596+0000 7f3b5978d640 1 --2- 172.21.15.46:0/141899751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b54108a70 0x7f3b5410ae90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.596+0000 7f3b5978d640 1 --2- 172.21.15.46:0/141899751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b54103d70 0x7f3b54104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.596+0000 7f3b5978d640 1 --2- 172.21.15.46:0/141899751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b54102b30 0x7f3b54103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.596+0000 7f3b5978d640 1 -- 172.21.15.46:0/141899751 >> 172.21.15.46:0/141899751 conn(0x7f3b540fdd20 msgr2=0x7f3b541001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:51.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.596+0000 7f3b5978d640 1 -- 172.21.15.46:0/141899751 shutdown_connections 2024-05-09T00:11:51.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.596+0000 7f3b5978d640 1 -- 172.21.15.46:0/141899751 wait complete. 2024-05-09T00:11:51.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.596+0000 7f3b5978d640 1 Processor -- start 2024-05-09T00:11:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.596+0000 7f3b5978d640 1 -- start start 2024-05-09T00:11:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b5978d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b54102b30 0x7f3b54075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b5978d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b54103d70 0x7f3b5419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b5978d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b54108a70 0x7f3b541a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b5978d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5410e690 con 0x7f3b54102b30 2024-05-09T00:11:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b5978d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5410e510 con 0x7f3b54108a70 2024-05-09T00:11:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b5978d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5410e810 con 0x7f3b54103d70 2024-05-09T00:11:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b52ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b54102b30 0x7f3b54075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b52ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b54102b30 0x7f3b54075990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41808/0 (socket says 172.21.15.46:41808) 2024-05-09T00:11:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b52ffd640 1 -- 172.21.15.46:0/1208044048 learned_addr learned my addr 172.21.15.46:0/1208044048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b4bfff640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b54103d70 0x7f3b5419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b52ffd640 1 -- 172.21.15.46:0/1208044048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b54103d70 msgr2=0x7f3b5419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b52ffd640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b54103d70 0x7f3b5419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b52ffd640 1 -- 172.21.15.46:0/1208044048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b54108a70 msgr2=0x7f3b541a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b52ffd640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b54108a70 0x7f3b541a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b52ffd640 1 -- 172.21.15.46:0/1208044048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b541a1d40 con 0x7f3b54102b30 2024-05-09T00:11:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b52ffd640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b54102b30 0x7f3b54075990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400c970 tx=0x7f3b4400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b50ff9640 1 -- 172.21.15.46:0/1208044048 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44007c70 con 0x7f3b54102b30 2024-05-09T00:11:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b50ff9640 1 -- 172.21.15.46:0/1208044048 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b44007e10 con 0x7f3b54102b30 2024-05-09T00:11:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b50ff9640 1 -- 172.21.15.46:0/1208044048 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b440187b0 con 0x7f3b54102b30 2024-05-09T00:11:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b5978d640 1 -- 172.21.15.46:0/1208044048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b541a1fd0 con 0x7f3b54102b30 2024-05-09T00:11:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.597+0000 7f3b5978d640 1 -- 172.21.15.46:0/1208044048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b541a2540 con 0x7f3b54102b30 2024-05-09T00:11:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.598+0000 7f3b5978d640 1 -- 172.21.15.46:0/1208044048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b54107880 con 0x7f3b54102b30 2024-05-09T00:11:51.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.601+0000 7f3b50ff9640 1 -- 172.21.15.46:0/1208044048 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3b4400fab0 con 0x7f3b54102b30 2024-05-09T00:11:51.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.602+0000 7f3b50ff9640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3b400b2870 0x7f3b400b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:51.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.602+0000 7f3b4bfff640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3b400b2870 0x7f3b400b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:51.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.602+0000 7f3b50ff9640 1 -- 172.21.15.46:0/1208044048 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f3b440e2370 con 0x7f3b54102b30 2024-05-09T00:11:51.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.602+0000 7f3b4bfff640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3b400b2870 0x7f3b400b4d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c005fd0 tx=0x7f3b3c009490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.606+0000 7f3b50ff9640 1 -- 172.21.15.46:0/1208044048 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b44014030 con 0x7f3b54102b30 2024-05-09T00:11:51.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:51 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1216217128' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T00:11:51.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.802+0000 7f3b5978d640 1 -- 172.21.15.46:0/1208044048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b5402de20 con 0x7f3b54102b30 2024-05-09T00:11:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.804+0000 7f3b50ff9640 1 -- 172.21.15.46:0/1208044048 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22694 (secure 0 0 0) 0x7f3b440aa1a0 con 0x7f3b54102b30 2024-05-09T00:11:51.802 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:51.802 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","created":"2024-05-09T00:00:14.356524+0000","modified":"2024-05-09T00:09:10.458026+0000","last_up_change":"2024-05-09T00:07:23.332796+0000","last_in_change":"2024-05-09T00:07:03.238539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:03:33.278910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:08:21.014416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:08:24.922968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"740d6388-5c1f-45eb-93b0-7583bfa5a225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6803","nonce":3727342125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6805","nonce":3727342125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6809","nonce":3727342125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6807","nonce":3727342125}]},"public_addr":"172.21.15.46:6803/3727342125","cluster_addr":"172.21.15.46:6805/3727342125","heartbeat_back_addr":"172.21.15.46:6809/3727342125","heartbeat_front_addr":"172.21.15.46:6807/3727342125","state":["exists","up"]},{"osd":1,"uuid":"85fbf3ae-0be8-4504-84d6-66fae0d2169d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6801","nonce":3156042469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6803","nonce":3156042469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6807","nonce":3156042469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6805","nonce":3156042469}]},"public_addr":"172.21.15.142:6801/3156042469","cluster_addr":"172.21.15.142:6803/3156042469","heartbeat_back_addr":"172.21.15.142:6807/3156042469","heartbeat_front_addr":"172.21.15.142:6805/3156042469","state":["exists","up"]},{"osd":2,"uuid":"05c070e7-83ad-4949-9834-4ce51ac0a2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6801","nonce":2680673064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6803","nonce":2680673064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6807","nonce":2680673064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6805","nonce":2680673064}]},"public_addr":"172.21.15.169:6801/2680673064","cluster_addr":"172.21.15.169:6803/2680673064","heartbeat_back_addr":"172.21.15.169:6807/2680673064","heartbeat_front_addr":"172.21.15.169:6805/2680673064","state":["exists","up"]},{"osd":3,"uuid":"09d7bbe1-58b6-4407-bef2-920f179dfb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6811","nonce":1158536261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6813","nonce":1158536261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6817","nonce":1158536261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6815","nonce":1158536261}]},"public_addr":"172.21.15.46:6811/1158536261","cluster_addr":"172.21.15.46:6813/1158536261","heartbeat_back_addr":"172.21.15.46:6817/1158536261","heartbeat_front_addr":"172.21.15.46:6815/1158536261","state":["exists","up"]},{"osd":4,"uuid":"66c9644b-6b0b-4fe7-aa63-b9d87bff7153","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6809","nonce":2299616865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6811","nonce":2299616865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6815","nonce":2299616865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6813","nonce":2299616865}]},"public_addr":"172.21.15.142:6809/2299616865","cluster_addr":"172.21.15.142:6811/2299616865","heartbeat_back_addr":"172.21.15.142:6815/2299616865","heartbeat_front_addr":"172.21.15.142:6813/2299616865","state":["exists","up"]},{"osd":5,"uuid":"96a72e01-1c5f-418e-bb9b-b790d9b3059b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6809","nonce":445044713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6811","nonce":445044713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6815","nonce":445044713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6813","nonce":445044713}]},"public_addr":"172.21.15.169:6809/445044713","cluster_addr":"172.21.15.169:6811/445044713","heartbeat_back_addr":"172.21.15.169:6815/445044713","heartbeat_front_addr":"172.21.15.169:6813/445044713","state":["exists","up"]},{"osd":6,"uuid":"6b459542-0e56-4369-b542-e36c46a18f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6819","nonce":1394035482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6821","nonce":1394035482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6825","nonce":1394035482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6823","nonce":1394035482}]},"public_addr":"172.21.15.46:6819/1394035482","cluster_addr":"172.21.15.46:6821/1394035482","heartbeat_back_addr":"172.21.15.46:6825/1394035482","heartbeat_front_addr":"172.21.15.46:6823/1394035482","state":["exists","up"]},{"osd":7,"uuid":"786b4517-4cbd-4fa3-b5a0-39cccf839893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6817","nonce":596805607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6819","nonce":596805607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6823","nonce":596805607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6821","nonce":596805607}]},"public_addr":"172.21.15.142:6817/596805607","cluster_addr":"172.21.15.142:6819/596805607","heartbeat_back_addr":"172.21.15.142:6823/596805607","heartbeat_front_addr":"172.21.15.142:6821/596805607","state":["exists","up"]},{"osd":8,"uuid":"b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6817","nonce":2650652376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6819","nonce":2650652376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6823","nonce":2650652376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6821","nonce":2650652376}]},"public_addr":"172.21.15.169:6817/2650652376","cluster_addr":"172.21.15.169:6819/2650652376","heartbeat_back_addr":"172.21.15.169:6823/2650652376","heartbeat_front_addr":"172.21.15.169:6821/2650652376","state":["exists","up"]},{"osd":9,"uuid":"32df0dab-5875-4beb-b733-60a915c94a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6827","nonce":374861248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6829","nonce":374861248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6833","nonce":374861248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6831","nonce":374861248}]},"public_addr":"172.21.15.46:6827/374861248","cluster_addr":"172.21.15.46:6829/374861248","heartbeat_back_addr":"172.21.15.46:6833/374861248","heartbeat_front_addr":"172.21.15.46:6831/374861248","state":["exists","up"]},{"osd":10,"uuid":"380a4f47-e528-4258-8836-32eac0e6e630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6825","nonce":1364891982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6827","nonce":1364891982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6831","nonce":1364891982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6829","nonce":1364891982}]},"public_addr":"172.21.15.142:6825/1364891982","cluster_addr":"172.21.15.142:6827/1364891982","heartbeat_back_addr":"172.21.15.142:6831/1364891982","heartbeat_front_addr":"172.21.15.142:6829/1364891982","state":["exists","up"]},{"osd":11,"uuid":"56b04b15-6fcd-497a-aa66-19da9624b656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6825","nonce":4262478976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6827","nonce":4262478976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6831","nonce":4262478976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6829","nonce":4262478976}]},"public_addr":"172.21.15.169:6825/4262478976","cluster_addr":"172.21.15.169:6827/4262478976","heartbeat_back_addr":"172.21.15.169:6831/4262478976","heartbeat_front_addr":"172.21.15.169:6829/4262478976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:02:42.159298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:06.053328+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:29.587137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:54.573782+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:20.276073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:45.320565+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:10.985522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:36.806137+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:02.266736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:29.788773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:55.827639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:07:21.762382+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/1072138393":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2786891364":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/7723180":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/626888490":"2024-05-10T00:01:09.191239+0000","172.21.15.46:6800/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/4236543921":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2168313126":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6800/953818590":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6801/953818590":"2024-05-10T00:00:41.833323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:11:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.806+0000 7f3b5978d640 1 -- 172.21.15.46:0/1208044048 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3b400b2870 msgr2=0x7f3b400b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.806+0000 7f3b5978d640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3b400b2870 0x7f3b400b4d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c005fd0 tx=0x7f3b3c009490 comp rx=0 tx=0).stop 2024-05-09T00:11:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.806+0000 7f3b5978d640 1 -- 172.21.15.46:0/1208044048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b54102b30 msgr2=0x7f3b54075990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.806+0000 7f3b5978d640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b54102b30 0x7f3b54075990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400c970 tx=0x7f3b4400ce30 comp rx=0 tx=0).stop 2024-05-09T00:11:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.806+0000 7f3b5978d640 1 -- 172.21.15.46:0/1208044048 shutdown_connections 2024-05-09T00:11:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.806+0000 7f3b5978d640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3b400b2870 0x7f3b400b4d80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.806+0000 7f3b5978d640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b54108a70 0x7f3b541a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.806+0000 7f3b5978d640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b54103d70 0x7f3b5419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.806+0000 7f3b5978d640 1 --2- 172.21.15.46:0/1208044048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b54102b30 0x7f3b54075990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.806+0000 7f3b5978d640 1 -- 172.21.15.46:0/1208044048 >> 172.21.15.46:0/1208044048 conn(0x7f3b540fdd20 msgr2=0x7f3b54104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.806+0000 7f3b5978d640 1 -- 172.21.15.46:0/1208044048 shutdown_connections 2024-05-09T00:11:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.806+0000 7f3b5978d640 1 -- 172.21.15.46:0/1208044048 wait complete. 2024-05-09T00:11:51.809 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:51 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1216217128' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T00:11:51.816 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T00:03:33.278910+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T00:08:21.014416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T00:08:24.922968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T00:11:51.816 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T00:11:51.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:51 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1216217128' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T00:11:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.942+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/3283190802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cb8102370 msgr2=0x7f7cb810bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.942+0000 7f7cbf8a8640 1 --2- 172.21.15.46:0/3283190802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cb8102370 0x7f7cb810bdd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009ca0 tx=0x7f7cb002a8c0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.942+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/3283190802 shutdown_connections 2024-05-09T00:11:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.942+0000 7f7cbf8a8640 1 --2- 172.21.15.46:0/3283190802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cb8102370 0x7f7cb810bdd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.942+0000 7f7cbf8a8640 1 --2- 172.21.15.46:0/3283190802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cb8101870 0x7f7cb8101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.942+0000 7f7cbf8a8640 1 --2- 172.21.15.46:0/3283190802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb81051c0 0x7f7cb81055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.942+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/3283190802 >> 172.21.15.46:0/3283190802 conn(0x7f7cb80fd470 msgr2=0x7f7cb80ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.942+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/3283190802 shutdown_connections 2024-05-09T00:11:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.942+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/3283190802 wait complete. 2024-05-09T00:11:51.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.942+0000 7f7cbf8a8640 1 Processor -- start 2024-05-09T00:11:51.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbf8a8640 1 -- start start 2024-05-09T00:11:51.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbf8a8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cb8101870 0x7f7cb819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:51.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbf8a8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb8102370 0x7f7cb819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbf8a8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cb81051c0 0x7f7cb81a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbf8a8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb810f750 con 0x7f7cb8101870 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbf8a8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb810f5d0 con 0x7f7cb81051c0 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbf8a8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb810f8d0 con 0x7f7cb8102370 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbd644640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cb8101870 0x7f7cb819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbd644640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cb8101870 0x7f7cb819c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41824/0 (socket says 172.21.15.46:41824) 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbd644640 1 -- 172.21.15.46:0/562144439 learned_addr learned my addr 172.21.15.46:0/562144439 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbde45640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cb81051c0 0x7f7cb81a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbce43640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb8102370 0x7f7cb819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbd644640 1 -- 172.21.15.46:0/562144439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb8102370 msgr2=0x7f7cb819ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbd644640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb8102370 0x7f7cb819ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbd644640 1 -- 172.21.15.46:0/562144439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cb81051c0 msgr2=0x7f7cb81a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbd644640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cb81051c0 0x7f7cb81a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbd644640 1 -- 172.21.15.46:0/562144439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb81a4020 con 0x7f7cb8101870 2024-05-09T00:11:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbce43640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb8102370 0x7f7cb819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.943+0000 7f7cbd644640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cb8101870 0x7f7cb819c4e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400e9f0 tx=0x7f7cb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.944+0000 7f7cae7fc640 1 -- 172.21.15.46:0/562144439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400ccc0 con 0x7f7cb8101870 2024-05-09T00:11:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.944+0000 7f7cae7fc640 1 -- 172.21.15.46:0/562144439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb40035f0 con 0x7f7cb8101870 2024-05-09T00:11:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.944+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/562144439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb81a4310 con 0x7f7cb8101870 2024-05-09T00:11:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.944+0000 7f7cae7fc640 1 -- 172.21.15.46:0/562144439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400ce60 con 0x7f7cb8101870 2024-05-09T00:11:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.944+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/562144439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb81a4810 con 0x7f7cb8101870 2024-05-09T00:11:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.945+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/562144439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb81a4b30 con 0x7f7cb8101870 2024-05-09T00:11:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.946+0000 7f7cae7fc640 1 -- 172.21.15.46:0/562144439 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7cb4004030 con 0x7f7cb8101870 2024-05-09T00:11:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.947+0000 7f7cae7fc640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7ca40b27f0 0x7f7ca40b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.947+0000 7f7cbce43640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7ca40b27f0 0x7f7ca40b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.947+0000 7f7cae7fc640 1 -- 172.21.15.46:0/562144439 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7cb40e2640 con 0x7f7cb8101870 2024-05-09T00:11:51.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.948+0000 7f7cbce43640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7ca40b27f0 0x7f7ca40b4d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7cb819d580 tx=0x7f7ca8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:51.952+0000 7f7cae7fc640 1 -- 172.21.15.46:0/562144439 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7cb40aa360 con 0x7f7cb8101870 2024-05-09T00:11:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.145+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/562144439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7cb81a4d00 con 0x7f7cb8101870 2024-05-09T00:11:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.146+0000 7f7cae7fc640 1 -- 172.21.15.46:0/562144439 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f7cb40a9c60 con 0x7f7cb8101870 2024-05-09T00:11:52.144 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-09T00:11:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.148+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/562144439 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7ca40b27f0 msgr2=0x7f7ca40b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.148+0000 7f7cbf8a8640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7ca40b27f0 0x7f7ca40b4d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7cb819d580 tx=0x7f7ca8005f90 comp rx=0 tx=0).stop 2024-05-09T00:11:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.148+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/562144439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cb8101870 msgr2=0x7f7cb819c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.148+0000 7f7cbf8a8640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cb8101870 0x7f7cb819c4e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400e9f0 tx=0x7f7cb400eeb0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.148+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/562144439 shutdown_connections 2024-05-09T00:11:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.148+0000 7f7cbf8a8640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7ca40b27f0 0x7f7ca40b4d00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.148+0000 7f7cbf8a8640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cb81051c0 0x7f7cb81a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.148+0000 7f7cbf8a8640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb8102370 0x7f7cb819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.148+0000 7f7cbf8a8640 1 --2- 172.21.15.46:0/562144439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cb8101870 0x7f7cb819c4e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.148+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/562144439 >> 172.21.15.46:0/562144439 conn(0x7f7cb80fd470 msgr2=0x7f7cb81028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.148+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/562144439 shutdown_connections 2024-05-09T00:11:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.148+0000 7f7cbf8a8640 1 -- 172.21.15.46:0/562144439 wait complete. 2024-05-09T00:11:52.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T00:11:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.278+0000 7f26bed5f640 1 -- 172.21.15.46:0/896734252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b8108a50 msgr2=0x7f26b810ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.278+0000 7f26bed5f640 1 --2- 172.21.15.46:0/896734252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b8108a50 0x7f26b810ae70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009ca0 tx=0x7f26b002a8c0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.278+0000 7f26bed5f640 1 -- 172.21.15.46:0/896734252 shutdown_connections 2024-05-09T00:11:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.278+0000 7f26bed5f640 1 --2- 172.21.15.46:0/896734252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b8108a50 0x7f26b810ae70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.278+0000 7f26bed5f640 1 --2- 172.21.15.46:0/896734252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26b8103d50 0x7f26b8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.278+0000 7f26bed5f640 1 --2- 172.21.15.46:0/896734252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b8102b10 0x7f26b8103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.278+0000 7f26bed5f640 1 -- 172.21.15.46:0/896734252 >> 172.21.15.46:0/896734252 conn(0x7f26b80fdd00 msgr2=0x7f26b81001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.278+0000 7f26bed5f640 1 -- 172.21.15.46:0/896734252 shutdown_connections 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.278+0000 7f26bed5f640 1 -- 172.21.15.46:0/896734252 wait complete. 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bed5f640 1 Processor -- start 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bed5f640 1 -- start start 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bed5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b8102b10 0x7f26b8199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bed5f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26b8103d50 0x7f26b8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bed5f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b8108a50 0x7f26b81a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bed5f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26b810e690 con 0x7f26b8108a50 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bed5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26b810e510 con 0x7f26b8102b10 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bed5f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26b810e810 con 0x7f26b8103d50 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bd2fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b8108a50 0x7f26b81a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bd2fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b8108a50 0x7f26b81a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41836/0 (socket says 172.21.15.46:41836) 2024-05-09T00:11:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bd2fc640 1 -- 172.21.15.46:0/3495578343 learned_addr learned my addr 172.21.15.46:0/3495578343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26b7fff640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26b8103d50 0x7f26b8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bcafb640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b8102b10 0x7f26b8199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bd2fc640 1 -- 172.21.15.46:0/3495578343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26b8103d50 msgr2=0x7f26b8199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bd2fc640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26b8103d50 0x7f26b8199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bd2fc640 1 -- 172.21.15.46:0/3495578343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b8102b10 msgr2=0x7f26b8199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.279+0000 7f26bd2fc640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b8102b10 0x7f26b8199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.280+0000 7f26bd2fc640 1 -- 172.21.15.46:0/3495578343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b8069fc0 con 0x7f26b8108a50 2024-05-09T00:11:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.280+0000 7f26bd2fc640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b8108a50 0x7f26b81a09d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009430 tx=0x7f26b00317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.280+0000 7f26b5ffb640 1 -- 172.21.15.46:0/3495578343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0003bf0 con 0x7f26b8108a50 2024-05-09T00:11:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.280+0000 7f26b5ffb640 1 -- 172.21.15.46:0/3495578343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26b0003d90 con 0x7f26b8108a50 2024-05-09T00:11:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.280+0000 7f26bed5f640 1 -- 172.21.15.46:0/3495578343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b806a250 con 0x7f26b8108a50 2024-05-09T00:11:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.280+0000 7f26b5ffb640 1 -- 172.21.15.46:0/3495578343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b002fc80 con 0x7f26b8108a50 2024-05-09T00:11:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.280+0000 7f26bed5f640 1 -- 172.21.15.46:0/3495578343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b806a7f0 con 0x7f26b8108a50 2024-05-09T00:11:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.281+0000 7f26bed5f640 1 -- 172.21.15.46:0/3495578343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b806c4f0 con 0x7f26b8108a50 2024-05-09T00:11:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.283+0000 7f26b5ffb640 1 -- 172.21.15.46:0/3495578343 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f26b0004710 con 0x7f26b8108a50 2024-05-09T00:11:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.284+0000 7f26b5ffb640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f26a00b2870 0x7f26a00b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.284+0000 7f26bcafb640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f26a00b2870 0x7f26a00b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.284+0000 7f26b5ffb640 1 -- 172.21.15.46:0/3495578343 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f26b0103800 con 0x7f26b8108a50 2024-05-09T00:11:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.284+0000 7f26bcafb640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f26a00b2870 0x7f26a00b4d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26b8103710 tx=0x7f26a800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.289+0000 7f26b5ffb640 1 -- 172.21.15.46:0/3495578343 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26b00cb520 con 0x7f26b8108a50 2024-05-09T00:11:52.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.483+0000 7f26bed5f640 1 -- 172.21.15.46:0/3495578343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f26b81a3250 con 0x7f26b8108a50 2024-05-09T00:11:52.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.483+0000 7f26b5ffb640 1 -- 172.21.15.46:0/3495578343 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f26b00cae20 con 0x7f26b8108a50 2024-05-09T00:11:52.482 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-09T00:11:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.486+0000 7f26bed5f640 1 -- 172.21.15.46:0/3495578343 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f26a00b2870 msgr2=0x7f26a00b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.486+0000 7f26bed5f640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f26a00b2870 0x7f26a00b4d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26b8103710 tx=0x7f26a800b390 comp rx=0 tx=0).stop 2024-05-09T00:11:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.486+0000 7f26bed5f640 1 -- 172.21.15.46:0/3495578343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b8108a50 msgr2=0x7f26b81a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.486+0000 7f26bed5f640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b8108a50 0x7f26b81a09d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009430 tx=0x7f26b00317c0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.486+0000 7f26bed5f640 1 -- 172.21.15.46:0/3495578343 shutdown_connections 2024-05-09T00:11:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.486+0000 7f26bed5f640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f26a00b2870 0x7f26a00b4d80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.486+0000 7f26bed5f640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b8108a50 0x7f26b81a09d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.486+0000 7f26bed5f640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26b8103d50 0x7f26b8199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.486+0000 7f26bed5f640 1 --2- 172.21.15.46:0/3495578343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26b8102b10 0x7f26b8199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.486+0000 7f26bed5f640 1 -- 172.21.15.46:0/3495578343 >> 172.21.15.46:0/3495578343 conn(0x7f26b80fdd00 msgr2=0x7f26b8104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.486+0000 7f26bed5f640 1 -- 172.21.15.46:0/3495578343 shutdown_connections 2024-05-09T00:11:52.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.486+0000 7f26bed5f640 1 -- 172.21.15.46:0/3495578343 wait complete. 2024-05-09T00:11:52.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:52 smithi046 ceph-mon[30151]: pgmap v391: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:52.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:52 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1216217128' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T00:11:52.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:52 smithi046 ceph-mon[30151]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:11:52.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:52 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1208044048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:11:52.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:52 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/562144439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:11:52.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T00:11:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.616+0000 7f75d4973640 1 -- 172.21.15.46:0/693429366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d0102b80 msgr2=0x7f75d0103880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.616+0000 7f75d4973640 1 --2- 172.21.15.46:0/693429366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d0102b80 0x7f75d0103880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009900 tx=0x7f75c002a410 comp rx=0 tx=0).stop 2024-05-09T00:11:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.616+0000 7f75d4973640 1 -- 172.21.15.46:0/693429366 shutdown_connections 2024-05-09T00:11:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.616+0000 7f75d4973640 1 --2- 172.21.15.46:0/693429366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75d0108ac0 0x7f75d010aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.616+0000 7f75d4973640 1 --2- 172.21.15.46:0/693429366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d0103dc0 0x7f75d01041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.616+0000 7f75d4973640 1 --2- 172.21.15.46:0/693429366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d0102b80 0x7f75d0103880 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.616+0000 7f75d4973640 1 -- 172.21.15.46:0/693429366 >> 172.21.15.46:0/693429366 conn(0x7f75d00fdd70 msgr2=0x7f75d0100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.616+0000 7f75d4973640 1 -- 172.21.15.46:0/693429366 shutdown_connections 2024-05-09T00:11:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.616+0000 7f75d4973640 1 -- 172.21.15.46:0/693429366 wait complete. 2024-05-09T00:11:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.617+0000 7f75d4973640 1 Processor -- start 2024-05-09T00:11:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.617+0000 7f75d4973640 1 -- start start 2024-05-09T00:11:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75d4973640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d0102b80 0x7f75d0075970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75d4973640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75d0103dc0 0x7f75d019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75d4973640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d0108ac0 0x7f75d01a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75d4973640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75d010e610 con 0x7f75d0102b80 2024-05-09T00:11:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75d4973640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75d010e490 con 0x7f75d0108ac0 2024-05-09T00:11:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75d4973640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f75d010e790 con 0x7f75d0103dc0 2024-05-09T00:11:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75ce59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d0102b80 0x7f75d0075970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75ce59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d0102b80 0x7f75d0075970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41852/0 (socket says 172.21.15.46:41852) 2024-05-09T00:11:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75ce59c640 1 -- 172.21.15.46:0/4256947576 learned_addr learned my addr 172.21.15.46:0/4256947576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75c5d9b640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75d0103dc0 0x7f75d019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75ced9d640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d0108ac0 0x7f75d01a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75ce59c640 1 -- 172.21.15.46:0/4256947576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75d0103dc0 msgr2=0x7f75d019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75ce59c640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75d0103dc0 0x7f75d019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75ce59c640 1 -- 172.21.15.46:0/4256947576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d0108ac0 msgr2=0x7f75d01a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75ce59c640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d0108ac0 0x7f75d01a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75ce59c640 1 -- 172.21.15.46:0/4256947576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d01a1ea0 con 0x7f75d0102b80 2024-05-09T00:11:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75ce59c640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d0102b80 0x7f75d0075970 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f75c0006010 tx=0x7f75c0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.618+0000 7f75c7fff640 1 -- 172.21.15.46:0/4256947576 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c003f030 con 0x7f75d0102b80 2024-05-09T00:11:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.619+0000 7f75d4973640 1 -- 172.21.15.46:0/4256947576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d01a2130 con 0x7f75d0102b80 2024-05-09T00:11:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.619+0000 7f75c7fff640 1 -- 172.21.15.46:0/4256947576 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c0003aa0 con 0x7f75d0102b80 2024-05-09T00:11:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.619+0000 7f75c7fff640 1 -- 172.21.15.46:0/4256947576 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0035070 con 0x7f75d0102b80 2024-05-09T00:11:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.619+0000 7f75d4973640 1 -- 172.21.15.46:0/4256947576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d01a2670 con 0x7f75d0102b80 2024-05-09T00:11:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.619+0000 7f75d4973640 1 -- 172.21.15.46:0/4256947576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d01078d0 con 0x7f75d0102b80 2024-05-09T00:11:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.621+0000 7f75c7fff640 1 -- 172.21.15.46:0/4256947576 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f75c0042030 con 0x7f75d0102b80 2024-05-09T00:11:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.622+0000 7f75c7fff640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f75bc0b27f0 0x7f75bc0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.622+0000 7f75c5d9b640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f75bc0b27f0 0x7f75bc0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.622+0000 7f75c7fff640 1 -- 172.21.15.46:0/4256947576 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f75c0102d70 con 0x7f75d0102b80 2024-05-09T00:11:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.622+0000 7f75c5d9b640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f75bc0b27f0 0x7f75bc0b4d00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f75b8005fd0 tx=0x7f75b8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.625+0000 7f75c7fff640 1 -- 172.21.15.46:0/4256947576 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f75c00caa50 con 0x7f75d0102b80 2024-05-09T00:11:52.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:52 smithi142 ceph-mon[34533]: pgmap v391: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:52.691 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:52 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1216217128' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T00:11:52.691 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:52 smithi142 ceph-mon[34533]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:11:52.691 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:52 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1208044048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:11:52.691 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:52 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/562144439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:11:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.818+0000 7f75d4973640 1 -- 172.21.15.46:0/4256947576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f75d002dc30 con 0x7f75d0102b80 2024-05-09T00:11:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.819+0000 7f75c7fff640 1 -- 172.21.15.46:0/4256947576 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f75c00ca350 con 0x7f75d0102b80 2024-05-09T00:11:52.817 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-09T00:11:52.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.821+0000 7f75d4973640 1 -- 172.21.15.46:0/4256947576 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f75bc0b27f0 msgr2=0x7f75bc0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.821+0000 7f75d4973640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f75bc0b27f0 0x7f75bc0b4d00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f75b8005fd0 tx=0x7f75b8005850 comp rx=0 tx=0).stop 2024-05-09T00:11:52.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.821+0000 7f75d4973640 1 -- 172.21.15.46:0/4256947576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d0102b80 msgr2=0x7f75d0075970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:52.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.821+0000 7f75d4973640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d0102b80 0x7f75d0075970 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f75c0006010 tx=0x7f75c0031a50 comp rx=0 tx=0).stop 2024-05-09T00:11:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.821+0000 7f75d4973640 1 -- 172.21.15.46:0/4256947576 shutdown_connections 2024-05-09T00:11:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.821+0000 7f75d4973640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f75bc0b27f0 0x7f75bc0b4d00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.821+0000 7f75d4973640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75d0108ac0 0x7f75d01a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.821+0000 7f75d4973640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75d0103dc0 0x7f75d019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.821+0000 7f75d4973640 1 --2- 172.21.15.46:0/4256947576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75d0102b80 0x7f75d0075970 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.821+0000 7f75d4973640 1 -- 172.21.15.46:0/4256947576 >> 172.21.15.46:0/4256947576 conn(0x7f75d00fdd70 msgr2=0x7f75d0104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.821+0000 7f75d4973640 1 -- 172.21.15.46:0/4256947576 shutdown_connections 2024-05-09T00:11:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:52.821+0000 7f75d4973640 1 -- 172.21.15.46:0/4256947576 wait complete. 2024-05-09T00:11:52.831 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-09T00:11:52.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:11:52.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:52 smithi169 ceph-mon[34777]: pgmap v391: 97 pgs: 97 active+clean; 641 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:11:52.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:52 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1216217128' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T00:11:52.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:52 smithi169 ceph-mon[34777]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T00:11:52.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:52 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1208044048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:11:52.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:52 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/562144439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:11:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:53.477 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:53 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3495578343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:11:53.477 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:53 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/4256947576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:11:53.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:53 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3495578343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:11:53.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:53 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/4256947576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:11:53.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:53 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3495578343' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:11:53.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:53 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/4256947576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:11:54.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:54 smithi046 ceph-mon[30151]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-09T00:11:54.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:54 smithi142 ceph-mon[34533]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-09T00:11:54.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:54 smithi169 ceph-mon[34777]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-09T00:11:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.879+0000 7fe4280bb700 1 -- 172.21.15.46:0/1625351166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe420108f40 msgr2=0x7fe420109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.879+0000 7fe4280bb700 1 --2- 172.21.15.46:0/1625351166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe420108f40 0x7fe420109350 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0099c0 tx=0x7fe41c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.879+0000 7fe4280bb700 1 -- 172.21.15.46:0/1625351166 shutdown_connections 2024-05-09T00:11:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.879+0000 7fe4280bb700 1 --2- 172.21.15.46:0/1625351166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe420109890 0x7fe4201111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.879+0000 7fe4280bb700 1 --2- 172.21.15.46:0/1625351166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe420104610 0x7fe420104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.879+0000 7fe4280bb700 1 --2- 172.21.15.46:0/1625351166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe420108f40 0x7fe420109350 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.879+0000 7fe4280bb700 1 -- 172.21.15.46:0/1625351166 >> 172.21.15.46:0/1625351166 conn(0x7fe42007af70 msgr2=0x7fe42007b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.879+0000 7fe4280bb700 1 -- 172.21.15.46:0/1625351166 shutdown_connections 2024-05-09T00:11:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.879+0000 7fe4280bb700 1 -- 172.21.15.46:0/1625351166 wait complete. 2024-05-09T00:11:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe4280bb700 1 Processor -- start 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe4280bb700 1 -- start start 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe4280bb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe420104610 0x7fe420075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe4280bb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe420108f40 0x7fe420076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe4280bb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe420109890 0x7fe420076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe4280bb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe4201144a0 con 0x7fe420109890 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe4280bb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe420114320 con 0x7fe420104610 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe4280bb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe420114620 con 0x7fe420108f40 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe426658700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe420109890 0x7fe420076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe426658700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe420109890 0x7fe420076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41864/0 (socket says 172.21.15.46:41864) 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe426658700 1 -- 172.21.15.46:0/725558801 learned_addr learned my addr 172.21.15.46:0/725558801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe425656700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe420108f40 0x7fe420076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.880+0000 7fe425e57700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe420104610 0x7fe420075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.881+0000 7fe426658700 1 -- 172.21.15.46:0/725558801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe420108f40 msgr2=0x7fe420076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.881+0000 7fe426658700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe420108f40 0x7fe420076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.881+0000 7fe426658700 1 -- 172.21.15.46:0/725558801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe420104610 msgr2=0x7fe420075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.881+0000 7fe426658700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe420104610 0x7fe420075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.881+0000 7fe426658700 1 -- 172.21.15.46:0/725558801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4201a9220 con 0x7fe420109890 2024-05-09T00:11:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.881+0000 7fe426658700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe420109890 0x7fe420076540 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe418007ee0 tx=0x7fe41800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.881+0000 7fe412ffd700 1 -- 172.21.15.46:0/725558801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418004bb0 con 0x7fe420109890 2024-05-09T00:11:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.881+0000 7fe412ffd700 1 -- 172.21.15.46:0/725558801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe418004d50 con 0x7fe420109890 2024-05-09T00:11:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.881+0000 7fe4280bb700 1 -- 172.21.15.46:0/725558801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4201a9510 con 0x7fe420109890 2024-05-09T00:11:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.881+0000 7fe412ffd700 1 -- 172.21.15.46:0/725558801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418005920 con 0x7fe420109890 2024-05-09T00:11:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.881+0000 7fe4280bb700 1 -- 172.21.15.46:0/725558801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4201a9a20 con 0x7fe420109890 2024-05-09T00:11:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.882+0000 7fe4280bb700 1 -- 172.21.15.46:0/725558801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4200512c0 con 0x7fe420109890 2024-05-09T00:11:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.884+0000 7fe412ffd700 1 -- 172.21.15.46:0/725558801 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe41801a020 con 0x7fe420109890 2024-05-09T00:11:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.885+0000 7fe412ffd700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe40c0b2900 0x7fe40c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.885+0000 7fe425e57700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe40c0b2900 0x7fe40c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.885+0000 7fe412ffd700 1 -- 172.21.15.46:0/725558801 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe4180e1e80 con 0x7fe420109890 2024-05-09T00:11:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.885+0000 7fe425e57700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe40c0b2900 0x7fe40c0b4e10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe41400ac50 tx=0x7fe41400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:54.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:54.888+0000 7fe412ffd700 1 -- 172.21.15.46:0/725558801 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe418010070 con 0x7fe420109890 2024-05-09T00:11:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.149+0000 7fe4280bb700 1 -- 172.21.15.46:0/725558801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe42002e5f0 con 0x7fe420109890 2024-05-09T00:11:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.150+0000 7fe412ffd700 1 -- 172.21.15.46:0/725558801 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7fe4180a9b60 con 0x7fe420109890 2024-05-09T00:11:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:55.149 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:11:55.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.152+0000 7fe4280bb700 1 -- 172.21.15.46:0/725558801 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe40c0b2900 msgr2=0x7fe40c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.152+0000 7fe4280bb700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe40c0b2900 0x7fe40c0b4e10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe41400ac50 tx=0x7fe41400a380 comp rx=0 tx=0).stop 2024-05-09T00:11:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.152+0000 7fe4280bb700 1 -- 172.21.15.46:0/725558801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe420109890 msgr2=0x7fe420076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.152+0000 7fe4280bb700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe420109890 0x7fe420076540 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe418007ee0 tx=0x7fe41800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:11:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.152+0000 7fe4280bb700 1 -- 172.21.15.46:0/725558801 shutdown_connections 2024-05-09T00:11:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.152+0000 7fe4280bb700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe40c0b2900 0x7fe40c0b4e10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.152+0000 7fe4280bb700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe420109890 0x7fe420076540 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.152+0000 7fe4280bb700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe420104610 0x7fe420075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.152+0000 7fe4280bb700 1 --2- 172.21.15.46:0/725558801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe420108f40 0x7fe420076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.152+0000 7fe4280bb700 1 -- 172.21.15.46:0/725558801 >> 172.21.15.46:0/725558801 conn(0x7fe42007af70 msgr2=0x7fe420101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.152+0000 7fe4280bb700 1 -- 172.21.15.46:0/725558801 shutdown_connections 2024-05-09T00:11:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.153+0000 7fe4280bb700 1 -- 172.21.15.46:0/725558801 wait complete. 2024-05-09T00:11:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:11:55.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:55 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/725558801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:55.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:55 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/725558801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:55.846 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-09T00:11:55.846 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-09T00:11:55.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T00:11:55.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:55 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/725558801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:55.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.967+0000 7f59b4700640 1 -- 172.21.15.46:0/1522264836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac101f90 msgr2=0x7f59ac106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:55.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.967+0000 7f59b4700640 1 --2- 172.21.15.46:0/1522264836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac101f90 0x7f59ac106c50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009900 tx=0x7f59a802a410 comp rx=0 tx=0).stop 2024-05-09T00:11:55.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.967+0000 7f59b4700640 1 -- 172.21.15.46:0/1522264836 shutdown_connections 2024-05-09T00:11:55.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.967+0000 7f59b4700640 1 --2- 172.21.15.46:0/1522264836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59ac1098b0 0x7f59ac10bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.967+0000 7f59b4700640 1 --2- 172.21.15.46:0/1522264836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59ac102490 0x7f59ac109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.967+0000 7f59b4700640 1 --2- 172.21.15.46:0/1522264836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac101f90 0x7f59ac106c50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.967+0000 7f59b4700640 1 -- 172.21.15.46:0/1522264836 >> 172.21.15.46:0/1522264836 conn(0x7f59ac0fd4e0 msgr2=0x7f59ac0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.967+0000 7f59b4700640 1 -- 172.21.15.46:0/1522264836 shutdown_connections 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.967+0000 7f59b4700640 1 -- 172.21.15.46:0/1522264836 wait complete. 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b4700640 1 Processor -- start 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b4700640 1 -- start start 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b4700640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59ac101f90 0x7f59ac104460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b4700640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac102490 0x7f59ac102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b4700640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59ac1098b0 0x7f59ac103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b4700640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac10f710 con 0x7f59ac102490 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b4700640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac10f590 con 0x7f59ac101f90 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b4700640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac10f890 con 0x7f59ac1098b0 2024-05-09T00:11:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b1c9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac102490 0x7f59ac102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b1c9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac102490 0x7f59ac102b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41878/0 (socket says 172.21.15.46:41878) 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b1c9b640 1 -- 172.21.15.46:0/1299514188 learned_addr learned my addr 172.21.15.46:0/1299514188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b249c640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59ac101f90 0x7f59ac104460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.968+0000 7f59b2c9d640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59ac1098b0 0x7f59ac103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.969+0000 7f59b1c9b640 1 -- 172.21.15.46:0/1299514188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59ac1098b0 msgr2=0x7f59ac103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.969+0000 7f59b1c9b640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59ac1098b0 0x7f59ac103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.969+0000 7f59b1c9b640 1 -- 172.21.15.46:0/1299514188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59ac101f90 msgr2=0x7f59ac104460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.969+0000 7f59b1c9b640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59ac101f90 0x7f59ac104460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.969+0000 7f59b1c9b640 1 -- 172.21.15.46:0/1299514188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac103770 con 0x7f59ac102490 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.969+0000 7f59b1c9b640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac102490 0x7f59ac102b00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f599c00d770 tx=0x7f599c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.969+0000 7f59a37fe640 1 -- 172.21.15.46:0/1299514188 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c017070 con 0x7f59ac102490 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.969+0000 7f59a37fe640 1 -- 172.21.15.46:0/1299514188 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f599c0054d0 con 0x7f59ac102490 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.969+0000 7f59b4700640 1 -- 172.21.15.46:0/1299514188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ac1a4400 con 0x7f59ac102490 2024-05-09T00:11:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.969+0000 7f59a37fe640 1 -- 172.21.15.46:0/1299514188 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c004c30 con 0x7f59ac102490 2024-05-09T00:11:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.969+0000 7f59b4700640 1 -- 172.21.15.46:0/1299514188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ac1a48c0 con 0x7f59ac102490 2024-05-09T00:11:55.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.970+0000 7f59b4700640 1 -- 172.21.15.46:0/1299514188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ac0ff000 con 0x7f59ac102490 2024-05-09T00:11:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.972+0000 7f59a37fe640 1 -- 172.21.15.46:0/1299514188 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f599c004dd0 con 0x7f59ac102490 2024-05-09T00:11:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.972+0000 7f59a37fe640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f59980b2870 0x7f59980b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.973+0000 7f59b249c640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f59980b2870 0x7f59980b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.973+0000 7f59a37fe640 1 -- 172.21.15.46:0/1299514188 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f599c0e1fe0 con 0x7f59ac102490 2024-05-09T00:11:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.973+0000 7f59b249c640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f59980b2870 0x7f59980b4d80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f59a8000c00 tx=0x7f59a8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:55.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:55.976+0000 7f59a37fe640 1 -- 172.21.15.46:0/1299514188 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f599c0a9e20 con 0x7f59ac102490 2024-05-09T00:11:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.165+0000 7f59b4700640 1 -- 172.21.15.46:0/1299514188 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f59ac063110 con 0x7f59980b2870 2024-05-09T00:11:56.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.172+0000 7f59a37fe640 1 -- 172.21.15.46:0/1299514188 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259083 (secure 0 0 0) 0x7f59ac063110 con 0x7f59980b2870 2024-05-09T00:11:56.173 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:56.175 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T00:11:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.175+0000 7f59b4700640 1 -- 172.21.15.46:0/1299514188 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f59980b2870 msgr2=0x7f59980b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.175+0000 7f59b4700640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f59980b2870 0x7f59980b4d80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f59a8000c00 tx=0x7f59a8031400 comp rx=0 tx=0).stop 2024-05-09T00:11:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.175+0000 7f59b4700640 1 -- 172.21.15.46:0/1299514188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac102490 msgr2=0x7f59ac102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.175+0000 7f59b4700640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac102490 0x7f59ac102b00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f599c00d770 tx=0x7f599c00dc30 comp rx=0 tx=0).stop 2024-05-09T00:11:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.175+0000 7f59b4700640 1 -- 172.21.15.46:0/1299514188 shutdown_connections 2024-05-09T00:11:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.175+0000 7f59b4700640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f59980b2870 0x7f59980b4d80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.175+0000 7f59b4700640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59ac1098b0 0x7f59ac103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.175+0000 7f59b4700640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac102490 0x7f59ac102b00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.175+0000 7f59b4700640 1 --2- 172.21.15.46:0/1299514188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59ac101f90 0x7f59ac104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.175+0000 7f59b4700640 1 -- 172.21.15.46:0/1299514188 >> 172.21.15.46:0/1299514188 conn(0x7f59ac0fd4e0 msgr2=0x7f59ac10a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.175+0000 7f59b4700640 1 -- 172.21.15.46:0/1299514188 shutdown_connections 2024-05-09T00:11:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.175+0000 7f59b4700640 1 -- 172.21.15.46:0/1299514188 wait complete. 2024-05-09T00:11:56.185 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-05-09T00:11:55.339263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661749,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1245,"num_read_kb":725,"num_write":306,"num_write_kb":2280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":747,"ondisk_log_size":747,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372308,"kb_used_data":10676,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569516,"statfs":{"total":1151940427776,"available":1151559184384,"internally_reserved":0,"allocated":10932224,"data_stored":7076649,"data_compressed":177897,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4930,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":52,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003242"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324383+0000","last_change":"2024-05-09T00:08:26.060455+0000","last_active":"2024-05-09T00:10:09.324383+0000","last_peered":"2024-05-09T00:10:09.324383+0000","last_clean":"2024-05-09T00:10:09.324383+0000","last_became_active":"2024-05-09T00:08:26.060068+0000","last_became_peered":"2024-05-09T00:08:26.060068+0000","last_unstale":"2024-05-09T00:10:09.324383+0000","last_undegraded":"2024-05-09T00:10:09.324383+0000","last_fullsized":"2024-05-09T00:10:09.324383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:48.354188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.314519+0000","last_change":"2024-05-09T00:08:26.055737+0000","last_active":"2024-05-09T00:10:09.314519+0000","last_peered":"2024-05-09T00:10:09.314519+0000","last_clean":"2024-05-09T00:10:09.314519+0000","last_became_active":"2024-05-09T00:08:26.055354+0000","last_became_peered":"2024-05-09T00:08:26.055354+0000","last_unstale":"2024-05-09T00:10:09.314519+0000","last_undegraded":"2024-05-09T00:10:09.314519+0000","last_fullsized":"2024-05-09T00:10:09.314519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:54.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324383+0000","last_change":"2024-05-09T00:08:26.050776+0000","last_active":"2024-05-09T00:10:09.324383+0000","last_peered":"2024-05-09T00:10:09.324383+0000","last_clean":"2024-05-09T00:10:09.324383+0000","last_became_active":"2024-05-09T00:08:26.050393+0000","last_became_peered":"2024-05-09T00:08:26.050393+0000","last_unstale":"2024-05-09T00:10:09.324383+0000","last_undegraded":"2024-05-09T00:10:09.324383+0000","last_fullsized":"2024-05-09T00:10:09.324383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:30.819219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.307617+0000","last_change":"2024-05-09T00:08:26.065946+0000","last_active":"2024-05-09T00:10:09.307617+0000","last_peered":"2024-05-09T00:10:09.307617+0000","last_clean":"2024-05-09T00:10:09.307617+0000","last_became_active":"2024-05-09T00:08:26.065498+0000","last_became_peered":"2024-05-09T00:08:26.065498+0000","last_unstale":"2024-05-09T00:10:09.307617+0000","last_undegraded":"2024-05-09T00:10:09.307617+0000","last_fullsized":"2024-05-09T00:10:09.307617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:04.444860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.321202+0000","last_change":"2024-05-09T00:08:26.051961+0000","last_active":"2024-05-09T00:10:09.321202+0000","last_peered":"2024-05-09T00:10:09.321202+0000","last_clean":"2024-05-09T00:10:09.321202+0000","last_became_active":"2024-05-09T00:08:26.051548+0000","last_became_peered":"2024-05-09T00:08:26.051548+0000","last_unstale":"2024-05-09T00:10:09.321202+0000","last_undegraded":"2024-05-09T00:10:09.321202+0000","last_fullsized":"2024-05-09T00:10:09.321202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:07.539117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.311478+0000","last_change":"2024-05-09T00:08:26.052012+0000","last_active":"2024-05-09T00:10:09.311478+0000","last_peered":"2024-05-09T00:10:09.311478+0000","last_clean":"2024-05-09T00:10:09.311478+0000","last_became_active":"2024-05-09T00:08:26.050886+0000","last_became_peered":"2024-05-09T00:08:26.050886+0000","last_unstale":"2024-05-09T00:10:09.311478+0000","last_undegraded":"2024-05-09T00:10:09.311478+0000","last_fullsized":"2024-05-09T00:10:09.311478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:14.651888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324357+0000","last_change":"2024-05-09T00:08:26.058418+0000","last_active":"2024-05-09T00:10:09.324357+0000","last_peered":"2024-05-09T00:10:09.324357+0000","last_clean":"2024-05-09T00:10:09.324357+0000","last_became_active":"2024-05-09T00:08:26.058039+0000","last_became_peered":"2024-05-09T00:08:26.058039+0000","last_unstale":"2024-05-09T00:10:09.324357+0000","last_undegraded":"2024-05-09T00:10:09.324357+0000","last_fullsized":"2024-05-09T00:10:09.324357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:13.678123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.322294+0000","last_change":"2024-05-09T00:08:26.057602+0000","last_active":"2024-05-09T00:10:09.322294+0000","last_peered":"2024-05-09T00:10:09.322294+0000","last_clean":"2024-05-09T00:10:09.322294+0000","last_became_active":"2024-05-09T00:08:26.057169+0000","last_became_peered":"2024-05-09T00:08:26.057169+0000","last_unstale":"2024-05-09T00:10:09.322294+0000","last_undegraded":"2024-05-09T00:10:09.322294+0000","last_fullsized":"2024-05-09T00:10:09.322294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:11.452083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327189+0000","last_change":"2024-05-09T00:08:26.067639+0000","last_active":"2024-05-09T00:10:09.327189+0000","last_peered":"2024-05-09T00:10:09.327189+0000","last_clean":"2024-05-09T00:10:09.327189+0000","last_became_active":"2024-05-09T00:08:26.067226+0000","last_became_peered":"2024-05-09T00:08:26.067226+0000","last_unstale":"2024-05-09T00:10:09.327189+0000","last_undegraded":"2024-05-09T00:10:09.327189+0000","last_fullsized":"2024-05-09T00:10:09.327189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:12.366500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.310873+0000","last_change":"2024-05-09T00:08:26.055234+0000","last_active":"2024-05-09T00:10:09.310873+0000","last_peered":"2024-05-09T00:10:09.310873+0000","last_clean":"2024-05-09T00:10:09.310873+0000","last_became_active":"2024-05-09T00:08:26.054851+0000","last_became_peered":"2024-05-09T00:08:26.054851+0000","last_unstale":"2024-05-09T00:10:09.310873+0000","last_undegraded":"2024-05-09T00:10:09.310873+0000","last_fullsized":"2024-05-09T00:10:09.310873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:54:52.121329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330090+0000","last_change":"2024-05-09T00:08:26.063375+0000","last_active":"2024-05-09T00:10:09.330090+0000","last_peered":"2024-05-09T00:10:09.330090+0000","last_clean":"2024-05-09T00:10:09.330090+0000","last_became_active":"2024-05-09T00:08:26.062962+0000","last_became_peered":"2024-05-09T00:08:26.062962+0000","last_unstale":"2024-05-09T00:10:09.330090+0000","last_undegraded":"2024-05-09T00:10:09.330090+0000","last_fullsized":"2024-05-09T00:10:09.330090+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:05.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.305381+0000","last_change":"2024-05-09T00:08:26.062034+0000","last_active":"2024-05-09T00:10:09.305381+0000","last_peered":"2024-05-09T00:10:09.305381+0000","last_clean":"2024-05-09T00:10:09.305381+0000","last_became_active":"2024-05-09T00:08:26.060376+0000","last_became_peered":"2024-05-09T00:08:26.060376+0000","last_unstale":"2024-05-09T00:10:09.305381+0000","last_undegraded":"2024-05-09T00:10:09.305381+0000","last_fullsized":"2024-05-09T00:10:09.305381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:15.830024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327845+0000","last_change":"2024-05-09T00:08:26.053227+0000","last_active":"2024-05-09T00:10:09.327845+0000","last_peered":"2024-05-09T00:10:09.327845+0000","last_clean":"2024-05-09T00:10:09.327845+0000","last_became_active":"2024-05-09T00:08:26.052878+0000","last_became_peered":"2024-05-09T00:08:26.052878+0000","last_unstale":"2024-05-09T00:10:09.327845+0000","last_undegraded":"2024-05-09T00:10:09.327845+0000","last_fullsized":"2024-05-09T00:10:09.327845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:24.385026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.315496+0000","last_change":"2024-05-09T00:08:26.053041+0000","last_active":"2024-05-09T00:10:09.315496+0000","last_peered":"2024-05-09T00:10:09.315496+0000","last_clean":"2024-05-09T00:10:09.315496+0000","last_became_active":"2024-05-09T00:08:26.052677+0000","last_became_peered":"2024-05-09T00:08:26.052677+0000","last_unstale":"2024-05-09T00:10:09.315496+0000","last_undegraded":"2024-05-09T00:10:09.315496+0000","last_fullsized":"2024-05-09T00:10:09.315496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:24.478270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.315918+0000","last_change":"2024-05-09T00:08:26.063391+0000","last_active":"2024-05-09T00:10:09.315918+0000","last_peered":"2024-05-09T00:10:09.315918+0000","last_clean":"2024-05-09T00:10:09.315918+0000","last_became_active":"2024-05-09T00:08:26.063033+0000","last_became_peered":"2024-05-09T00:08:26.063033+0000","last_unstale":"2024-05-09T00:10:09.315918+0000","last_undegraded":"2024-05-09T00:10:09.315918+0000","last_fullsized":"2024-05-09T00:10:09.315918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:41.900453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328915+0000","last_change":"2024-05-09T00:08:26.065018+0000","last_active":"2024-05-09T00:10:09.328915+0000","last_peered":"2024-05-09T00:10:09.328915+0000","last_clean":"2024-05-09T00:10:09.328915+0000","last_became_active":"2024-05-09T00:08:26.064642+0000","last_became_peered":"2024-05-09T00:08:26.064642+0000","last_unstale":"2024-05-09T00:10:09.328915+0000","last_undegraded":"2024-05-09T00:10:09.328915+0000","last_fullsized":"2024-05-09T00:10:09.328915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:02.403704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324263+0000","last_change":"2024-05-09T00:08:26.049770+0000","last_active":"2024-05-09T00:10:09.324263+0000","last_peered":"2024-05-09T00:10:09.324263+0000","last_clean":"2024-05-09T00:10:09.324263+0000","last_became_active":"2024-05-09T00:08:26.048555+0000","last_became_peered":"2024-05-09T00:08:26.048555+0000","last_unstale":"2024-05-09T00:10:09.324263+0000","last_undegraded":"2024-05-09T00:10:09.324263+0000","last_fullsized":"2024-05-09T00:10:09.324263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:16.009200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.317473+0000","last_change":"2024-05-09T00:08:26.048587+0000","last_active":"2024-05-09T00:10:09.317473+0000","last_peered":"2024-05-09T00:10:09.317473+0000","last_clean":"2024-05-09T00:10:09.317473+0000","last_became_active":"2024-05-09T00:08:26.048148+0000","last_became_peered":"2024-05-09T00:08:26.048148+0000","last_unstale":"2024-05-09T00:10:09.317473+0000","last_undegraded":"2024-05-09T00:10:09.317473+0000","last_fullsized":"2024-05-09T00:10:09.317473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:42.113829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.322240+0000","last_change":"2024-05-09T00:08:26.053607+0000","last_active":"2024-05-09T00:10:09.322240+0000","last_peered":"2024-05-09T00:10:09.322240+0000","last_clean":"2024-05-09T00:10:09.322240+0000","last_became_active":"2024-05-09T00:08:26.053212+0000","last_became_peered":"2024-05-09T00:08:26.053212+0000","last_unstale":"2024-05-09T00:10:09.322240+0000","last_undegraded":"2024-05-09T00:10:09.322240+0000","last_fullsized":"2024-05-09T00:10:09.322240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:32.937049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469352+0000","last_change":"2024-05-09T00:08:23.024481+0000","last_active":"2024-05-09T00:09:10.469352+0000","last_peered":"2024-05-09T00:09:10.469352+0000","last_clean":"2024-05-09T00:09:10.469352+0000","last_became_active":"2024-05-09T00:08:23.024132+0000","last_became_peered":"2024-05-09T00:08:23.024132+0000","last_unstale":"2024-05-09T00:09:10.469352+0000","last_undegraded":"2024-05-09T00:09:10.469352+0000","last_fullsized":"2024-05-09T00:09:10.469352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:11.188505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325280+0000","last_change":"2024-05-09T00:08:26.054772+0000","last_active":"2024-05-09T00:10:09.325280+0000","last_peered":"2024-05-09T00:10:09.325280+0000","last_clean":"2024-05-09T00:10:09.325280+0000","last_became_active":"2024-05-09T00:08:26.054433+0000","last_became_peered":"2024-05-09T00:08:26.054433+0000","last_unstale":"2024-05-09T00:10:09.325280+0000","last_undegraded":"2024-05-09T00:10:09.325280+0000","last_fullsized":"2024-05-09T00:10:09.325280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:44.283240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468356+0000","last_change":"2024-05-09T00:08:23.118778+0000","last_active":"2024-05-09T00:09:10.468356+0000","last_peered":"2024-05-09T00:09:10.468356+0000","last_clean":"2024-05-09T00:09:10.468356+0000","last_became_active":"2024-05-09T00:08:23.118377+0000","last_became_peered":"2024-05-09T00:08:23.118377+0000","last_unstale":"2024-05-09T00:09:10.468356+0000","last_undegraded":"2024-05-09T00:09:10.468356+0000","last_fullsized":"2024-05-09T00:09:10.468356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:51.352442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325452+0000","last_change":"2024-05-09T00:08:26.059953+0000","last_active":"2024-05-09T00:10:09.325452+0000","last_peered":"2024-05-09T00:10:09.325452+0000","last_clean":"2024-05-09T00:10:09.325452+0000","last_became_active":"2024-05-09T00:08:26.059571+0000","last_became_peered":"2024-05-09T00:08:26.059571+0000","last_unstale":"2024-05-09T00:10:09.325452+0000","last_undegraded":"2024-05-09T00:10:09.325452+0000","last_fullsized":"2024-05-09T00:10:09.325452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:48.056798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475294+0000","last_change":"2024-05-09T00:08:23.021244+0000","last_active":"2024-05-09T00:09:10.475294+0000","last_peered":"2024-05-09T00:09:10.475294+0000","last_clean":"2024-05-09T00:09:10.475294+0000","last_became_active":"2024-05-09T00:08:23.020597+0000","last_became_peered":"2024-05-09T00:08:23.020597+0000","last_unstale":"2024-05-09T00:09:10.475294+0000","last_undegraded":"2024-05-09T00:09:10.475294+0000","last_fullsized":"2024-05-09T00:09:10.475294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.654261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328461+0000","last_change":"2024-05-09T00:08:26.059834+0000","last_active":"2024-05-09T00:10:09.328461+0000","last_peered":"2024-05-09T00:10:09.328461+0000","last_clean":"2024-05-09T00:10:09.328461+0000","last_became_active":"2024-05-09T00:08:26.059349+0000","last_became_peered":"2024-05-09T00:08:26.059349+0000","last_unstale":"2024-05-09T00:10:09.328461+0000","last_undegraded":"2024-05-09T00:10:09.328461+0000","last_fullsized":"2024-05-09T00:10:09.328461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:42.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474235+0000","last_change":"2024-05-09T00:08:23.019568+0000","last_active":"2024-05-09T00:09:10.474235+0000","last_peered":"2024-05-09T00:09:10.474235+0000","last_clean":"2024-05-09T00:09:10.474235+0000","last_became_active":"2024-05-09T00:08:23.019149+0000","last_became_peered":"2024-05-09T00:08:23.019149+0000","last_unstale":"2024-05-09T00:09:10.474235+0000","last_undegraded":"2024-05-09T00:09:10.474235+0000","last_fullsized":"2024-05-09T00:09:10.474235+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:44.569911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328628+0000","last_change":"2024-05-09T00:08:26.061308+0000","last_active":"2024-05-09T00:10:09.328628+0000","last_peered":"2024-05-09T00:10:09.328628+0000","last_clean":"2024-05-09T00:10:09.328628+0000","last_became_active":"2024-05-09T00:08:26.060973+0000","last_became_peered":"2024-05-09T00:08:26.060973+0000","last_unstale":"2024-05-09T00:10:09.328628+0000","last_undegraded":"2024-05-09T00:10:09.328628+0000","last_fullsized":"2024-05-09T00:10:09.328628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:16.845074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":221,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:50.666958+0000","last_change":"2024-05-09T00:08:23.020886+0000","last_active":"2024-05-09T00:11:50.666958+0000","last_peered":"2024-05-09T00:11:50.666958+0000","last_clean":"2024-05-09T00:11:50.666958+0000","last_became_active":"2024-05-09T00:08:23.020530+0000","last_became_peered":"2024-05-09T00:08:23.020530+0000","last_unstale":"2024-05-09T00:11:50.666958+0000","last_undegraded":"2024-05-09T00:11:50.666958+0000","last_fullsized":"2024-05-09T00:11:50.666958+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:34.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318827+0000","last_change":"2024-05-09T00:08:26.051018+0000","last_active":"2024-05-09T00:10:09.318827+0000","last_peered":"2024-05-09T00:10:09.318827+0000","last_clean":"2024-05-09T00:10:09.318827+0000","last_became_active":"2024-05-09T00:08:26.050495+0000","last_became_peered":"2024-05-09T00:08:26.050495+0000","last_unstale":"2024-05-09T00:10:09.318827+0000","last_undegraded":"2024-05-09T00:10:09.318827+0000","last_fullsized":"2024-05-09T00:10:09.318827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:07.862339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472149+0000","last_change":"2024-05-09T00:08:23.021187+0000","last_active":"2024-05-09T00:09:10.472149+0000","last_peered":"2024-05-09T00:09:10.472149+0000","last_clean":"2024-05-09T00:09:10.472149+0000","last_became_active":"2024-05-09T00:08:23.020770+0000","last_became_peered":"2024-05-09T00:08:23.020770+0000","last_unstale":"2024-05-09T00:09:10.472149+0000","last_undegraded":"2024-05-09T00:09:10.472149+0000","last_fullsized":"2024-05-09T00:09:10.472149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:08.992770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.316888+0000","last_change":"2024-05-09T00:09:10.797392+0000","last_active":"2024-05-09T00:10:09.316888+0000","last_peered":"2024-05-09T00:10:09.316888+0000","last_clean":"2024-05-09T00:10:09.316888+0000","last_became_active":"2024-05-09T00:09:10.796969+0000","last_became_peered":"2024-05-09T00:09:10.796969+0000","last_unstale":"2024-05-09T00:10:09.316888+0000","last_undegraded":"2024-05-09T00:10:09.316888+0000","last_fullsized":"2024-05-09T00:10:09.316888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:18.852678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548815+0000","last_change":"2024-05-09T00:08:23.012226+0000","last_active":"2024-05-09T00:09:10.548815+0000","last_peered":"2024-05-09T00:09:10.548815+0000","last_clean":"2024-05-09T00:09:10.548815+0000","last_became_active":"2024-05-09T00:08:23.011781+0000","last_became_peered":"2024-05-09T00:08:23.011781+0000","last_unstale":"2024-05-09T00:09:10.548815+0000","last_undegraded":"2024-05-09T00:09:10.548815+0000","last_fullsized":"2024-05-09T00:09:10.548815+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:44.452206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.321641+0000","last_change":"2024-05-09T00:08:26.043035+0000","last_active":"2024-05-09T00:10:09.321641+0000","last_peered":"2024-05-09T00:10:09.321641+0000","last_clean":"2024-05-09T00:10:09.321641+0000","last_became_active":"2024-05-09T00:08:26.042686+0000","last_became_peered":"2024-05-09T00:08:26.042686+0000","last_unstale":"2024-05-09T00:10:09.321641+0000","last_undegraded":"2024-05-09T00:10:09.321641+0000","last_fullsized":"2024-05-09T00:10:09.321641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:24.511309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469113+0000","last_change":"2024-05-09T00:08:23.022017+0000","last_active":"2024-05-09T00:09:10.469113+0000","last_peered":"2024-05-09T00:09:10.469113+0000","last_clean":"2024-05-09T00:09:10.469113+0000","last_became_active":"2024-05-09T00:08:23.021540+0000","last_became_peered":"2024-05-09T00:08:23.021540+0000","last_unstale":"2024-05-09T00:09:10.469113+0000","last_undegraded":"2024-05-09T00:09:10.469113+0000","last_fullsized":"2024-05-09T00:09:10.469113+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:50.773211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326086+0000","last_change":"2024-05-09T00:08:26.048039+0000","last_active":"2024-05-09T00:10:09.326086+0000","last_peered":"2024-05-09T00:10:09.326086+0000","last_clean":"2024-05-09T00:10:09.326086+0000","last_became_active":"2024-05-09T00:08:26.047695+0000","last_became_peered":"2024-05-09T00:08:26.047695+0000","last_unstale":"2024-05-09T00:10:09.326086+0000","last_undegraded":"2024-05-09T00:10:09.326086+0000","last_fullsized":"2024-05-09T00:10:09.326086+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:27.363847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":220,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:52.669314+0000","last_change":"2024-05-09T00:08:23.019565+0000","last_active":"2024-05-09T00:11:52.669314+0000","last_peered":"2024-05-09T00:11:52.669314+0000","last_clean":"2024-05-09T00:11:52.669314+0000","last_became_active":"2024-05-09T00:08:23.019113+0000","last_became_peered":"2024-05-09T00:08:23.019113+0000","last_unstale":"2024-05-09T00:11:52.669314+0000","last_undegraded":"2024-05-09T00:11:52.669314+0000","last_fullsized":"2024-05-09T00:11:52.669314+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:45.578934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323960+0000","last_change":"2024-05-09T00:08:26.058869+0000","last_active":"2024-05-09T00:10:09.323960+0000","last_peered":"2024-05-09T00:10:09.323960+0000","last_clean":"2024-05-09T00:10:09.323960+0000","last_became_active":"2024-05-09T00:08:26.058482+0000","last_became_peered":"2024-05-09T00:08:26.058482+0000","last_unstale":"2024-05-09T00:10:09.323960+0000","last_undegraded":"2024-05-09T00:10:09.323960+0000","last_fullsized":"2024-05-09T00:10:09.323960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:33.252208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472217+0000","last_change":"2024-05-09T00:08:23.024425+0000","last_active":"2024-05-09T00:09:10.472217+0000","last_peered":"2024-05-09T00:09:10.472217+0000","last_clean":"2024-05-09T00:09:10.472217+0000","last_became_active":"2024-05-09T00:08:23.023991+0000","last_became_peered":"2024-05-09T00:08:23.023991+0000","last_unstale":"2024-05-09T00:09:10.472217+0000","last_undegraded":"2024-05-09T00:09:10.472217+0000","last_fullsized":"2024-05-09T00:09:10.472217+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:47.907851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330931+0000","last_change":"2024-05-09T00:09:10.475008+0000","last_active":"2024-05-09T00:10:09.330931+0000","last_peered":"2024-05-09T00:10:09.330931+0000","last_clean":"2024-05-09T00:10:09.330931+0000","last_became_active":"2024-05-09T00:09:10.474663+0000","last_became_peered":"2024-05-09T00:09:10.474663+0000","last_unstale":"2024-05-09T00:10:09.330931+0000","last_undegraded":"2024-05-09T00:10:09.330931+0000","last_fullsized":"2024-05-09T00:10:09.330931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:36.990362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470179+0000","last_change":"2024-05-09T00:08:23.118516+0000","last_active":"2024-05-09T00:09:10.470179+0000","last_peered":"2024-05-09T00:09:10.470179+0000","last_clean":"2024-05-09T00:09:10.470179+0000","last_became_active":"2024-05-09T00:08:23.118147+0000","last_became_peered":"2024-05-09T00:08:23.118147+0000","last_unstale":"2024-05-09T00:09:10.470179+0000","last_undegraded":"2024-05-09T00:09:10.470179+0000","last_fullsized":"2024-05-09T00:09:10.470179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:24.205818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327118+0000","last_change":"2024-05-09T00:08:26.067063+0000","last_active":"2024-05-09T00:10:09.327118+0000","last_peered":"2024-05-09T00:10:09.327118+0000","last_clean":"2024-05-09T00:10:09.327118+0000","last_became_active":"2024-05-09T00:08:26.066440+0000","last_became_peered":"2024-05-09T00:08:26.066440+0000","last_unstale":"2024-05-09T00:10:09.327118+0000","last_undegraded":"2024-05-09T00:10:09.327118+0000","last_fullsized":"2024-05-09T00:10:09.327118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:48.648485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'36","reported_seq":455,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:53.672263+0000","last_change":"2024-05-09T00:08:23.020757+0000","last_active":"2024-05-09T00:11:53.672263+0000","last_peered":"2024-05-09T00:11:53.672263+0000","last_clean":"2024-05-09T00:11:53.672263+0000","last_became_active":"2024-05-09T00:08:23.020355+0000","last_became_peered":"2024-05-09T00:08:23.020355+0000","last_unstale":"2024-05-09T00:11:53.672263+0000","last_undegraded":"2024-05-09T00:11:53.672263+0000","last_fullsized":"2024-05-09T00:11:53.672263+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:00.605691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66697,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":392,"num_read_kb":261,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328259+0000","last_change":"2024-05-09T00:08:26.057402+0000","last_active":"2024-05-09T00:10:09.328259+0000","last_peered":"2024-05-09T00:10:09.328259+0000","last_clean":"2024-05-09T00:10:09.328259+0000","last_became_active":"2024-05-09T00:08:26.056974+0000","last_became_peered":"2024-05-09T00:08:26.056974+0000","last_unstale":"2024-05-09T00:10:09.328259+0000","last_undegraded":"2024-05-09T00:10:09.328259+0000","last_fullsized":"2024-05-09T00:10:09.328259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:47.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473953+0000","last_change":"2024-05-09T00:08:23.020208+0000","last_active":"2024-05-09T00:09:10.473953+0000","last_peered":"2024-05-09T00:09:10.473953+0000","last_clean":"2024-05-09T00:09:10.473953+0000","last_became_active":"2024-05-09T00:08:23.019809+0000","last_became_peered":"2024-05-09T00:08:23.019809+0000","last_unstale":"2024-05-09T00:09:10.473953+0000","last_undegraded":"2024-05-09T00:09:10.473953+0000","last_fullsized":"2024-05-09T00:09:10.473953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:47.900731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330506+0000","last_change":"2024-05-09T00:08:26.054273+0000","last_active":"2024-05-09T00:10:09.330506+0000","last_peered":"2024-05-09T00:10:09.330506+0000","last_clean":"2024-05-09T00:10:09.330506+0000","last_became_active":"2024-05-09T00:08:26.053840+0000","last_became_peered":"2024-05-09T00:08:26.053840+0000","last_unstale":"2024-05-09T00:10:09.330506+0000","last_undegraded":"2024-05-09T00:10:09.330506+0000","last_fullsized":"2024-05-09T00:10:09.330506+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:27.128582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469520+0000","last_change":"2024-05-09T00:08:23.021594+0000","last_active":"2024-05-09T00:09:10.469520+0000","last_peered":"2024-05-09T00:09:10.469520+0000","last_clean":"2024-05-09T00:09:10.469520+0000","last_became_active":"2024-05-09T00:08:23.021152+0000","last_became_peered":"2024-05-09T00:08:23.021152+0000","last_unstale":"2024-05-09T00:09:10.469520+0000","last_undegraded":"2024-05-09T00:09:10.469520+0000","last_fullsized":"2024-05-09T00:09:10.469520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:58.505781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330209+0000","last_change":"2024-05-09T00:08:26.064464+0000","last_active":"2024-05-09T00:10:09.330209+0000","last_peered":"2024-05-09T00:10:09.330209+0000","last_clean":"2024-05-09T00:10:09.330209+0000","last_became_active":"2024-05-09T00:08:26.064000+0000","last_became_peered":"2024-05-09T00:08:26.064000+0000","last_unstale":"2024-05-09T00:10:09.330209+0000","last_undegraded":"2024-05-09T00:10:09.330209+0000","last_fullsized":"2024-05-09T00:10:09.330209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:01.455979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":227,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:51.668686+0000","last_change":"2024-05-09T00:08:23.020823+0000","last_active":"2024-05-09T00:11:51.668686+0000","last_peered":"2024-05-09T00:11:51.668686+0000","last_clean":"2024-05-09T00:11:51.668686+0000","last_became_active":"2024-05-09T00:08:23.020412+0000","last_became_peered":"2024-05-09T00:08:23.020412+0000","last_unstale":"2024-05-09T00:11:51.668686+0000","last_undegraded":"2024-05-09T00:11:51.668686+0000","last_fullsized":"2024-05-09T00:11:51.668686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:05.457452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":195,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318663+0000","last_change":"2024-05-09T00:09:10.474540+0000","last_active":"2024-05-09T00:10:09.318663+0000","last_peered":"2024-05-09T00:10:09.318663+0000","last_clean":"2024-05-09T00:10:09.318663+0000","last_became_active":"2024-05-09T00:09:10.474068+0000","last_became_peered":"2024-05-09T00:09:10.474068+0000","last_unstale":"2024-05-09T00:10:09.318663+0000","last_undegraded":"2024-05-09T00:10:09.318663+0000","last_fullsized":"2024-05-09T00:10:09.318663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:37.296140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474415+0000","last_change":"2024-05-09T00:08:23.118857+0000","last_active":"2024-05-09T00:09:10.474415+0000","last_peered":"2024-05-09T00:09:10.474415+0000","last_clean":"2024-05-09T00:09:10.474415+0000","last_became_active":"2024-05-09T00:08:23.118433+0000","last_became_peered":"2024-05-09T00:08:23.118433+0000","last_unstale":"2024-05-09T00:09:10.474415+0000","last_undegraded":"2024-05-09T00:09:10.474415+0000","last_fullsized":"2024-05-09T00:09:10.474415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:10.617987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327227+0000","last_change":"2024-05-09T00:08:26.066911+0000","last_active":"2024-05-09T00:10:09.327227+0000","last_peered":"2024-05-09T00:10:09.327227+0000","last_clean":"2024-05-09T00:10:09.327227+0000","last_became_active":"2024-05-09T00:08:26.066548+0000","last_became_peered":"2024-05-09T00:08:26.066548+0000","last_unstale":"2024-05-09T00:10:09.327227+0000","last_undegraded":"2024-05-09T00:10:09.327227+0000","last_fullsized":"2024-05-09T00:10:09.327227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:32.391390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:39.175360+0000","last_change":"2024-05-09T00:08:23.019829+0000","last_active":"2024-05-09T00:10:39.175360+0000","last_peered":"2024-05-09T00:10:39.175360+0000","last_clean":"2024-05-09T00:10:39.175360+0000","last_became_active":"2024-05-09T00:08:23.019336+0000","last_became_peered":"2024-05-09T00:08:23.019336+0000","last_unstale":"2024-05-09T00:10:39.175360+0000","last_undegraded":"2024-05-09T00:10:39.175360+0000","last_fullsized":"2024-05-09T00:10:39.175360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:33.802738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318804+0000","last_change":"2024-05-09T00:08:26.055975+0000","last_active":"2024-05-09T00:10:09.318804+0000","last_peered":"2024-05-09T00:10:09.318804+0000","last_clean":"2024-05-09T00:10:09.318804+0000","last_became_active":"2024-05-09T00:08:26.055595+0000","last_became_peered":"2024-05-09T00:08:26.055595+0000","last_unstale":"2024-05-09T00:10:09.318804+0000","last_undegraded":"2024-05-09T00:10:09.318804+0000","last_fullsized":"2024-05-09T00:10:09.318804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:20.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469423+0000","last_change":"2024-05-09T00:08:23.120353+0000","last_active":"2024-05-09T00:09:10.469423+0000","last_peered":"2024-05-09T00:09:10.469423+0000","last_clean":"2024-05-09T00:09:10.469423+0000","last_became_active":"2024-05-09T00:08:23.120004+0000","last_became_peered":"2024-05-09T00:08:23.120004+0000","last_unstale":"2024-05-09T00:09:10.469423+0000","last_undegraded":"2024-05-09T00:09:10.469423+0000","last_fullsized":"2024-05-09T00:09:10.469423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:12.168671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.320556+0000","last_change":"2024-05-09T00:08:26.046554+0000","last_active":"2024-05-09T00:10:09.320556+0000","last_peered":"2024-05-09T00:10:09.320556+0000","last_clean":"2024-05-09T00:10:09.320556+0000","last_became_active":"2024-05-09T00:08:26.045924+0000","last_became_peered":"2024-05-09T00:08:26.045924+0000","last_unstale":"2024-05-09T00:10:09.320556+0000","last_undegraded":"2024-05-09T00:10:09.320556+0000","last_fullsized":"2024-05-09T00:10:09.320556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:01.208980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472462+0000","last_change":"2024-05-09T00:08:23.019778+0000","last_active":"2024-05-09T00:09:10.472462+0000","last_peered":"2024-05-09T00:09:10.472462+0000","last_clean":"2024-05-09T00:09:10.472462+0000","last_became_active":"2024-05-09T00:08:23.019400+0000","last_became_peered":"2024-05-09T00:08:23.019400+0000","last_unstale":"2024-05-09T00:09:10.472462+0000","last_undegraded":"2024-05-09T00:09:10.472462+0000","last_fullsized":"2024-05-09T00:09:10.472462+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:06.883167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.308957+0000","last_change":"2024-05-09T00:09:10.474352+0000","last_active":"2024-05-09T00:10:09.308957+0000","last_peered":"2024-05-09T00:10:09.308957+0000","last_clean":"2024-05-09T00:10:09.308957+0000","last_became_active":"2024-05-09T00:09:10.473906+0000","last_became_peered":"2024-05-09T00:09:10.473906+0000","last_unstale":"2024-05-09T00:10:09.308957+0000","last_undegraded":"2024-05-09T00:10:09.308957+0000","last_fullsized":"2024-05-09T00:10:09.308957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:35:18.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.319595+0000","last_change":"2024-05-09T00:09:10.476484+0000","last_active":"2024-05-09T00:10:09.319595+0000","last_peered":"2024-05-09T00:10:09.319595+0000","last_clean":"2024-05-09T00:10:09.319595+0000","last_became_active":"2024-05-09T00:09:10.476001+0000","last_became_peered":"2024-05-09T00:09:10.476001+0000","last_unstale":"2024-05-09T00:10:09.319595+0000","last_undegraded":"2024-05-09T00:10:09.319595+0000","last_fullsized":"2024-05-09T00:10:09.319595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:32.018749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'6","reported_seq":223,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:52.669427+0000","last_change":"2024-05-09T00:08:23.118322+0000","last_active":"2024-05-09T00:11:52.669427+0000","last_peered":"2024-05-09T00:11:52.669427+0000","last_clean":"2024-05-09T00:11:52.669427+0000","last_became_active":"2024-05-09T00:08:23.117943+0000","last_became_peered":"2024-05-09T00:08:23.117943+0000","last_unstale":"2024-05-09T00:11:52.669427+0000","last_undegraded":"2024-05-09T00:11:52.669427+0000","last_fullsized":"2024-05-09T00:11:52.669427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:07.890437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323361+0000","last_change":"2024-05-09T00:08:26.052294+0000","last_active":"2024-05-09T00:10:09.323361+0000","last_peered":"2024-05-09T00:10:09.323361+0000","last_clean":"2024-05-09T00:10:09.323361+0000","last_became_active":"2024-05-09T00:08:26.051926+0000","last_became_peered":"2024-05-09T00:08:26.051926+0000","last_unstale":"2024-05-09T00:10:09.323361+0000","last_undegraded":"2024-05-09T00:10:09.323361+0000","last_fullsized":"2024-05-09T00:10:09.323361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:53.651401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796358+0000","last_change":"2024-05-09T00:08:23.015007+0000","last_active":"2024-05-09T00:09:10.796358+0000","last_peered":"2024-05-09T00:09:10.796358+0000","last_clean":"2024-05-09T00:09:10.796358+0000","last_became_active":"2024-05-09T00:08:23.014545+0000","last_became_peered":"2024-05-09T00:08:23.014545+0000","last_unstale":"2024-05-09T00:09:10.796358+0000","last_undegraded":"2024-05-09T00:09:10.796358+0000","last_fullsized":"2024-05-09T00:09:10.796358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:19.850200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:43.888672+0000","last_change":"2024-05-09T00:07:00.733233+0000","last_active":"2024-05-09T00:09:43.888672+0000","last_peered":"2024-05-09T00:09:43.888672+0000","last_clean":"2024-05-09T00:09:43.888672+0000","last_became_active":"2024-05-09T00:07:00.727365+0000","last_became_peered":"2024-05-09T00:07:00.727365+0000","last_unstale":"2024-05-09T00:09:43.888672+0000","last_undegraded":"2024-05-09T00:09:43.888672+0000","last_fullsized":"2024-05-09T00:09:43.888672+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_clean_scrub_stamp":"2024-05-09T00:03:33.285220+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:08.597290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326209+0000","last_change":"2024-05-09T00:08:26.064000+0000","last_active":"2024-05-09T00:10:09.326209+0000","last_peered":"2024-05-09T00:10:09.326209+0000","last_clean":"2024-05-09T00:10:09.326209+0000","last_became_active":"2024-05-09T00:08:26.063636+0000","last_became_peered":"2024-05-09T00:08:26.063636+0000","last_unstale":"2024-05-09T00:10:09.326209+0000","last_undegraded":"2024-05-09T00:10:09.326209+0000","last_fullsized":"2024-05-09T00:10:09.326209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:59.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472596+0000","last_change":"2024-05-09T00:08:23.023459+0000","last_active":"2024-05-09T00:09:10.472596+0000","last_peered":"2024-05-09T00:09:10.472596+0000","last_clean":"2024-05-09T00:09:10.472596+0000","last_became_active":"2024-05-09T00:08:23.023089+0000","last_became_peered":"2024-05-09T00:08:23.023089+0000","last_unstale":"2024-05-09T00:09:10.472596+0000","last_undegraded":"2024-05-09T00:09:10.472596+0000","last_fullsized":"2024-05-09T00:09:10.472596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:41.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:14.346903+0000","last_change":"2024-05-09T00:09:10.478686+0000","last_active":"2024-05-09T00:09:14.346903+0000","last_peered":"2024-05-09T00:09:14.346903+0000","last_clean":"2024-05-09T00:09:14.346903+0000","last_became_active":"2024-05-09T00:09:10.474890+0000","last_became_peered":"2024-05-09T00:09:10.474890+0000","last_unstale":"2024-05-09T00:09:14.346903+0000","last_undegraded":"2024-05-09T00:09:14.346903+0000","last_fullsized":"2024-05-09T00:09:14.346903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:41.041247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327716+0000","last_change":"2024-05-09T00:08:26.050685+0000","last_active":"2024-05-09T00:10:09.327716+0000","last_peered":"2024-05-09T00:10:09.327716+0000","last_clean":"2024-05-09T00:10:09.327716+0000","last_became_active":"2024-05-09T00:08:26.050312+0000","last_became_peered":"2024-05-09T00:08:26.050312+0000","last_unstale":"2024-05-09T00:10:09.327716+0000","last_undegraded":"2024-05-09T00:10:09.327716+0000","last_fullsized":"2024-05-09T00:10:09.327716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:31.623182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:32.811512+0000","last_change":"2024-05-09T00:08:23.019663+0000","last_active":"2024-05-09T00:10:32.811512+0000","last_peered":"2024-05-09T00:10:32.811512+0000","last_clean":"2024-05-09T00:10:32.811512+0000","last_became_active":"2024-05-09T00:08:23.019301+0000","last_became_peered":"2024-05-09T00:08:23.019301+0000","last_unstale":"2024-05-09T00:10:32.811512+0000","last_undegraded":"2024-05-09T00:10:32.811512+0000","last_fullsized":"2024-05-09T00:10:32.811512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:08.291227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330686+0000","last_change":"2024-05-09T00:08:26.058825+0000","last_active":"2024-05-09T00:10:09.330686+0000","last_peered":"2024-05-09T00:10:09.330686+0000","last_clean":"2024-05-09T00:10:09.330686+0000","last_became_active":"2024-05-09T00:08:26.058469+0000","last_became_peered":"2024-05-09T00:08:26.058469+0000","last_unstale":"2024-05-09T00:10:09.330686+0000","last_undegraded":"2024-05-09T00:10:09.330686+0000","last_fullsized":"2024-05-09T00:10:09.330686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:08.118819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796594+0000","last_change":"2024-05-09T00:08:23.054366+0000","last_active":"2024-05-09T00:09:10.796594+0000","last_peered":"2024-05-09T00:09:10.796594+0000","last_clean":"2024-05-09T00:09:10.796594+0000","last_became_active":"2024-05-09T00:08:23.053977+0000","last_became_peered":"2024-05-09T00:08:23.053977+0000","last_unstale":"2024-05-09T00:09:10.796594+0000","last_undegraded":"2024-05-09T00:09:10.796594+0000","last_fullsized":"2024-05-09T00:09:10.796594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:37.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.316437+0000","last_change":"2024-05-09T00:08:26.057864+0000","last_active":"2024-05-09T00:10:09.316437+0000","last_peered":"2024-05-09T00:10:09.316437+0000","last_clean":"2024-05-09T00:10:09.316437+0000","last_became_active":"2024-05-09T00:08:26.057436+0000","last_became_peered":"2024-05-09T00:08:26.057436+0000","last_unstale":"2024-05-09T00:10:09.316437+0000","last_undegraded":"2024-05-09T00:10:09.316437+0000","last_fullsized":"2024-05-09T00:10:09.316437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:19.976265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468672+0000","last_change":"2024-05-09T00:08:23.118803+0000","last_active":"2024-05-09T00:09:10.468672+0000","last_peered":"2024-05-09T00:09:10.468672+0000","last_clean":"2024-05-09T00:09:10.468672+0000","last_became_active":"2024-05-09T00:08:23.118432+0000","last_became_peered":"2024-05-09T00:08:23.118432+0000","last_unstale":"2024-05-09T00:09:10.468672+0000","last_undegraded":"2024-05-09T00:09:10.468672+0000","last_fullsized":"2024-05-09T00:09:10.468672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:50.253699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.310200+0000","last_change":"2024-05-09T00:08:26.054281+0000","last_active":"2024-05-09T00:10:09.310200+0000","last_peered":"2024-05-09T00:10:09.310200+0000","last_clean":"2024-05-09T00:10:09.310200+0000","last_became_active":"2024-05-09T00:08:26.053874+0000","last_became_peered":"2024-05-09T00:08:26.053874+0000","last_unstale":"2024-05-09T00:10:09.310200+0000","last_undegraded":"2024-05-09T00:10:09.310200+0000","last_fullsized":"2024-05-09T00:10:09.310200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:11.566337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467767+0000","last_change":"2024-05-09T00:08:23.017905+0000","last_active":"2024-05-09T00:09:10.467767+0000","last_peered":"2024-05-09T00:09:10.467767+0000","last_clean":"2024-05-09T00:09:10.467767+0000","last_became_active":"2024-05-09T00:08:23.017481+0000","last_became_peered":"2024-05-09T00:08:23.017481+0000","last_unstale":"2024-05-09T00:09:10.467767+0000","last_undegraded":"2024-05-09T00:09:10.467767+0000","last_fullsized":"2024-05-09T00:09:10.467767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:40.096415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.317514+0000","last_change":"2024-05-09T00:08:26.053392+0000","last_active":"2024-05-09T00:10:09.317514+0000","last_peered":"2024-05-09T00:10:09.317514+0000","last_clean":"2024-05-09T00:10:09.317514+0000","last_became_active":"2024-05-09T00:08:26.052628+0000","last_became_peered":"2024-05-09T00:08:26.052628+0000","last_unstale":"2024-05-09T00:10:09.317514+0000","last_undegraded":"2024-05-09T00:10:09.317514+0000","last_fullsized":"2024-05-09T00:10:09.317514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:28.013955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:14.346880+0000","last_change":"2024-05-09T00:09:10.481233+0000","last_active":"2024-05-09T00:09:14.346880+0000","last_peered":"2024-05-09T00:09:14.346880+0000","last_clean":"2024-05-09T00:09:14.346880+0000","last_became_active":"2024-05-09T00:09:10.475321+0000","last_became_peered":"2024-05-09T00:09:10.475321+0000","last_unstale":"2024-05-09T00:09:14.346880+0000","last_undegraded":"2024-05-09T00:09:14.346880+0000","last_fullsized":"2024-05-09T00:09:14.346880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:58.859277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.308335+0000","last_change":"2024-05-09T00:08:26.047812+0000","last_active":"2024-05-09T00:10:09.308335+0000","last_peered":"2024-05-09T00:10:09.308335+0000","last_clean":"2024-05-09T00:10:09.308335+0000","last_became_active":"2024-05-09T00:08:26.047459+0000","last_became_peered":"2024-05-09T00:08:26.047459+0000","last_unstale":"2024-05-09T00:10:09.308335+0000","last_undegraded":"2024-05-09T00:10:09.308335+0000","last_fullsized":"2024-05-09T00:10:09.308335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.257831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473816+0000","last_change":"2024-05-09T00:08:23.023800+0000","last_active":"2024-05-09T00:09:10.473816+0000","last_peered":"2024-05-09T00:09:10.473816+0000","last_clean":"2024-05-09T00:09:10.473816+0000","last_became_active":"2024-05-09T00:08:23.023387+0000","last_became_peered":"2024-05-09T00:08:23.023387+0000","last_unstale":"2024-05-09T00:09:10.473816+0000","last_undegraded":"2024-05-09T00:09:10.473816+0000","last_fullsized":"2024-05-09T00:09:10.473816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:33.714683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330035+0000","last_change":"2024-05-09T00:08:26.056981+0000","last_active":"2024-05-09T00:10:09.330035+0000","last_peered":"2024-05-09T00:10:09.330035+0000","last_clean":"2024-05-09T00:10:09.330035+0000","last_became_active":"2024-05-09T00:08:26.056605+0000","last_became_peered":"2024-05-09T00:08:26.056605+0000","last_unstale":"2024-05-09T00:10:09.330035+0000","last_undegraded":"2024-05-09T00:10:09.330035+0000","last_fullsized":"2024-05-09T00:10:09.330035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:32.897929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473675+0000","last_change":"2024-05-09T00:08:23.120612+0000","last_active":"2024-05-09T00:09:10.473675+0000","last_peered":"2024-05-09T00:09:10.473675+0000","last_clean":"2024-05-09T00:09:10.473675+0000","last_became_active":"2024-05-09T00:08:23.120157+0000","last_became_peered":"2024-05-09T00:08:23.120157+0000","last_unstale":"2024-05-09T00:09:10.473675+0000","last_undegraded":"2024-05-09T00:09:10.473675+0000","last_fullsized":"2024-05-09T00:09:10.473675+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:08.027214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330272+0000","last_change":"2024-05-09T00:08:26.061594+0000","last_active":"2024-05-09T00:10:09.330272+0000","last_peered":"2024-05-09T00:10:09.330272+0000","last_clean":"2024-05-09T00:10:09.330272+0000","last_became_active":"2024-05-09T00:08:26.061201+0000","last_became_peered":"2024-05-09T00:08:26.061201+0000","last_unstale":"2024-05-09T00:10:09.330272+0000","last_undegraded":"2024-05-09T00:10:09.330272+0000","last_fullsized":"2024-05-09T00:10:09.330272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:50.183648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470328+0000","last_change":"2024-05-09T00:08:23.017693+0000","last_active":"2024-05-09T00:09:10.470328+0000","last_peered":"2024-05-09T00:09:10.470328+0000","last_clean":"2024-05-09T00:09:10.470328+0000","last_became_active":"2024-05-09T00:08:23.017211+0000","last_became_peered":"2024-05-09T00:08:23.017211+0000","last_unstale":"2024-05-09T00:09:10.470328+0000","last_undegraded":"2024-05-09T00:09:10.470328+0000","last_fullsized":"2024-05-09T00:09:10.470328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:53.569932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327590+0000","last_change":"2024-05-09T00:08:26.063922+0000","last_active":"2024-05-09T00:10:09.327590+0000","last_peered":"2024-05-09T00:10:09.327590+0000","last_clean":"2024-05-09T00:10:09.327590+0000","last_became_active":"2024-05-09T00:08:26.063554+0000","last_became_peered":"2024-05-09T00:08:26.063554+0000","last_unstale":"2024-05-09T00:10:09.327590+0000","last_undegraded":"2024-05-09T00:10:09.327590+0000","last_fullsized":"2024-05-09T00:10:09.327590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:01.754688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548651+0000","last_change":"2024-05-09T00:08:23.020367+0000","last_active":"2024-05-09T00:09:10.548651+0000","last_peered":"2024-05-09T00:09:10.548651+0000","last_clean":"2024-05-09T00:09:10.548651+0000","last_became_active":"2024-05-09T00:08:23.019801+0000","last_became_peered":"2024-05-09T00:08:23.019801+0000","last_unstale":"2024-05-09T00:09:10.548651+0000","last_undegraded":"2024-05-09T00:09:10.548651+0000","last_fullsized":"2024-05-09T00:09:10.548651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:14.936021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330156+0000","last_change":"2024-05-09T00:08:26.063863+0000","last_active":"2024-05-09T00:10:09.330156+0000","last_peered":"2024-05-09T00:10:09.330156+0000","last_clean":"2024-05-09T00:10:09.330156+0000","last_became_active":"2024-05-09T00:08:26.061938+0000","last_became_peered":"2024-05-09T00:08:26.061938+0000","last_unstale":"2024-05-09T00:10:09.330156+0000","last_undegraded":"2024-05-09T00:10:09.330156+0000","last_fullsized":"2024-05-09T00:10:09.330156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:04.597991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.319694+0000","last_change":"2024-05-09T00:08:26.050837+0000","last_active":"2024-05-09T00:10:09.319694+0000","last_peered":"2024-05-09T00:10:09.319694+0000","last_clean":"2024-05-09T00:10:09.319694+0000","last_became_active":"2024-05-09T00:08:26.050424+0000","last_became_peered":"2024-05-09T00:08:26.050424+0000","last_unstale":"2024-05-09T00:10:09.319694+0000","last_undegraded":"2024-05-09T00:10:09.319694+0000","last_fullsized":"2024-05-09T00:10:09.319694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:45.134811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323117+0000","last_change":"2024-05-09T00:09:10.549989+0000","last_active":"2024-05-09T00:10:09.323117+0000","last_peered":"2024-05-09T00:10:09.323117+0000","last_clean":"2024-05-09T00:10:09.323117+0000","last_became_active":"2024-05-09T00:09:10.549502+0000","last_became_peered":"2024-05-09T00:09:10.549502+0000","last_unstale":"2024-05-09T00:10:09.323117+0000","last_undegraded":"2024-05-09T00:10:09.323117+0000","last_fullsized":"2024-05-09T00:10:09.323117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:10.654452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323905+0000","last_change":"2024-05-09T00:08:26.058012+0000","last_active":"2024-05-09T00:10:09.323905+0000","last_peered":"2024-05-09T00:10:09.323905+0000","last_clean":"2024-05-09T00:10:09.323905+0000","last_became_active":"2024-05-09T00:08:26.057282+0000","last_became_peered":"2024-05-09T00:08:26.057282+0000","last_unstale":"2024-05-09T00:10:09.323905+0000","last_undegraded":"2024-05-09T00:10:09.323905+0000","last_fullsized":"2024-05-09T00:10:09.323905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:53.520593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.320691+0000","last_change":"2024-05-09T00:08:26.063331+0000","last_active":"2024-05-09T00:10:09.320691+0000","last_peered":"2024-05-09T00:10:09.320691+0000","last_clean":"2024-05-09T00:10:09.320691+0000","last_became_active":"2024-05-09T00:08:26.062866+0000","last_became_peered":"2024-05-09T00:08:26.062866+0000","last_unstale":"2024-05-09T00:10:09.320691+0000","last_undegraded":"2024-05-09T00:10:09.320691+0000","last_fullsized":"2024-05-09T00:10:09.320691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:10.387852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326309+0000","last_change":"2024-05-09T00:08:26.063735+0000","last_active":"2024-05-09T00:10:09.326309+0000","last_peered":"2024-05-09T00:10:09.326309+0000","last_clean":"2024-05-09T00:10:09.326309+0000","last_became_active":"2024-05-09T00:08:26.063358+0000","last_became_peered":"2024-05-09T00:08:26.063358+0000","last_unstale":"2024-05-09T00:10:09.326309+0000","last_undegraded":"2024-05-09T00:10:09.326309+0000","last_fullsized":"2024-05-09T00:10:09.326309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:06.832071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318431+0000","last_change":"2024-05-09T00:08:26.052746+0000","last_active":"2024-05-09T00:10:09.318431+0000","last_peered":"2024-05-09T00:10:09.318431+0000","last_clean":"2024-05-09T00:10:09.318431+0000","last_became_active":"2024-05-09T00:08:26.051544+0000","last_became_peered":"2024-05-09T00:08:26.051544+0000","last_unstale":"2024-05-09T00:10:09.318431+0000","last_undegraded":"2024-05-09T00:10:09.318431+0000","last_fullsized":"2024-05-09T00:10:09.318431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:31.963268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323267+0000","last_change":"2024-05-09T00:08:26.058075+0000","last_active":"2024-05-09T00:10:09.323267+0000","last_peered":"2024-05-09T00:10:09.323267+0000","last_clean":"2024-05-09T00:10:09.323267+0000","last_became_active":"2024-05-09T00:08:26.057349+0000","last_became_peered":"2024-05-09T00:08:26.057349+0000","last_unstale":"2024-05-09T00:10:09.323267+0000","last_undegraded":"2024-05-09T00:10:09.323267+0000","last_fullsized":"2024-05-09T00:10:09.323267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:59.322903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324094+0000","last_change":"2024-05-09T00:08:26.056429+0000","last_active":"2024-05-09T00:10:09.324094+0000","last_peered":"2024-05-09T00:10:09.324094+0000","last_clean":"2024-05-09T00:10:09.324094+0000","last_became_active":"2024-05-09T00:08:26.054852+0000","last_became_peered":"2024-05-09T00:08:26.054852+0000","last_unstale":"2024-05-09T00:10:09.324094+0000","last_undegraded":"2024-05-09T00:10:09.324094+0000","last_fullsized":"2024-05-09T00:10:09.324094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:05.337636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327096+0000","last_change":"2024-05-09T00:09:10.549880+0000","last_active":"2024-05-09T00:10:09.327096+0000","last_peered":"2024-05-09T00:10:09.327096+0000","last_clean":"2024-05-09T00:10:09.327096+0000","last_became_active":"2024-05-09T00:09:10.549448+0000","last_became_peered":"2024-05-09T00:09:10.549448+0000","last_unstale":"2024-05-09T00:10:09.327096+0000","last_undegraded":"2024-05-09T00:10:09.327096+0000","last_fullsized":"2024-05-09T00:10:09.327096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:16.990729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325082+0000","last_change":"2024-05-09T00:08:26.054579+0000","last_active":"2024-05-09T00:10:09.325082+0000","last_peered":"2024-05-09T00:10:09.325082+0000","last_clean":"2024-05-09T00:10:09.325082+0000","last_became_active":"2024-05-09T00:08:26.053186+0000","last_became_peered":"2024-05-09T00:08:26.053186+0000","last_unstale":"2024-05-09T00:10:09.325082+0000","last_undegraded":"2024-05-09T00:10:09.325082+0000","last_fullsized":"2024-05-09T00:10:09.325082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:37.561691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328708+0000","last_change":"2024-05-09T00:08:26.063608+0000","last_active":"2024-05-09T00:10:09.328708+0000","last_peered":"2024-05-09T00:10:09.328708+0000","last_clean":"2024-05-09T00:10:09.328708+0000","last_became_active":"2024-05-09T00:08:26.063078+0000","last_became_peered":"2024-05-09T00:08:26.063078+0000","last_unstale":"2024-05-09T00:10:09.328708+0000","last_undegraded":"2024-05-09T00:10:09.328708+0000","last_fullsized":"2024-05-09T00:10:09.328708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:06.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.331031+0000","last_change":"2024-05-09T00:08:26.055058+0000","last_active":"2024-05-09T00:10:09.331031+0000","last_peered":"2024-05-09T00:10:09.331031+0000","last_clean":"2024-05-09T00:10:09.331031+0000","last_became_active":"2024-05-09T00:08:26.054695+0000","last_became_peered":"2024-05-09T00:08:26.054695+0000","last_unstale":"2024-05-09T00:10:09.331031+0000","last_undegraded":"2024-05-09T00:10:09.331031+0000","last_fullsized":"2024-05-09T00:10:09.331031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:58.925990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.312854+0000","last_change":"2024-05-09T00:08:26.055366+0000","last_active":"2024-05-09T00:10:09.312854+0000","last_peered":"2024-05-09T00:10:09.312854+0000","last_clean":"2024-05-09T00:10:09.312854+0000","last_became_active":"2024-05-09T00:08:26.054904+0000","last_became_peered":"2024-05-09T00:08:26.054904+0000","last_unstale":"2024-05-09T00:10:09.312854+0000","last_undegraded":"2024-05-09T00:10:09.312854+0000","last_fullsized":"2024-05-09T00:10:09.312854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:40.754878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285241,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71147,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1169,"num_read_kb":661,"num_write":181,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":213441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710777,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":425247,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1084243,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070531,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":880640,"data_stored":425461,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266761,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":425359,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":1017760,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":868352,"data_stored":424850,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":491340,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":888832,"data_stored":426121,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":488376,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":425161,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986270,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":426127,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822744,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":1016604,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:11:56.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T00:11:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.343+0000 7fb58d096640 1 -- 172.21.15.46:0/1712664445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb588108a30 msgr2=0x7fb58810ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.343+0000 7fb58d096640 1 --2- 172.21.15.46:0/1712664445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb588108a30 0x7fb58810ae50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb578009ca0 tx=0x7fb57802a8c0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.343+0000 7fb58d096640 1 -- 172.21.15.46:0/1712664445 shutdown_connections 2024-05-09T00:11:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.343+0000 7fb58d096640 1 --2- 172.21.15.46:0/1712664445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb588108a30 0x7fb58810ae50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.343+0000 7fb58d096640 1 --2- 172.21.15.46:0/1712664445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb588103d30 0x7fb588104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.343+0000 7fb58d096640 1 --2- 172.21.15.46:0/1712664445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb588102af0 0x7fb5881037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.343+0000 7fb58d096640 1 -- 172.21.15.46:0/1712664445 >> 172.21.15.46:0/1712664445 conn(0x7fb5880fdd00 msgr2=0x7fb588100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.343+0000 7fb58d096640 1 -- 172.21.15.46:0/1712664445 shutdown_connections 2024-05-09T00:11:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.343+0000 7fb58d096640 1 -- 172.21.15.46:0/1712664445 wait complete. 2024-05-09T00:11:56.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb58d096640 1 Processor -- start 2024-05-09T00:11:56.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb58d096640 1 -- start start 2024-05-09T00:11:56.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb58d096640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb588102af0 0x7fb5881991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:56.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb58d096640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb588103d30 0x7fb588199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:56.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb58d096640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb588108a30 0x7fb5881a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:56.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb58d096640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810e650 con 0x7fb588102af0 2024-05-09T00:11:56.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb58d096640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810e4d0 con 0x7fb588103d30 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb58d096640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810e7d0 con 0x7fb588108a30 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb586d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb588102af0 0x7fb5881991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb586d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb588102af0 0x7fb5881991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59458/0 (socket says 172.21.15.46:59458) 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb586d9d640 1 -- 172.21.15.46:0/1045750475 learned_addr learned my addr 172.21.15.46:0/1045750475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb58659c640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb588103d30 0x7fb588199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.344+0000 7fb58759e640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb588108a30 0x7fb5881a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.345+0000 7fb586d9d640 1 -- 172.21.15.46:0/1045750475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb588108a30 msgr2=0x7fb5881a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.345+0000 7fb586d9d640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb588108a30 0x7fb5881a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.345+0000 7fb586d9d640 1 -- 172.21.15.46:0/1045750475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb588103d30 msgr2=0x7fb588199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.345+0000 7fb586d9d640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb588103d30 0x7fb588199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.345+0000 7fb586d9d640 1 -- 172.21.15.46:0/1045750475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5881a0d30 con 0x7fb588102af0 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.345+0000 7fb586d9d640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb588102af0 0x7fb5881991f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb58000d740 tx=0x7fb58000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.345+0000 7fb57ffff640 1 -- 172.21.15.46:0/1045750475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580017070 con 0x7fb588102af0 2024-05-09T00:11:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.345+0000 7fb58d096640 1 -- 172.21.15.46:0/1045750475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5881a0fc0 con 0x7fb588102af0 2024-05-09T00:11:56.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.345+0000 7fb57ffff640 1 -- 172.21.15.46:0/1045750475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb58000de50 con 0x7fb588102af0 2024-05-09T00:11:56.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.345+0000 7fb57ffff640 1 -- 172.21.15.46:0/1045750475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580003c30 con 0x7fb588102af0 2024-05-09T00:11:56.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.345+0000 7fb58d096640 1 -- 172.21.15.46:0/1045750475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5881a1480 con 0x7fb588102af0 2024-05-09T00:11:56.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.346+0000 7fb58d096640 1 -- 172.21.15.46:0/1045750475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb588107840 con 0x7fb588102af0 2024-05-09T00:11:56.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.348+0000 7fb57ffff640 1 -- 172.21.15.46:0/1045750475 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb580003dd0 con 0x7fb588102af0 2024-05-09T00:11:56.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.348+0000 7fb57ffff640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb5740b27f0 0x7fb5740b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:56.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.349+0000 7fb58659c640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb5740b27f0 0x7fb5740b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:56.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.349+0000 7fb57ffff640 1 -- 172.21.15.46:0/1045750475 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb5800e1f90 con 0x7fb588102af0 2024-05-09T00:11:56.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.349+0000 7fb58659c640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb5740b27f0 0x7fb5740b4d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb58819a290 tx=0x7fb570005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:56.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.352+0000 7fb57ffff640 1 -- 172.21.15.46:0/1045750475 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb5800a9660 con 0x7fb588102af0 2024-05-09T00:11:56.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.541+0000 7fb58d096640 1 -- 172.21.15.46:0/1045750475 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb588063110 con 0x7fb5740b27f0 2024-05-09T00:11:56.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.549+0000 7fb57ffff640 1 -- 172.21.15.46:0/1045750475 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259083 (secure 0 0 0) 0x7fb588063110 con 0x7fb5740b27f0 2024-05-09T00:11:56.549 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:56.550 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T00:11:56.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.552+0000 7fb58d096640 1 -- 172.21.15.46:0/1045750475 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb5740b27f0 msgr2=0x7fb5740b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:56.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.552+0000 7fb58d096640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb5740b27f0 0x7fb5740b4d00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb58819a290 tx=0x7fb570005f90 comp rx=0 tx=0).stop 2024-05-09T00:11:56.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.552+0000 7fb58d096640 1 -- 172.21.15.46:0/1045750475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb588102af0 msgr2=0x7fb5881991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:56.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.552+0000 7fb58d096640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb588102af0 0x7fb5881991f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb58000d740 tx=0x7fb58000dc00 comp rx=0 tx=0).stop 2024-05-09T00:11:56.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.552+0000 7fb58d096640 1 -- 172.21.15.46:0/1045750475 shutdown_connections 2024-05-09T00:11:56.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.552+0000 7fb58d096640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb5740b27f0 0x7fb5740b4d00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.552+0000 7fb58d096640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb588108a30 0x7fb5881a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.552+0000 7fb58d096640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb588103d30 0x7fb588199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.552+0000 7fb58d096640 1 --2- 172.21.15.46:0/1045750475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb588102af0 0x7fb5881991f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.552+0000 7fb58d096640 1 -- 172.21.15.46:0/1045750475 >> 172.21.15.46:0/1045750475 conn(0x7fb5880fdd00 msgr2=0x7fb588104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:56.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.552+0000 7fb58d096640 1 -- 172.21.15.46:0/1045750475 shutdown_connections 2024-05-09T00:11:56.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.552+0000 7fb58d096640 1 -- 172.21.15.46:0/1045750475 wait complete. 2024-05-09T00:11:56.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:56 smithi046 ceph-mon[30151]: pgmap v393: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-09T00:11:56.562 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-05-09T00:11:55.339263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661749,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1245,"num_read_kb":725,"num_write":306,"num_write_kb":2280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":747,"ondisk_log_size":747,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372308,"kb_used_data":10676,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569516,"statfs":{"total":1151940427776,"available":1151559184384,"internally_reserved":0,"allocated":10932224,"data_stored":7076649,"data_compressed":177897,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4930,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":52,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003242"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324383+0000","last_change":"2024-05-09T00:08:26.060455+0000","last_active":"2024-05-09T00:10:09.324383+0000","last_peered":"2024-05-09T00:10:09.324383+0000","last_clean":"2024-05-09T00:10:09.324383+0000","last_became_active":"2024-05-09T00:08:26.060068+0000","last_became_peered":"2024-05-09T00:08:26.060068+0000","last_unstale":"2024-05-09T00:10:09.324383+0000","last_undegraded":"2024-05-09T00:10:09.324383+0000","last_fullsized":"2024-05-09T00:10:09.324383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:48.354188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.314519+0000","last_change":"2024-05-09T00:08:26.055737+0000","last_active":"2024-05-09T00:10:09.314519+0000","last_peered":"2024-05-09T00:10:09.314519+0000","last_clean":"2024-05-09T00:10:09.314519+0000","last_became_active":"2024-05-09T00:08:26.055354+0000","last_became_peered":"2024-05-09T00:08:26.055354+0000","last_unstale":"2024-05-09T00:10:09.314519+0000","last_undegraded":"2024-05-09T00:10:09.314519+0000","last_fullsized":"2024-05-09T00:10:09.314519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:54.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324383+0000","last_change":"2024-05-09T00:08:26.050776+0000","last_active":"2024-05-09T00:10:09.324383+0000","last_peered":"2024-05-09T00:10:09.324383+0000","last_clean":"2024-05-09T00:10:09.324383+0000","last_became_active":"2024-05-09T00:08:26.050393+0000","last_became_peered":"2024-05-09T00:08:26.050393+0000","last_unstale":"2024-05-09T00:10:09.324383+0000","last_undegraded":"2024-05-09T00:10:09.324383+0000","last_fullsized":"2024-05-09T00:10:09.324383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:30.819219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.307617+0000","last_change":"2024-05-09T00:08:26.065946+0000","last_active":"2024-05-09T00:10:09.307617+0000","last_peered":"2024-05-09T00:10:09.307617+0000","last_clean":"2024-05-09T00:10:09.307617+0000","last_became_active":"2024-05-09T00:08:26.065498+0000","last_became_peered":"2024-05-09T00:08:26.065498+0000","last_unstale":"2024-05-09T00:10:09.307617+0000","last_undegraded":"2024-05-09T00:10:09.307617+0000","last_fullsized":"2024-05-09T00:10:09.307617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:04.444860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.321202+0000","last_change":"2024-05-09T00:08:26.051961+0000","last_active":"2024-05-09T00:10:09.321202+0000","last_peered":"2024-05-09T00:10:09.321202+0000","last_clean":"2024-05-09T00:10:09.321202+0000","last_became_active":"2024-05-09T00:08:26.051548+0000","last_became_peered":"2024-05-09T00:08:26.051548+0000","last_unstale":"2024-05-09T00:10:09.321202+0000","last_undegraded":"2024-05-09T00:10:09.321202+0000","last_fullsized":"2024-05-09T00:10:09.321202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:07.539117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.311478+0000","last_change":"2024-05-09T00:08:26.052012+0000","last_active":"2024-05-09T00:10:09.311478+0000","last_peered":"2024-05-09T00:10:09.311478+0000","last_clean":"2024-05-09T00:10:09.311478+0000","last_became_active":"2024-05-09T00:08:26.050886+0000","last_became_peered":"2024-05-09T00:08:26.050886+0000","last_unstale":"2024-05-09T00:10:09.311478+0000","last_undegraded":"2024-05-09T00:10:09.311478+0000","last_fullsized":"2024-05-09T00:10:09.311478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:14.651888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324357+0000","last_change":"2024-05-09T00:08:26.058418+0000","last_active":"2024-05-09T00:10:09.324357+0000","last_peered":"2024-05-09T00:10:09.324357+0000","last_clean":"2024-05-09T00:10:09.324357+0000","last_became_active":"2024-05-09T00:08:26.058039+0000","last_became_peered":"2024-05-09T00:08:26.058039+0000","last_unstale":"2024-05-09T00:10:09.324357+0000","last_undegraded":"2024-05-09T00:10:09.324357+0000","last_fullsized":"2024-05-09T00:10:09.324357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:13.678123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.322294+0000","last_change":"2024-05-09T00:08:26.057602+0000","last_active":"2024-05-09T00:10:09.322294+0000","last_peered":"2024-05-09T00:10:09.322294+0000","last_clean":"2024-05-09T00:10:09.322294+0000","last_became_active":"2024-05-09T00:08:26.057169+0000","last_became_peered":"2024-05-09T00:08:26.057169+0000","last_unstale":"2024-05-09T00:10:09.322294+0000","last_undegraded":"2024-05-09T00:10:09.322294+0000","last_fullsized":"2024-05-09T00:10:09.322294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:11.452083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327189+0000","last_change":"2024-05-09T00:08:26.067639+0000","last_active":"2024-05-09T00:10:09.327189+0000","last_peered":"2024-05-09T00:10:09.327189+0000","last_clean":"2024-05-09T00:10:09.327189+0000","last_became_active":"2024-05-09T00:08:26.067226+0000","last_became_peered":"2024-05-09T00:08:26.067226+0000","last_unstale":"2024-05-09T00:10:09.327189+0000","last_undegraded":"2024-05-09T00:10:09.327189+0000","last_fullsized":"2024-05-09T00:10:09.327189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:12.366500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.310873+0000","last_change":"2024-05-09T00:08:26.055234+0000","last_active":"2024-05-09T00:10:09.310873+0000","last_peered":"2024-05-09T00:10:09.310873+0000","last_clean":"2024-05-09T00:10:09.310873+0000","last_became_active":"2024-05-09T00:08:26.054851+0000","last_became_peered":"2024-05-09T00:08:26.054851+0000","last_unstale":"2024-05-09T00:10:09.310873+0000","last_undegraded":"2024-05-09T00:10:09.310873+0000","last_fullsized":"2024-05-09T00:10:09.310873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:54:52.121329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330090+0000","last_change":"2024-05-09T00:08:26.063375+0000","last_active":"2024-05-09T00:10:09.330090+0000","last_peered":"2024-05-09T00:10:09.330090+0000","last_clean":"2024-05-09T00:10:09.330090+0000","last_became_active":"2024-05-09T00:08:26.062962+0000","last_became_peered":"2024-05-09T00:08:26.062962+0000","last_unstale":"2024-05-09T00:10:09.330090+0000","last_undegraded":"2024-05-09T00:10:09.330090+0000","last_fullsized":"2024-05-09T00:10:09.330090+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:05.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.305381+0000","last_change":"2024-05-09T00:08:26.062034+0000","last_active":"2024-05-09T00:10:09.305381+0000","last_peered":"2024-05-09T00:10:09.305381+0000","last_clean":"2024-05-09T00:10:09.305381+0000","last_became_active":"2024-05-09T00:08:26.060376+0000","last_became_peered":"2024-05-09T00:08:26.060376+0000","last_unstale":"2024-05-09T00:10:09.305381+0000","last_undegraded":"2024-05-09T00:10:09.305381+0000","last_fullsized":"2024-05-09T00:10:09.305381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:15.830024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327845+0000","last_change":"2024-05-09T00:08:26.053227+0000","last_active":"2024-05-09T00:10:09.327845+0000","last_peered":"2024-05-09T00:10:09.327845+0000","last_clean":"2024-05-09T00:10:09.327845+0000","last_became_active":"2024-05-09T00:08:26.052878+0000","last_became_peered":"2024-05-09T00:08:26.052878+0000","last_unstale":"2024-05-09T00:10:09.327845+0000","last_undegraded":"2024-05-09T00:10:09.327845+0000","last_fullsized":"2024-05-09T00:10:09.327845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:24.385026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.315496+0000","last_change":"2024-05-09T00:08:26.053041+0000","last_active":"2024-05-09T00:10:09.315496+0000","last_peered":"2024-05-09T00:10:09.315496+0000","last_clean":"2024-05-09T00:10:09.315496+0000","last_became_active":"2024-05-09T00:08:26.052677+0000","last_became_peered":"2024-05-09T00:08:26.052677+0000","last_unstale":"2024-05-09T00:10:09.315496+0000","last_undegraded":"2024-05-09T00:10:09.315496+0000","last_fullsized":"2024-05-09T00:10:09.315496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:24.478270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.315918+0000","last_change":"2024-05-09T00:08:26.063391+0000","last_active":"2024-05-09T00:10:09.315918+0000","last_peered":"2024-05-09T00:10:09.315918+0000","last_clean":"2024-05-09T00:10:09.315918+0000","last_became_active":"2024-05-09T00:08:26.063033+0000","last_became_peered":"2024-05-09T00:08:26.063033+0000","last_unstale":"2024-05-09T00:10:09.315918+0000","last_undegraded":"2024-05-09T00:10:09.315918+0000","last_fullsized":"2024-05-09T00:10:09.315918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:41.900453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328915+0000","last_change":"2024-05-09T00:08:26.065018+0000","last_active":"2024-05-09T00:10:09.328915+0000","last_peered":"2024-05-09T00:10:09.328915+0000","last_clean":"2024-05-09T00:10:09.328915+0000","last_became_active":"2024-05-09T00:08:26.064642+0000","last_became_peered":"2024-05-09T00:08:26.064642+0000","last_unstale":"2024-05-09T00:10:09.328915+0000","last_undegraded":"2024-05-09T00:10:09.328915+0000","last_fullsized":"2024-05-09T00:10:09.328915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:02.403704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324263+0000","last_change":"2024-05-09T00:08:26.049770+0000","last_active":"2024-05-09T00:10:09.324263+0000","last_peered":"2024-05-09T00:10:09.324263+0000","last_clean":"2024-05-09T00:10:09.324263+0000","last_became_active":"2024-05-09T00:08:26.048555+0000","last_became_peered":"2024-05-09T00:08:26.048555+0000","last_unstale":"2024-05-09T00:10:09.324263+0000","last_undegraded":"2024-05-09T00:10:09.324263+0000","last_fullsized":"2024-05-09T00:10:09.324263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:16.009200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.317473+0000","last_change":"2024-05-09T00:08:26.048587+0000","last_active":"2024-05-09T00:10:09.317473+0000","last_peered":"2024-05-09T00:10:09.317473+0000","last_clean":"2024-05-09T00:10:09.317473+0000","last_became_active":"2024-05-09T00:08:26.048148+0000","last_became_peered":"2024-05-09T00:08:26.048148+0000","last_unstale":"2024-05-09T00:10:09.317473+0000","last_undegraded":"2024-05-09T00:10:09.317473+0000","last_fullsized":"2024-05-09T00:10:09.317473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:42.113829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.322240+0000","last_change":"2024-05-09T00:08:26.053607+0000","last_active":"2024-05-09T00:10:09.322240+0000","last_peered":"2024-05-09T00:10:09.322240+0000","last_clean":"2024-05-09T00:10:09.322240+0000","last_became_active":"2024-05-09T00:08:26.053212+0000","last_became_peered":"2024-05-09T00:08:26.053212+0000","last_unstale":"2024-05-09T00:10:09.322240+0000","last_undegraded":"2024-05-09T00:10:09.322240+0000","last_fullsized":"2024-05-09T00:10:09.322240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:32.937049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469352+0000","last_change":"2024-05-09T00:08:23.024481+0000","last_active":"2024-05-09T00:09:10.469352+0000","last_peered":"2024-05-09T00:09:10.469352+0000","last_clean":"2024-05-09T00:09:10.469352+0000","last_became_active":"2024-05-09T00:08:23.024132+0000","last_became_peered":"2024-05-09T00:08:23.024132+0000","last_unstale":"2024-05-09T00:09:10.469352+0000","last_undegraded":"2024-05-09T00:09:10.469352+0000","last_fullsized":"2024-05-09T00:09:10.469352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:11.188505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325280+0000","last_change":"2024-05-09T00:08:26.054772+0000","last_active":"2024-05-09T00:10:09.325280+0000","last_peered":"2024-05-09T00:10:09.325280+0000","last_clean":"2024-05-09T00:10:09.325280+0000","last_became_active":"2024-05-09T00:08:26.054433+0000","last_became_peered":"2024-05-09T00:08:26.054433+0000","last_unstale":"2024-05-09T00:10:09.325280+0000","last_undegraded":"2024-05-09T00:10:09.325280+0000","last_fullsized":"2024-05-09T00:10:09.325280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:44.283240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468356+0000","last_change":"2024-05-09T00:08:23.118778+0000","last_active":"2024-05-09T00:09:10.468356+0000","last_peered":"2024-05-09T00:09:10.468356+0000","last_clean":"2024-05-09T00:09:10.468356+0000","last_became_active":"2024-05-09T00:08:23.118377+0000","last_became_peered":"2024-05-09T00:08:23.118377+0000","last_unstale":"2024-05-09T00:09:10.468356+0000","last_undegraded":"2024-05-09T00:09:10.468356+0000","last_fullsized":"2024-05-09T00:09:10.468356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:51.352442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325452+0000","last_change":"2024-05-09T00:08:26.059953+0000","last_active":"2024-05-09T00:10:09.325452+0000","last_peered":"2024-05-09T00:10:09.325452+0000","last_clean":"2024-05-09T00:10:09.325452+0000","last_became_active":"2024-05-09T00:08:26.059571+0000","last_became_peered":"2024-05-09T00:08:26.059571+0000","last_unstale":"2024-05-09T00:10:09.325452+0000","last_undegraded":"2024-05-09T00:10:09.325452+0000","last_fullsized":"2024-05-09T00:10:09.325452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:48.056798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475294+0000","last_change":"2024-05-09T00:08:23.021244+0000","last_active":"2024-05-09T00:09:10.475294+0000","last_peered":"2024-05-09T00:09:10.475294+0000","last_clean":"2024-05-09T00:09:10.475294+0000","last_became_active":"2024-05-09T00:08:23.020597+0000","last_became_peered":"2024-05-09T00:08:23.020597+0000","last_unstale":"2024-05-09T00:09:10.475294+0000","last_undegraded":"2024-05-09T00:09:10.475294+0000","last_fullsized":"2024-05-09T00:09:10.475294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.654261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328461+0000","last_change":"2024-05-09T00:08:26.059834+0000","last_active":"2024-05-09T00:10:09.328461+0000","last_peered":"2024-05-09T00:10:09.328461+0000","last_clean":"2024-05-09T00:10:09.328461+0000","last_became_active":"2024-05-09T00:08:26.059349+0000","last_became_peered":"2024-05-09T00:08:26.059349+0000","last_unstale":"2024-05-09T00:10:09.328461+0000","last_undegraded":"2024-05-09T00:10:09.328461+0000","last_fullsized":"2024-05-09T00:10:09.328461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:42.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474235+0000","last_change":"2024-05-09T00:08:23.019568+0000","last_active":"2024-05-09T00:09:10.474235+0000","last_peered":"2024-05-09T00:09:10.474235+0000","last_clean":"2024-05-09T00:09:10.474235+0000","last_became_active":"2024-05-09T00:08:23.019149+0000","last_became_peered":"2024-05-09T00:08:23.019149+0000","last_unstale":"2024-05-09T00:09:10.474235+0000","last_undegraded":"2024-05-09T00:09:10.474235+0000","last_fullsized":"2024-05-09T00:09:10.474235+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:44.569911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328628+0000","last_change":"2024-05-09T00:08:26.061308+0000","last_active":"2024-05-09T00:10:09.328628+0000","last_peered":"2024-05-09T00:10:09.328628+0000","last_clean":"2024-05-09T00:10:09.328628+0000","last_became_active":"2024-05-09T00:08:26.060973+0000","last_became_peered":"2024-05-09T00:08:26.060973+0000","last_unstale":"2024-05-09T00:10:09.328628+0000","last_undegraded":"2024-05-09T00:10:09.328628+0000","last_fullsized":"2024-05-09T00:10:09.328628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:16.845074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":221,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:50.666958+0000","last_change":"2024-05-09T00:08:23.020886+0000","last_active":"2024-05-09T00:11:50.666958+0000","last_peered":"2024-05-09T00:11:50.666958+0000","last_clean":"2024-05-09T00:11:50.666958+0000","last_became_active":"2024-05-09T00:08:23.020530+0000","last_became_peered":"2024-05-09T00:08:23.020530+0000","last_unstale":"2024-05-09T00:11:50.666958+0000","last_undegraded":"2024-05-09T00:11:50.666958+0000","last_fullsized":"2024-05-09T00:11:50.666958+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:34.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318827+0000","last_change":"2024-05-09T00:08:26.051018+0000","last_active":"2024-05-09T00:10:09.318827+0000","last_peered":"2024-05-09T00:10:09.318827+0000","last_clean":"2024-05-09T00:10:09.318827+0000","last_became_active":"2024-05-09T00:08:26.050495+0000","last_became_peered":"2024-05-09T00:08:26.050495+0000","last_unstale":"2024-05-09T00:10:09.318827+0000","last_undegraded":"2024-05-09T00:10:09.318827+0000","last_fullsized":"2024-05-09T00:10:09.318827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:07.862339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472149+0000","last_change":"2024-05-09T00:08:23.021187+0000","last_active":"2024-05-09T00:09:10.472149+0000","last_peered":"2024-05-09T00:09:10.472149+0000","last_clean":"2024-05-09T00:09:10.472149+0000","last_became_active":"2024-05-09T00:08:23.020770+0000","last_became_peered":"2024-05-09T00:08:23.020770+0000","last_unstale":"2024-05-09T00:09:10.472149+0000","last_undegraded":"2024-05-09T00:09:10.472149+0000","last_fullsized":"2024-05-09T00:09:10.472149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:08.992770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.316888+0000","last_change":"2024-05-09T00:09:10.797392+0000","last_active":"2024-05-09T00:10:09.316888+0000","last_peered":"2024-05-09T00:10:09.316888+0000","last_clean":"2024-05-09T00:10:09.316888+0000","last_became_active":"2024-05-09T00:09:10.796969+0000","last_became_peered":"2024-05-09T00:09:10.796969+0000","last_unstale":"2024-05-09T00:10:09.316888+0000","last_undegraded":"2024-05-09T00:10:09.316888+0000","last_fullsized":"2024-05-09T00:10:09.316888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:18.852678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548815+0000","last_change":"2024-05-09T00:08:23.012226+0000","last_active":"2024-05-09T00:09:10.548815+0000","last_peered":"2024-05-09T00:09:10.548815+0000","last_clean":"2024-05-09T00:09:10.548815+0000","last_became_active":"2024-05-09T00:08:23.011781+0000","last_became_peered":"2024-05-09T00:08:23.011781+0000","last_unstale":"2024-05-09T00:09:10.548815+0000","last_undegraded":"2024-05-09T00:09:10.548815+0000","last_fullsized":"2024-05-09T00:09:10.548815+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:44.452206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.321641+0000","last_change":"2024-05-09T00:08:26.043035+0000","last_active":"2024-05-09T00:10:09.321641+0000","last_peered":"2024-05-09T00:10:09.321641+0000","last_clean":"2024-05-09T00:10:09.321641+0000","last_became_active":"2024-05-09T00:08:26.042686+0000","last_became_peered":"2024-05-09T00:08:26.042686+0000","last_unstale":"2024-05-09T00:10:09.321641+0000","last_undegraded":"2024-05-09T00:10:09.321641+0000","last_fullsized":"2024-05-09T00:10:09.321641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:24.511309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469113+0000","last_change":"2024-05-09T00:08:23.022017+0000","last_active":"2024-05-09T00:09:10.469113+0000","last_peered":"2024-05-09T00:09:10.469113+0000","last_clean":"2024-05-09T00:09:10.469113+0000","last_became_active":"2024-05-09T00:08:23.021540+0000","last_became_peered":"2024-05-09T00:08:23.021540+0000","last_unstale":"2024-05-09T00:09:10.469113+0000","last_undegraded":"2024-05-09T00:09:10.469113+0000","last_fullsized":"2024-05-09T00:09:10.469113+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:50.773211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326086+0000","last_change":"2024-05-09T00:08:26.048039+0000","last_active":"2024-05-09T00:10:09.326086+0000","last_peered":"2024-05-09T00:10:09.326086+0000","last_clean":"2024-05-09T00:10:09.326086+0000","last_became_active":"2024-05-09T00:08:26.047695+0000","last_became_peered":"2024-05-09T00:08:26.047695+0000","last_unstale":"2024-05-09T00:10:09.326086+0000","last_undegraded":"2024-05-09T00:10:09.326086+0000","last_fullsized":"2024-05-09T00:10:09.326086+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:27.363847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":220,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:52.669314+0000","last_change":"2024-05-09T00:08:23.019565+0000","last_active":"2024-05-09T00:11:52.669314+0000","last_peered":"2024-05-09T00:11:52.669314+0000","last_clean":"2024-05-09T00:11:52.669314+0000","last_became_active":"2024-05-09T00:08:23.019113+0000","last_became_peered":"2024-05-09T00:08:23.019113+0000","last_unstale":"2024-05-09T00:11:52.669314+0000","last_undegraded":"2024-05-09T00:11:52.669314+0000","last_fullsized":"2024-05-09T00:11:52.669314+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:45.578934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323960+0000","last_change":"2024-05-09T00:08:26.058869+0000","last_active":"2024-05-09T00:10:09.323960+0000","last_peered":"2024-05-09T00:10:09.323960+0000","last_clean":"2024-05-09T00:10:09.323960+0000","last_became_active":"2024-05-09T00:08:26.058482+0000","last_became_peered":"2024-05-09T00:08:26.058482+0000","last_unstale":"2024-05-09T00:10:09.323960+0000","last_undegraded":"2024-05-09T00:10:09.323960+0000","last_fullsized":"2024-05-09T00:10:09.323960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:33.252208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472217+0000","last_change":"2024-05-09T00:08:23.024425+0000","last_active":"2024-05-09T00:09:10.472217+0000","last_peered":"2024-05-09T00:09:10.472217+0000","last_clean":"2024-05-09T00:09:10.472217+0000","last_became_active":"2024-05-09T00:08:23.023991+0000","last_became_peered":"2024-05-09T00:08:23.023991+0000","last_unstale":"2024-05-09T00:09:10.472217+0000","last_undegraded":"2024-05-09T00:09:10.472217+0000","last_fullsized":"2024-05-09T00:09:10.472217+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:47.907851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330931+0000","last_change":"2024-05-09T00:09:10.475008+0000","last_active":"2024-05-09T00:10:09.330931+0000","last_peered":"2024-05-09T00:10:09.330931+0000","last_clean":"2024-05-09T00:10:09.330931+0000","last_became_active":"2024-05-09T00:09:10.474663+0000","last_became_peered":"2024-05-09T00:09:10.474663+0000","last_unstale":"2024-05-09T00:10:09.330931+0000","last_undegraded":"2024-05-09T00:10:09.330931+0000","last_fullsized":"2024-05-09T00:10:09.330931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:36.990362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470179+0000","last_change":"2024-05-09T00:08:23.118516+0000","last_active":"2024-05-09T00:09:10.470179+0000","last_peered":"2024-05-09T00:09:10.470179+0000","last_clean":"2024-05-09T00:09:10.470179+0000","last_became_active":"2024-05-09T00:08:23.118147+0000","last_became_peered":"2024-05-09T00:08:23.118147+0000","last_unstale":"2024-05-09T00:09:10.470179+0000","last_undegraded":"2024-05-09T00:09:10.470179+0000","last_fullsized":"2024-05-09T00:09:10.470179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:24.205818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327118+0000","last_change":"2024-05-09T00:08:26.067063+0000","last_active":"2024-05-09T00:10:09.327118+0000","last_peered":"2024-05-09T00:10:09.327118+0000","last_clean":"2024-05-09T00:10:09.327118+0000","last_became_active":"2024-05-09T00:08:26.066440+0000","last_became_peered":"2024-05-09T00:08:26.066440+0000","last_unstale":"2024-05-09T00:10:09.327118+0000","last_undegraded":"2024-05-09T00:10:09.327118+0000","last_fullsized":"2024-05-09T00:10:09.327118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:48.648485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'36","reported_seq":455,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:53.672263+0000","last_change":"2024-05-09T00:08:23.020757+0000","last_active":"2024-05-09T00:11:53.672263+0000","last_peered":"2024-05-09T00:11:53.672263+0000","last_clean":"2024-05-09T00:11:53.672263+0000","last_became_active":"2024-05-09T00:08:23.020355+0000","last_became_peered":"2024-05-09T00:08:23.020355+0000","last_unstale":"2024-05-09T00:11:53.672263+0000","last_undegraded":"2024-05-09T00:11:53.672263+0000","last_fullsized":"2024-05-09T00:11:53.672263+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:00.605691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66697,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":392,"num_read_kb":261,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328259+0000","last_change":"2024-05-09T00:08:26.057402+0000","last_active":"2024-05-09T00:10:09.328259+0000","last_peered":"2024-05-09T00:10:09.328259+0000","last_clean":"2024-05-09T00:10:09.328259+0000","last_became_active":"2024-05-09T00:08:26.056974+0000","last_became_peered":"2024-05-09T00:08:26.056974+0000","last_unstale":"2024-05-09T00:10:09.328259+0000","last_undegraded":"2024-05-09T00:10:09.328259+0000","last_fullsized":"2024-05-09T00:10:09.328259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:47.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473953+0000","last_change":"2024-05-09T00:08:23.020208+0000","last_active":"2024-05-09T00:09:10.473953+0000","last_peered":"2024-05-09T00:09:10.473953+0000","last_clean":"2024-05-09T00:09:10.473953+0000","last_became_active":"2024-05-09T00:08:23.019809+0000","last_became_peered":"2024-05-09T00:08:23.019809+0000","last_unstale":"2024-05-09T00:09:10.473953+0000","last_undegraded":"2024-05-09T00:09:10.473953+0000","last_fullsized":"2024-05-09T00:09:10.473953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:47.900731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330506+0000","last_change":"2024-05-09T00:08:26.054273+0000","last_active":"2024-05-09T00:10:09.330506+0000","last_peered":"2024-05-09T00:10:09.330506+0000","last_clean":"2024-05-09T00:10:09.330506+0000","last_became_active":"2024-05-09T00:08:26.053840+0000","last_became_peered":"2024-05-09T00:08:26.053840+0000","last_unstale":"2024-05-09T00:10:09.330506+0000","last_undegraded":"2024-05-09T00:10:09.330506+0000","last_fullsized":"2024-05-09T00:10:09.330506+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:27.128582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469520+0000","last_change":"2024-05-09T00:08:23.021594+0000","last_active":"2024-05-09T00:09:10.469520+0000","last_peered":"2024-05-09T00:09:10.469520+0000","last_clean":"2024-05-09T00:09:10.469520+0000","last_became_active":"2024-05-09T00:08:23.021152+0000","last_became_peered":"2024-05-09T00:08:23.021152+0000","last_unstale":"2024-05-09T00:09:10.469520+0000","last_undegraded":"2024-05-09T00:09:10.469520+0000","last_fullsized":"2024-05-09T00:09:10.469520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:58.505781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330209+0000","last_change":"2024-05-09T00:08:26.064464+0000","last_active":"2024-05-09T00:10:09.330209+0000","last_peered":"2024-05-09T00:10:09.330209+0000","last_clean":"2024-05-09T00:10:09.330209+0000","last_became_active":"2024-05-09T00:08:26.064000+0000","last_became_peered":"2024-05-09T00:08:26.064000+0000","last_unstale":"2024-05-09T00:10:09.330209+0000","last_undegraded":"2024-05-09T00:10:09.330209+0000","last_fullsized":"2024-05-09T00:10:09.330209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:01.455979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":227,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:51.668686+0000","last_change":"2024-05-09T00:08:23.020823+0000","last_active":"2024-05-09T00:11:51.668686+0000","last_peered":"2024-05-09T00:11:51.668686+0000","last_clean":"2024-05-09T00:11:51.668686+0000","last_became_active":"2024-05-09T00:08:23.020412+0000","last_became_peered":"2024-05-09T00:08:23.020412+0000","last_unstale":"2024-05-09T00:11:51.668686+0000","last_undegraded":"2024-05-09T00:11:51.668686+0000","last_fullsized":"2024-05-09T00:11:51.668686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:05.457452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":195,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318663+0000","last_change":"2024-05-09T00:09:10.474540+0000","last_active":"2024-05-09T00:10:09.318663+0000","last_peered":"2024-05-09T00:10:09.318663+0000","last_clean":"2024-05-09T00:10:09.318663+0000","last_became_active":"2024-05-09T00:09:10.474068+0000","last_became_peered":"2024-05-09T00:09:10.474068+0000","last_unstale":"2024-05-09T00:10:09.318663+0000","last_undegraded":"2024-05-09T00:10:09.318663+0000","last_fullsized":"2024-05-09T00:10:09.318663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:37.296140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474415+0000","last_change":"2024-05-09T00:08:23.118857+0000","last_active":"2024-05-09T00:09:10.474415+0000","last_peered":"2024-05-09T00:09:10.474415+0000","last_clean":"2024-05-09T00:09:10.474415+0000","last_became_active":"2024-05-09T00:08:23.118433+0000","last_became_peered":"2024-05-09T00:08:23.118433+0000","last_unstale":"2024-05-09T00:09:10.474415+0000","last_undegraded":"2024-05-09T00:09:10.474415+0000","last_fullsized":"2024-05-09T00:09:10.474415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:10.617987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327227+0000","last_change":"2024-05-09T00:08:26.066911+0000","last_active":"2024-05-09T00:10:09.327227+0000","last_peered":"2024-05-09T00:10:09.327227+0000","last_clean":"2024-05-09T00:10:09.327227+0000","last_became_active":"2024-05-09T00:08:26.066548+0000","last_became_peered":"2024-05-09T00:08:26.066548+0000","last_unstale":"2024-05-09T00:10:09.327227+0000","last_undegraded":"2024-05-09T00:10:09.327227+0000","last_fullsized":"2024-05-09T00:10:09.327227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:32.391390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:39.175360+0000","last_change":"2024-05-09T00:08:23.019829+0000","last_active":"2024-05-09T00:10:39.175360+0000","last_peered":"2024-05-09T00:10:39.175360+0000","last_clean":"2024-05-09T00:10:39.175360+0000","last_became_active":"2024-05-09T00:08:23.019336+0000","last_became_peered":"2024-05-09T00:08:23.019336+0000","last_unstale":"2024-05-09T00:10:39.175360+0000","last_undegraded":"2024-05-09T00:10:39.175360+0000","last_fullsized":"2024-05-09T00:10:39.175360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:33.802738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318804+0000","last_change":"2024-05-09T00:08:26.055975+0000","last_active":"2024-05-09T00:10:09.318804+0000","last_peered":"2024-05-09T00:10:09.318804+0000","last_clean":"2024-05-09T00:10:09.318804+0000","last_became_active":"2024-05-09T00:08:26.055595+0000","last_became_peered":"2024-05-09T00:08:26.055595+0000","last_unstale":"2024-05-09T00:10:09.318804+0000","last_undegraded":"2024-05-09T00:10:09.318804+0000","last_fullsized":"2024-05-09T00:10:09.318804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:20.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469423+0000","last_change":"2024-05-09T00:08:23.120353+0000","last_active":"2024-05-09T00:09:10.469423+0000","last_peered":"2024-05-09T00:09:10.469423+0000","last_clean":"2024-05-09T00:09:10.469423+0000","last_became_active":"2024-05-09T00:08:23.120004+0000","last_became_peered":"2024-05-09T00:08:23.120004+0000","last_unstale":"2024-05-09T00:09:10.469423+0000","last_undegraded":"2024-05-09T00:09:10.469423+0000","last_fullsized":"2024-05-09T00:09:10.469423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:12.168671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.320556+0000","last_change":"2024-05-09T00:08:26.046554+0000","last_active":"2024-05-09T00:10:09.320556+0000","last_peered":"2024-05-09T00:10:09.320556+0000","last_clean":"2024-05-09T00:10:09.320556+0000","last_became_active":"2024-05-09T00:08:26.045924+0000","last_became_peered":"2024-05-09T00:08:26.045924+0000","last_unstale":"2024-05-09T00:10:09.320556+0000","last_undegraded":"2024-05-09T00:10:09.320556+0000","last_fullsized":"2024-05-09T00:10:09.320556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:01.208980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472462+0000","last_change":"2024-05-09T00:08:23.019778+0000","last_active":"2024-05-09T00:09:10.472462+0000","last_peered":"2024-05-09T00:09:10.472462+0000","last_clean":"2024-05-09T00:09:10.472462+0000","last_became_active":"2024-05-09T00:08:23.019400+0000","last_became_peered":"2024-05-09T00:08:23.019400+0000","last_unstale":"2024-05-09T00:09:10.472462+0000","last_undegraded":"2024-05-09T00:09:10.472462+0000","last_fullsized":"2024-05-09T00:09:10.472462+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:06.883167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.308957+0000","last_change":"2024-05-09T00:09:10.474352+0000","last_active":"2024-05-09T00:10:09.308957+0000","last_peered":"2024-05-09T00:10:09.308957+0000","last_clean":"2024-05-09T00:10:09.308957+0000","last_became_active":"2024-05-09T00:09:10.473906+0000","last_became_peered":"2024-05-09T00:09:10.473906+0000","last_unstale":"2024-05-09T00:10:09.308957+0000","last_undegraded":"2024-05-09T00:10:09.308957+0000","last_fullsized":"2024-05-09T00:10:09.308957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:35:18.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.319595+0000","last_change":"2024-05-09T00:09:10.476484+0000","last_active":"2024-05-09T00:10:09.319595+0000","last_peered":"2024-05-09T00:10:09.319595+0000","last_clean":"2024-05-09T00:10:09.319595+0000","last_became_active":"2024-05-09T00:09:10.476001+0000","last_became_peered":"2024-05-09T00:09:10.476001+0000","last_unstale":"2024-05-09T00:10:09.319595+0000","last_undegraded":"2024-05-09T00:10:09.319595+0000","last_fullsized":"2024-05-09T00:10:09.319595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:32.018749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'6","reported_seq":223,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:52.669427+0000","last_change":"2024-05-09T00:08:23.118322+0000","last_active":"2024-05-09T00:11:52.669427+0000","last_peered":"2024-05-09T00:11:52.669427+0000","last_clean":"2024-05-09T00:11:52.669427+0000","last_became_active":"2024-05-09T00:08:23.117943+0000","last_became_peered":"2024-05-09T00:08:23.117943+0000","last_unstale":"2024-05-09T00:11:52.669427+0000","last_undegraded":"2024-05-09T00:11:52.669427+0000","last_fullsized":"2024-05-09T00:11:52.669427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:07.890437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323361+0000","last_change":"2024-05-09T00:08:26.052294+0000","last_active":"2024-05-09T00:10:09.323361+0000","last_peered":"2024-05-09T00:10:09.323361+0000","last_clean":"2024-05-09T00:10:09.323361+0000","last_became_active":"2024-05-09T00:08:26.051926+0000","last_became_peered":"2024-05-09T00:08:26.051926+0000","last_unstale":"2024-05-09T00:10:09.323361+0000","last_undegraded":"2024-05-09T00:10:09.323361+0000","last_fullsized":"2024-05-09T00:10:09.323361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:53.651401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796358+0000","last_change":"2024-05-09T00:08:23.015007+0000","last_active":"2024-05-09T00:09:10.796358+0000","last_peered":"2024-05-09T00:09:10.796358+0000","last_clean":"2024-05-09T00:09:10.796358+0000","last_became_active":"2024-05-09T00:08:23.014545+0000","last_became_peered":"2024-05-09T00:08:23.014545+0000","last_unstale":"2024-05-09T00:09:10.796358+0000","last_undegraded":"2024-05-09T00:09:10.796358+0000","last_fullsized":"2024-05-09T00:09:10.796358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:19.850200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:43.888672+0000","last_change":"2024-05-09T00:07:00.733233+0000","last_active":"2024-05-09T00:09:43.888672+0000","last_peered":"2024-05-09T00:09:43.888672+0000","last_clean":"2024-05-09T00:09:43.888672+0000","last_became_active":"2024-05-09T00:07:00.727365+0000","last_became_peered":"2024-05-09T00:07:00.727365+0000","last_unstale":"2024-05-09T00:09:43.888672+0000","last_undegraded":"2024-05-09T00:09:43.888672+0000","last_fullsized":"2024-05-09T00:09:43.888672+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_clean_scrub_stamp":"2024-05-09T00:03:33.285220+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:08.597290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326209+0000","last_change":"2024-05-09T00:08:26.064000+0000","last_active":"2024-05-09T00:10:09.326209+0000","last_peered":"2024-05-09T00:10:09.326209+0000","last_clean":"2024-05-09T00:10:09.326209+0000","last_became_active":"2024-05-09T00:08:26.063636+0000","last_became_peered":"2024-05-09T00:08:26.063636+0000","last_unstale":"2024-05-09T00:10:09.326209+0000","last_undegraded":"2024-05-09T00:10:09.326209+0000","last_fullsized":"2024-05-09T00:10:09.326209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:59.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472596+0000","last_change":"2024-05-09T00:08:23.023459+0000","last_active":"2024-05-09T00:09:10.472596+0000","last_peered":"2024-05-09T00:09:10.472596+0000","last_clean":"2024-05-09T00:09:10.472596+0000","last_became_active":"2024-05-09T00:08:23.023089+0000","last_became_peered":"2024-05-09T00:08:23.023089+0000","last_unstale":"2024-05-09T00:09:10.472596+0000","last_undegraded":"2024-05-09T00:09:10.472596+0000","last_fullsized":"2024-05-09T00:09:10.472596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:41.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:14.346903+0000","last_change":"2024-05-09T00:09:10.478686+0000","last_active":"2024-05-09T00:09:14.346903+0000","last_peered":"2024-05-09T00:09:14.346903+0000","last_clean":"2024-05-09T00:09:14.346903+0000","last_became_active":"2024-05-09T00:09:10.474890+0000","last_became_peered":"2024-05-09T00:09:10.474890+0000","last_unstale":"2024-05-09T00:09:14.346903+0000","last_undegraded":"2024-05-09T00:09:14.346903+0000","last_fullsized":"2024-05-09T00:09:14.346903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:41.041247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327716+0000","last_change":"2024-05-09T00:08:26.050685+0000","last_active":"2024-05-09T00:10:09.327716+0000","last_peered":"2024-05-09T00:10:09.327716+0000","last_clean":"2024-05-09T00:10:09.327716+0000","last_became_active":"2024-05-09T00:08:26.050312+0000","last_became_peered":"2024-05-09T00:08:26.050312+0000","last_unstale":"2024-05-09T00:10:09.327716+0000","last_undegraded":"2024-05-09T00:10:09.327716+0000","last_fullsized":"2024-05-09T00:10:09.327716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:31.623182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:32.811512+0000","last_change":"2024-05-09T00:08:23.019663+0000","last_active":"2024-05-09T00:10:32.811512+0000","last_peered":"2024-05-09T00:10:32.811512+0000","last_clean":"2024-05-09T00:10:32.811512+0000","last_became_active":"2024-05-09T00:08:23.019301+0000","last_became_peered":"2024-05-09T00:08:23.019301+0000","last_unstale":"2024-05-09T00:10:32.811512+0000","last_undegraded":"2024-05-09T00:10:32.811512+0000","last_fullsized":"2024-05-09T00:10:32.811512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:08.291227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330686+0000","last_change":"2024-05-09T00:08:26.058825+0000","last_active":"2024-05-09T00:10:09.330686+0000","last_peered":"2024-05-09T00:10:09.330686+0000","last_clean":"2024-05-09T00:10:09.330686+0000","last_became_active":"2024-05-09T00:08:26.058469+0000","last_became_peered":"2024-05-09T00:08:26.058469+0000","last_unstale":"2024-05-09T00:10:09.330686+0000","last_undegraded":"2024-05-09T00:10:09.330686+0000","last_fullsized":"2024-05-09T00:10:09.330686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:08.118819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796594+0000","last_change":"2024-05-09T00:08:23.054366+0000","last_active":"2024-05-09T00:09:10.796594+0000","last_peered":"2024-05-09T00:09:10.796594+0000","last_clean":"2024-05-09T00:09:10.796594+0000","last_became_active":"2024-05-09T00:08:23.053977+0000","last_became_peered":"2024-05-09T00:08:23.053977+0000","last_unstale":"2024-05-09T00:09:10.796594+0000","last_undegraded":"2024-05-09T00:09:10.796594+0000","last_fullsized":"2024-05-09T00:09:10.796594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:37.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.316437+0000","last_change":"2024-05-09T00:08:26.057864+0000","last_active":"2024-05-09T00:10:09.316437+0000","last_peered":"2024-05-09T00:10:09.316437+0000","last_clean":"2024-05-09T00:10:09.316437+0000","last_became_active":"2024-05-09T00:08:26.057436+0000","last_became_peered":"2024-05-09T00:08:26.057436+0000","last_unstale":"2024-05-09T00:10:09.316437+0000","last_undegraded":"2024-05-09T00:10:09.316437+0000","last_fullsized":"2024-05-09T00:10:09.316437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:19.976265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468672+0000","last_change":"2024-05-09T00:08:23.118803+0000","last_active":"2024-05-09T00:09:10.468672+0000","last_peered":"2024-05-09T00:09:10.468672+0000","last_clean":"2024-05-09T00:09:10.468672+0000","last_became_active":"2024-05-09T00:08:23.118432+0000","last_became_peered":"2024-05-09T00:08:23.118432+0000","last_unstale":"2024-05-09T00:09:10.468672+0000","last_undegraded":"2024-05-09T00:09:10.468672+0000","last_fullsized":"2024-05-09T00:09:10.468672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:50.253699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.310200+0000","last_change":"2024-05-09T00:08:26.054281+0000","last_active":"2024-05-09T00:10:09.310200+0000","last_peered":"2024-05-09T00:10:09.310200+0000","last_clean":"2024-05-09T00:10:09.310200+0000","last_became_active":"2024-05-09T00:08:26.053874+0000","last_became_peered":"2024-05-09T00:08:26.053874+0000","last_unstale":"2024-05-09T00:10:09.310200+0000","last_undegraded":"2024-05-09T00:10:09.310200+0000","last_fullsized":"2024-05-09T00:10:09.310200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:11.566337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467767+0000","last_change":"2024-05-09T00:08:23.017905+0000","last_active":"2024-05-09T00:09:10.467767+0000","last_peered":"2024-05-09T00:09:10.467767+0000","last_clean":"2024-05-09T00:09:10.467767+0000","last_became_active":"2024-05-09T00:08:23.017481+0000","last_became_peered":"2024-05-09T00:08:23.017481+0000","last_unstale":"2024-05-09T00:09:10.467767+0000","last_undegraded":"2024-05-09T00:09:10.467767+0000","last_fullsized":"2024-05-09T00:09:10.467767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:40.096415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.317514+0000","last_change":"2024-05-09T00:08:26.053392+0000","last_active":"2024-05-09T00:10:09.317514+0000","last_peered":"2024-05-09T00:10:09.317514+0000","last_clean":"2024-05-09T00:10:09.317514+0000","last_became_active":"2024-05-09T00:08:26.052628+0000","last_became_peered":"2024-05-09T00:08:26.052628+0000","last_unstale":"2024-05-09T00:10:09.317514+0000","last_undegraded":"2024-05-09T00:10:09.317514+0000","last_fullsized":"2024-05-09T00:10:09.317514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:28.013955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:14.346880+0000","last_change":"2024-05-09T00:09:10.481233+0000","last_active":"2024-05-09T00:09:14.346880+0000","last_peered":"2024-05-09T00:09:14.346880+0000","last_clean":"2024-05-09T00:09:14.346880+0000","last_became_active":"2024-05-09T00:09:10.475321+0000","last_became_peered":"2024-05-09T00:09:10.475321+0000","last_unstale":"2024-05-09T00:09:14.346880+0000","last_undegraded":"2024-05-09T00:09:14.346880+0000","last_fullsized":"2024-05-09T00:09:14.346880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:58.859277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.308335+0000","last_change":"2024-05-09T00:08:26.047812+0000","last_active":"2024-05-09T00:10:09.308335+0000","last_peered":"2024-05-09T00:10:09.308335+0000","last_clean":"2024-05-09T00:10:09.308335+0000","last_became_active":"2024-05-09T00:08:26.047459+0000","last_became_peered":"2024-05-09T00:08:26.047459+0000","last_unstale":"2024-05-09T00:10:09.308335+0000","last_undegraded":"2024-05-09T00:10:09.308335+0000","last_fullsized":"2024-05-09T00:10:09.308335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.257831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473816+0000","last_change":"2024-05-09T00:08:23.023800+0000","last_active":"2024-05-09T00:09:10.473816+0000","last_peered":"2024-05-09T00:09:10.473816+0000","last_clean":"2024-05-09T00:09:10.473816+0000","last_became_active":"2024-05-09T00:08:23.023387+0000","last_became_peered":"2024-05-09T00:08:23.023387+0000","last_unstale":"2024-05-09T00:09:10.473816+0000","last_undegraded":"2024-05-09T00:09:10.473816+0000","last_fullsized":"2024-05-09T00:09:10.473816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:33.714683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330035+0000","last_change":"2024-05-09T00:08:26.056981+0000","last_active":"2024-05-09T00:10:09.330035+0000","last_peered":"2024-05-09T00:10:09.330035+0000","last_clean":"2024-05-09T00:10:09.330035+0000","last_became_active":"2024-05-09T00:08:26.056605+0000","last_became_peered":"2024-05-09T00:08:26.056605+0000","last_unstale":"2024-05-09T00:10:09.330035+0000","last_undegraded":"2024-05-09T00:10:09.330035+0000","last_fullsized":"2024-05-09T00:10:09.330035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:32.897929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473675+0000","last_change":"2024-05-09T00:08:23.120612+0000","last_active":"2024-05-09T00:09:10.473675+0000","last_peered":"2024-05-09T00:09:10.473675+0000","last_clean":"2024-05-09T00:09:10.473675+0000","last_became_active":"2024-05-09T00:08:23.120157+0000","last_became_peered":"2024-05-09T00:08:23.120157+0000","last_unstale":"2024-05-09T00:09:10.473675+0000","last_undegraded":"2024-05-09T00:09:10.473675+0000","last_fullsized":"2024-05-09T00:09:10.473675+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:08.027214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330272+0000","last_change":"2024-05-09T00:08:26.061594+0000","last_active":"2024-05-09T00:10:09.330272+0000","last_peered":"2024-05-09T00:10:09.330272+0000","last_clean":"2024-05-09T00:10:09.330272+0000","last_became_active":"2024-05-09T00:08:26.061201+0000","last_became_peered":"2024-05-09T00:08:26.061201+0000","last_unstale":"2024-05-09T00:10:09.330272+0000","last_undegraded":"2024-05-09T00:10:09.330272+0000","last_fullsized":"2024-05-09T00:10:09.330272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:50.183648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470328+0000","last_change":"2024-05-09T00:08:23.017693+0000","last_active":"2024-05-09T00:09:10.470328+0000","last_peered":"2024-05-09T00:09:10.470328+0000","last_clean":"2024-05-09T00:09:10.470328+0000","last_became_active":"2024-05-09T00:08:23.017211+0000","last_became_peered":"2024-05-09T00:08:23.017211+0000","last_unstale":"2024-05-09T00:09:10.470328+0000","last_undegraded":"2024-05-09T00:09:10.470328+0000","last_fullsized":"2024-05-09T00:09:10.470328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:53.569932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327590+0000","last_change":"2024-05-09T00:08:26.063922+0000","last_active":"2024-05-09T00:10:09.327590+0000","last_peered":"2024-05-09T00:10:09.327590+0000","last_clean":"2024-05-09T00:10:09.327590+0000","last_became_active":"2024-05-09T00:08:26.063554+0000","last_became_peered":"2024-05-09T00:08:26.063554+0000","last_unstale":"2024-05-09T00:10:09.327590+0000","last_undegraded":"2024-05-09T00:10:09.327590+0000","last_fullsized":"2024-05-09T00:10:09.327590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:01.754688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548651+0000","last_change":"2024-05-09T00:08:23.020367+0000","last_active":"2024-05-09T00:09:10.548651+0000","last_peered":"2024-05-09T00:09:10.548651+0000","last_clean":"2024-05-09T00:09:10.548651+0000","last_became_active":"2024-05-09T00:08:23.019801+0000","last_became_peered":"2024-05-09T00:08:23.019801+0000","last_unstale":"2024-05-09T00:09:10.548651+0000","last_undegraded":"2024-05-09T00:09:10.548651+0000","last_fullsized":"2024-05-09T00:09:10.548651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:14.936021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330156+0000","last_change":"2024-05-09T00:08:26.063863+0000","last_active":"2024-05-09T00:10:09.330156+0000","last_peered":"2024-05-09T00:10:09.330156+0000","last_clean":"2024-05-09T00:10:09.330156+0000","last_became_active":"2024-05-09T00:08:26.061938+0000","last_became_peered":"2024-05-09T00:08:26.061938+0000","last_unstale":"2024-05-09T00:10:09.330156+0000","last_undegraded":"2024-05-09T00:10:09.330156+0000","last_fullsized":"2024-05-09T00:10:09.330156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:04.597991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.319694+0000","last_change":"2024-05-09T00:08:26.050837+0000","last_active":"2024-05-09T00:10:09.319694+0000","last_peered":"2024-05-09T00:10:09.319694+0000","last_clean":"2024-05-09T00:10:09.319694+0000","last_became_active":"2024-05-09T00:08:26.050424+0000","last_became_peered":"2024-05-09T00:08:26.050424+0000","last_unstale":"2024-05-09T00:10:09.319694+0000","last_undegraded":"2024-05-09T00:10:09.319694+0000","last_fullsized":"2024-05-09T00:10:09.319694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:45.134811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323117+0000","last_change":"2024-05-09T00:09:10.549989+0000","last_active":"2024-05-09T00:10:09.323117+0000","last_peered":"2024-05-09T00:10:09.323117+0000","last_clean":"2024-05-09T00:10:09.323117+0000","last_became_active":"2024-05-09T00:09:10.549502+0000","last_became_peered":"2024-05-09T00:09:10.549502+0000","last_unstale":"2024-05-09T00:10:09.323117+0000","last_undegraded":"2024-05-09T00:10:09.323117+0000","last_fullsized":"2024-05-09T00:10:09.323117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:10.654452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323905+0000","last_change":"2024-05-09T00:08:26.058012+0000","last_active":"2024-05-09T00:10:09.323905+0000","last_peered":"2024-05-09T00:10:09.323905+0000","last_clean":"2024-05-09T00:10:09.323905+0000","last_became_active":"2024-05-09T00:08:26.057282+0000","last_became_peered":"2024-05-09T00:08:26.057282+0000","last_unstale":"2024-05-09T00:10:09.323905+0000","last_undegraded":"2024-05-09T00:10:09.323905+0000","last_fullsized":"2024-05-09T00:10:09.323905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:53.520593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.320691+0000","last_change":"2024-05-09T00:08:26.063331+0000","last_active":"2024-05-09T00:10:09.320691+0000","last_peered":"2024-05-09T00:10:09.320691+0000","last_clean":"2024-05-09T00:10:09.320691+0000","last_became_active":"2024-05-09T00:08:26.062866+0000","last_became_peered":"2024-05-09T00:08:26.062866+0000","last_unstale":"2024-05-09T00:10:09.320691+0000","last_undegraded":"2024-05-09T00:10:09.320691+0000","last_fullsized":"2024-05-09T00:10:09.320691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:10.387852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326309+0000","last_change":"2024-05-09T00:08:26.063735+0000","last_active":"2024-05-09T00:10:09.326309+0000","last_peered":"2024-05-09T00:10:09.326309+0000","last_clean":"2024-05-09T00:10:09.326309+0000","last_became_active":"2024-05-09T00:08:26.063358+0000","last_became_peered":"2024-05-09T00:08:26.063358+0000","last_unstale":"2024-05-09T00:10:09.326309+0000","last_undegraded":"2024-05-09T00:10:09.326309+0000","last_fullsized":"2024-05-09T00:10:09.326309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:06.832071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318431+0000","last_change":"2024-05-09T00:08:26.052746+0000","last_active":"2024-05-09T00:10:09.318431+0000","last_peered":"2024-05-09T00:10:09.318431+0000","last_clean":"2024-05-09T00:10:09.318431+0000","last_became_active":"2024-05-09T00:08:26.051544+0000","last_became_peered":"2024-05-09T00:08:26.051544+0000","last_unstale":"2024-05-09T00:10:09.318431+0000","last_undegraded":"2024-05-09T00:10:09.318431+0000","last_fullsized":"2024-05-09T00:10:09.318431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:31.963268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323267+0000","last_change":"2024-05-09T00:08:26.058075+0000","last_active":"2024-05-09T00:10:09.323267+0000","last_peered":"2024-05-09T00:10:09.323267+0000","last_clean":"2024-05-09T00:10:09.323267+0000","last_became_active":"2024-05-09T00:08:26.057349+0000","last_became_peered":"2024-05-09T00:08:26.057349+0000","last_unstale":"2024-05-09T00:10:09.323267+0000","last_undegraded":"2024-05-09T00:10:09.323267+0000","last_fullsized":"2024-05-09T00:10:09.323267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:59.322903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324094+0000","last_change":"2024-05-09T00:08:26.056429+0000","last_active":"2024-05-09T00:10:09.324094+0000","last_peered":"2024-05-09T00:10:09.324094+0000","last_clean":"2024-05-09T00:10:09.324094+0000","last_became_active":"2024-05-09T00:08:26.054852+0000","last_became_peered":"2024-05-09T00:08:26.054852+0000","last_unstale":"2024-05-09T00:10:09.324094+0000","last_undegraded":"2024-05-09T00:10:09.324094+0000","last_fullsized":"2024-05-09T00:10:09.324094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:05.337636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327096+0000","last_change":"2024-05-09T00:09:10.549880+0000","last_active":"2024-05-09T00:10:09.327096+0000","last_peered":"2024-05-09T00:10:09.327096+0000","last_clean":"2024-05-09T00:10:09.327096+0000","last_became_active":"2024-05-09T00:09:10.549448+0000","last_became_peered":"2024-05-09T00:09:10.549448+0000","last_unstale":"2024-05-09T00:10:09.327096+0000","last_undegraded":"2024-05-09T00:10:09.327096+0000","last_fullsized":"2024-05-09T00:10:09.327096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:16.990729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325082+0000","last_change":"2024-05-09T00:08:26.054579+0000","last_active":"2024-05-09T00:10:09.325082+0000","last_peered":"2024-05-09T00:10:09.325082+0000","last_clean":"2024-05-09T00:10:09.325082+0000","last_became_active":"2024-05-09T00:08:26.053186+0000","last_became_peered":"2024-05-09T00:08:26.053186+0000","last_unstale":"2024-05-09T00:10:09.325082+0000","last_undegraded":"2024-05-09T00:10:09.325082+0000","last_fullsized":"2024-05-09T00:10:09.325082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:37.561691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328708+0000","last_change":"2024-05-09T00:08:26.063608+0000","last_active":"2024-05-09T00:10:09.328708+0000","last_peered":"2024-05-09T00:10:09.328708+0000","last_clean":"2024-05-09T00:10:09.328708+0000","last_became_active":"2024-05-09T00:08:26.063078+0000","last_became_peered":"2024-05-09T00:08:26.063078+0000","last_unstale":"2024-05-09T00:10:09.328708+0000","last_undegraded":"2024-05-09T00:10:09.328708+0000","last_fullsized":"2024-05-09T00:10:09.328708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:06.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.331031+0000","last_change":"2024-05-09T00:08:26.055058+0000","last_active":"2024-05-09T00:10:09.331031+0000","last_peered":"2024-05-09T00:10:09.331031+0000","last_clean":"2024-05-09T00:10:09.331031+0000","last_became_active":"2024-05-09T00:08:26.054695+0000","last_became_peered":"2024-05-09T00:08:26.054695+0000","last_unstale":"2024-05-09T00:10:09.331031+0000","last_undegraded":"2024-05-09T00:10:09.331031+0000","last_fullsized":"2024-05-09T00:10:09.331031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:58.925990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.312854+0000","last_change":"2024-05-09T00:08:26.055366+0000","last_active":"2024-05-09T00:10:09.312854+0000","last_peered":"2024-05-09T00:10:09.312854+0000","last_clean":"2024-05-09T00:10:09.312854+0000","last_became_active":"2024-05-09T00:08:26.054904+0000","last_became_peered":"2024-05-09T00:08:26.054904+0000","last_unstale":"2024-05-09T00:10:09.312854+0000","last_undegraded":"2024-05-09T00:10:09.312854+0000","last_fullsized":"2024-05-09T00:10:09.312854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:40.754878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285241,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71147,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1169,"num_read_kb":661,"num_write":181,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":213441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710777,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":425247,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1084243,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070531,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":880640,"data_stored":425461,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266761,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":425359,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":1017760,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":868352,"data_stored":424850,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":491340,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":888832,"data_stored":426121,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":488376,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":425161,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986270,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":426127,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822744,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":1016604,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:11:56.566 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-09T00:11:56.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T00:11:56.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:56 smithi142 ceph-mon[34533]: pgmap v393: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-09T00:11:56.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.719+0000 7fafd0815640 1 -- 172.21.15.46:0/4228197193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafcc102b30 msgr2=0x7fafcc103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:56.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.719+0000 7fafd0815640 1 --2- 172.21.15.46:0/4228197193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafcc102b30 0x7fafcc103830 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009900 tx=0x7fafbc02a410 comp rx=0 tx=0).stop 2024-05-09T00:11:56.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.719+0000 7fafd0815640 1 -- 172.21.15.46:0/4228197193 shutdown_connections 2024-05-09T00:11:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.719+0000 7fafd0815640 1 --2- 172.21.15.46:0/4228197193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafcc108a70 0x7fafcc10ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.719+0000 7fafd0815640 1 --2- 172.21.15.46:0/4228197193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc103d70 0x7fafcc104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.719+0000 7fafd0815640 1 --2- 172.21.15.46:0/4228197193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafcc102b30 0x7fafcc103830 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.719+0000 7fafd0815640 1 -- 172.21.15.46:0/4228197193 >> 172.21.15.46:0/4228197193 conn(0x7fafcc0fdd20 msgr2=0x7fafcc1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.719+0000 7fafd0815640 1 -- 172.21.15.46:0/4228197193 shutdown_connections 2024-05-09T00:11:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.719+0000 7fafd0815640 1 -- 172.21.15.46:0/4228197193 wait complete. 2024-05-09T00:11:56.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafd0815640 1 Processor -- start 2024-05-09T00:11:56.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafd0815640 1 -- start start 2024-05-09T00:11:56.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafd0815640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102b30 0x7fafcc19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:56.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafd0815640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafcc103d70 0x7fafcc19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:56.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafd0815640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafcc108a70 0x7fafcc1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafd0815640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc10f760 con 0x7fafcc103d70 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafd0815640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc10f5e0 con 0x7fafcc102b30 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafd0815640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc10f8e0 con 0x7fafcc108a70 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafc3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafcc103d70 0x7fafcc19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafc3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafcc103d70 0x7fafcc19a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59486/0 (socket says 172.21.15.46:59486) 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafc3fff640 1 -- 172.21.15.46:0/3191935053 learned_addr learned my addr 172.21.15.46:0/3191935053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafcad9d640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafcc108a70 0x7fafcc1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafca59c640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102b30 0x7fafcc19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafc3fff640 1 -- 172.21.15.46:0/3191935053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafcc108a70 msgr2=0x7fafcc1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafc3fff640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafcc108a70 0x7fafcc1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafc3fff640 1 -- 172.21.15.46:0/3191935053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102b30 msgr2=0x7fafcc19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafc3fff640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102b30 0x7fafcc19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafc3fff640 1 -- 172.21.15.46:0/3191935053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc1a1e80 con 0x7fafcc103d70 2024-05-09T00:11:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.720+0000 7fafcad9d640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafcc108a70 0x7fafcc1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:11:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.721+0000 7fafc3fff640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafcc103d70 0x7fafcc19a880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e9f0 tx=0x7fafb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.721+0000 7fafc37fe640 1 -- 172.21.15.46:0/3191935053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb400cd80 con 0x7fafcc103d70 2024-05-09T00:11:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.721+0000 7fafc37fe640 1 -- 172.21.15.46:0/3191935053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb40108e0 con 0x7fafcc103d70 2024-05-09T00:11:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.721+0000 7fafd0815640 1 -- 172.21.15.46:0/3191935053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafcc1a2170 con 0x7fafcc103d70 2024-05-09T00:11:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.721+0000 7fafc37fe640 1 -- 172.21.15.46:0/3191935053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4018980 con 0x7fafcc103d70 2024-05-09T00:11:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.721+0000 7fafd0815640 1 -- 172.21.15.46:0/3191935053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafcc1a2710 con 0x7fafcc103d70 2024-05-09T00:11:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.722+0000 7fafd0815640 1 -- 172.21.15.46:0/3191935053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafcc107880 con 0x7fafcc103d70 2024-05-09T00:11:56.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.723+0000 7fafc37fe640 1 -- 172.21.15.46:0/3191935053 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fafb4010430 con 0x7fafcc103d70 2024-05-09T00:11:56.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.724+0000 7fafc37fe640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fafb80b2870 0x7fafb80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:56.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.724+0000 7fafca59c640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fafb80b2870 0x7fafb80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:56.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.724+0000 7fafc37fe640 1 -- 172.21.15.46:0/3191935053 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fafb40e2ae0 con 0x7fafcc103d70 2024-05-09T00:11:56.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.725+0000 7fafca59c640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fafb80b2870 0x7fafb80b4d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009230 tx=0x7fafbc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:56.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.728+0000 7fafc37fe640 1 -- 172.21.15.46:0/3191935053 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fafb40aa7c0 con 0x7fafcc103d70 2024-05-09T00:11:56.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:56 smithi169 ceph-mon[34777]: pgmap v393: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-09T00:11:56.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.916+0000 7fafd0815640 1 -- 172.21.15.46:0/3191935053 --> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fafcc063110 con 0x7fafb80b2870 2024-05-09T00:11:56.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.923+0000 7fafc37fe640 1 -- 172.21.15.46:0/3191935053 <== mgr.14152 v2:172.21.15.46:6800/286117730 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259083 (secure 0 0 0) 0x7fafcc063110 con 0x7fafb80b2870 2024-05-09T00:11:56.924 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T00:11:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.926+0000 7fafd0815640 1 -- 172.21.15.46:0/3191935053 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fafb80b2870 msgr2=0x7fafb80b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.926+0000 7fafd0815640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fafb80b2870 0x7fafb80b4d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009230 tx=0x7fafbc031400 comp rx=0 tx=0).stop 2024-05-09T00:11:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.926+0000 7fafd0815640 1 -- 172.21.15.46:0/3191935053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafcc103d70 msgr2=0x7fafcc19a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.926+0000 7fafd0815640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafcc103d70 0x7fafcc19a880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e9f0 tx=0x7fafb400eeb0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.926+0000 7fafd0815640 1 -- 172.21.15.46:0/3191935053 shutdown_connections 2024-05-09T00:11:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.926+0000 7fafd0815640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fafb80b2870 0x7fafb80b4d80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.926+0000 7fafd0815640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafcc108a70 0x7fafcc1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.926+0000 7fafd0815640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafcc103d70 0x7fafcc19a880 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.926+0000 7fafd0815640 1 --2- 172.21.15.46:0/3191935053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafcc102b30 0x7fafcc19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.926+0000 7fafd0815640 1 -- 172.21.15.46:0/3191935053 >> 172.21.15.46:0/3191935053 conn(0x7fafcc0fdd20 msgr2=0x7fafcc104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.926+0000 7fafd0815640 1 -- 172.21.15.46:0/3191935053 shutdown_connections 2024-05-09T00:11:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:56.926+0000 7fafd0815640 1 -- 172.21.15.46:0/3191935053 wait complete. 2024-05-09T00:11:56.936 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-05-09T00:11:55.339263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661749,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1245,"num_read_kb":725,"num_write":306,"num_write_kb":2280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":747,"ondisk_log_size":747,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372308,"kb_used_data":10676,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569516,"statfs":{"total":1151940427776,"available":1151559184384,"internally_reserved":0,"allocated":10932224,"data_stored":7076649,"data_compressed":177897,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4930,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":52,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003242"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324383+0000","last_change":"2024-05-09T00:08:26.060455+0000","last_active":"2024-05-09T00:10:09.324383+0000","last_peered":"2024-05-09T00:10:09.324383+0000","last_clean":"2024-05-09T00:10:09.324383+0000","last_became_active":"2024-05-09T00:08:26.060068+0000","last_became_peered":"2024-05-09T00:08:26.060068+0000","last_unstale":"2024-05-09T00:10:09.324383+0000","last_undegraded":"2024-05-09T00:10:09.324383+0000","last_fullsized":"2024-05-09T00:10:09.324383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:48.354188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.314519+0000","last_change":"2024-05-09T00:08:26.055737+0000","last_active":"2024-05-09T00:10:09.314519+0000","last_peered":"2024-05-09T00:10:09.314519+0000","last_clean":"2024-05-09T00:10:09.314519+0000","last_became_active":"2024-05-09T00:08:26.055354+0000","last_became_peered":"2024-05-09T00:08:26.055354+0000","last_unstale":"2024-05-09T00:10:09.314519+0000","last_undegraded":"2024-05-09T00:10:09.314519+0000","last_fullsized":"2024-05-09T00:10:09.314519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:54.068256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324383+0000","last_change":"2024-05-09T00:08:26.050776+0000","last_active":"2024-05-09T00:10:09.324383+0000","last_peered":"2024-05-09T00:10:09.324383+0000","last_clean":"2024-05-09T00:10:09.324383+0000","last_became_active":"2024-05-09T00:08:26.050393+0000","last_became_peered":"2024-05-09T00:08:26.050393+0000","last_unstale":"2024-05-09T00:10:09.324383+0000","last_undegraded":"2024-05-09T00:10:09.324383+0000","last_fullsized":"2024-05-09T00:10:09.324383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:30.819219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.307617+0000","last_change":"2024-05-09T00:08:26.065946+0000","last_active":"2024-05-09T00:10:09.307617+0000","last_peered":"2024-05-09T00:10:09.307617+0000","last_clean":"2024-05-09T00:10:09.307617+0000","last_became_active":"2024-05-09T00:08:26.065498+0000","last_became_peered":"2024-05-09T00:08:26.065498+0000","last_unstale":"2024-05-09T00:10:09.307617+0000","last_undegraded":"2024-05-09T00:10:09.307617+0000","last_fullsized":"2024-05-09T00:10:09.307617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:04.444860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.321202+0000","last_change":"2024-05-09T00:08:26.051961+0000","last_active":"2024-05-09T00:10:09.321202+0000","last_peered":"2024-05-09T00:10:09.321202+0000","last_clean":"2024-05-09T00:10:09.321202+0000","last_became_active":"2024-05-09T00:08:26.051548+0000","last_became_peered":"2024-05-09T00:08:26.051548+0000","last_unstale":"2024-05-09T00:10:09.321202+0000","last_undegraded":"2024-05-09T00:10:09.321202+0000","last_fullsized":"2024-05-09T00:10:09.321202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:07.539117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.311478+0000","last_change":"2024-05-09T00:08:26.052012+0000","last_active":"2024-05-09T00:10:09.311478+0000","last_peered":"2024-05-09T00:10:09.311478+0000","last_clean":"2024-05-09T00:10:09.311478+0000","last_became_active":"2024-05-09T00:08:26.050886+0000","last_became_peered":"2024-05-09T00:08:26.050886+0000","last_unstale":"2024-05-09T00:10:09.311478+0000","last_undegraded":"2024-05-09T00:10:09.311478+0000","last_fullsized":"2024-05-09T00:10:09.311478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:14.651888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324357+0000","last_change":"2024-05-09T00:08:26.058418+0000","last_active":"2024-05-09T00:10:09.324357+0000","last_peered":"2024-05-09T00:10:09.324357+0000","last_clean":"2024-05-09T00:10:09.324357+0000","last_became_active":"2024-05-09T00:08:26.058039+0000","last_became_peered":"2024-05-09T00:08:26.058039+0000","last_unstale":"2024-05-09T00:10:09.324357+0000","last_undegraded":"2024-05-09T00:10:09.324357+0000","last_fullsized":"2024-05-09T00:10:09.324357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:13.678123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.322294+0000","last_change":"2024-05-09T00:08:26.057602+0000","last_active":"2024-05-09T00:10:09.322294+0000","last_peered":"2024-05-09T00:10:09.322294+0000","last_clean":"2024-05-09T00:10:09.322294+0000","last_became_active":"2024-05-09T00:08:26.057169+0000","last_became_peered":"2024-05-09T00:08:26.057169+0000","last_unstale":"2024-05-09T00:10:09.322294+0000","last_undegraded":"2024-05-09T00:10:09.322294+0000","last_fullsized":"2024-05-09T00:10:09.322294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:11.452083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327189+0000","last_change":"2024-05-09T00:08:26.067639+0000","last_active":"2024-05-09T00:10:09.327189+0000","last_peered":"2024-05-09T00:10:09.327189+0000","last_clean":"2024-05-09T00:10:09.327189+0000","last_became_active":"2024-05-09T00:08:26.067226+0000","last_became_peered":"2024-05-09T00:08:26.067226+0000","last_unstale":"2024-05-09T00:10:09.327189+0000","last_undegraded":"2024-05-09T00:10:09.327189+0000","last_fullsized":"2024-05-09T00:10:09.327189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:12.366500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.310873+0000","last_change":"2024-05-09T00:08:26.055234+0000","last_active":"2024-05-09T00:10:09.310873+0000","last_peered":"2024-05-09T00:10:09.310873+0000","last_clean":"2024-05-09T00:10:09.310873+0000","last_became_active":"2024-05-09T00:08:26.054851+0000","last_became_peered":"2024-05-09T00:08:26.054851+0000","last_unstale":"2024-05-09T00:10:09.310873+0000","last_undegraded":"2024-05-09T00:10:09.310873+0000","last_fullsized":"2024-05-09T00:10:09.310873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:54:52.121329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330090+0000","last_change":"2024-05-09T00:08:26.063375+0000","last_active":"2024-05-09T00:10:09.330090+0000","last_peered":"2024-05-09T00:10:09.330090+0000","last_clean":"2024-05-09T00:10:09.330090+0000","last_became_active":"2024-05-09T00:08:26.062962+0000","last_became_peered":"2024-05-09T00:08:26.062962+0000","last_unstale":"2024-05-09T00:10:09.330090+0000","last_undegraded":"2024-05-09T00:10:09.330090+0000","last_fullsized":"2024-05-09T00:10:09.330090+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:05.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.305381+0000","last_change":"2024-05-09T00:08:26.062034+0000","last_active":"2024-05-09T00:10:09.305381+0000","last_peered":"2024-05-09T00:10:09.305381+0000","last_clean":"2024-05-09T00:10:09.305381+0000","last_became_active":"2024-05-09T00:08:26.060376+0000","last_became_peered":"2024-05-09T00:08:26.060376+0000","last_unstale":"2024-05-09T00:10:09.305381+0000","last_undegraded":"2024-05-09T00:10:09.305381+0000","last_fullsized":"2024-05-09T00:10:09.305381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:22:15.830024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327845+0000","last_change":"2024-05-09T00:08:26.053227+0000","last_active":"2024-05-09T00:10:09.327845+0000","last_peered":"2024-05-09T00:10:09.327845+0000","last_clean":"2024-05-09T00:10:09.327845+0000","last_became_active":"2024-05-09T00:08:26.052878+0000","last_became_peered":"2024-05-09T00:08:26.052878+0000","last_unstale":"2024-05-09T00:10:09.327845+0000","last_undegraded":"2024-05-09T00:10:09.327845+0000","last_fullsized":"2024-05-09T00:10:09.327845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:24.385026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.315496+0000","last_change":"2024-05-09T00:08:26.053041+0000","last_active":"2024-05-09T00:10:09.315496+0000","last_peered":"2024-05-09T00:10:09.315496+0000","last_clean":"2024-05-09T00:10:09.315496+0000","last_became_active":"2024-05-09T00:08:26.052677+0000","last_became_peered":"2024-05-09T00:08:26.052677+0000","last_unstale":"2024-05-09T00:10:09.315496+0000","last_undegraded":"2024-05-09T00:10:09.315496+0000","last_fullsized":"2024-05-09T00:10:09.315496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:24.478270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.315918+0000","last_change":"2024-05-09T00:08:26.063391+0000","last_active":"2024-05-09T00:10:09.315918+0000","last_peered":"2024-05-09T00:10:09.315918+0000","last_clean":"2024-05-09T00:10:09.315918+0000","last_became_active":"2024-05-09T00:08:26.063033+0000","last_became_peered":"2024-05-09T00:08:26.063033+0000","last_unstale":"2024-05-09T00:10:09.315918+0000","last_undegraded":"2024-05-09T00:10:09.315918+0000","last_fullsized":"2024-05-09T00:10:09.315918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:41.900453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328915+0000","last_change":"2024-05-09T00:08:26.065018+0000","last_active":"2024-05-09T00:10:09.328915+0000","last_peered":"2024-05-09T00:10:09.328915+0000","last_clean":"2024-05-09T00:10:09.328915+0000","last_became_active":"2024-05-09T00:08:26.064642+0000","last_became_peered":"2024-05-09T00:08:26.064642+0000","last_unstale":"2024-05-09T00:10:09.328915+0000","last_undegraded":"2024-05-09T00:10:09.328915+0000","last_fullsized":"2024-05-09T00:10:09.328915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:02.403704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324263+0000","last_change":"2024-05-09T00:08:26.049770+0000","last_active":"2024-05-09T00:10:09.324263+0000","last_peered":"2024-05-09T00:10:09.324263+0000","last_clean":"2024-05-09T00:10:09.324263+0000","last_became_active":"2024-05-09T00:08:26.048555+0000","last_became_peered":"2024-05-09T00:08:26.048555+0000","last_unstale":"2024-05-09T00:10:09.324263+0000","last_undegraded":"2024-05-09T00:10:09.324263+0000","last_fullsized":"2024-05-09T00:10:09.324263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:16.009200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.317473+0000","last_change":"2024-05-09T00:08:26.048587+0000","last_active":"2024-05-09T00:10:09.317473+0000","last_peered":"2024-05-09T00:10:09.317473+0000","last_clean":"2024-05-09T00:10:09.317473+0000","last_became_active":"2024-05-09T00:08:26.048148+0000","last_became_peered":"2024-05-09T00:08:26.048148+0000","last_unstale":"2024-05-09T00:10:09.317473+0000","last_undegraded":"2024-05-09T00:10:09.317473+0000","last_fullsized":"2024-05-09T00:10:09.317473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:42.113829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.322240+0000","last_change":"2024-05-09T00:08:26.053607+0000","last_active":"2024-05-09T00:10:09.322240+0000","last_peered":"2024-05-09T00:10:09.322240+0000","last_clean":"2024-05-09T00:10:09.322240+0000","last_became_active":"2024-05-09T00:08:26.053212+0000","last_became_peered":"2024-05-09T00:08:26.053212+0000","last_unstale":"2024-05-09T00:10:09.322240+0000","last_undegraded":"2024-05-09T00:10:09.322240+0000","last_fullsized":"2024-05-09T00:10:09.322240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:52:32.937049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469352+0000","last_change":"2024-05-09T00:08:23.024481+0000","last_active":"2024-05-09T00:09:10.469352+0000","last_peered":"2024-05-09T00:09:10.469352+0000","last_clean":"2024-05-09T00:09:10.469352+0000","last_became_active":"2024-05-09T00:08:23.024132+0000","last_became_peered":"2024-05-09T00:08:23.024132+0000","last_unstale":"2024-05-09T00:09:10.469352+0000","last_undegraded":"2024-05-09T00:09:10.469352+0000","last_fullsized":"2024-05-09T00:09:10.469352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:11.188505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325280+0000","last_change":"2024-05-09T00:08:26.054772+0000","last_active":"2024-05-09T00:10:09.325280+0000","last_peered":"2024-05-09T00:10:09.325280+0000","last_clean":"2024-05-09T00:10:09.325280+0000","last_became_active":"2024-05-09T00:08:26.054433+0000","last_became_peered":"2024-05-09T00:08:26.054433+0000","last_unstale":"2024-05-09T00:10:09.325280+0000","last_undegraded":"2024-05-09T00:10:09.325280+0000","last_fullsized":"2024-05-09T00:10:09.325280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:05:44.283240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468356+0000","last_change":"2024-05-09T00:08:23.118778+0000","last_active":"2024-05-09T00:09:10.468356+0000","last_peered":"2024-05-09T00:09:10.468356+0000","last_clean":"2024-05-09T00:09:10.468356+0000","last_became_active":"2024-05-09T00:08:23.118377+0000","last_became_peered":"2024-05-09T00:08:23.118377+0000","last_unstale":"2024-05-09T00:09:10.468356+0000","last_undegraded":"2024-05-09T00:09:10.468356+0000","last_fullsized":"2024-05-09T00:09:10.468356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:51.352442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325452+0000","last_change":"2024-05-09T00:08:26.059953+0000","last_active":"2024-05-09T00:10:09.325452+0000","last_peered":"2024-05-09T00:10:09.325452+0000","last_clean":"2024-05-09T00:10:09.325452+0000","last_became_active":"2024-05-09T00:08:26.059571+0000","last_became_peered":"2024-05-09T00:08:26.059571+0000","last_unstale":"2024-05-09T00:10:09.325452+0000","last_undegraded":"2024-05-09T00:10:09.325452+0000","last_fullsized":"2024-05-09T00:10:09.325452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:48.056798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.475294+0000","last_change":"2024-05-09T00:08:23.021244+0000","last_active":"2024-05-09T00:09:10.475294+0000","last_peered":"2024-05-09T00:09:10.475294+0000","last_clean":"2024-05-09T00:09:10.475294+0000","last_became_active":"2024-05-09T00:08:23.020597+0000","last_became_peered":"2024-05-09T00:08:23.020597+0000","last_unstale":"2024-05-09T00:09:10.475294+0000","last_undegraded":"2024-05-09T00:09:10.475294+0000","last_fullsized":"2024-05-09T00:09:10.475294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.654261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328461+0000","last_change":"2024-05-09T00:08:26.059834+0000","last_active":"2024-05-09T00:10:09.328461+0000","last_peered":"2024-05-09T00:10:09.328461+0000","last_clean":"2024-05-09T00:10:09.328461+0000","last_became_active":"2024-05-09T00:08:26.059349+0000","last_became_peered":"2024-05-09T00:08:26.059349+0000","last_unstale":"2024-05-09T00:10:09.328461+0000","last_undegraded":"2024-05-09T00:10:09.328461+0000","last_fullsized":"2024-05-09T00:10:09.328461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:42.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474235+0000","last_change":"2024-05-09T00:08:23.019568+0000","last_active":"2024-05-09T00:09:10.474235+0000","last_peered":"2024-05-09T00:09:10.474235+0000","last_clean":"2024-05-09T00:09:10.474235+0000","last_became_active":"2024-05-09T00:08:23.019149+0000","last_became_peered":"2024-05-09T00:08:23.019149+0000","last_unstale":"2024-05-09T00:09:10.474235+0000","last_undegraded":"2024-05-09T00:09:10.474235+0000","last_fullsized":"2024-05-09T00:09:10.474235+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:44.569911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328628+0000","last_change":"2024-05-09T00:08:26.061308+0000","last_active":"2024-05-09T00:10:09.328628+0000","last_peered":"2024-05-09T00:10:09.328628+0000","last_clean":"2024-05-09T00:10:09.328628+0000","last_became_active":"2024-05-09T00:08:26.060973+0000","last_became_peered":"2024-05-09T00:08:26.060973+0000","last_unstale":"2024-05-09T00:10:09.328628+0000","last_undegraded":"2024-05-09T00:10:09.328628+0000","last_fullsized":"2024-05-09T00:10:09.328628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:16.845074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":221,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:50.666958+0000","last_change":"2024-05-09T00:08:23.020886+0000","last_active":"2024-05-09T00:11:50.666958+0000","last_peered":"2024-05-09T00:11:50.666958+0000","last_clean":"2024-05-09T00:11:50.666958+0000","last_became_active":"2024-05-09T00:08:23.020530+0000","last_became_peered":"2024-05-09T00:08:23.020530+0000","last_unstale":"2024-05-09T00:11:50.666958+0000","last_undegraded":"2024-05-09T00:11:50.666958+0000","last_fullsized":"2024-05-09T00:11:50.666958+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:34.978057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318827+0000","last_change":"2024-05-09T00:08:26.051018+0000","last_active":"2024-05-09T00:10:09.318827+0000","last_peered":"2024-05-09T00:10:09.318827+0000","last_clean":"2024-05-09T00:10:09.318827+0000","last_became_active":"2024-05-09T00:08:26.050495+0000","last_became_peered":"2024-05-09T00:08:26.050495+0000","last_unstale":"2024-05-09T00:10:09.318827+0000","last_undegraded":"2024-05-09T00:10:09.318827+0000","last_fullsized":"2024-05-09T00:10:09.318827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:07.862339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472149+0000","last_change":"2024-05-09T00:08:23.021187+0000","last_active":"2024-05-09T00:09:10.472149+0000","last_peered":"2024-05-09T00:09:10.472149+0000","last_clean":"2024-05-09T00:09:10.472149+0000","last_became_active":"2024-05-09T00:08:23.020770+0000","last_became_peered":"2024-05-09T00:08:23.020770+0000","last_unstale":"2024-05-09T00:09:10.472149+0000","last_undegraded":"2024-05-09T00:09:10.472149+0000","last_fullsized":"2024-05-09T00:09:10.472149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:45:08.992770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.316888+0000","last_change":"2024-05-09T00:09:10.797392+0000","last_active":"2024-05-09T00:10:09.316888+0000","last_peered":"2024-05-09T00:10:09.316888+0000","last_clean":"2024-05-09T00:10:09.316888+0000","last_became_active":"2024-05-09T00:09:10.796969+0000","last_became_peered":"2024-05-09T00:09:10.796969+0000","last_unstale":"2024-05-09T00:10:09.316888+0000","last_undegraded":"2024-05-09T00:10:09.316888+0000","last_fullsized":"2024-05-09T00:10:09.316888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:18.852678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548815+0000","last_change":"2024-05-09T00:08:23.012226+0000","last_active":"2024-05-09T00:09:10.548815+0000","last_peered":"2024-05-09T00:09:10.548815+0000","last_clean":"2024-05-09T00:09:10.548815+0000","last_became_active":"2024-05-09T00:08:23.011781+0000","last_became_peered":"2024-05-09T00:08:23.011781+0000","last_unstale":"2024-05-09T00:09:10.548815+0000","last_undegraded":"2024-05-09T00:09:10.548815+0000","last_fullsized":"2024-05-09T00:09:10.548815+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:44.452206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.321641+0000","last_change":"2024-05-09T00:08:26.043035+0000","last_active":"2024-05-09T00:10:09.321641+0000","last_peered":"2024-05-09T00:10:09.321641+0000","last_clean":"2024-05-09T00:10:09.321641+0000","last_became_active":"2024-05-09T00:08:26.042686+0000","last_became_peered":"2024-05-09T00:08:26.042686+0000","last_unstale":"2024-05-09T00:10:09.321641+0000","last_undegraded":"2024-05-09T00:10:09.321641+0000","last_fullsized":"2024-05-09T00:10:09.321641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:39:24.511309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469113+0000","last_change":"2024-05-09T00:08:23.022017+0000","last_active":"2024-05-09T00:09:10.469113+0000","last_peered":"2024-05-09T00:09:10.469113+0000","last_clean":"2024-05-09T00:09:10.469113+0000","last_became_active":"2024-05-09T00:08:23.021540+0000","last_became_peered":"2024-05-09T00:08:23.021540+0000","last_unstale":"2024-05-09T00:09:10.469113+0000","last_undegraded":"2024-05-09T00:09:10.469113+0000","last_fullsized":"2024-05-09T00:09:10.469113+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:50.773211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326086+0000","last_change":"2024-05-09T00:08:26.048039+0000","last_active":"2024-05-09T00:10:09.326086+0000","last_peered":"2024-05-09T00:10:09.326086+0000","last_clean":"2024-05-09T00:10:09.326086+0000","last_became_active":"2024-05-09T00:08:26.047695+0000","last_became_peered":"2024-05-09T00:08:26.047695+0000","last_unstale":"2024-05-09T00:10:09.326086+0000","last_undegraded":"2024-05-09T00:10:09.326086+0000","last_fullsized":"2024-05-09T00:10:09.326086+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:27.363847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":220,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:52.669314+0000","last_change":"2024-05-09T00:08:23.019565+0000","last_active":"2024-05-09T00:11:52.669314+0000","last_peered":"2024-05-09T00:11:52.669314+0000","last_clean":"2024-05-09T00:11:52.669314+0000","last_became_active":"2024-05-09T00:08:23.019113+0000","last_became_peered":"2024-05-09T00:08:23.019113+0000","last_unstale":"2024-05-09T00:11:52.669314+0000","last_undegraded":"2024-05-09T00:11:52.669314+0000","last_fullsized":"2024-05-09T00:11:52.669314+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:45.578934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323960+0000","last_change":"2024-05-09T00:08:26.058869+0000","last_active":"2024-05-09T00:10:09.323960+0000","last_peered":"2024-05-09T00:10:09.323960+0000","last_clean":"2024-05-09T00:10:09.323960+0000","last_became_active":"2024-05-09T00:08:26.058482+0000","last_became_peered":"2024-05-09T00:08:26.058482+0000","last_unstale":"2024-05-09T00:10:09.323960+0000","last_undegraded":"2024-05-09T00:10:09.323960+0000","last_fullsized":"2024-05-09T00:10:09.323960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:33.252208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472217+0000","last_change":"2024-05-09T00:08:23.024425+0000","last_active":"2024-05-09T00:09:10.472217+0000","last_peered":"2024-05-09T00:09:10.472217+0000","last_clean":"2024-05-09T00:09:10.472217+0000","last_became_active":"2024-05-09T00:08:23.023991+0000","last_became_peered":"2024-05-09T00:08:23.023991+0000","last_unstale":"2024-05-09T00:09:10.472217+0000","last_undegraded":"2024-05-09T00:09:10.472217+0000","last_fullsized":"2024-05-09T00:09:10.472217+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:47.907851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330931+0000","last_change":"2024-05-09T00:09:10.475008+0000","last_active":"2024-05-09T00:10:09.330931+0000","last_peered":"2024-05-09T00:10:09.330931+0000","last_clean":"2024-05-09T00:10:09.330931+0000","last_became_active":"2024-05-09T00:09:10.474663+0000","last_became_peered":"2024-05-09T00:09:10.474663+0000","last_unstale":"2024-05-09T00:10:09.330931+0000","last_undegraded":"2024-05-09T00:10:09.330931+0000","last_fullsized":"2024-05-09T00:10:09.330931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:36.990362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470179+0000","last_change":"2024-05-09T00:08:23.118516+0000","last_active":"2024-05-09T00:09:10.470179+0000","last_peered":"2024-05-09T00:09:10.470179+0000","last_clean":"2024-05-09T00:09:10.470179+0000","last_became_active":"2024-05-09T00:08:23.118147+0000","last_became_peered":"2024-05-09T00:08:23.118147+0000","last_unstale":"2024-05-09T00:09:10.470179+0000","last_undegraded":"2024-05-09T00:09:10.470179+0000","last_fullsized":"2024-05-09T00:09:10.470179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:24.205818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327118+0000","last_change":"2024-05-09T00:08:26.067063+0000","last_active":"2024-05-09T00:10:09.327118+0000","last_peered":"2024-05-09T00:10:09.327118+0000","last_clean":"2024-05-09T00:10:09.327118+0000","last_became_active":"2024-05-09T00:08:26.066440+0000","last_became_peered":"2024-05-09T00:08:26.066440+0000","last_unstale":"2024-05-09T00:10:09.327118+0000","last_undegraded":"2024-05-09T00:10:09.327118+0000","last_fullsized":"2024-05-09T00:10:09.327118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:48.648485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'36","reported_seq":455,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:53.672263+0000","last_change":"2024-05-09T00:08:23.020757+0000","last_active":"2024-05-09T00:11:53.672263+0000","last_peered":"2024-05-09T00:11:53.672263+0000","last_clean":"2024-05-09T00:11:53.672263+0000","last_became_active":"2024-05-09T00:08:23.020355+0000","last_became_peered":"2024-05-09T00:08:23.020355+0000","last_unstale":"2024-05-09T00:11:53.672263+0000","last_undegraded":"2024-05-09T00:11:53.672263+0000","last_fullsized":"2024-05-09T00:11:53.672263+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:00.605691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66697,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":392,"num_read_kb":261,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328259+0000","last_change":"2024-05-09T00:08:26.057402+0000","last_active":"2024-05-09T00:10:09.328259+0000","last_peered":"2024-05-09T00:10:09.328259+0000","last_clean":"2024-05-09T00:10:09.328259+0000","last_became_active":"2024-05-09T00:08:26.056974+0000","last_became_peered":"2024-05-09T00:08:26.056974+0000","last_unstale":"2024-05-09T00:10:09.328259+0000","last_undegraded":"2024-05-09T00:10:09.328259+0000","last_fullsized":"2024-05-09T00:10:09.328259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:47.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473953+0000","last_change":"2024-05-09T00:08:23.020208+0000","last_active":"2024-05-09T00:09:10.473953+0000","last_peered":"2024-05-09T00:09:10.473953+0000","last_clean":"2024-05-09T00:09:10.473953+0000","last_became_active":"2024-05-09T00:08:23.019809+0000","last_became_peered":"2024-05-09T00:08:23.019809+0000","last_unstale":"2024-05-09T00:09:10.473953+0000","last_undegraded":"2024-05-09T00:09:10.473953+0000","last_fullsized":"2024-05-09T00:09:10.473953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:47.900731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330506+0000","last_change":"2024-05-09T00:08:26.054273+0000","last_active":"2024-05-09T00:10:09.330506+0000","last_peered":"2024-05-09T00:10:09.330506+0000","last_clean":"2024-05-09T00:10:09.330506+0000","last_became_active":"2024-05-09T00:08:26.053840+0000","last_became_peered":"2024-05-09T00:08:26.053840+0000","last_unstale":"2024-05-09T00:10:09.330506+0000","last_undegraded":"2024-05-09T00:10:09.330506+0000","last_fullsized":"2024-05-09T00:10:09.330506+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:03:27.128582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469520+0000","last_change":"2024-05-09T00:08:23.021594+0000","last_active":"2024-05-09T00:09:10.469520+0000","last_peered":"2024-05-09T00:09:10.469520+0000","last_clean":"2024-05-09T00:09:10.469520+0000","last_became_active":"2024-05-09T00:08:23.021152+0000","last_became_peered":"2024-05-09T00:08:23.021152+0000","last_unstale":"2024-05-09T00:09:10.469520+0000","last_undegraded":"2024-05-09T00:09:10.469520+0000","last_fullsized":"2024-05-09T00:09:10.469520+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:58.505781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330209+0000","last_change":"2024-05-09T00:08:26.064464+0000","last_active":"2024-05-09T00:10:09.330209+0000","last_peered":"2024-05-09T00:10:09.330209+0000","last_clean":"2024-05-09T00:10:09.330209+0000","last_became_active":"2024-05-09T00:08:26.064000+0000","last_became_peered":"2024-05-09T00:08:26.064000+0000","last_unstale":"2024-05-09T00:10:09.330209+0000","last_undegraded":"2024-05-09T00:10:09.330209+0000","last_fullsized":"2024-05-09T00:10:09.330209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:01.455979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":227,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:51.668686+0000","last_change":"2024-05-09T00:08:23.020823+0000","last_active":"2024-05-09T00:11:51.668686+0000","last_peered":"2024-05-09T00:11:51.668686+0000","last_clean":"2024-05-09T00:11:51.668686+0000","last_became_active":"2024-05-09T00:08:23.020412+0000","last_became_peered":"2024-05-09T00:08:23.020412+0000","last_unstale":"2024-05-09T00:11:51.668686+0000","last_undegraded":"2024-05-09T00:11:51.668686+0000","last_fullsized":"2024-05-09T00:11:51.668686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:05.457452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":195,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318663+0000","last_change":"2024-05-09T00:09:10.474540+0000","last_active":"2024-05-09T00:10:09.318663+0000","last_peered":"2024-05-09T00:10:09.318663+0000","last_clean":"2024-05-09T00:10:09.318663+0000","last_became_active":"2024-05-09T00:09:10.474068+0000","last_became_peered":"2024-05-09T00:09:10.474068+0000","last_unstale":"2024-05-09T00:10:09.318663+0000","last_undegraded":"2024-05-09T00:10:09.318663+0000","last_fullsized":"2024-05-09T00:10:09.318663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:37.296140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.474415+0000","last_change":"2024-05-09T00:08:23.118857+0000","last_active":"2024-05-09T00:09:10.474415+0000","last_peered":"2024-05-09T00:09:10.474415+0000","last_clean":"2024-05-09T00:09:10.474415+0000","last_became_active":"2024-05-09T00:08:23.118433+0000","last_became_peered":"2024-05-09T00:08:23.118433+0000","last_unstale":"2024-05-09T00:09:10.474415+0000","last_undegraded":"2024-05-09T00:09:10.474415+0000","last_fullsized":"2024-05-09T00:09:10.474415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:37:10.617987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327227+0000","last_change":"2024-05-09T00:08:26.066911+0000","last_active":"2024-05-09T00:10:09.327227+0000","last_peered":"2024-05-09T00:10:09.327227+0000","last_clean":"2024-05-09T00:10:09.327227+0000","last_became_active":"2024-05-09T00:08:26.066548+0000","last_became_peered":"2024-05-09T00:08:26.066548+0000","last_unstale":"2024-05-09T00:10:09.327227+0000","last_undegraded":"2024-05-09T00:10:09.327227+0000","last_fullsized":"2024-05-09T00:10:09.327227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:32.391390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:39.175360+0000","last_change":"2024-05-09T00:08:23.019829+0000","last_active":"2024-05-09T00:10:39.175360+0000","last_peered":"2024-05-09T00:10:39.175360+0000","last_clean":"2024-05-09T00:10:39.175360+0000","last_became_active":"2024-05-09T00:08:23.019336+0000","last_became_peered":"2024-05-09T00:08:23.019336+0000","last_unstale":"2024-05-09T00:10:39.175360+0000","last_undegraded":"2024-05-09T00:10:39.175360+0000","last_fullsized":"2024-05-09T00:10:39.175360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:33:33.802738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318804+0000","last_change":"2024-05-09T00:08:26.055975+0000","last_active":"2024-05-09T00:10:09.318804+0000","last_peered":"2024-05-09T00:10:09.318804+0000","last_clean":"2024-05-09T00:10:09.318804+0000","last_became_active":"2024-05-09T00:08:26.055595+0000","last_became_peered":"2024-05-09T00:08:26.055595+0000","last_unstale":"2024-05-09T00:10:09.318804+0000","last_undegraded":"2024-05-09T00:10:09.318804+0000","last_fullsized":"2024-05-09T00:10:09.318804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:51:20.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.469423+0000","last_change":"2024-05-09T00:08:23.120353+0000","last_active":"2024-05-09T00:09:10.469423+0000","last_peered":"2024-05-09T00:09:10.469423+0000","last_clean":"2024-05-09T00:09:10.469423+0000","last_became_active":"2024-05-09T00:08:23.120004+0000","last_became_peered":"2024-05-09T00:08:23.120004+0000","last_unstale":"2024-05-09T00:09:10.469423+0000","last_undegraded":"2024-05-09T00:09:10.469423+0000","last_fullsized":"2024-05-09T00:09:10.469423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:12.168671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.320556+0000","last_change":"2024-05-09T00:08:26.046554+0000","last_active":"2024-05-09T00:10:09.320556+0000","last_peered":"2024-05-09T00:10:09.320556+0000","last_clean":"2024-05-09T00:10:09.320556+0000","last_became_active":"2024-05-09T00:08:26.045924+0000","last_became_peered":"2024-05-09T00:08:26.045924+0000","last_unstale":"2024-05-09T00:10:09.320556+0000","last_undegraded":"2024-05-09T00:10:09.320556+0000","last_fullsized":"2024-05-09T00:10:09.320556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:01.208980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472462+0000","last_change":"2024-05-09T00:08:23.019778+0000","last_active":"2024-05-09T00:09:10.472462+0000","last_peered":"2024-05-09T00:09:10.472462+0000","last_clean":"2024-05-09T00:09:10.472462+0000","last_became_active":"2024-05-09T00:08:23.019400+0000","last_became_peered":"2024-05-09T00:08:23.019400+0000","last_unstale":"2024-05-09T00:09:10.472462+0000","last_undegraded":"2024-05-09T00:09:10.472462+0000","last_fullsized":"2024-05-09T00:09:10.472462+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:06.883167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.308957+0000","last_change":"2024-05-09T00:09:10.474352+0000","last_active":"2024-05-09T00:10:09.308957+0000","last_peered":"2024-05-09T00:10:09.308957+0000","last_clean":"2024-05-09T00:10:09.308957+0000","last_became_active":"2024-05-09T00:09:10.473906+0000","last_became_peered":"2024-05-09T00:09:10.473906+0000","last_unstale":"2024-05-09T00:10:09.308957+0000","last_undegraded":"2024-05-09T00:10:09.308957+0000","last_fullsized":"2024-05-09T00:10:09.308957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:35:18.930585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.319595+0000","last_change":"2024-05-09T00:09:10.476484+0000","last_active":"2024-05-09T00:10:09.319595+0000","last_peered":"2024-05-09T00:10:09.319595+0000","last_clean":"2024-05-09T00:10:09.319595+0000","last_became_active":"2024-05-09T00:09:10.476001+0000","last_became_peered":"2024-05-09T00:09:10.476001+0000","last_unstale":"2024-05-09T00:10:09.319595+0000","last_undegraded":"2024-05-09T00:10:09.319595+0000","last_fullsized":"2024-05-09T00:10:09.319595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:32.018749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'6","reported_seq":223,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:11:52.669427+0000","last_change":"2024-05-09T00:08:23.118322+0000","last_active":"2024-05-09T00:11:52.669427+0000","last_peered":"2024-05-09T00:11:52.669427+0000","last_clean":"2024-05-09T00:11:52.669427+0000","last_became_active":"2024-05-09T00:08:23.117943+0000","last_became_peered":"2024-05-09T00:08:23.117943+0000","last_unstale":"2024-05-09T00:11:52.669427+0000","last_undegraded":"2024-05-09T00:11:52.669427+0000","last_fullsized":"2024-05-09T00:11:52.669427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:07.890437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323361+0000","last_change":"2024-05-09T00:08:26.052294+0000","last_active":"2024-05-09T00:10:09.323361+0000","last_peered":"2024-05-09T00:10:09.323361+0000","last_clean":"2024-05-09T00:10:09.323361+0000","last_became_active":"2024-05-09T00:08:26.051926+0000","last_became_peered":"2024-05-09T00:08:26.051926+0000","last_unstale":"2024-05-09T00:10:09.323361+0000","last_undegraded":"2024-05-09T00:10:09.323361+0000","last_fullsized":"2024-05-09T00:10:09.323361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:53.651401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796358+0000","last_change":"2024-05-09T00:08:23.015007+0000","last_active":"2024-05-09T00:09:10.796358+0000","last_peered":"2024-05-09T00:09:10.796358+0000","last_clean":"2024-05-09T00:09:10.796358+0000","last_became_active":"2024-05-09T00:08:23.014545+0000","last_became_peered":"2024-05-09T00:08:23.014545+0000","last_unstale":"2024-05-09T00:09:10.796358+0000","last_undegraded":"2024-05-09T00:09:10.796358+0000","last_fullsized":"2024-05-09T00:09:10.796358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:19.850200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:43.888672+0000","last_change":"2024-05-09T00:07:00.733233+0000","last_active":"2024-05-09T00:09:43.888672+0000","last_peered":"2024-05-09T00:09:43.888672+0000","last_clean":"2024-05-09T00:09:43.888672+0000","last_became_active":"2024-05-09T00:07:00.727365+0000","last_became_peered":"2024-05-09T00:07:00.727365+0000","last_unstale":"2024-05-09T00:09:43.888672+0000","last_undegraded":"2024-05-09T00:09:43.888672+0000","last_fullsized":"2024-05-09T00:09:43.888672+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:03:33.285220+0000","last_clean_scrub_stamp":"2024-05-09T00:03:33.285220+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:08.597290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326209+0000","last_change":"2024-05-09T00:08:26.064000+0000","last_active":"2024-05-09T00:10:09.326209+0000","last_peered":"2024-05-09T00:10:09.326209+0000","last_clean":"2024-05-09T00:10:09.326209+0000","last_became_active":"2024-05-09T00:08:26.063636+0000","last_became_peered":"2024-05-09T00:08:26.063636+0000","last_unstale":"2024-05-09T00:10:09.326209+0000","last_undegraded":"2024-05-09T00:10:09.326209+0000","last_fullsized":"2024-05-09T00:10:09.326209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:59.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.472596+0000","last_change":"2024-05-09T00:08:23.023459+0000","last_active":"2024-05-09T00:09:10.472596+0000","last_peered":"2024-05-09T00:09:10.472596+0000","last_clean":"2024-05-09T00:09:10.472596+0000","last_became_active":"2024-05-09T00:08:23.023089+0000","last_became_peered":"2024-05-09T00:08:23.023089+0000","last_unstale":"2024-05-09T00:09:10.472596+0000","last_undegraded":"2024-05-09T00:09:10.472596+0000","last_fullsized":"2024-05-09T00:09:10.472596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:54:41.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:14.346903+0000","last_change":"2024-05-09T00:09:10.478686+0000","last_active":"2024-05-09T00:09:14.346903+0000","last_peered":"2024-05-09T00:09:14.346903+0000","last_clean":"2024-05-09T00:09:14.346903+0000","last_became_active":"2024-05-09T00:09:10.474890+0000","last_became_peered":"2024-05-09T00:09:10.474890+0000","last_unstale":"2024-05-09T00:09:14.346903+0000","last_undegraded":"2024-05-09T00:09:14.346903+0000","last_fullsized":"2024-05-09T00:09:14.346903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:41.041247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327716+0000","last_change":"2024-05-09T00:08:26.050685+0000","last_active":"2024-05-09T00:10:09.327716+0000","last_peered":"2024-05-09T00:10:09.327716+0000","last_clean":"2024-05-09T00:10:09.327716+0000","last_became_active":"2024-05-09T00:08:26.050312+0000","last_became_peered":"2024-05-09T00:08:26.050312+0000","last_unstale":"2024-05-09T00:10:09.327716+0000","last_undegraded":"2024-05-09T00:10:09.327716+0000","last_fullsized":"2024-05-09T00:10:09.327716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:24:31.623182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:32.811512+0000","last_change":"2024-05-09T00:08:23.019663+0000","last_active":"2024-05-09T00:10:32.811512+0000","last_peered":"2024-05-09T00:10:32.811512+0000","last_clean":"2024-05-09T00:10:32.811512+0000","last_became_active":"2024-05-09T00:08:23.019301+0000","last_became_peered":"2024-05-09T00:08:23.019301+0000","last_unstale":"2024-05-09T00:10:32.811512+0000","last_undegraded":"2024-05-09T00:10:32.811512+0000","last_fullsized":"2024-05-09T00:10:32.811512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:08.291227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330686+0000","last_change":"2024-05-09T00:08:26.058825+0000","last_active":"2024-05-09T00:10:09.330686+0000","last_peered":"2024-05-09T00:10:09.330686+0000","last_clean":"2024-05-09T00:10:09.330686+0000","last_became_active":"2024-05-09T00:08:26.058469+0000","last_became_peered":"2024-05-09T00:08:26.058469+0000","last_unstale":"2024-05-09T00:10:09.330686+0000","last_undegraded":"2024-05-09T00:10:09.330686+0000","last_fullsized":"2024-05-09T00:10:09.330686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:08.118819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.796594+0000","last_change":"2024-05-09T00:08:23.054366+0000","last_active":"2024-05-09T00:09:10.796594+0000","last_peered":"2024-05-09T00:09:10.796594+0000","last_clean":"2024-05-09T00:09:10.796594+0000","last_became_active":"2024-05-09T00:08:23.053977+0000","last_became_peered":"2024-05-09T00:08:23.053977+0000","last_unstale":"2024-05-09T00:09:10.796594+0000","last_undegraded":"2024-05-09T00:09:10.796594+0000","last_fullsized":"2024-05-09T00:09:10.796594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:37.526043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.316437+0000","last_change":"2024-05-09T00:08:26.057864+0000","last_active":"2024-05-09T00:10:09.316437+0000","last_peered":"2024-05-09T00:10:09.316437+0000","last_clean":"2024-05-09T00:10:09.316437+0000","last_became_active":"2024-05-09T00:08:26.057436+0000","last_became_peered":"2024-05-09T00:08:26.057436+0000","last_unstale":"2024-05-09T00:10:09.316437+0000","last_undegraded":"2024-05-09T00:10:09.316437+0000","last_fullsized":"2024-05-09T00:10:09.316437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:47:19.976265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.468672+0000","last_change":"2024-05-09T00:08:23.118803+0000","last_active":"2024-05-09T00:09:10.468672+0000","last_peered":"2024-05-09T00:09:10.468672+0000","last_clean":"2024-05-09T00:09:10.468672+0000","last_became_active":"2024-05-09T00:08:23.118432+0000","last_became_peered":"2024-05-09T00:08:23.118432+0000","last_unstale":"2024-05-09T00:09:10.468672+0000","last_undegraded":"2024-05-09T00:09:10.468672+0000","last_fullsized":"2024-05-09T00:09:10.468672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:50.253699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.310200+0000","last_change":"2024-05-09T00:08:26.054281+0000","last_active":"2024-05-09T00:10:09.310200+0000","last_peered":"2024-05-09T00:10:09.310200+0000","last_clean":"2024-05-09T00:10:09.310200+0000","last_became_active":"2024-05-09T00:08:26.053874+0000","last_became_peered":"2024-05-09T00:08:26.053874+0000","last_unstale":"2024-05-09T00:10:09.310200+0000","last_undegraded":"2024-05-09T00:10:09.310200+0000","last_fullsized":"2024-05-09T00:10:09.310200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:11.566337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.467767+0000","last_change":"2024-05-09T00:08:23.017905+0000","last_active":"2024-05-09T00:09:10.467767+0000","last_peered":"2024-05-09T00:09:10.467767+0000","last_clean":"2024-05-09T00:09:10.467767+0000","last_became_active":"2024-05-09T00:08:23.017481+0000","last_became_peered":"2024-05-09T00:08:23.017481+0000","last_unstale":"2024-05-09T00:09:10.467767+0000","last_undegraded":"2024-05-09T00:09:10.467767+0000","last_fullsized":"2024-05-09T00:09:10.467767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:40.096415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.317514+0000","last_change":"2024-05-09T00:08:26.053392+0000","last_active":"2024-05-09T00:10:09.317514+0000","last_peered":"2024-05-09T00:10:09.317514+0000","last_clean":"2024-05-09T00:10:09.317514+0000","last_became_active":"2024-05-09T00:08:26.052628+0000","last_became_peered":"2024-05-09T00:08:26.052628+0000","last_unstale":"2024-05-09T00:10:09.317514+0000","last_undegraded":"2024-05-09T00:10:09.317514+0000","last_fullsized":"2024-05-09T00:10:09.317514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:28.013955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:14.346880+0000","last_change":"2024-05-09T00:09:10.481233+0000","last_active":"2024-05-09T00:09:14.346880+0000","last_peered":"2024-05-09T00:09:14.346880+0000","last_clean":"2024-05-09T00:09:14.346880+0000","last_became_active":"2024-05-09T00:09:10.475321+0000","last_became_peered":"2024-05-09T00:09:10.475321+0000","last_unstale":"2024-05-09T00:09:14.346880+0000","last_undegraded":"2024-05-09T00:09:14.346880+0000","last_fullsized":"2024-05-09T00:09:14.346880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:58.859277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.308335+0000","last_change":"2024-05-09T00:08:26.047812+0000","last_active":"2024-05-09T00:10:09.308335+0000","last_peered":"2024-05-09T00:10:09.308335+0000","last_clean":"2024-05-09T00:10:09.308335+0000","last_became_active":"2024-05-09T00:08:26.047459+0000","last_became_peered":"2024-05-09T00:08:26.047459+0000","last_unstale":"2024-05-09T00:10:09.308335+0000","last_undegraded":"2024-05-09T00:10:09.308335+0000","last_fullsized":"2024-05-09T00:10:09.308335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:02:37.257831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473816+0000","last_change":"2024-05-09T00:08:23.023800+0000","last_active":"2024-05-09T00:09:10.473816+0000","last_peered":"2024-05-09T00:09:10.473816+0000","last_clean":"2024-05-09T00:09:10.473816+0000","last_became_active":"2024-05-09T00:08:23.023387+0000","last_became_peered":"2024-05-09T00:08:23.023387+0000","last_unstale":"2024-05-09T00:09:10.473816+0000","last_undegraded":"2024-05-09T00:09:10.473816+0000","last_fullsized":"2024-05-09T00:09:10.473816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:33.714683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330035+0000","last_change":"2024-05-09T00:08:26.056981+0000","last_active":"2024-05-09T00:10:09.330035+0000","last_peered":"2024-05-09T00:10:09.330035+0000","last_clean":"2024-05-09T00:10:09.330035+0000","last_became_active":"2024-05-09T00:08:26.056605+0000","last_became_peered":"2024-05-09T00:08:26.056605+0000","last_unstale":"2024-05-09T00:10:09.330035+0000","last_undegraded":"2024-05-09T00:10:09.330035+0000","last_fullsized":"2024-05-09T00:10:09.330035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:32.897929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.473675+0000","last_change":"2024-05-09T00:08:23.120612+0000","last_active":"2024-05-09T00:09:10.473675+0000","last_peered":"2024-05-09T00:09:10.473675+0000","last_clean":"2024-05-09T00:09:10.473675+0000","last_became_active":"2024-05-09T00:08:23.120157+0000","last_became_peered":"2024-05-09T00:08:23.120157+0000","last_unstale":"2024-05-09T00:09:10.473675+0000","last_undegraded":"2024-05-09T00:09:10.473675+0000","last_fullsized":"2024-05-09T00:09:10.473675+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:06:08.027214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330272+0000","last_change":"2024-05-09T00:08:26.061594+0000","last_active":"2024-05-09T00:10:09.330272+0000","last_peered":"2024-05-09T00:10:09.330272+0000","last_clean":"2024-05-09T00:10:09.330272+0000","last_became_active":"2024-05-09T00:08:26.061201+0000","last_became_peered":"2024-05-09T00:08:26.061201+0000","last_unstale":"2024-05-09T00:10:09.330272+0000","last_undegraded":"2024-05-09T00:10:09.330272+0000","last_fullsized":"2024-05-09T00:10:09.330272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:40:50.183648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.470328+0000","last_change":"2024-05-09T00:08:23.017693+0000","last_active":"2024-05-09T00:09:10.470328+0000","last_peered":"2024-05-09T00:09:10.470328+0000","last_clean":"2024-05-09T00:09:10.470328+0000","last_became_active":"2024-05-09T00:08:23.017211+0000","last_became_peered":"2024-05-09T00:08:23.017211+0000","last_unstale":"2024-05-09T00:09:10.470328+0000","last_undegraded":"2024-05-09T00:09:10.470328+0000","last_fullsized":"2024-05-09T00:09:10.470328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:53.569932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327590+0000","last_change":"2024-05-09T00:08:26.063922+0000","last_active":"2024-05-09T00:10:09.327590+0000","last_peered":"2024-05-09T00:10:09.327590+0000","last_clean":"2024-05-09T00:10:09.327590+0000","last_became_active":"2024-05-09T00:08:26.063554+0000","last_became_peered":"2024-05-09T00:08:26.063554+0000","last_unstale":"2024-05-09T00:10:09.327590+0000","last_undegraded":"2024-05-09T00:10:09.327590+0000","last_fullsized":"2024-05-09T00:10:09.327590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:01.754688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:09:10.548651+0000","last_change":"2024-05-09T00:08:23.020367+0000","last_active":"2024-05-09T00:09:10.548651+0000","last_peered":"2024-05-09T00:09:10.548651+0000","last_clean":"2024-05-09T00:09:10.548651+0000","last_became_active":"2024-05-09T00:08:23.019801+0000","last_became_peered":"2024-05-09T00:08:23.019801+0000","last_unstale":"2024-05-09T00:09:10.548651+0000","last_undegraded":"2024-05-09T00:09:10.548651+0000","last_fullsized":"2024-05-09T00:09:10.548651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:21.998445+0000","last_clean_scrub_stamp":"2024-05-09T00:08:21.998445+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:14.936021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.330156+0000","last_change":"2024-05-09T00:08:26.063863+0000","last_active":"2024-05-09T00:10:09.330156+0000","last_peered":"2024-05-09T00:10:09.330156+0000","last_clean":"2024-05-09T00:10:09.330156+0000","last_became_active":"2024-05-09T00:08:26.061938+0000","last_became_peered":"2024-05-09T00:08:26.061938+0000","last_unstale":"2024-05-09T00:10:09.330156+0000","last_undegraded":"2024-05-09T00:10:09.330156+0000","last_fullsized":"2024-05-09T00:10:09.330156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:04.597991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.319694+0000","last_change":"2024-05-09T00:08:26.050837+0000","last_active":"2024-05-09T00:10:09.319694+0000","last_peered":"2024-05-09T00:10:09.319694+0000","last_clean":"2024-05-09T00:10:09.319694+0000","last_became_active":"2024-05-09T00:08:26.050424+0000","last_became_peered":"2024-05-09T00:08:26.050424+0000","last_unstale":"2024-05-09T00:10:09.319694+0000","last_undegraded":"2024-05-09T00:10:09.319694+0000","last_fullsized":"2024-05-09T00:10:09.319694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:45.134811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323117+0000","last_change":"2024-05-09T00:09:10.549989+0000","last_active":"2024-05-09T00:10:09.323117+0000","last_peered":"2024-05-09T00:10:09.323117+0000","last_clean":"2024-05-09T00:10:09.323117+0000","last_became_active":"2024-05-09T00:09:10.549502+0000","last_became_peered":"2024-05-09T00:09:10.549502+0000","last_unstale":"2024-05-09T00:10:09.323117+0000","last_undegraded":"2024-05-09T00:10:09.323117+0000","last_fullsized":"2024-05-09T00:10:09.323117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:10.654452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323905+0000","last_change":"2024-05-09T00:08:26.058012+0000","last_active":"2024-05-09T00:10:09.323905+0000","last_peered":"2024-05-09T00:10:09.323905+0000","last_clean":"2024-05-09T00:10:09.323905+0000","last_became_active":"2024-05-09T00:08:26.057282+0000","last_became_peered":"2024-05-09T00:08:26.057282+0000","last_unstale":"2024-05-09T00:10:09.323905+0000","last_undegraded":"2024-05-09T00:10:09.323905+0000","last_fullsized":"2024-05-09T00:10:09.323905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:38:53.520593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.320691+0000","last_change":"2024-05-09T00:08:26.063331+0000","last_active":"2024-05-09T00:10:09.320691+0000","last_peered":"2024-05-09T00:10:09.320691+0000","last_clean":"2024-05-09T00:10:09.320691+0000","last_became_active":"2024-05-09T00:08:26.062866+0000","last_became_peered":"2024-05-09T00:08:26.062866+0000","last_unstale":"2024-05-09T00:10:09.320691+0000","last_undegraded":"2024-05-09T00:10:09.320691+0000","last_fullsized":"2024-05-09T00:10:09.320691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:10.387852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.326309+0000","last_change":"2024-05-09T00:08:26.063735+0000","last_active":"2024-05-09T00:10:09.326309+0000","last_peered":"2024-05-09T00:10:09.326309+0000","last_clean":"2024-05-09T00:10:09.326309+0000","last_became_active":"2024-05-09T00:08:26.063358+0000","last_became_peered":"2024-05-09T00:08:26.063358+0000","last_unstale":"2024-05-09T00:10:09.326309+0000","last_undegraded":"2024-05-09T00:10:09.326309+0000","last_fullsized":"2024-05-09T00:10:09.326309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:06.832071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.318431+0000","last_change":"2024-05-09T00:08:26.052746+0000","last_active":"2024-05-09T00:10:09.318431+0000","last_peered":"2024-05-09T00:10:09.318431+0000","last_clean":"2024-05-09T00:10:09.318431+0000","last_became_active":"2024-05-09T00:08:26.051544+0000","last_became_peered":"2024-05-09T00:08:26.051544+0000","last_unstale":"2024-05-09T00:10:09.318431+0000","last_undegraded":"2024-05-09T00:10:09.318431+0000","last_fullsized":"2024-05-09T00:10:09.318431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:31.963268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.323267+0000","last_change":"2024-05-09T00:08:26.058075+0000","last_active":"2024-05-09T00:10:09.323267+0000","last_peered":"2024-05-09T00:10:09.323267+0000","last_clean":"2024-05-09T00:10:09.323267+0000","last_became_active":"2024-05-09T00:08:26.057349+0000","last_became_peered":"2024-05-09T00:08:26.057349+0000","last_unstale":"2024-05-09T00:10:09.323267+0000","last_undegraded":"2024-05-09T00:10:09.323267+0000","last_fullsized":"2024-05-09T00:10:09.323267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:59.322903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.324094+0000","last_change":"2024-05-09T00:08:26.056429+0000","last_active":"2024-05-09T00:10:09.324094+0000","last_peered":"2024-05-09T00:10:09.324094+0000","last_clean":"2024-05-09T00:10:09.324094+0000","last_became_active":"2024-05-09T00:08:26.054852+0000","last_became_peered":"2024-05-09T00:08:26.054852+0000","last_unstale":"2024-05-09T00:10:09.324094+0000","last_undegraded":"2024-05-09T00:10:09.324094+0000","last_fullsized":"2024-05-09T00:10:09.324094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:05.337636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.327096+0000","last_change":"2024-05-09T00:09:10.549880+0000","last_active":"2024-05-09T00:10:09.327096+0000","last_peered":"2024-05-09T00:10:09.327096+0000","last_clean":"2024-05-09T00:10:09.327096+0000","last_became_active":"2024-05-09T00:09:10.549448+0000","last_became_peered":"2024-05-09T00:09:10.549448+0000","last_unstale":"2024-05-09T00:10:09.327096+0000","last_undegraded":"2024-05-09T00:10:09.327096+0000","last_fullsized":"2024-05-09T00:10:09.327096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:16.990729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.325082+0000","last_change":"2024-05-09T00:08:26.054579+0000","last_active":"2024-05-09T00:10:09.325082+0000","last_peered":"2024-05-09T00:10:09.325082+0000","last_clean":"2024-05-09T00:10:09.325082+0000","last_became_active":"2024-05-09T00:08:26.053186+0000","last_became_peered":"2024-05-09T00:08:26.053186+0000","last_unstale":"2024-05-09T00:10:09.325082+0000","last_undegraded":"2024-05-09T00:10:09.325082+0000","last_fullsized":"2024-05-09T00:10:09.325082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:37.561691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.328708+0000","last_change":"2024-05-09T00:08:26.063608+0000","last_active":"2024-05-09T00:10:09.328708+0000","last_peered":"2024-05-09T00:10:09.328708+0000","last_clean":"2024-05-09T00:10:09.328708+0000","last_became_active":"2024-05-09T00:08:26.063078+0000","last_became_peered":"2024-05-09T00:08:26.063078+0000","last_unstale":"2024-05-09T00:10:09.328708+0000","last_undegraded":"2024-05-09T00:10:09.328708+0000","last_fullsized":"2024-05-09T00:10:09.328708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:18:06.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.331031+0000","last_change":"2024-05-09T00:08:26.055058+0000","last_active":"2024-05-09T00:10:09.331031+0000","last_peered":"2024-05-09T00:10:09.331031+0000","last_clean":"2024-05-09T00:10:09.331031+0000","last_became_active":"2024-05-09T00:08:26.054695+0000","last_became_peered":"2024-05-09T00:08:26.054695+0000","last_unstale":"2024-05-09T00:10:09.331031+0000","last_undegraded":"2024-05-09T00:10:09.331031+0000","last_fullsized":"2024-05-09T00:10:09.331031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:58.925990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-09T00:10:09.312854+0000","last_change":"2024-05-09T00:08:26.055366+0000","last_active":"2024-05-09T00:10:09.312854+0000","last_peered":"2024-05-09T00:10:09.312854+0000","last_clean":"2024-05-09T00:10:09.312854+0000","last_became_active":"2024-05-09T00:08:26.054904+0000","last_became_peered":"2024-05-09T00:08:26.054904+0000","last_unstale":"2024-05-09T00:10:09.312854+0000","last_undegraded":"2024-05-09T00:10:09.312854+0000","last_fullsized":"2024-05-09T00:10:09.312854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T00:08:25.019616+0000","last_clean_scrub_stamp":"2024-05-09T00:08:25.019616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:30:40.754878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285241,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71147,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1169,"num_read_kb":661,"num_write":181,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":213441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710777,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":425247,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1084243,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070531,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":880640,"data_stored":425461,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266761,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":425359,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":1017760,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":868352,"data_stored":424850,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":491340,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":888832,"data_stored":426121,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":488376,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":425161,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986270,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":426127,"data_compressed":5759,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822744,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":1016604,"data_compressed":42022,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34642,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36263,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T00:11:56.940 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:11:57.226 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:11:57.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:57 smithi046 ceph-mon[30151]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:11:57.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:57 smithi046 ceph-mon[30151]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:11:57.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:57 smithi142 ceph-mon[34533]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:11:57.691 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:57 smithi142 ceph-mon[34533]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:11:57.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:57 smithi169 ceph-mon[34777]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:11:57.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:57 smithi169 ceph-mon[34777]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:11:58.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:58 smithi142 ceph-mon[34533]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:11:58.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:58 smithi142 ceph-mon[34533]: pgmap v394: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-09T00:11:58.698 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:58 smithi046 ceph-mon[30151]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:11:58.698 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:58 smithi046 ceph-mon[30151]: pgmap v394: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-09T00:11:58.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:58 smithi169 ceph-mon[34777]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T00:11:58.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:58 smithi169 ceph-mon[34777]: pgmap v394: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-09T00:11:58.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.949+0000 7f5940397700 1 -- 172.21.15.46:0/1565619943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593810e930 msgr2=0x7f5938110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:58.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.949+0000 7f5940397700 1 --2- 172.21.15.46:0/1565619943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593810e930 0x7f5938110dd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f593000b0a0 tx=0x7f593000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:11:58.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.949+0000 7f5940397700 1 -- 172.21.15.46:0/1565619943 shutdown_connections 2024-05-09T00:11:58.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.949+0000 7f5940397700 1 --2- 172.21.15.46:0/1565619943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593810e930 0x7f5938110dd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:58.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.949+0000 7f5940397700 1 --2- 172.21.15.46:0/1565619943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938105500 0x7f5938105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:58.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.949+0000 7f5940397700 1 --2- 172.21.15.46:0/1565619943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5938105e50 0x7f593810e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:58.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.949+0000 7f5940397700 1 -- 172.21.15.46:0/1565619943 >> 172.21.15.46:0/1565619943 conn(0x7f59380fedf0 msgr2=0x7f5938101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:58.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.949+0000 7f5940397700 1 -- 172.21.15.46:0/1565619943 shutdown_connections 2024-05-09T00:11:58.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.949+0000 7f5940397700 1 -- 172.21.15.46:0/1565619943 wait complete. 2024-05-09T00:11:58.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f5940397700 1 Processor -- start 2024-05-09T00:11:58.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f5940397700 1 -- start start 2024-05-09T00:11:58.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f5940397700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5938105500 0x7f593819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f5940397700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938105e50 0x7f593819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f5940397700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593810e930 0x7f593819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f5940397700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5938114450 con 0x7f5938105500 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f5940397700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59381142d0 con 0x7f593810e930 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f5940397700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f59381145d0 con 0x7f5938105e50 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f593e133700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5938105500 0x7f593819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f593e133700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5938105500 0x7f593819ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59516/0 (socket says 172.21.15.46:59516) 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f593e133700 1 -- 172.21.15.46:0/3623788357 learned_addr learned my addr 172.21.15.46:0/3623788357 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f593d932700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938105e50 0x7f593819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.950+0000 7f593e934700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593810e930 0x7f593819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.951+0000 7f593e133700 1 -- 172.21.15.46:0/3623788357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938105e50 msgr2=0x7f593819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.951+0000 7f593e133700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938105e50 0x7f593819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.951+0000 7f593e133700 1 -- 172.21.15.46:0/3623788357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593810e930 msgr2=0x7f593819f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.951+0000 7f593e133700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593810e930 0x7f593819f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.951+0000 7f593e133700 1 -- 172.21.15.46:0/3623788357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593819fc30 con 0x7f5938105500 2024-05-09T00:11:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.951+0000 7f593e133700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5938105500 0x7f593819ad60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f593400d980 tx=0x7f593400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.951+0000 7f592f7fe700 1 -- 172.21.15.46:0/3623788357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934017030 con 0x7f5938105500 2024-05-09T00:11:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.951+0000 7f5940397700 1 -- 172.21.15.46:0/3623788357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593819ff20 con 0x7f5938105500 2024-05-09T00:11:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.951+0000 7f592f7fe700 1 -- 172.21.15.46:0/3623788357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59340044d0 con 0x7f5938105500 2024-05-09T00:11:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.952+0000 7f5940397700 1 -- 172.21.15.46:0/3623788357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59381a7840 con 0x7f5938105500 2024-05-09T00:11:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.952+0000 7f592f7fe700 1 -- 172.21.15.46:0/3623788357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934003970 con 0x7f5938105500 2024-05-09T00:11:58.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.953+0000 7f5940397700 1 -- 172.21.15.46:0/3623788357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59380512c0 con 0x7f5938105500 2024-05-09T00:11:58.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.954+0000 7f592f7fe700 1 -- 172.21.15.46:0/3623788357 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f593401a070 con 0x7f5938105500 2024-05-09T00:11:58.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.955+0000 7f592f7fe700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f59240b2900 0x7f59240b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:11:58.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.955+0000 7f593d932700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f59240b2900 0x7f59240b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:11:58.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.955+0000 7f592f7fe700 1 -- 172.21.15.46:0/3623788357 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f59340e1a90 con 0x7f5938105500 2024-05-09T00:11:58.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.955+0000 7f593d932700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f59240b2900 0x7f59240b4e10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59381022a0 tx=0x7f592800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:11:58.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:58.960+0000 7f592f7fe700 1 -- 172.21.15.46:0/3623788357 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f59340a97b0 con 0x7f5938105500 2024-05-09T00:11:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.226+0000 7f5940397700 1 -- 172.21.15.46:0/3623788357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f593819c0c0 con 0x7f5938105500 2024-05-09T00:11:59.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.227+0000 7f592f7fe700 1 -- 172.21.15.46:0/3623788357 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f59340a90b0 con 0x7f5938105500 2024-05-09T00:11:59.225 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:11:59.225 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:11:59.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.229+0000 7f5940397700 1 -- 172.21.15.46:0/3623788357 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f59240b2900 msgr2=0x7f59240b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:59.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.229+0000 7f5940397700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f59240b2900 0x7f59240b4e10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59381022a0 tx=0x7f592800b410 comp rx=0 tx=0).stop 2024-05-09T00:11:59.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.229+0000 7f5940397700 1 -- 172.21.15.46:0/3623788357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5938105500 msgr2=0x7f593819ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:11:59.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.229+0000 7f5940397700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5938105500 0x7f593819ad60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f593400d980 tx=0x7f593400dc90 comp rx=0 tx=0).stop 2024-05-09T00:11:59.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.229+0000 7f5940397700 1 -- 172.21.15.46:0/3623788357 shutdown_connections 2024-05-09T00:11:59.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.229+0000 7f5940397700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f59240b2900 0x7f59240b4e10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:59.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.229+0000 7f5940397700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593810e930 0x7f593819f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.229+0000 7f5940397700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5938105500 0x7f593819ad60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.229+0000 7f5940397700 1 --2- 172.21.15.46:0/3623788357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938105e50 0x7f593819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:11:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.229+0000 7f5940397700 1 -- 172.21.15.46:0/3623788357 >> 172.21.15.46:0/3623788357 conn(0x7f59380fedf0 msgr2=0x7f59380ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:11:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.229+0000 7f5940397700 1 -- 172.21.15.46:0/3623788357 shutdown_connections 2024-05-09T00:11:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:11:59.230+0000 7f5940397700 1 -- 172.21.15.46:0/3623788357 wait complete. 2024-05-09T00:11:59.229 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:11:59.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:11:59 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3623788357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:59.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:11:59 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3623788357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:59.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:11:59 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3623788357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:11:59.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:12:00.188 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:00.562 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:00 smithi046 ceph-mon[30151]: pgmap v395: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-09T00:12:00.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:00 smithi142 ceph-mon[34533]: pgmap v395: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-09T00:12:00.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:00 smithi169 ceph-mon[34777]: pgmap v395: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-09T00:12:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.942+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/982669781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3b4108740 msgr2=0x7fb3b41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.942+0000 7fb3bc4ee700 1 --2- 172.21.15.46:0/982669781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3b4108740 0x7fb3b41100a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00b0d0 tx=0x7fb3ac00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:12:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.942+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/982669781 shutdown_connections 2024-05-09T00:12:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.942+0000 7fb3bc4ee700 1 --2- 172.21.15.46:0/982669781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3b4108740 0x7fb3b41100a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.942+0000 7fb3bc4ee700 1 --2- 172.21.15.46:0/982669781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b41034c0 0x7fb3b41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.942+0000 7fb3bc4ee700 1 --2- 172.21.15.46:0/982669781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b4107df0 0x7fb3b4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.942+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/982669781 >> 172.21.15.46:0/982669781 conn(0x7fb3b40fedf0 msgr2=0x7fb3b4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.942+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/982669781 shutdown_connections 2024-05-09T00:12:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.943+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/982669781 wait complete. 2024-05-09T00:12:01.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.944+0000 7fb3bc4ee700 1 Processor -- start 2024-05-09T00:12:01.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.944+0000 7fb3bc4ee700 1 -- start start 2024-05-09T00:12:01.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.944+0000 7fb3bc4ee700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b41034c0 0x7fb3b419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:01.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.944+0000 7fb3bc4ee700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3b4107df0 0x7fb3b419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:01.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.944+0000 7fb3bc4ee700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b4108740 0x7fb3b419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:01.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.944+0000 7fb3bc4ee700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41133c0 con 0x7fb3b4107df0 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.944+0000 7fb3bc4ee700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4113240 con 0x7fb3b4108740 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.944+0000 7fb3bc4ee700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4113540 con 0x7fb3b41034c0 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3b9a89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3b4107df0 0x7fb3b419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3b9a89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3b4107df0 0x7fb3b419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59544/0 (socket says 172.21.15.46:59544) 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3b9a89700 1 -- 172.21.15.46:0/4236162971 learned_addr learned my addr 172.21.15.46:0/4236162971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3ba28a700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b41034c0 0x7fb3b419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3baa8b700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b4108740 0x7fb3b419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3b9a89700 1 -- 172.21.15.46:0/4236162971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b41034c0 msgr2=0x7fb3b419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3b9a89700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b41034c0 0x7fb3b419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3b9a89700 1 -- 172.21.15.46:0/4236162971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b4108740 msgr2=0x7fb3b419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3b9a89700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b4108740 0x7fb3b419f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3b9a89700 1 -- 172.21.15.46:0/4236162971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b419fcb0 con 0x7fb3b4107df0 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3b9a89700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3b4107df0 0x7fb3b419b380 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e970 tx=0x7fb3a400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3ab7fe700 1 -- 172.21.15.46:0/4236162971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4013be0 con 0x7fb3b4107df0 2024-05-09T00:12:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3ab7fe700 1 -- 172.21.15.46:0/4236162971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3a4013d80 con 0x7fb3b4107df0 2024-05-09T00:12:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/4236162971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b419ffa0 con 0x7fb3b4107df0 2024-05-09T00:12:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3ab7fe700 1 -- 172.21.15.46:0/4236162971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4005020 con 0x7fb3b4107df0 2024-05-09T00:12:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.945+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/4236162971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b410a070 con 0x7fb3b4107df0 2024-05-09T00:12:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.947+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/4236162971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b4104970 con 0x7fb3b4107df0 2024-05-09T00:12:01.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.949+0000 7fb3ab7fe700 1 -- 172.21.15.46:0/4236162971 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb3a4004750 con 0x7fb3b4107df0 2024-05-09T00:12:01.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.950+0000 7fb3ab7fe700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb3a00b2880 0x7fb3a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:01.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.950+0000 7fb3ba28a700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb3a00b2880 0x7fb3a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:01.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.950+0000 7fb3ab7fe700 1 -- 172.21.15.46:0/4236162971 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb3a40e6900 con 0x7fb3b4107df0 2024-05-09T00:12:01.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.951+0000 7fb3ba28a700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb3a00b2880 0x7fb3a00b4d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0005fd0 tx=0x7fb3b0005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:01.954+0000 7fb3ab7fe700 1 -- 172.21.15.46:0/4236162971 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3a40ae620 con 0x7fb3b4107df0 2024-05-09T00:12:02.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.175+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/4236162971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3b419c140 con 0x7fb3b4107df0 2024-05-09T00:12:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.177+0000 7fb3ab7fe700 1 -- 172.21.15.46:0/4236162971 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22694 (secure 0 0 0) 0x7fb3a40adf20 con 0x7fb3b4107df0 2024-05-09T00:12:02.174 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:12:02.175 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","created":"2024-05-09T00:00:14.356524+0000","modified":"2024-05-09T00:09:10.458026+0000","last_up_change":"2024-05-09T00:07:23.332796+0000","last_in_change":"2024-05-09T00:07:03.238539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:03:33.278910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:08:21.014416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:08:24.922968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"740d6388-5c1f-45eb-93b0-7583bfa5a225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6803","nonce":3727342125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6805","nonce":3727342125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6809","nonce":3727342125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6807","nonce":3727342125}]},"public_addr":"172.21.15.46:6803/3727342125","cluster_addr":"172.21.15.46:6805/3727342125","heartbeat_back_addr":"172.21.15.46:6809/3727342125","heartbeat_front_addr":"172.21.15.46:6807/3727342125","state":["exists","up"]},{"osd":1,"uuid":"85fbf3ae-0be8-4504-84d6-66fae0d2169d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6801","nonce":3156042469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6803","nonce":3156042469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6807","nonce":3156042469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6805","nonce":3156042469}]},"public_addr":"172.21.15.142:6801/3156042469","cluster_addr":"172.21.15.142:6803/3156042469","heartbeat_back_addr":"172.21.15.142:6807/3156042469","heartbeat_front_addr":"172.21.15.142:6805/3156042469","state":["exists","up"]},{"osd":2,"uuid":"05c070e7-83ad-4949-9834-4ce51ac0a2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6801","nonce":2680673064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6803","nonce":2680673064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6807","nonce":2680673064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6805","nonce":2680673064}]},"public_addr":"172.21.15.169:6801/2680673064","cluster_addr":"172.21.15.169:6803/2680673064","heartbeat_back_addr":"172.21.15.169:6807/2680673064","heartbeat_front_addr":"172.21.15.169:6805/2680673064","state":["exists","up"]},{"osd":3,"uuid":"09d7bbe1-58b6-4407-bef2-920f179dfb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6811","nonce":1158536261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6813","nonce":1158536261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6817","nonce":1158536261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6815","nonce":1158536261}]},"public_addr":"172.21.15.46:6811/1158536261","cluster_addr":"172.21.15.46:6813/1158536261","heartbeat_back_addr":"172.21.15.46:6817/1158536261","heartbeat_front_addr":"172.21.15.46:6815/1158536261","state":["exists","up"]},{"osd":4,"uuid":"66c9644b-6b0b-4fe7-aa63-b9d87bff7153","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6809","nonce":2299616865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6811","nonce":2299616865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6815","nonce":2299616865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6813","nonce":2299616865}]},"public_addr":"172.21.15.142:6809/2299616865","cluster_addr":"172.21.15.142:6811/2299616865","heartbeat_back_addr":"172.21.15.142:6815/2299616865","heartbeat_front_addr":"172.21.15.142:6813/2299616865","state":["exists","up"]},{"osd":5,"uuid":"96a72e01-1c5f-418e-bb9b-b790d9b3059b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6809","nonce":445044713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6811","nonce":445044713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6815","nonce":445044713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6813","nonce":445044713}]},"public_addr":"172.21.15.169:6809/445044713","cluster_addr":"172.21.15.169:6811/445044713","heartbeat_back_addr":"172.21.15.169:6815/445044713","heartbeat_front_addr":"172.21.15.169:6813/445044713","state":["exists","up"]},{"osd":6,"uuid":"6b459542-0e56-4369-b542-e36c46a18f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6819","nonce":1394035482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6821","nonce":1394035482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6825","nonce":1394035482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6823","nonce":1394035482}]},"public_addr":"172.21.15.46:6819/1394035482","cluster_addr":"172.21.15.46:6821/1394035482","heartbeat_back_addr":"172.21.15.46:6825/1394035482","heartbeat_front_addr":"172.21.15.46:6823/1394035482","state":["exists","up"]},{"osd":7,"uuid":"786b4517-4cbd-4fa3-b5a0-39cccf839893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6817","nonce":596805607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6819","nonce":596805607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6823","nonce":596805607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6821","nonce":596805607}]},"public_addr":"172.21.15.142:6817/596805607","cluster_addr":"172.21.15.142:6819/596805607","heartbeat_back_addr":"172.21.15.142:6823/596805607","heartbeat_front_addr":"172.21.15.142:6821/596805607","state":["exists","up"]},{"osd":8,"uuid":"b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6817","nonce":2650652376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6819","nonce":2650652376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6823","nonce":2650652376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6821","nonce":2650652376}]},"public_addr":"172.21.15.169:6817/2650652376","cluster_addr":"172.21.15.169:6819/2650652376","heartbeat_back_addr":"172.21.15.169:6823/2650652376","heartbeat_front_addr":"172.21.15.169:6821/2650652376","state":["exists","up"]},{"osd":9,"uuid":"32df0dab-5875-4beb-b733-60a915c94a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6827","nonce":374861248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6829","nonce":374861248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6833","nonce":374861248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6831","nonce":374861248}]},"public_addr":"172.21.15.46:6827/374861248","cluster_addr":"172.21.15.46:6829/374861248","heartbeat_back_addr":"172.21.15.46:6833/374861248","heartbeat_front_addr":"172.21.15.46:6831/374861248","state":["exists","up"]},{"osd":10,"uuid":"380a4f47-e528-4258-8836-32eac0e6e630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6825","nonce":1364891982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6827","nonce":1364891982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6831","nonce":1364891982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6829","nonce":1364891982}]},"public_addr":"172.21.15.142:6825/1364891982","cluster_addr":"172.21.15.142:6827/1364891982","heartbeat_back_addr":"172.21.15.142:6831/1364891982","heartbeat_front_addr":"172.21.15.142:6829/1364891982","state":["exists","up"]},{"osd":11,"uuid":"56b04b15-6fcd-497a-aa66-19da9624b656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6825","nonce":4262478976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6827","nonce":4262478976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6831","nonce":4262478976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6829","nonce":4262478976}]},"public_addr":"172.21.15.169:6825/4262478976","cluster_addr":"172.21.15.169:6827/4262478976","heartbeat_back_addr":"172.21.15.169:6831/4262478976","heartbeat_front_addr":"172.21.15.169:6829/4262478976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:02:42.159298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:06.053328+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:29.587137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:54.573782+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:20.276073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:45.320565+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:10.985522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:36.806137+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:02.266736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:29.788773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:55.827639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:07:21.762382+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/1072138393":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2786891364":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/7723180":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/626888490":"2024-05-10T00:01:09.191239+0000","172.21.15.46:6800/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/4236543921":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2168313126":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6800/953818590":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6801/953818590":"2024-05-10T00:00:41.833323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:12:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.179+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/4236162971 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb3a00b2880 msgr2=0x7fb3a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.179+0000 7fb3bc4ee700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb3a00b2880 0x7fb3a00b4d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0005fd0 tx=0x7fb3b0005f60 comp rx=0 tx=0).stop 2024-05-09T00:12:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.179+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/4236162971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3b4107df0 msgr2=0x7fb3b419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.179+0000 7fb3bc4ee700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3b4107df0 0x7fb3b419b380 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e970 tx=0x7fb3a400ed30 comp rx=0 tx=0).stop 2024-05-09T00:12:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.180+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/4236162971 shutdown_connections 2024-05-09T00:12:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.180+0000 7fb3bc4ee700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb3a00b2880 0x7fb3a00b4d90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.180+0000 7fb3bc4ee700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b4108740 0x7fb3b419f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.180+0000 7fb3bc4ee700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3b41034c0 0x7fb3b419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.180+0000 7fb3bc4ee700 1 --2- 172.21.15.46:0/4236162971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3b4107df0 0x7fb3b419b380 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.180+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/4236162971 >> 172.21.15.46:0/4236162971 conn(0x7fb3b40fedf0 msgr2=0x7fb3b4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.180+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/4236162971 shutdown_connections 2024-05-09T00:12:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:02.180+0000 7fb3bc4ee700 1 -- 172.21.15.46:0/4236162971 wait complete. 2024-05-09T00:12:02.476 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:02 smithi046 ceph-mon[30151]: pgmap v396: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-09T00:12:02.476 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:02 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/4236162971' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:12:02.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:02 smithi142 ceph-mon[34533]: pgmap v396: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-09T00:12:02.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:02 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/4236162971' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:12:02.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:02 smithi169 ceph-mon[34777]: pgmap v396: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-09T00:12:02.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:02 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/4236162971' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:12:02.873 DEBUG:tasks.fwd_scrub:Yielding 2024-05-09T00:12:02.874 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T00:12:02.887 INFO:tasks.workunit:Pulling workunits from ref c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:12:02.888 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T00:12:02.888 INFO:tasks.workunit:timeout=3h 2024-05-09T00:12:02.888 INFO:tasks.workunit:cleanup=True 2024-05-09T00:12:02.888 DEBUG:teuthology.orchestra.run.smithi046:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T00:12:02.889 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-09T00:12:02.889 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:12:02.889 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:12:02.907 INFO:teuthology.orchestra.run.smithi046.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-09T00:12:02.907 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-09T00:12:02.908 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-05-09T00:12:02.908 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T00:12:02.908 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-09T00:12:02.908 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-09 00:09:00.573120069 +0000 2024-05-09T00:12:02.908 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-09 00:09:00.573120069 +0000 2024-05-09T00:12:02.908 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-09 00:11:50.066685201 +0000 2024-05-09T00:12:02.908 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-09 00:09:00.573120069 +0000 2024-05-09T00:12:02.908 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-09T00:12:02.908 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-09T00:12:02.983 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T00:12:03.039 INFO:tasks.workunit.client.0.smithi046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T00:12:03.152 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:04.684 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:04 smithi046 ceph-mon[30151]: pgmap v397: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-09T00:12:04.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:04 smithi142 ceph-mon[34533]: pgmap v397: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-09T00:12:04.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:04 smithi169 ceph-mon[34777]: pgmap v397: 97 pgs: 97 active+clean; 646 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-09T00:12:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.987+0000 7f65f20a9700 1 -- 172.21.15.46:0/2783492829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ec1034c0 msgr2=0x7f65ec1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.987+0000 7f65f20a9700 1 --2- 172.21.15.46:0/2783492829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ec1034c0 0x7f65ec1038d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0099c0 tx=0x7f65dc009cd0 comp rx=0 tx=0).stop 2024-05-09T00:12:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.987+0000 7f65f20a9700 1 -- 172.21.15.46:0/2783492829 shutdown_connections 2024-05-09T00:12:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.987+0000 7f65f20a9700 1 --2- 172.21.15.46:0/2783492829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65ec108740 0x7f65ec1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.987+0000 7f65f20a9700 1 --2- 172.21.15.46:0/2783492829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ec1034c0 0x7f65ec1038d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.987+0000 7f65f20a9700 1 --2- 172.21.15.46:0/2783492829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65ec107df0 0x7f65ec108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.987+0000 7f65f20a9700 1 -- 172.21.15.46:0/2783492829 >> 172.21.15.46:0/2783492829 conn(0x7f65ec0fedf0 msgr2=0x7f65ec101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.987+0000 7f65f20a9700 1 -- 172.21.15.46:0/2783492829 shutdown_connections 2024-05-09T00:12:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.987+0000 7f65f20a9700 1 -- 172.21.15.46:0/2783492829 wait complete. 2024-05-09T00:12:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.988+0000 7f65f20a9700 1 Processor -- start 2024-05-09T00:12:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.988+0000 7f65f20a9700 1 -- start start 2024-05-09T00:12:04.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.988+0000 7f65f20a9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65ec1034c0 0x7f65ec19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:04.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.988+0000 7f65f20a9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ec107df0 0x7f65ec19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:04.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.988+0000 7f65f20a9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65ec108740 0x7f65ec19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:04.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.988+0000 7f65f20a9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1144e0 con 0x7f65ec107df0 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.988+0000 7f65f20a9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec114360 con 0x7f65ec108740 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.988+0000 7f65f20a9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec114660 con 0x7f65ec1034c0 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.988+0000 7f65eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ec107df0 0x7f65ec19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.988+0000 7f65eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ec107df0 0x7f65ec19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59564/0 (socket says 172.21.15.46:59564) 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.988+0000 7f65eaffd700 1 -- 172.21.15.46:0/3328663436 learned_addr learned my addr 172.21.15.46:0/3328663436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.988+0000 7f65ebfff700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65ec108740 0x7f65ec19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.989+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65ec1034c0 0x7f65ec19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.989+0000 7f65eaffd700 1 -- 172.21.15.46:0/3328663436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65ec1034c0 msgr2=0x7f65ec19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.989+0000 7f65eaffd700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65ec1034c0 0x7f65ec19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.989+0000 7f65eaffd700 1 -- 172.21.15.46:0/3328663436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65ec108740 msgr2=0x7f65ec19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.989+0000 7f65eaffd700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65ec108740 0x7f65ec19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.989+0000 7f65eaffd700 1 -- 172.21.15.46:0/3328663436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec19fd60 con 0x7f65ec107df0 2024-05-09T00:12:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.989+0000 7f65eaffd700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ec107df0 0x7f65ec19b430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f65e000d980 tx=0x7f65e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.989+0000 7f65e8ff9700 1 -- 172.21.15.46:0/3328663436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0017070 con 0x7f65ec107df0 2024-05-09T00:12:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.989+0000 7f65e8ff9700 1 -- 172.21.15.46:0/3328663436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e0009de0 con 0x7f65ec107df0 2024-05-09T00:12:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.989+0000 7f65e8ff9700 1 -- 172.21.15.46:0/3328663436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e000f3c0 con 0x7f65ec107df0 2024-05-09T00:12:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.989+0000 7f65f20a9700 1 -- 172.21.15.46:0/3328663436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec1a0050 con 0x7f65ec107df0 2024-05-09T00:12:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.989+0000 7f65f20a9700 1 -- 172.21.15.46:0/3328663436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec1a7970 con 0x7f65ec107df0 2024-05-09T00:12:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.992+0000 7f65e8ff9700 1 -- 172.21.15.46:0/3328663436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f65e000f560 con 0x7f65ec107df0 2024-05-09T00:12:04.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.993+0000 7f65e8ff9700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f65d40b2900 0x7f65d40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:04.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.993+0000 7f6600abfb80 1 Processor -- start 2024-05-09T00:12:04.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.993+0000 7f6600abfb80 1 -- start start 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.993+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f65d40b2900 0x7f65d40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.993+0000 7f65e8ff9700 1 -- 172.21.15.46:0/3328663436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f65e00e1d80 con 0x7f65ec107df0 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.993+0000 7f6600abfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e2c41e0 0x55e93e35bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.993+0000 7f6600abfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 0x55e93e35c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.993+0000 7f6600abfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e360480 0x55e93e360890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f6600abfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e93e27a070 con 0x55e93e1da520 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f6600abfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e93e218870 con 0x55e93e2c41e0 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f6600abfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e93e276030 con 0x55e93e360480 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 0x55e93e35c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 0x55e93e35c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59568/0 (socket says 172.21.15.46:59568) 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65eaffd700 1 -- 172.21.15.46:0/85376146 learned_addr learned my addr 172.21.15.46:0/85376146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/85376146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e2c41e0 0x55e93e35bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65ebfff700 1 --2- 172.21.15.46:0/85376146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e360480 0x55e93e360890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f65d40b2900 0x7f65d40b4e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f65dc005b40 tx=0x7f65dc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65eaffd700 1 -- 172.21.15.46:0/85376146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e360480 msgr2=0x55e93e360890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65eaffd700 1 --2- 172.21.15.46:0/85376146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e360480 0x55e93e360890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65eaffd700 1 -- 172.21.15.46:0/85376146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e2c41e0 msgr2=0x55e93e35bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65eaffd700 1 --2- 172.21.15.46:0/85376146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e2c41e0 0x55e93e35bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65eaffd700 1 -- 172.21.15.46:0/85376146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93e2103f0 con 0x55e93e1da520 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65ebfff700 1 --2- 172.21.15.46:0/85376146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e360480 0x55e93e360890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/85376146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e2c41e0 0x55e93e35bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65eaffd700 1 --2- 172.21.15.46:0/85376146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 0x55e93e35c370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f65e00a9060 tx=0x7f65e00a9a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39643d8862776120 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65d1ffb700 1 -- 172.21.15.46:0/85376146 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0020030 con 0x55e93e1da520 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65d1ffb700 1 -- 172.21.15.46:0/85376146 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e00ad710 con 0x55e93e1da520 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.994+0000 7f65d1ffb700 1 -- 172.21.15.46:0/85376146 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e00a8820 con 0x55e93e1da520 2024-05-09T00:12:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.995+0000 7f6600abfb80 1 -- 172.21.15.46:0/85376146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 msgr2=0x55e93e35c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:04.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.995+0000 7f6600abfb80 1 --2- 172.21.15.46:0/85376146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 0x55e93e35c370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f65e00a9060 tx=0x7f65e00a9a60 comp rx=0 tx=0).stop 2024-05-09T00:12:04.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.995+0000 7f6600abfb80 1 -- 172.21.15.46:0/85376146 shutdown_connections 2024-05-09T00:12:04.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.995+0000 7f6600abfb80 1 --2- 172.21.15.46:0/85376146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e360480 0x55e93e360890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:04.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.995+0000 7f6600abfb80 1 --2- 172.21.15.46:0/85376146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e2c41e0 0x55e93e35bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:04.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.995+0000 7f6600abfb80 1 --2- 172.21.15.46:0/85376146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 0x55e93e35c370 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:04.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.995+0000 7f6600abfb80 1 -- 172.21.15.46:0/85376146 >> 172.21.15.46:0/85376146 conn(0x55e93e23a720 msgr2=0x55e93e352480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:04.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.996+0000 7f6600abfb80 1 -- 172.21.15.46:0/85376146 shutdown_connections 2024-05-09T00:12:04.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.996+0000 7f6600abfb80 1 -- 172.21.15.46:0/85376146 wait complete. 2024-05-09T00:12:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.996+0000 7f6600abfb80 1 -- start start 2024-05-09T00:12:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.996+0000 7f6600abfb80 1 Processor -- start 2024-05-09T00:12:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f6600abfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e1da520 0x55e93e365790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f6600abfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e2c41e0 0x55e93e365cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f6600abfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e360480 0x55e93e366210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f6600abfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e93e0f5c20 con 0x55e93e360480 2024-05-09T00:12:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f6600abfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e93e2341d0 con 0x55e93e2c41e0 2024-05-09T00:12:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f6600abfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e93e3611d0 con 0x55e93e1da520 2024-05-09T00:12:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f6600abfb80 10 client.0 _ll_register_callbacks cb 0x55e93e0eb740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f65ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e360480 0x55e93e366210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f65ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e360480 0x55e93e366210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59584/0 (socket says 172.21.15.46:59584) 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f65ebfff700 1 -- 172.21.15.46:0/534110540 learned_addr learned my addr 172.21.15.46:0/534110540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f65eaffd700 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e2c41e0 0x55e93e365cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e1da520 0x55e93e365790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f65ebfff700 1 -- 172.21.15.46:0/534110540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e1da520 msgr2=0x55e93e365790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f65ebfff700 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e1da520 0x55e93e365790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f65ebfff700 1 -- 172.21.15.46:0/534110540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e2c41e0 msgr2=0x55e93e365cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f65ebfff700 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e2c41e0 0x55e93e365cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f65ebfff700 1 -- 172.21.15.46:0/534110540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93e0f5510 con 0x55e93e360480 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f65ebfff700 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e360480 0x55e93e366210 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f65e400af10 tx=0x7f65e400af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53120e6740987c3a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.997+0000 7f65caffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f65caffd700 1 -- 172.21.15.46:0/534110540 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4007b30 con 0x55e93e360480 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f65caffd700 1 -- 172.21.15.46:0/534110540 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f65e4007cd0 con 0x55e93e360480 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f6600abfb80 1 -- 172.21.15.46:0/534110540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e00ad710 con 0x55e93e360480 2024-05-09T00:12:05.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f65caffd700 1 -- 172.21.15.46:0/534110540 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f65e4020440 con 0x55e93e360480 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f6600abfb80 10 client.15201 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f6600abfb80 1 -- 172.21.15.46:0/534110540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e93e373010 con 0x55e93e360480 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f65caffd700 1 -- 172.21.15.46:0/534110540 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e00ad710 con 0x55e93e360480 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f65caffd700 1 -- 172.21.15.46:0/534110540 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f65e4015510 con 0x55e93e360480 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f6600abfb80 10 client.15201 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f6600abfb80 10 client.15201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f6600abfb80 20 client.15201 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f6600abfb80 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e93e3756a0 0x55e93e377b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f6600abfb80 4 client.15201 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.998+0000 7f6600abfb80 1 -- 172.21.15.46:0/534110540 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e93e287a50 con 0x55e93e3756a0 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.999+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e93e3756a0 0x55e93e377b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.999+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e93e3756a0 0x55e93e377b00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37c9b26056d0677 server_cookie=ceb989b7b31e1a14 in_seq=0 out_seq=0 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:04.999+0000 7f65caffd700 10 client.15201 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.000+0000 7f65caffd700 1 -- 172.21.15.46:0/534110540 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e93e287a50 con 0x55e93e3756a0 2024-05-09T00:12:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.000+0000 7f65caffd700 10 client.15201 handle_command_reply: tid=0 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.000+0000 7f6600abfb80 1 client.15201 shutdown 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.000+0000 7f6600abfb80 1 -- 172.21.15.46:0/534110540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e360480 msgr2=0x55e93e366210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.000+0000 7f6600abfb80 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e360480 0x55e93e366210 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f65e400af10 tx=0x7f65e400af40 comp rx=0 tx=0).stop 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.000+0000 7f6600abfb80 1 -- 172.21.15.46:0/534110540 shutdown_connections 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.000+0000 7f6600abfb80 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e93e3756a0 0x55e93e377b00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.000+0000 7f6600abfb80 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e360480 0x55e93e366210 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.000+0000 7f6600abfb80 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e1da520 0x55e93e365790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.000+0000 7f6600abfb80 1 --2- 172.21.15.46:0/534110540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e2c41e0 0x55e93e365cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.000+0000 7f65caffd700 0 client.15201 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.000+0000 7f6600abfb80 1 -- 172.21.15.46:0/534110540 >> 172.21.15.46:0/534110540 conn(0x55e93e23a720 msgr2=0x55e93e352240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.001+0000 7f6600abfb80 1 -- 172.21.15.46:0/534110540 shutdown_connections 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.001+0000 7f6600abfb80 1 -- 172.21.15.46:0/534110540 wait complete. 2024-05-09T00:12:05.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.001+0000 7f6600abfb80 20 client.15201 trim_cache size 0 max 16384 2024-05-09T00:12:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.022+0000 7f6600abfb80 1 Processor -- start 2024-05-09T00:12:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.022+0000 7f6600abfb80 1 -- start start 2024-05-09T00:12:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.022+0000 7f6600abfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 0x55e93e357140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.022+0000 7f6600abfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e2c41e0 0x55e93e3710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.022+0000 7f6600abfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e360480 0x55e93e366050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.022+0000 7f6600abfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1af3c0 con 0x55e93e1da520 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f6600abfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1acc50 con 0x55e93e360480 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f6600abfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1aaa80 con 0x55e93e2c41e0 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65eb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 0x55e93e357140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65eb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 0x55e93e357140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59600/0 (socket says 172.21.15.46:59600) 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65eb7fe700 1 -- 172.21.15.46:0/232808627 learned_addr learned my addr 172.21.15.46:0/232808627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65ebfff700 1 --2- 172.21.15.46:0/232808627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e360480 0x55e93e366050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65eaffd700 1 --2- 172.21.15.46:0/232808627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e2c41e0 0x55e93e3710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65eb7fe700 1 -- 172.21.15.46:0/232808627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e2c41e0 msgr2=0x55e93e3710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/232808627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e2c41e0 0x55e93e3710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65eb7fe700 1 -- 172.21.15.46:0/232808627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e360480 msgr2=0x55e93e366050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/232808627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e360480 0x55e93e366050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65eb7fe700 1 -- 172.21.15.46:0/232808627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93e368030 con 0x55e93e1da520 2024-05-09T00:12:05.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/232808627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 0x55e93e357140 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f65dc003fd0 tx=0x7f65dc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c50b5959c41b510 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:05.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65d17fa700 1 -- 172.21.15.46:0/232808627 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc034070 con 0x55e93e1da520 2024-05-09T00:12:05.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65d17fa700 1 -- 172.21.15.46:0/232808627 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65dc02fe50 con 0x55e93e1da520 2024-05-09T00:12:05.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.023+0000 7f65d17fa700 1 -- 172.21.15.46:0/232808627 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00f6c0 con 0x55e93e1da520 2024-05-09T00:12:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.024+0000 7f6600abfb80 1 -- 172.21.15.46:0/232808627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 msgr2=0x55e93e357140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.024+0000 7f6600abfb80 1 --2- 172.21.15.46:0/232808627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 0x55e93e357140 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f65dc003fd0 tx=0x7f65dc01e000 comp rx=0 tx=0).stop 2024-05-09T00:12:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.024+0000 7f6600abfb80 1 -- 172.21.15.46:0/232808627 shutdown_connections 2024-05-09T00:12:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.024+0000 7f6600abfb80 1 --2- 172.21.15.46:0/232808627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e360480 0x55e93e366050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.024+0000 7f6600abfb80 1 --2- 172.21.15.46:0/232808627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e1da520 0x55e93e357140 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.024+0000 7f6600abfb80 1 --2- 172.21.15.46:0/232808627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e2c41e0 0x55e93e3710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.024+0000 7f6600abfb80 1 -- 172.21.15.46:0/232808627 >> 172.21.15.46:0/232808627 conn(0x55e93e23a720 msgr2=0x55e93e361a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.024+0000 7f6600abfb80 1 -- 172.21.15.46:0/232808627 shutdown_connections 2024-05-09T00:12:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.024+0000 7f6600abfb80 1 -- 172.21.15.46:0/232808627 wait complete. 2024-05-09T00:12:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.024+0000 7f6600abfb80 1 -- start start 2024-05-09T00:12:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.025+0000 7f6600abfb80 1 Processor -- start 2024-05-09T00:12:05.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.025+0000 7f6600abfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e1da520 0x55e93e368670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:05.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.025+0000 7f6600abfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e2c41e0 0x55e93e368bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:05.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.025+0000 7f6600abfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e360480 0x55e93e3dfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:05.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.025+0000 7f6600abfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1ab110 con 0x55e93e2c41e0 2024-05-09T00:12:05.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.025+0000 7f6600abfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1ac580 con 0x55e93e1da520 2024-05-09T00:12:05.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.025+0000 7f6600abfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1ab7e0 con 0x55e93e360480 2024-05-09T00:12:05.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.025+0000 7f6600abfb80 10 client.0 _ll_register_callbacks cb 0x55e93e0eb740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:05.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.025+0000 7f65eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e2c41e0 0x55e93e368bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:05.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.025+0000 7f65eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e2c41e0 0x55e93e368bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59616/0 (socket says 172.21.15.46:59616) 2024-05-09T00:12:05.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.025+0000 7f65eaffd700 1 -- 172.21.15.46:0/2272177484 learned_addr learned my addr 172.21.15.46:0/2272177484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:05.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.025+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e1da520 0x55e93e368670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f65ebfff700 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e360480 0x55e93e3dfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f65eaffd700 1 -- 172.21.15.46:0/2272177484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e360480 msgr2=0x55e93e3dfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f65eaffd700 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e360480 0x55e93e3dfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f65eaffd700 1 -- 172.21.15.46:0/2272177484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e1da520 msgr2=0x55e93e368670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f65eaffd700 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e1da520 0x55e93e368670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f65eaffd700 1 -- 172.21.15.46:0/2272177484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93e3e03b0 con 0x55e93e2c41e0 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f65eaffd700 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e2c41e0 0x55e93e368bb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f65e00a65d0 tx=0x7f65e00b1d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c990466f1b88cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f65caffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f65caffd700 1 -- 172.21.15.46:0/2272177484 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e00c8060 con 0x55e93e2c41e0 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f65caffd700 1 -- 172.21.15.46:0/2272177484 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f65e00b1e40 con 0x55e93e2c41e0 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f6600abfb80 1 -- 172.21.15.46:0/2272177484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65dc02fe50 con 0x55e93e2c41e0 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f65caffd700 1 -- 172.21.15.46:0/2272177484 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f65e0020040 con 0x55e93e2c41e0 2024-05-09T00:12:05.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f6600abfb80 10 client.15207 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f6600abfb80 1 -- 172.21.15.46:0/2272177484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e93e3e14d0 con 0x55e93e2c41e0 2024-05-09T00:12:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.026+0000 7f65caffd700 1 -- 172.21.15.46:0/2272177484 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65dc02fe50 con 0x55e93e2c41e0 2024-05-09T00:12:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.027+0000 7f65caffd700 1 -- 172.21.15.46:0/2272177484 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f65e00a8630 con 0x55e93e2c41e0 2024-05-09T00:12:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.027+0000 7f6600abfb80 10 client.15207 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.027+0000 7f6600abfb80 10 client.15207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:12:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.027+0000 7f6600abfb80 20 client.15207 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.027+0000 7f6600abfb80 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e93e3e17b0 0x55e93e3e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.027+0000 7f6600abfb80 4 client.15207 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:12:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.027+0000 7f6600abfb80 1 -- 172.21.15.46:0/2272177484 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e93e281af0 con 0x55e93e3e17b0 2024-05-09T00:12:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.027+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e93e3e17b0 0x55e93e3e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.028+0000 7f65caffd700 10 client.15207 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:05.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.028+0000 7f65eb7fe700 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e93e3e17b0 0x55e93e3e3bd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f8c361c7e4fc09 server_cookie=e5c3b27a8eea04ba in_seq=0 out_seq=0 2024-05-09T00:12:05.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f65caffd700 1 -- 172.21.15.46:0/2272177484 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e93e281af0 con 0x55e93e3e17b0 2024-05-09T00:12:05.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f65caffd700 10 client.15207 handle_command_reply: tid=0 2024-05-09T00:12:05.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f6600abfb80 1 client.15207 shutdown 2024-05-09T00:12:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f6600abfb80 1 -- 172.21.15.46:0/2272177484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e2c41e0 msgr2=0x55e93e368bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f6600abfb80 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e2c41e0 0x55e93e368bb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f65e00a65d0 tx=0x7f65e00b1d30 comp rx=0 tx=0).stop 2024-05-09T00:12:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f6600abfb80 1 -- 172.21.15.46:0/2272177484 shutdown_connections 2024-05-09T00:12:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f6600abfb80 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e93e3e17b0 0x55e93e3e3bd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f6600abfb80 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e93e360480 0x55e93e3dfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f65caffd700 0 client.15207 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f6600abfb80 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e1da520 0x55e93e368670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f6600abfb80 1 --2- 172.21.15.46:0/2272177484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e93e2c41e0 0x55e93e368bb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f6600abfb80 1 -- 172.21.15.46:0/2272177484 >> 172.21.15.46:0/2272177484 conn(0x55e93e23a720 msgr2=0x55e93e362640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f6600abfb80 1 -- 172.21.15.46:0/2272177484 shutdown_connections 2024-05-09T00:12:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.029+0000 7f6600abfb80 1 -- 172.21.15.46:0/2272177484 wait complete. 2024-05-09T00:12:05.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f6600abfb80 20 client.15207 trim_cache size 0 max 16384 2024-05-09T00:12:05.031 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:12:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:12:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "6ea84373-fdda-498f-bebf-64ce18b2f498", 2024-05-09T00:12:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:12:05.031 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:12:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f65f20a9700 1 -- 172.21.15.46:0/3328663436 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f65d40b2900 msgr2=0x7f65d40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f65f20a9700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f65d40b2900 0x7f65d40b4e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f65dc005b40 tx=0x7f65dc01a040 comp rx=0 tx=0).stop 2024-05-09T00:12:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f65f20a9700 1 -- 172.21.15.46:0/3328663436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ec107df0 msgr2=0x7f65ec19b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f65f20a9700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ec107df0 0x7f65ec19b430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f65e000d980 tx=0x7f65e000dc90 comp rx=0 tx=0).stop 2024-05-09T00:12:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f65f20a9700 1 -- 172.21.15.46:0/3328663436 shutdown_connections 2024-05-09T00:12:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f65f20a9700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f65d40b2900 0x7f65d40b4e10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f65f20a9700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65ec108740 0x7f65ec19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f65f20a9700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65ec1034c0 0x7f65ec19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f65f20a9700 1 --2- 172.21.15.46:0/3328663436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ec107df0 0x7f65ec19b430 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f65f20a9700 1 -- 172.21.15.46:0/3328663436 >> 172.21.15.46:0/3328663436 conn(0x7f65ec0fedf0 msgr2=0x7f65ec1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f65f20a9700 1 -- 172.21.15.46:0/3328663436 shutdown_connections 2024-05-09T00:12:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:05.030+0000 7f65f20a9700 1 -- 172.21.15.46:0/3328663436 wait complete. 2024-05-09T00:12:05.826 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:12:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:06.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:06 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:12:06.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:06 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:12:06.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:06 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:12:06.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:06 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:12:06.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:06 smithi046 ceph-mon[30151]: pgmap v398: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-09T00:12:06.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:06 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:12:06.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:06 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:12:06.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:06 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:12:06.691 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:06 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:12:06.691 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:06 smithi142 ceph-mon[34533]: pgmap v398: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-09T00:12:06.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:06 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:12:06.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:06 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:12:06.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:06 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:12:06.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:06 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:12:06.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:06 smithi169 ceph-mon[34777]: pgmap v398: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-09T00:12:07.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.896+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/4000559341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a4103550 msgr2=0x7fe5a4103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:07.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.896+0000 7fe5aa7a9700 1 --2- 172.21.15.46:0/4000559341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a4103550 0x7fe5a4103960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe5940099c0 tx=0x7fe594009cd0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.896+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/4000559341 shutdown_connections 2024-05-09T00:12:07.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.896+0000 7fe5aa7a9700 1 --2- 172.21.15.46:0/4000559341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5a4108780 0x7fe5a41100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.896+0000 7fe5aa7a9700 1 --2- 172.21.15.46:0/4000559341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a4103550 0x7fe5a4103960 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.896+0000 7fe5aa7a9700 1 --2- 172.21.15.46:0/4000559341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4107e30 0x7fe5a4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.896+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/4000559341 >> 172.21.15.46:0/4000559341 conn(0x7fe5a4078e00 msgr2=0x7fe5a4079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.896+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/4000559341 shutdown_connections 2024-05-09T00:12:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.896+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/4000559341 wait complete. 2024-05-09T00:12:07.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.897+0000 7fe5aa7a9700 1 Processor -- start 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.897+0000 7fe5aa7a9700 1 -- start start 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.897+0000 7fe5aa7a9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4103550 0x7fe5a419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.897+0000 7fe5aa7a9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5a4107e30 0x7fe5a419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.897+0000 7fe5aa7a9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a4108780 0x7fe5a419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.897+0000 7fe5aa7a9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41133b0 con 0x7fe5a4108780 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.897+0000 7fe5aa7a9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a4113230 con 0x7fe5a4107e30 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.897+0000 7fe5aa7a9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a4113530 con 0x7fe5a4103550 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.897+0000 7fe5a8d46700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a4108780 0x7fe5a419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.897+0000 7fe5a8d46700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a4108780 0x7fe5a419c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47348/0 (socket says 172.21.15.46:47348) 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.897+0000 7fe5a8d46700 1 -- 172.21.15.46:0/607786994 learned_addr learned my addr 172.21.15.46:0/607786994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.897+0000 7fe59bfff700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5a4107e30 0x7fe5a419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.898+0000 7fe5a3fff700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4103550 0x7fe5a419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.898+0000 7fe5a8d46700 1 -- 172.21.15.46:0/607786994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4103550 msgr2=0x7fe5a419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.898+0000 7fe5a8d46700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4103550 0x7fe5a419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.898+0000 7fe5a8d46700 1 -- 172.21.15.46:0/607786994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5a4107e30 msgr2=0x7fe5a419c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.898+0000 7fe5a8d46700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5a4107e30 0x7fe5a419c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.898+0000 7fe5a8d46700 1 -- 172.21.15.46:0/607786994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a41a6fc0 con 0x7fe5a4108780 2024-05-09T00:12:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.898+0000 7fe5a8d46700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a4108780 0x7fe5a419c710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe59c007ee0 tx=0x7fe59c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.898+0000 7fe5a1ffb700 1 -- 172.21.15.46:0/607786994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00dbd0 con 0x7fe5a4108780 2024-05-09T00:12:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.898+0000 7fe5a1ffb700 1 -- 172.21.15.46:0/607786994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe59c00dd70 con 0x7fe5a4108780 2024-05-09T00:12:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.898+0000 7fe5a1ffb700 1 -- 172.21.15.46:0/607786994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c005990 con 0x7fe5a4108780 2024-05-09T00:12:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.898+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/607786994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a41a72b0 con 0x7fe5a4108780 2024-05-09T00:12:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.898+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/607786994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a41a77a0 con 0x7fe5a4108780 2024-05-09T00:12:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.901+0000 7fe5a1ffb700 1 -- 172.21.15.46:0/607786994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe59c01a020 con 0x7fe5a4108780 2024-05-09T00:12:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.901+0000 7fe5a1ffb700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe5900b2900 0x7fe5900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.902+0000 7fe5a3fff700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe5900b2900 0x7fe5900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.902+0000 7fe5a1ffb700 1 -- 172.21.15.46:0/607786994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe59c0e1980 con 0x7fe5a4108780 2024-05-09T00:12:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.902+0000 7fe5b91bfb80 1 Processor -- start 2024-05-09T00:12:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.902+0000 7fe5b91bfb80 1 -- start start 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.902+0000 7fe5a3fff700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe5900b2900 0x7fe5900b4e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe594003800 tx=0x7fe59401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.902+0000 7fe5b91bfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x556404801ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.902+0000 7fe5b91bfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046804a0 0x556404802030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.902+0000 7fe5b91bfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 0x5564048065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.902+0000 7fe5b91bfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5564047200a0 con 0x5564048061e0 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.902+0000 7fe5b91bfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5564046c59a0 con 0x5564046804a0 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.902+0000 7fe5b91bfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55640471c060 con 0x5564046fad00 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe5a8d46700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 0x5564048065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe5a3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x556404801ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe5a8d46700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 0x5564048065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47354/0 (socket says 172.21.15.46:47354) 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe5a3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x556404801ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:50230/0 (socket says 172.21.15.46:50230) 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe5a8d46700 1 -- 172.21.15.46:0/1601807057 learned_addr learned my addr 172.21.15.46:0/1601807057 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe59bfff700 1 --2- 172.21.15.46:0/1601807057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046804a0 0x556404802030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe5a8d46700 1 -- 172.21.15.46:0/1601807057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 msgr2=0x556404801ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe5a8d46700 1 --2- 172.21.15.46:0/1601807057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x556404801ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe5a8d46700 1 -- 172.21.15.46:0/1601807057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046804a0 msgr2=0x556404802030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe5a8d46700 1 --2- 172.21.15.46:0/1601807057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046804a0 0x556404802030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe5a8d46700 1 -- 172.21.15.46:0/1601807057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564046b6500 con 0x5564048061e0 2024-05-09T00:12:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe5a8d46700 1 --2- 172.21.15.46:0/1601807057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 0x5564048065f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0a6210 tx=0x7fe59c0a8a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472ed96c8cceae7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe59a7fc700 1 -- 172.21.15.46:0/1601807057 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0a96a0 con 0x5564048061e0 2024-05-09T00:12:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe59a7fc700 1 -- 172.21.15.46:0/1601807057 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe59c0a8fa0 con 0x5564048061e0 2024-05-09T00:12:07.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.903+0000 7fe59a7fc700 1 -- 172.21.15.46:0/1601807057 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0b1970 con 0x5564048061e0 2024-05-09T00:12:07.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.904+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/1601807057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 msgr2=0x5564048065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:07.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.904+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/1601807057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 0x5564048065f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0a6210 tx=0x7fe59c0a8a20 comp rx=0 tx=0).stop 2024-05-09T00:12:07.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.904+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/1601807057 shutdown_connections 2024-05-09T00:12:07.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.904+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/1601807057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 0x5564048065f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.904+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/1601807057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x556404801ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.904+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/1601807057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046804a0 0x556404802030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.904+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/1601807057 >> 172.21.15.46:0/1601807057 conn(0x556404683f70 msgr2=0x5564047f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.904+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/1601807057 shutdown_connections 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.904+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/1601807057 wait complete. 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.905+0000 7fe5b91bfb80 1 -- start start 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.905+0000 7fe5b91bfb80 1 Processor -- start 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.905+0000 7fe5b91bfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 0x55640480b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.905+0000 7fe5b91bfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x55640480b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5b91bfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564048061e0 0x55640480bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5b91bfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556404806f00 con 0x5564046804a0 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5b91bfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5564046bab20 con 0x5564048061e0 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5b91bfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55640480e470 con 0x5564046fad00 2024-05-09T00:12:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5b91bfb80 10 client.0 _ll_register_callbacks cb 0x5564045916c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 0x55640480b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 0x55640480b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47362/0 (socket says 172.21.15.46:47362) 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5a3fff700 1 -- 172.21.15.46:0/2334538948 learned_addr learned my addr 172.21.15.46:0/2334538948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe59bfff700 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x55640480b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5a8d46700 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564048061e0 0x55640480bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5a3fff700 1 -- 172.21.15.46:0/2334538948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 msgr2=0x55640480b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5a3fff700 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x55640480b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5a3fff700 1 -- 172.21.15.46:0/2334538948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564048061e0 msgr2=0x55640480bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5a3fff700 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564048061e0 0x55640480bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5a3fff700 1 -- 172.21.15.46:0/2334538948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55640459b490 con 0x5564046804a0 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe59bfff700 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x55640480b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5a3fff700 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 0x55640480b440 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe5940099c0 tx=0x7fe594009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4911a5e026fce44c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5837fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5837fe700 1 -- 172.21.15.46:0/2334538948 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59402e850 con 0x5564046804a0 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5837fe700 1 -- 172.21.15.46:0/2334538948 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe59402e9f0 con 0x5564046804a0 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.906+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/2334538948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe59c0a8fa0 con 0x5564046804a0 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.907+0000 7fe5837fe700 1 -- 172.21.15.46:0/2334538948 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe59400be50 con 0x5564046804a0 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.907+0000 7fe5b91bfb80 10 client.15219 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.907+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/2334538948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556404818dd0 con 0x5564046804a0 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.907+0000 7fe5837fe700 1 -- 172.21.15.46:0/2334538948 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe59c0a8fa0 con 0x5564046804a0 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.907+0000 7fe5837fe700 1 -- 172.21.15.46:0/2334538948 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe594022c90 con 0x5564046804a0 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.907+0000 7fe5b91bfb80 10 client.15219 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.907+0000 7fe5b91bfb80 10 client.15219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.907+0000 7fe5b91bfb80 20 client.15219 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.907+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55640481b4b0 0x55640481d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.907+0000 7fe5b91bfb80 4 client.15219 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.907+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/2334538948 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55640472da80 con 0x55640481b4b0 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.907+0000 7fe59bfff700 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55640481b4b0 0x55640481d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.908+0000 7fe59bfff700 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55640481b4b0 0x55640481d980 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99beff6f2e29492 server_cookie=3170c26a7e15684c in_seq=0 out_seq=0 2024-05-09T00:12:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.908+0000 7fe5837fe700 10 client.15219 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5837fe700 1 -- 172.21.15.46:0/2334538948 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55640472da80 con 0x55640481b4b0 2024-05-09T00:12:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5837fe700 10 client.15219 handle_command_reply: tid=0 2024-05-09T00:12:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5b91bfb80 1 client.15219 shutdown 2024-05-09T00:12:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/2334538948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 msgr2=0x55640480b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 0x55640480b440 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe5940099c0 tx=0x7fe594009310 comp rx=0 tx=0).stop 2024-05-09T00:12:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/2334538948 shutdown_connections 2024-05-09T00:12:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55640481b4b0 0x55640481d980 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564048061e0 0x55640480bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 0x55640480b440 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/2334538948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x55640480b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5837fe700 0 client.15219 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:07.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/2334538948 >> 172.21.15.46:0/2334538948 conn(0x556404683f70 msgr2=0x5564047fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:07.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/2334538948 shutdown_connections 2024-05-09T00:12:07.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/2334538948 wait complete. 2024-05-09T00:12:07.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.909+0000 7fe5b91bfb80 20 client.15219 trim_cache size 0 max 16384 2024-05-09T00:12:07.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5b91bfb80 1 Processor -- start 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5b91bfb80 1 -- start start 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5b91bfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046804a0 0x55640480a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5b91bfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x55640480bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5b91bfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 0x5564047f2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5b91bfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41af280 con 0x5564048061e0 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5b91bfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41acb10 con 0x5564046804a0 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5b91bfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41aa940 con 0x5564046fad00 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5a8d46700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 0x5564047f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5a8d46700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 0x5564047f2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47366/0 (socket says 172.21.15.46:47366) 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5a8d46700 1 -- 172.21.15.46:0/3106897717 learned_addr learned my addr 172.21.15.46:0/3106897717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe59bfff700 1 --2- 172.21.15.46:0/3106897717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x55640480bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5a3fff700 1 --2- 172.21.15.46:0/3106897717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046804a0 0x55640480a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5a8d46700 1 -- 172.21.15.46:0/3106897717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 msgr2=0x55640480bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5a8d46700 1 --2- 172.21.15.46:0/3106897717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x55640480bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5a8d46700 1 -- 172.21.15.46:0/3106897717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046804a0 msgr2=0x55640480a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5a8d46700 1 --2- 172.21.15.46:0/3106897717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046804a0 0x55640480a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.930+0000 7fe5a8d46700 1 -- 172.21.15.46:0/3106897717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564047f9c80 con 0x5564048061e0 2024-05-09T00:12:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.931+0000 7fe5a8d46700 1 --2- 172.21.15.46:0/3106897717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 0x5564047f2700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009460 tx=0x7fe59c019f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fef06bfcb8cf861 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.931+0000 7fe599ffb700 1 -- 172.21.15.46:0/3106897717 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0c7070 con 0x5564048061e0 2024-05-09T00:12:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.931+0000 7fe599ffb700 1 -- 172.21.15.46:0/3106897717 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe59c0ad350 con 0x5564048061e0 2024-05-09T00:12:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.931+0000 7fe599ffb700 1 -- 172.21.15.46:0/3106897717 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0a9460 con 0x5564048061e0 2024-05-09T00:12:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.932+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/3106897717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 msgr2=0x5564047f2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.932+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/3106897717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 0x5564047f2700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009460 tx=0x7fe59c019f80 comp rx=0 tx=0).stop 2024-05-09T00:12:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.932+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/3106897717 shutdown_connections 2024-05-09T00:12:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.932+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/3106897717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564048061e0 0x5564047f2700 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.932+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/3106897717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046804a0 0x55640480a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.932+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/3106897717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564046fad00 0x55640480bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.932+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/3106897717 >> 172.21.15.46:0/3106897717 conn(0x556404683f70 msgr2=0x5564047f1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.932+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/3106897717 shutdown_connections 2024-05-09T00:12:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.932+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/3106897717 wait complete. 2024-05-09T00:12:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.932+0000 7fe5b91bfb80 1 -- start start 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.932+0000 7fe5b91bfb80 1 Processor -- start 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5b91bfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 0x556404803dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5b91bfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046fad00 0x556404804310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5b91bfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564048061e0 0x556404804850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5b91bfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41aafd0 con 0x5564046804a0 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5b91bfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41ac440 con 0x5564046fad00 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5b91bfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41ab6a0 con 0x5564048061e0 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5b91bfb80 10 client.0 _ll_register_callbacks cb 0x5564045916c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 0x556404803dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 0x556404803dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47378/0 (socket says 172.21.15.46:47378) 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5a3fff700 1 -- 172.21.15.46:0/825956185 learned_addr learned my addr 172.21.15.46:0/825956185 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe59bfff700 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046fad00 0x556404804310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5a8d46700 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564048061e0 0x556404804850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5a3fff700 1 -- 172.21.15.46:0/825956185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564048061e0 msgr2=0x556404804850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5a3fff700 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564048061e0 0x556404804850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5a3fff700 1 -- 172.21.15.46:0/825956185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046fad00 msgr2=0x556404804310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5a3fff700 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046fad00 0x556404804310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5a3fff700 1 -- 172.21.15.46:0/825956185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556404884df0 con 0x5564046804a0 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5a3fff700 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 0x556404803dd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe594037500 tx=0x7fe594037f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c857d28b5a235b2b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.933+0000 7fe5837fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5837fe700 1 -- 172.21.15.46:0/825956185 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594049070 con 0x5564046804a0 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5837fe700 1 -- 172.21.15.46:0/825956185 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe59402eba0 con 0x5564046804a0 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/825956185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556404727b20 con 0x5564046804a0 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5837fe700 1 -- 172.21.15.46:0/825956185 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe59402eda0 con 0x5564046804a0 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5b91bfb80 10 client.15225 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/825956185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556404886060 con 0x5564046804a0 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5837fe700 1 -- 172.21.15.46:0/825956185 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556404727b20 con 0x5564046804a0 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5837fe700 1 -- 172.21.15.46:0/825956185 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe59404c030 con 0x5564046804a0 2024-05-09T00:12:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5b91bfb80 10 client.15225 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5b91bfb80 10 client.15225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:12:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5b91bfb80 20 client.15225 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x556404886640 0x556404888a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5b91bfb80 4 client.15225 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:12:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.934+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/825956185 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55640459b970 con 0x556404886640 2024-05-09T00:12:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.935+0000 7fe59bfff700 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x556404886640 0x556404888a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.935+0000 7fe59bfff700 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x556404886640 0x556404888a60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5486b5065d9404ae server_cookie=9b690ae4acf2c027 in_seq=0 out_seq=0 2024-05-09T00:12:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.935+0000 7fe5837fe700 10 client.15225 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.935+0000 7fe5837fe700 1 -- 172.21.15.46:0/825956185 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55640459b970 con 0x556404886640 2024-05-09T00:12:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.935+0000 7fe5837fe700 10 client.15225 handle_command_reply: tid=0 2024-05-09T00:12:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.935+0000 7fe5b91bfb80 1 client.15225 shutdown 2024-05-09T00:12:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/825956185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 msgr2=0x556404803dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:07.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 0x556404803dd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe594037500 tx=0x7fe594037f40 comp rx=0 tx=0).stop 2024-05-09T00:12:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/825956185 shutdown_connections 2024-05-09T00:12:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x556404886640 0x556404888a60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5564048061e0 0x556404804850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564046804a0 0x556404803dd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5b91bfb80 1 --2- 172.21.15.46:0/825956185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564046fad00 0x556404804310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/825956185 >> 172.21.15.46:0/825956185 conn(0x556404683f70 msgr2=0x5564047fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5837fe700 0 client.15225 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/825956185 shutdown_connections 2024-05-09T00:12:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5b91bfb80 1 -- 172.21.15.46:0/825956185 wait complete. 2024-05-09T00:12:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5b91bfb80 20 client.15225 trim_cache size 0 max 16384 2024-05-09T00:12:07.937 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:12:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:12:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T00:12:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "6ea84373-fdda-498f-bebf-64ce18b2f498": { 2024-05-09T00:12:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T00:12:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "6ea84373-fdda-498f-bebf-64ce18b2f498", 2024-05-09T00:12:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T00:12:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:12:07.938 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:12:07.938 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:12:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/607786994 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe5900b2900 msgr2=0x7fe5900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5aa7a9700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe5900b2900 0x7fe5900b4e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe594003800 tx=0x7fe59401a040 comp rx=0 tx=0).stop 2024-05-09T00:12:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/607786994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a4108780 msgr2=0x7fe5a419c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.936+0000 7fe5aa7a9700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a4108780 0x7fe5a419c710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe59c007ee0 tx=0x7fe59c00d3b0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.937+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/607786994 shutdown_connections 2024-05-09T00:12:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.937+0000 7fe5aa7a9700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe5900b2900 0x7fe5900b4e10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.937+0000 7fe5aa7a9700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a4108780 0x7fe5a419c710 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.937+0000 7fe5aa7a9700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4103550 0x7fe5a419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.937+0000 7fe5aa7a9700 1 --2- 172.21.15.46:0/607786994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5a4107e30 0x7fe5a419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.937+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/607786994 >> 172.21.15.46:0/607786994 conn(0x7fe5a4078e00 msgr2=0x7fe5a4101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.937+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/607786994 shutdown_connections 2024-05-09T00:12:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:07.937+0000 7fe5aa7a9700 1 -- 172.21.15.46:0/607786994 wait complete. 2024-05-09T00:12:08.681 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:08 smithi046 ceph-mon[30151]: pgmap v399: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-09T00:12:08.690 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:08 smithi142 ceph-mon[34533]: pgmap v399: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-09T00:12:08.710 INFO:tasks.cephfs.filesystem:scrub status for tag:6ea84373-fdda-498f-bebf-64ce18b2f498 - {'path': '/', 'tag': '6ea84373-fdda-498f-bebf-64ce18b2f498', 'options': 'recursive,force'} 2024-05-09T00:12:08.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:08 smithi169 ceph-mon[34777]: pgmap v399: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-09T00:12:10.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:10 smithi046 ceph-mon[30151]: pgmap v400: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-09T00:12:10.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:10 smithi169 ceph-mon[34777]: pgmap v400: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-09T00:12:10.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:10 smithi142 ceph-mon[34533]: pgmap v400: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-09T00:12:12.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:12 smithi046 ceph-mon[30151]: pgmap v401: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-09T00:12:12.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:12 smithi169 ceph-mon[34777]: pgmap v401: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-09T00:12:12.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:12 smithi142 ceph-mon[34533]: pgmap v401: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-09T00:12:14.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:14 smithi046 ceph-mon[30151]: pgmap v402: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-09T00:12:14.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:14 smithi169 ceph-mon[34777]: pgmap v402: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-09T00:12:14.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:14 smithi142 ceph-mon[34533]: pgmap v402: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-09T00:12:16.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:16 smithi046 ceph-mon[30151]: pgmap v403: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-09T00:12:16.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:16 smithi169 ceph-mon[34777]: pgmap v403: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-09T00:12:16.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:16 smithi142 ceph-mon[34533]: pgmap v403: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-09T00:12:17.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:17 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:12:17.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:17 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:12:17.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:17 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:12:18.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:18 smithi046 ceph-mon[30151]: pgmap v404: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-09T00:12:18.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:18 smithi169 ceph-mon[34777]: pgmap v404: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-09T00:12:18.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:18 smithi142 ceph-mon[34533]: pgmap v404: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-09T00:12:20.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:20 smithi046 ceph-mon[30151]: pgmap v405: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T00:12:20.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:20 smithi169 ceph-mon[34777]: pgmap v405: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T00:12:20.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:20 smithi142 ceph-mon[34533]: pgmap v405: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T00:12:22.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:22 smithi046 ceph-mon[30151]: pgmap v406: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:12:22.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:22 smithi169 ceph-mon[34777]: pgmap v406: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:12:22.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:22 smithi142 ceph-mon[34533]: pgmap v406: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:12:24.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:24 smithi046 ceph-mon[30151]: pgmap v407: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:24.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:24 smithi169 ceph-mon[34777]: pgmap v407: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:24.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:24 smithi142 ceph-mon[34533]: pgmap v407: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:26.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:26 smithi046 ceph-mon[30151]: pgmap v408: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:12:26.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:26 smithi169 ceph-mon[34777]: pgmap v408: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:12:26.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:26 smithi142 ceph-mon[34533]: pgmap v408: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:12:28.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:28 smithi046 ceph-mon[30151]: pgmap v409: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:28.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:28 smithi169 ceph-mon[34777]: pgmap v409: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:28.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:28 smithi142 ceph-mon[34533]: pgmap v409: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:30.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:30 smithi046 ceph-mon[30151]: pgmap v410: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:12:30.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:30 smithi169 ceph-mon[34777]: pgmap v410: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:12:30.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:30 smithi142 ceph-mon[34533]: pgmap v410: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:12:32.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:32 smithi046 ceph-mon[30151]: pgmap v411: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:12:32.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:32 smithi169 ceph-mon[34777]: pgmap v411: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:12:32.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:32 smithi142 ceph-mon[34533]: pgmap v411: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:12:34.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:34 smithi046 ceph-mon[30151]: pgmap v412: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:34.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:34 smithi169 ceph-mon[34777]: pgmap v412: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:34.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:34 smithi142 ceph-mon[34533]: pgmap v412: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:36.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:36 smithi046 ceph-mon[30151]: pgmap v413: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:12:36.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:36 smithi169 ceph-mon[34777]: pgmap v413: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:12:36.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:36 smithi142 ceph-mon[34533]: pgmap v413: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:12:38.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:12:38.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:38 smithi046 ceph-mon[30151]: pgmap v414: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:38.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:38 smithi169 ceph-mon[34777]: pgmap v414: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:38.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:38 smithi142 ceph-mon[34533]: pgmap v414: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:38.981 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:40.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:40 smithi046 ceph-mon[30151]: pgmap v415: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:12:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.782+0000 7f7e36f19700 1 -- 172.21.15.46:0/4206403074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e30108f00 msgr2=0x7f7e30109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.782+0000 7f7e36f19700 1 --2- 172.21.15.46:0/4206403074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e30108f00 0x7f7e30109310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0099c0 tx=0x7f7e1c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.782+0000 7f7e36f19700 1 -- 172.21.15.46:0/4206403074 shutdown_connections 2024-05-09T00:12:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.782+0000 7f7e36f19700 1 --2- 172.21.15.46:0/4206403074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e30109850 0x7f7e301111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.782+0000 7f7e36f19700 1 --2- 172.21.15.46:0/4206403074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e301045d0 0x7f7e301049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.782+0000 7f7e36f19700 1 --2- 172.21.15.46:0/4206403074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e30108f00 0x7f7e30109310 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.782+0000 7f7e36f19700 1 -- 172.21.15.46:0/4206403074 >> 172.21.15.46:0/4206403074 conn(0x7f7e300ffea0 msgr2=0x7f7e301022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.782+0000 7f7e36f19700 1 -- 172.21.15.46:0/4206403074 shutdown_connections 2024-05-09T00:12:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.782+0000 7f7e36f19700 1 -- 172.21.15.46:0/4206403074 wait complete. 2024-05-09T00:12:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.782+0000 7f7e36f19700 1 Processor -- start 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.782+0000 7f7e36f19700 1 -- start start 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e36f19700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e301045d0 0x7f7e3019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e36f19700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e30108f00 0x7f7e3019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e36f19700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e30109850 0x7f7e3019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e36f19700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e301144d0 con 0x7f7e30108f00 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e36f19700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e30114350 con 0x7f7e30109850 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e36f19700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e30114650 con 0x7f7e301045d0 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e30108f00 0x7f7e3019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e30108f00 0x7f7e3019b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58756/0 (socket says 172.21.15.46:58756) 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e2ffff700 1 -- 172.21.15.46:0/4272476510 learned_addr learned my addr 172.21.15.46:0/4272476510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e301045d0 0x7f7e3019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e354b6700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e30109850 0x7f7e3019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e2ffff700 1 -- 172.21.15.46:0/4272476510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e301045d0 msgr2=0x7f7e3019af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e2ffff700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e301045d0 0x7f7e3019af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e2ffff700 1 -- 172.21.15.46:0/4272476510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e30109850 msgr2=0x7f7e3019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e2ffff700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e30109850 0x7f7e3019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e2ffff700 1 -- 172.21.15.46:0/4272476510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3019fd20 con 0x7f7e30108f00 2024-05-09T00:12:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.783+0000 7f7e2ffff700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e30108f00 0x7f7e3019b4a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009f50 tx=0x7f7e1c0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.784+0000 7f7e2dffb700 1 -- 172.21.15.46:0/4272476510 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c027070 con 0x7f7e30108f00 2024-05-09T00:12:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.784+0000 7f7e2dffb700 1 -- 172.21.15.46:0/4272476510 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e1c004090 con 0x7f7e30108f00 2024-05-09T00:12:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.784+0000 7f7e36f19700 1 -- 172.21.15.46:0/4272476510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3019ffb0 con 0x7f7e30108f00 2024-05-09T00:12:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.784+0000 7f7e2dffb700 1 -- 172.21.15.46:0/4272476510 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c022e60 con 0x7f7e30108f00 2024-05-09T00:12:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.784+0000 7f7e36f19700 1 -- 172.21.15.46:0/4272476510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e301a04e0 con 0x7f7e30108f00 2024-05-09T00:12:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.787+0000 7f7e2dffb700 1 -- 172.21.15.46:0/4272476510 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7e1c02a070 con 0x7f7e30108f00 2024-05-09T00:12:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e2dffb700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e200b2900 0x7f7e200b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e4592fb80 1 Processor -- start 2024-05-09T00:12:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e200b2900 0x7f7e200b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e4592fb80 1 -- start start 2024-05-09T00:12:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e2dffb700 1 -- 172.21.15.46:0/4272476510 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7e1c0e2eb0 con 0x7f7e30108f00 2024-05-09T00:12:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e4592fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8062b160 0x55ed806c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e4592fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed805414a0 0x55ed806c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e4592fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed806be6a0 0x55ed806bead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e4592fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ed805e0ff0 con 0x55ed8062b160 2024-05-09T00:12:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e4592fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ed8057f7f0 con 0x55ed805414a0 2024-05-09T00:12:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e4592fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ed805dcfb0 con 0x55ed806be6a0 2024-05-09T00:12:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e34cb5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8062b160 0x55ed806c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e34cb5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8062b160 0x55ed806c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58760/0 (socket says 172.21.15.46:58760) 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e34cb5700 1 -- 172.21.15.46:0/1939270930 learned_addr learned my addr 172.21.15.46:0/1939270930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e354b6700 1 --2- 172.21.15.46:0/1939270930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed806be6a0 0x55ed806bead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.788+0000 7f7e2ffff700 1 --2- 172.21.15.46:0/1939270930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed805414a0 0x55ed806c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.789+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e200b2900 0x7f7e200b4e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009f80 tx=0x7f7e24015690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.789+0000 7f7e34cb5700 1 -- 172.21.15.46:0/1939270930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed806be6a0 msgr2=0x55ed806bead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.789+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/1939270930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed806be6a0 0x55ed806bead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.789+0000 7f7e34cb5700 1 -- 172.21.15.46:0/1939270930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed805414a0 msgr2=0x55ed806c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.789+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/1939270930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed805414a0 0x55ed806c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.789+0000 7f7e34cb5700 1 -- 172.21.15.46:0/1939270930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed80577370 con 0x55ed8062b160 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.789+0000 7f7e354b6700 1 --2- 172.21.15.46:0/1939270930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed806be6a0 0x55ed806bead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.789+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/1939270930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8062b160 0x55ed806c44f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7e24015a90 tx=0x7f7e2401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7989acc59536c680 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.789+0000 7f7e1affd700 1 -- 172.21.15.46:0/1939270930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24018d90 con 0x55ed8062b160 2024-05-09T00:12:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.789+0000 7f7e1affd700 1 -- 172.21.15.46:0/1939270930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e240128d0 con 0x55ed8062b160 2024-05-09T00:12:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.789+0000 7f7e1affd700 1 -- 172.21.15.46:0/1939270930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24023830 con 0x55ed8062b160 2024-05-09T00:12:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.790+0000 7f7e4592fb80 1 -- 172.21.15.46:0/1939270930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8062b160 msgr2=0x55ed806c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.790+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/1939270930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8062b160 0x55ed806c44f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7e24015a90 tx=0x7f7e2401d000 comp rx=0 tx=0).stop 2024-05-09T00:12:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.790+0000 7f7e4592fb80 1 -- 172.21.15.46:0/1939270930 shutdown_connections 2024-05-09T00:12:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.790+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/1939270930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed806be6a0 0x55ed806bead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.790+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/1939270930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed8062b160 0x55ed806c44f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.790+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/1939270930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed805414a0 0x55ed806c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.790+0000 7f7e4592fb80 1 -- 172.21.15.46:0/1939270930 >> 172.21.15.46:0/1939270930 conn(0x55ed805a16a0 msgr2=0x55ed806b9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.790+0000 7f7e4592fb80 1 -- 172.21.15.46:0/1939270930 shutdown_connections 2024-05-09T00:12:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.790+0000 7f7e4592fb80 1 -- 172.21.15.46:0/1939270930 wait complete. 2024-05-09T00:12:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.791+0000 7f7e4592fb80 1 -- start start 2024-05-09T00:12:40.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.791+0000 7f7e4592fb80 1 Processor -- start 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.791+0000 7f7e4592fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 0x55ed806c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.791+0000 7f7e4592fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed8062b160 0x55ed806c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.791+0000 7f7e4592fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed806be6a0 0x55ed806c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.791+0000 7f7e4592fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ed8045cba0 con 0x55ed805414a0 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.791+0000 7f7e4592fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ed8059b150 con 0x55ed806be6a0 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.791+0000 7f7e4592fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ed806bf4e0 con 0x55ed8062b160 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.791+0000 7f7e4592fb80 10 client.0 _ll_register_callbacks cb 0x55ed804526c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.791+0000 7f7e34cb5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 0x55ed806c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.791+0000 7f7e34cb5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 0x55ed806c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58762/0 (socket says 172.21.15.46:58762) 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.791+0000 7f7e34cb5700 1 -- 172.21.15.46:0/2952577604 learned_addr learned my addr 172.21.15.46:0/2952577604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e354b6700 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed806be6a0 0x55ed806c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e2ffff700 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed8062b160 0x55ed806c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e34cb5700 1 -- 172.21.15.46:0/2952577604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed8062b160 msgr2=0x55ed806c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed8062b160 0x55ed806c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e34cb5700 1 -- 172.21.15.46:0/2952577604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed806be6a0 msgr2=0x55ed806c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed806be6a0 0x55ed806c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e34cb5700 1 -- 172.21.15.46:0/2952577604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8045c490 con 0x55ed805414a0 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e2ffff700 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed8062b160 0x55ed806c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 0x55ed806c81a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7e240098b0 tx=0x7f7e24004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fdec391f4936cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e0ffff700 1 -- 172.21.15.46:0/2952577604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24032020 con 0x55ed805414a0 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e4592fb80 1 -- 172.21.15.46:0/2952577604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e240128d0 con 0x55ed805414a0 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e0ffff700 1 -- 172.21.15.46:0/2952577604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7e2400f040 con 0x55ed805414a0 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e0ffff700 1 -- 172.21.15.46:0/2952577604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e24012710 con 0x55ed805414a0 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e4592fb80 10 client.15237 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.792+0000 7f7e4592fb80 1 -- 172.21.15.46:0/2952577604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed806d5b00 con 0x55ed805414a0 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.793+0000 7f7e0ffff700 1 -- 172.21.15.46:0/2952577604 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e240128d0 con 0x55ed805414a0 2024-05-09T00:12:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.793+0000 7f7e0ffff700 1 -- 172.21.15.46:0/2952577604 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7e2402d450 con 0x55ed805414a0 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.793+0000 7f7e4592fb80 10 client.15237 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.793+0000 7f7e4592fb80 10 client.15237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.793+0000 7f7e4592fb80 20 client.15237 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.793+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ed806d8030 0x55ed806da450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.793+0000 7f7e4592fb80 4 client.15237 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.793+0000 7f7e4592fb80 1 -- 172.21.15.46:0/2952577604 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed805ee9d0 con 0x55ed806d8030 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.793+0000 7f7e2ffff700 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ed806d8030 0x55ed806da450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.794+0000 7f7e2ffff700 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ed806d8030 0x55ed806da450 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c8a4876034b805 server_cookie=40ffbf4ba8d23798 in_seq=0 out_seq=0 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.794+0000 7f7e0ffff700 10 client.15237 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.794+0000 7f7e0ffff700 1 -- 172.21.15.46:0/2952577604 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ed805ee9d0 con 0x55ed806d8030 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.794+0000 7f7e0ffff700 10 client.15237 handle_command_reply: tid=0 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e4592fb80 1 client.15237 shutdown 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e4592fb80 1 -- 172.21.15.46:0/2952577604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 msgr2=0x55ed806c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:40.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 0x55ed806c81a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7e240098b0 tx=0x7f7e24004860 comp rx=0 tx=0).stop 2024-05-09T00:12:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e4592fb80 1 -- 172.21.15.46:0/2952577604 shutdown_connections 2024-05-09T00:12:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ed806d8030 0x55ed806da450 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed806be6a0 0x55ed806c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 0x55ed806c81a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e0ffff700 0 client.15237 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/2952577604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed8062b160 0x55ed806c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e4592fb80 1 -- 172.21.15.46:0/2952577604 >> 172.21.15.46:0/2952577604 conn(0x55ed805a16a0 msgr2=0x55ed806b91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e4592fb80 1 -- 172.21.15.46:0/2952577604 shutdown_connections 2024-05-09T00:12:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e4592fb80 1 -- 172.21.15.46:0/2952577604 wait complete. 2024-05-09T00:12:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.795+0000 7f7e4592fb80 20 client.15237 trim_cache size 0 max 16384 2024-05-09T00:12:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e4592fb80 1 Processor -- start 2024-05-09T00:12:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e4592fb80 1 -- start start 2024-05-09T00:12:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e4592fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 0x55ed806be0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e4592fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8062b160 0x55ed806d42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e4592fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed806be6a0 0x55ed806c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e4592fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e301af1f0 con 0x55ed805414a0 2024-05-09T00:12:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e4592fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e301aca80 con 0x55ed8062b160 2024-05-09T00:12:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e4592fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e301aa8d0 con 0x55ed806be6a0 2024-05-09T00:12:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e34cb5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 0x55ed806be0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e34cb5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 0x55ed806be0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58766/0 (socket says 172.21.15.46:58766) 2024-05-09T00:12:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e34cb5700 1 -- 172.21.15.46:0/997722369 learned_addr learned my addr 172.21.15.46:0/997722369 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e34cb5700 1 -- 172.21.15.46:0/997722369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed806be6a0 msgr2=0x55ed806c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/997722369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed806be6a0 0x55ed806c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e34cb5700 1 -- 172.21.15.46:0/997722369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8062b160 msgr2=0x55ed806d42b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T00:12:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/997722369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8062b160 0x55ed806d42b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.816+0000 7f7e34cb5700 1 -- 172.21.15.46:0/997722369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed806ba110 con 0x55ed805414a0 2024-05-09T00:12:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.817+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/997722369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 0x55ed806be0c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7e24006fa0 tx=0x7f7e24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aed4ae335ec7ab9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.817+0000 7f7e1a7fc700 1 -- 172.21.15.46:0/997722369 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24046070 con 0x55ed805414a0 2024-05-09T00:12:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.817+0000 7f7e1a7fc700 1 -- 172.21.15.46:0/997722369 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e240128b0 con 0x55ed805414a0 2024-05-09T00:12:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.817+0000 7f7e1a7fc700 1 -- 172.21.15.46:0/997722369 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24032050 con 0x55ed805414a0 2024-05-09T00:12:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.817+0000 7f7e4592fb80 1 -- 172.21.15.46:0/997722369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 msgr2=0x55ed806be0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.817+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/997722369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 0x55ed806be0c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7e24006fa0 tx=0x7f7e24009840 comp rx=0 tx=0).stop 2024-05-09T00:12:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.818+0000 7f7e4592fb80 1 -- 172.21.15.46:0/997722369 shutdown_connections 2024-05-09T00:12:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.818+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/997722369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed806be6a0 0x55ed806c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.818+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/997722369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed805414a0 0x55ed806be0c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.818+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/997722369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8062b160 0x55ed806d42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.818+0000 7f7e4592fb80 1 -- 172.21.15.46:0/997722369 >> 172.21.15.46:0/997722369 conn(0x55ed805a16a0 msgr2=0x55ed806c3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.818+0000 7f7e4592fb80 1 -- 172.21.15.46:0/997722369 shutdown_connections 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.818+0000 7f7e4592fb80 1 -- 172.21.15.46:0/997722369 wait complete. 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.818+0000 7f7e4592fb80 1 -- start start 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.818+0000 7f7e4592fb80 1 Processor -- start 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e4592fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed805414a0 0x55ed806db710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e4592fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8062b160 0x55ed806dbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e4592fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed806be6a0 0x55ed80741580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e4592fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e301aaf40 con 0x55ed806be6a0 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e4592fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e301ac3b0 con 0x55ed8062b160 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e4592fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e301ab610 con 0x55ed805414a0 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e4592fb80 10 client.0 _ll_register_callbacks cb 0x55ed806c3ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e354b6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed806be6a0 0x55ed80741580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e354b6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed806be6a0 0x55ed80741580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58780/0 (socket says 172.21.15.46:58780) 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e354b6700 1 -- 172.21.15.46:0/716897717 learned_addr learned my addr 172.21.15.46:0/716897717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e2ffff700 1 --2- 172.21.15.46:0/716897717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8062b160 0x55ed806dbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e354b6700 1 -- 172.21.15.46:0/716897717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed805414a0 msgr2=0x55ed806db710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e354b6700 1 --2- 172.21.15.46:0/716897717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed805414a0 0x55ed806db710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e354b6700 1 -- 172.21.15.46:0/716897717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8062b160 msgr2=0x55ed806dbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e354b6700 1 --2- 172.21.15.46:0/716897717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8062b160 0x55ed806dbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e354b6700 1 -- 172.21.15.46:0/716897717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed806b2f60 con 0x55ed806be6a0 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e354b6700 1 --2- 172.21.15.46:0/716897717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed806be6a0 0x55ed80741580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7e28013d80 tx=0x7f7e28011530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afef75606464744 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e0ffff700 1 -- 172.21.15.46:0/716897717 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2801c070 con 0x55ed806be6a0 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e4592fb80 1 -- 172.21.15.46:0/716897717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e240128b0 con 0x55ed806be6a0 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.819+0000 7f7e0ffff700 1 -- 172.21.15.46:0/716897717 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7e28004bb0 con 0x55ed806be6a0 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.820+0000 7f7e0ffff700 1 -- 172.21.15.46:0/716897717 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e28004df0 con 0x55ed806be6a0 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.820+0000 7f7e4592fb80 10 client.15243 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.820+0000 7f7e4592fb80 1 -- 172.21.15.46:0/716897717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed80742ae0 con 0x55ed806be6a0 2024-05-09T00:12:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.820+0000 7f7e0ffff700 1 -- 172.21.15.46:0/716897717 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e240128b0 con 0x55ed806be6a0 2024-05-09T00:12:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.820+0000 7f7e0ffff700 1 -- 172.21.15.46:0/716897717 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7e280041e0 con 0x55ed806be6a0 2024-05-09T00:12:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.820+0000 7f7e4592fb80 10 client.15243 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.820+0000 7f7e4592fb80 10 client.15243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:12:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.820+0000 7f7e4592fb80 20 client.15243 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.820+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/716897717 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ed807431e0 0x55ed80745600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.820+0000 7f7e4592fb80 4 client.15243 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:12:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.820+0000 7f7e4592fb80 1 -- 172.21.15.46:0/716897717 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed805e8a70 con 0x55ed807431e0 2024-05-09T00:12:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.822+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/716897717 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ed807431e0 0x55ed80745600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.822+0000 7f7e34cb5700 1 --2- 172.21.15.46:0/716897717 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ed807431e0 0x55ed80745600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91cc6707bdd14169 server_cookie=eed4cf8defbd702a in_seq=0 out_seq=0 2024-05-09T00:12:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.822+0000 7f7e0ffff700 10 client.15243 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e0ffff700 1 -- 172.21.15.46:0/716897717 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ed805e8a70 con 0x55ed807431e0 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e0ffff700 10 client.15243 handle_command_reply: tid=0 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e4592fb80 1 client.15243 shutdown 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e4592fb80 1 -- 172.21.15.46:0/716897717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed806be6a0 msgr2=0x55ed80741580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/716897717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed806be6a0 0x55ed80741580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7e28013d80 tx=0x7f7e28011530 comp rx=0 tx=0).stop 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e4592fb80 1 -- 172.21.15.46:0/716897717 shutdown_connections 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/716897717 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ed807431e0 0x55ed80745600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/716897717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ed806be6a0 0x55ed80741580 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/716897717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ed805414a0 0x55ed806db710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e0ffff700 0 client.15243 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e4592fb80 1 --2- 172.21.15.46:0/716897717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed8062b160 0x55ed806dbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e4592fb80 1 -- 172.21.15.46:0/716897717 >> 172.21.15.46:0/716897717 conn(0x55ed805a16a0 msgr2=0x55ed806c3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e4592fb80 1 -- 172.21.15.46:0/716897717 shutdown_connections 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e4592fb80 1 -- 172.21.15.46:0/716897717 wait complete. 2024-05-09T00:12:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.823+0000 7f7e4592fb80 20 client.15243 trim_cache size 0 max 16384 2024-05-09T00:12:40.824 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:12:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:12:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:12:40.824 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:12:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.824+0000 7f7e36f19700 1 -- 172.21.15.46:0/4272476510 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e200b2900 msgr2=0x7f7e200b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.824+0000 7f7e36f19700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e200b2900 0x7f7e200b4e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009f80 tx=0x7f7e24015690 comp rx=0 tx=0).stop 2024-05-09T00:12:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.824+0000 7f7e36f19700 1 -- 172.21.15.46:0/4272476510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e30108f00 msgr2=0x7f7e3019b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.824+0000 7f7e36f19700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e30108f00 0x7f7e3019b4a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009f50 tx=0x7f7e1c0037c0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.824+0000 7f7e36f19700 1 -- 172.21.15.46:0/4272476510 shutdown_connections 2024-05-09T00:12:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.824+0000 7f7e36f19700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e200b2900 0x7f7e200b4e10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.824+0000 7f7e36f19700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e30109850 0x7f7e3019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.824+0000 7f7e36f19700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e301045d0 0x7f7e3019af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.824+0000 7f7e36f19700 1 --2- 172.21.15.46:0/4272476510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e30108f00 0x7f7e3019b4a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.824+0000 7f7e36f19700 1 -- 172.21.15.46:0/4272476510 >> 172.21.15.46:0/4272476510 conn(0x7f7e300ffea0 msgr2=0x7f7e3010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.824+0000 7f7e36f19700 1 -- 172.21.15.46:0/4272476510 shutdown_connections 2024-05-09T00:12:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:40.824+0000 7f7e36f19700 1 -- 172.21.15.46:0/4272476510 wait complete. 2024-05-09T00:12:40.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:40 smithi169 ceph-mon[34777]: pgmap v415: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:12:40.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:40 smithi142 ceph-mon[34533]: pgmap v415: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:12:41.657 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:12:41.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:12:41.913 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:42.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:42 smithi046 ceph-mon[30151]: pgmap v416: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:12:42.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:42 smithi169 ceph-mon[34777]: pgmap v416: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:12:42.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:42 smithi142 ceph-mon[34533]: pgmap v416: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:12:43.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.739+0000 7f289ee70700 1 -- 172.21.15.46:0/1808742240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2898076e40 msgr2=0x7f2898077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:43.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.739+0000 7f289ee70700 1 --2- 172.21.15.46:0/1808742240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2898076e40 0x7f2898077250 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f288c009990 tx=0x7f288c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:12:43.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.739+0000 7f289ee70700 1 -- 172.21.15.46:0/1808742240 shutdown_connections 2024-05-09T00:12:43.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.739+0000 7f289ee70700 1 --2- 172.21.15.46:0/1808742240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2898110ba0 0x7f2898113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:43.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.739+0000 7f289ee70700 1 --2- 172.21.15.46:0/1808742240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2898076e40 0x7f2898077250 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:43.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.739+0000 7f289ee70700 1 --2- 172.21.15.46:0/1808742240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2898077790 0x7f2898110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:43.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.739+0000 7f289ee70700 1 -- 172.21.15.46:0/1808742240 >> 172.21.15.46:0/1808742240 conn(0x7f28980ffec0 msgr2=0x7f2898102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:43.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.739+0000 7f289ee70700 1 -- 172.21.15.46:0/1808742240 shutdown_connections 2024-05-09T00:12:43.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.739+0000 7f289ee70700 1 -- 172.21.15.46:0/1808742240 wait complete. 2024-05-09T00:12:43.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.740+0000 7f289ee70700 1 Processor -- start 2024-05-09T00:12:43.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.740+0000 7f289ee70700 1 -- start start 2024-05-09T00:12:43.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.740+0000 7f289ee70700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2898076e40 0x7f289819d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:43.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.740+0000 7f289ee70700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2898077790 0x7f289819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:43.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.740+0000 7f289ee70700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2898110ba0 0x7f28981a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:43.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.740+0000 7f289ee70700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2898116790 con 0x7f2898077790 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.740+0000 7f289ee70700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2898116610 con 0x7f2898076e40 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.740+0000 7f289ee70700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2898116910 con 0x7f2898110ba0 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.740+0000 7f2897fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2898077790 0x7f289819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f2897fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2898077790 0x7f289819d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58794/0 (socket says 172.21.15.46:58794) 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f2897fff700 1 -- 172.21.15.46:0/3296514432 learned_addr learned my addr 172.21.15.46:0/3296514432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f289d40d700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2898110ba0 0x7f28981a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f2897fff700 1 -- 172.21.15.46:0/3296514432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2898110ba0 msgr2=0x7f28981a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f2897fff700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2898110ba0 0x7f28981a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f2897fff700 1 -- 172.21.15.46:0/3296514432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2898076e40 msgr2=0x7f289819d180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f2897fff700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2898076e40 0x7f289819d180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f2897fff700 1 -- 172.21.15.46:0/3296514432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28981a2050 con 0x7f2898077790 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f2897fff700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2898077790 0x7f289819d720 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f288400b530 tx=0x7f288400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f2895ffb700 1 -- 172.21.15.46:0/3296514432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884015020 con 0x7f2898077790 2024-05-09T00:12:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f2895ffb700 1 -- 172.21.15.46:0/3296514432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2884011ce0 con 0x7f2898077790 2024-05-09T00:12:43.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f2895ffb700 1 -- 172.21.15.46:0/3296514432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884010d20 con 0x7f2898077790 2024-05-09T00:12:43.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.741+0000 7f289ee70700 1 -- 172.21.15.46:0/3296514432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28981a2340 con 0x7f2898077790 2024-05-09T00:12:43.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.742+0000 7f289ee70700 1 -- 172.21.15.46:0/3296514432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28981a9c60 con 0x7f2898077790 2024-05-09T00:12:43.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.743+0000 7f289ee70700 1 -- 172.21.15.46:0/3296514432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28980512c0 con 0x7f2898077790 2024-05-09T00:12:43.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.744+0000 7f2895ffb700 1 -- 172.21.15.46:0/3296514432 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2884018020 con 0x7f2898077790 2024-05-09T00:12:43.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.745+0000 7f2895ffb700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28880b2900 0x7f28880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:43.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.745+0000 7f289cc0c700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28880b2900 0x7f28880b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:43.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.745+0000 7f2895ffb700 1 -- 172.21.15.46:0/3296514432 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f28840e1a20 con 0x7f2898077790 2024-05-09T00:12:43.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.746+0000 7f289cc0c700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28880b2900 0x7f28880b4e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f288c005f50 tx=0x7f288c005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:43.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:43.750+0000 7f2895ffb700 1 -- 172.21.15.46:0/3296514432 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f288401f020 con 0x7f2898077790 2024-05-09T00:12:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.010+0000 7f289ee70700 1 -- 172.21.15.46:0/3296514432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f289819e4e0 con 0x7f2898077790 2024-05-09T00:12:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.011+0000 7f2895ffb700 1 -- 172.21.15.46:0/3296514432 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f28840a9700 con 0x7f2898077790 2024-05-09T00:12:44.009 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:12:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:12:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.014+0000 7f289ee70700 1 -- 172.21.15.46:0/3296514432 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28880b2900 msgr2=0x7f28880b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.014+0000 7f289ee70700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28880b2900 0x7f28880b4e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f288c005f50 tx=0x7f288c005ea0 comp rx=0 tx=0).stop 2024-05-09T00:12:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.014+0000 7f289ee70700 1 -- 172.21.15.46:0/3296514432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2898077790 msgr2=0x7f289819d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.014+0000 7f289ee70700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2898077790 0x7f289819d720 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f288400b530 tx=0x7f288400b8f0 comp rx=0 tx=0).stop 2024-05-09T00:12:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.014+0000 7f289ee70700 1 -- 172.21.15.46:0/3296514432 shutdown_connections 2024-05-09T00:12:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.014+0000 7f289ee70700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28880b2900 0x7f28880b4e10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.014+0000 7f289ee70700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2898110ba0 0x7f28981a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.014+0000 7f289ee70700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2898076e40 0x7f289819d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.014+0000 7f289ee70700 1 --2- 172.21.15.46:0/3296514432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2898077790 0x7f289819d720 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.014+0000 7f289ee70700 1 -- 172.21.15.46:0/3296514432 >> 172.21.15.46:0/3296514432 conn(0x7f28980ffec0 msgr2=0x7f28981005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.014+0000 7f289ee70700 1 -- 172.21.15.46:0/3296514432 shutdown_connections 2024-05-09T00:12:44.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:44.014+0000 7f289ee70700 1 -- 172.21.15.46:0/3296514432 wait complete. 2024-05-09T00:12:44.013 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:12:44.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:44 smithi046 ceph-mon[30151]: pgmap v417: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T00:12:44.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:44 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3296514432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:12:44.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:44 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:12:44.850 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:12:44.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:44 smithi169 ceph-mon[34777]: pgmap v417: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T00:12:44.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:44 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3296514432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:12:44.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:44 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:12:44.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:44 smithi142 ceph-mon[34533]: pgmap v417: 97 pgs: 97 active+clean; 652 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T00:12:44.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:44 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3296514432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:12:44.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:44 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:12:45.109 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:45.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:45 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:12:45.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:45 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:12:45.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:45 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:12:45.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:45 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:12:45.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:45 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:12:45.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:45 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:12:45.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:45 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:12:45.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:45 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:12:45.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:45 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:12:46.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:46 smithi046 ceph-mon[30151]: pgmap v418: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T00:12:46.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:46 smithi169 ceph-mon[34777]: pgmap v418: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T00:12:46.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:46 smithi142 ceph-mon[34533]: pgmap v418: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T00:12:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.965+0000 7f9776c48700 1 -- 172.21.15.46:0/209323943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9770108740 msgr2=0x7f97701100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.965+0000 7f9776c48700 1 --2- 172.21.15.46:0/209323943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9770108740 0x7f97701100a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f976800b0d0 tx=0x7f976800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.965+0000 7f9776c48700 1 -- 172.21.15.46:0/209323943 shutdown_connections 2024-05-09T00:12:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.965+0000 7f9776c48700 1 --2- 172.21.15.46:0/209323943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9770108740 0x7f97701100a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.965+0000 7f9776c48700 1 --2- 172.21.15.46:0/209323943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97701034c0 0x7f97701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.965+0000 7f9776c48700 1 --2- 172.21.15.46:0/209323943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9770107df0 0x7f9770108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.965+0000 7f9776c48700 1 -- 172.21.15.46:0/209323943 >> 172.21.15.46:0/209323943 conn(0x7f97700fedc0 msgr2=0x7f97701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:46.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.965+0000 7f9776c48700 1 -- 172.21.15.46:0/209323943 shutdown_connections 2024-05-09T00:12:46.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.965+0000 7f9776c48700 1 -- 172.21.15.46:0/209323943 wait complete. 2024-05-09T00:12:46.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f9776c48700 1 Processor -- start 2024-05-09T00:12:46.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f9776c48700 1 -- start start 2024-05-09T00:12:46.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f9776c48700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97701034c0 0x7f977019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f9776c48700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9770107df0 0x7f977019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f9776c48700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9770108740 0x7f977019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f9776c48700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97701144f0 con 0x7f9770107df0 2024-05-09T00:12:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f9776c48700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9770114370 con 0x7f9770108740 2024-05-09T00:12:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f9776c48700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9770114670 con 0x7f97701034c0 2024-05-09T00:12:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f976ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9770107df0 0x7f977019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f976ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9770107df0 0x7f977019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33968/0 (socket says 172.21.15.46:33968) 2024-05-09T00:12:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f976ffff700 1 -- 172.21.15.46:0/1798724899 learned_addr learned my addr 172.21.15.46:0/1798724899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f97751e5700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9770108740 0x7f977019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.966+0000 7f97749e4700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97701034c0 0x7f977019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.967+0000 7f976ffff700 1 -- 172.21.15.46:0/1798724899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97701034c0 msgr2=0x7f977019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.967+0000 7f976ffff700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97701034c0 0x7f977019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.967+0000 7f976ffff700 1 -- 172.21.15.46:0/1798724899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9770108740 msgr2=0x7f977019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.967+0000 7f976ffff700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9770108740 0x7f977019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.967+0000 7f976ffff700 1 -- 172.21.15.46:0/1798724899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977019fcd0 con 0x7f9770107df0 2024-05-09T00:12:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.967+0000 7f97749e4700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97701034c0 0x7f977019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.967+0000 7f976ffff700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9770107df0 0x7f977019b3a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f975c00c8f0 tx=0x7f975c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.967+0000 7f976dffb700 1 -- 172.21.15.46:0/1798724899 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c007940 con 0x7f9770107df0 2024-05-09T00:12:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.967+0000 7f976dffb700 1 -- 172.21.15.46:0/1798724899 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c00f450 con 0x7f9770107df0 2024-05-09T00:12:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.967+0000 7f9776c48700 1 -- 172.21.15.46:0/1798724899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977019ffc0 con 0x7f9770107df0 2024-05-09T00:12:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.967+0000 7f976dffb700 1 -- 172.21.15.46:0/1798724899 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c018660 con 0x7f9770107df0 2024-05-09T00:12:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.967+0000 7f9776c48700 1 -- 172.21.15.46:0/1798724899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97701a78e0 con 0x7f9770107df0 2024-05-09T00:12:46.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.970+0000 7f976dffb700 1 -- 172.21.15.46:0/1798724899 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f975c00f890 con 0x7f9770107df0 2024-05-09T00:12:46.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.971+0000 7f976dffb700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97600b2880 0x7f97600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.971+0000 7f97749e4700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97600b2880 0x7f97600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.971+0000 7f976dffb700 1 -- 172.21.15.46:0/1798724899 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f975c0e20f0 con 0x7f9770107df0 2024-05-09T00:12:46.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.971+0000 7f978565eb80 1 Processor -- start 2024-05-09T00:12:46.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f978565eb80 1 -- start start 2024-05-09T00:12:46.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f97749e4700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97600b2880 0x7f97600b4d90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f97640099d0 tx=0x7f9764008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f978565eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edfa9b160 0x559edfb38900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f978565eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edf9b14a0 0x559edfb38e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f978565eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 0x559edfb32ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f978565eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559edfa50ff0 con 0x559edfb32ab0 2024-05-09T00:12:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f978565eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559edf9ef7f0 con 0x559edf9b14a0 2024-05-09T00:12:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f978565eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559edfa4cfb0 con 0x559edfa9b160 2024-05-09T00:12:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f97751e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 0x559edfb32ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f97749e4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edfa9b160 0x559edfb38900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f97751e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 0x559edfb32ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33974/0 (socket says 172.21.15.46:33974) 2024-05-09T00:12:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f97749e4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edfa9b160 0x559edfb38900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:42378/0 (socket says 172.21.15.46:42378) 2024-05-09T00:12:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f97751e5700 1 -- 172.21.15.46:0/4087060914 learned_addr learned my addr 172.21.15.46:0/4087060914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.972+0000 7f976ffff700 1 --2- 172.21.15.46:0/4087060914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edf9b14a0 0x559edfb38e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.973+0000 7f97751e5700 1 -- 172.21.15.46:0/4087060914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edfa9b160 msgr2=0x559edfb38900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.973+0000 7f97751e5700 1 --2- 172.21.15.46:0/4087060914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edfa9b160 0x559edfb38900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.973+0000 7f97751e5700 1 -- 172.21.15.46:0/4087060914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edf9b14a0 msgr2=0x559edfb38e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.973+0000 7f97751e5700 1 --2- 172.21.15.46:0/4087060914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edf9b14a0 0x559edfb38e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.973+0000 7f97751e5700 1 -- 172.21.15.46:0/4087060914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559edf9e7370 con 0x559edfb32ab0 2024-05-09T00:12:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.973+0000 7f97751e5700 1 --2- 172.21.15.46:0/4087060914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 0x559edfb32ee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9768005b80 tx=0x7f97680090d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f11170139b8d9a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.973+0000 7f975affd700 1 -- 172.21.15.46:0/4087060914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768003a00 con 0x559edfb32ab0 2024-05-09T00:12:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.973+0000 7f975affd700 1 -- 172.21.15.46:0/4087060914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9768003ba0 con 0x559edfb32ab0 2024-05-09T00:12:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.973+0000 7f975affd700 1 -- 172.21.15.46:0/4087060914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97680052b0 con 0x559edfb32ab0 2024-05-09T00:12:46.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.974+0000 7f978565eb80 1 -- 172.21.15.46:0/4087060914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 msgr2=0x559edfb32ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.974+0000 7f978565eb80 1 --2- 172.21.15.46:0/4087060914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 0x559edfb32ee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9768005b80 tx=0x7f97680090d0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.974+0000 7f978565eb80 1 -- 172.21.15.46:0/4087060914 shutdown_connections 2024-05-09T00:12:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.974+0000 7f978565eb80 1 --2- 172.21.15.46:0/4087060914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 0x559edfb32ee0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.974+0000 7f978565eb80 1 --2- 172.21.15.46:0/4087060914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edfa9b160 0x559edfb38900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.974+0000 7f978565eb80 1 --2- 172.21.15.46:0/4087060914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edf9b14a0 0x559edfb38e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.974+0000 7f978565eb80 1 -- 172.21.15.46:0/4087060914 >> 172.21.15.46:0/4087060914 conn(0x559edfa116a0 msgr2=0x559edfb29400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.974+0000 7f978565eb80 1 -- 172.21.15.46:0/4087060914 shutdown_connections 2024-05-09T00:12:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.974+0000 7f978565eb80 1 -- 172.21.15.46:0/4087060914 wait complete. 2024-05-09T00:12:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f978565eb80 1 -- start start 2024-05-09T00:12:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f978565eb80 1 Processor -- start 2024-05-09T00:12:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f978565eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edf9b14a0 0x559edfb3c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f978565eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edfa9b160 0x559edfb3cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f978565eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfb32ab0 0x559edfb3d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f978565eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559edf8ccba0 con 0x559edf9b14a0 2024-05-09T00:12:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f978565eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559edfa0b150 con 0x559edfb32ab0 2024-05-09T00:12:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f978565eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559edfb3f5e0 con 0x559edfa9b160 2024-05-09T00:12:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f978565eb80 10 client.0 _ll_register_callbacks cb 0x559edf8c26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f97749e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edf9b14a0 0x559edfb3c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f97749e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edf9b14a0 0x559edfb3c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33982/0 (socket says 172.21.15.46:33982) 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f97749e4700 1 -- 172.21.15.46:0/3573438210 learned_addr learned my addr 172.21.15.46:0/3573438210 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f97751e5700 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfb32ab0 0x559edfb3d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.975+0000 7f976ffff700 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edfa9b160 0x559edfb3cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f97749e4700 1 -- 172.21.15.46:0/3573438210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edfa9b160 msgr2=0x559edfb3cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f97749e4700 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edfa9b160 0x559edfb3cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f97749e4700 1 -- 172.21.15.46:0/3573438210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfb32ab0 msgr2=0x559edfb3d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f97749e4700 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfb32ab0 0x559edfb3d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f97749e4700 1 -- 172.21.15.46:0/3573438210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559edf8cc490 con 0x559edf9b14a0 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f97749e4700 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edf9b14a0 0x559edfb3c640 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f976400df60 tx=0x7f9764004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b66598d4ba422 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f974ffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f974ffff700 1 -- 172.21.15.46:0/3573438210 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764024020 con 0x559edf9b14a0 2024-05-09T00:12:46.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f974ffff700 1 -- 172.21.15.46:0/3573438210 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f976400f040 con 0x559edf9b14a0 2024-05-09T00:12:46.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f978565eb80 1 -- 172.21.15.46:0/3573438210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9768003ba0 con 0x559edf9b14a0 2024-05-09T00:12:46.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f974ffff700 1 -- 172.21.15.46:0/3573438210 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9764014db0 con 0x559edf9b14a0 2024-05-09T00:12:46.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f978565eb80 10 client.15261 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:46.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.976+0000 7f978565eb80 1 -- 172.21.15.46:0/3573438210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559edfb49f90 con 0x559edf9b14a0 2024-05-09T00:12:46.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.977+0000 7f974ffff700 1 -- 172.21.15.46:0/3573438210 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9768003ba0 con 0x559edf9b14a0 2024-05-09T00:12:46.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.977+0000 7f974ffff700 1 -- 172.21.15.46:0/3573438210 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9764013070 con 0x559edf9b14a0 2024-05-09T00:12:46.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.977+0000 7f978565eb80 10 client.15261 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:46.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.977+0000 7f978565eb80 10 client.15261 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:12:46.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.977+0000 7f978565eb80 20 client.15261 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.977+0000 7f978565eb80 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559edfb4c670 0x559edfb4eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.977+0000 7f978565eb80 4 client.15261 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.977+0000 7f978565eb80 1 -- 172.21.15.46:0/3573438210 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559edfa5e9d0 con 0x559edfb4c670 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.977+0000 7f976ffff700 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559edfb4c670 0x559edfb4eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.978+0000 7f976ffff700 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559edfb4c670 0x559edfb4eab0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb70caa6865128cc server_cookie=81b4b428f66f1dcb in_seq=0 out_seq=0 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.978+0000 7f974ffff700 10 client.15261 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.978+0000 7f974ffff700 1 -- 172.21.15.46:0/3573438210 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559edfa5e9d0 con 0x559edfb4c670 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.978+0000 7f974ffff700 10 client.15261 handle_command_reply: tid=0 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.978+0000 7f978565eb80 1 client.15261 shutdown 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.979+0000 7f978565eb80 1 -- 172.21.15.46:0/3573438210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edf9b14a0 msgr2=0x559edfb3c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.979+0000 7f978565eb80 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edf9b14a0 0x559edfb3c640 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f976400df60 tx=0x7f9764004860 comp rx=0 tx=0).stop 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.979+0000 7f978565eb80 1 -- 172.21.15.46:0/3573438210 shutdown_connections 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.979+0000 7f978565eb80 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559edfb4c670 0x559edfb4eab0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.979+0000 7f978565eb80 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfb32ab0 0x559edfb3d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.979+0000 7f978565eb80 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edf9b14a0 0x559edfb3c640 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.979+0000 7f974ffff700 0 client.15261 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:12:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.979+0000 7f978565eb80 1 --2- 172.21.15.46:0/3573438210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edfa9b160 0x559edfb3cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.979+0000 7f978565eb80 1 -- 172.21.15.46:0/3573438210 >> 172.21.15.46:0/3573438210 conn(0x559edfa116a0 msgr2=0x559edfb291c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.979+0000 7f978565eb80 1 -- 172.21.15.46:0/3573438210 shutdown_connections 2024-05-09T00:12:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.979+0000 7f978565eb80 1 -- 172.21.15.46:0/3573438210 wait complete. 2024-05-09T00:12:46.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.979+0000 7f978565eb80 20 client.15261 trim_cache size 0 max 16384 2024-05-09T00:12:46.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.994+0000 7f978565eb80 1 Processor -- start 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.994+0000 7f978565eb80 1 -- start start 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.994+0000 7f978565eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edf9b14a0 0x559edfb37d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f978565eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfa9b160 0x559edfb36bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f978565eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 0x559edfb3cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f978565eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97701af280 con 0x559edfb32ab0 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f978565eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f97701acb10 con 0x559edfa9b160 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f978565eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f97701aa940 con 0x559edf9b14a0 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f97751e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 0x559edfb3cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f97751e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 0x559edfb3cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33996/0 (socket says 172.21.15.46:33996) 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f97751e5700 1 -- 172.21.15.46:0/739480421 learned_addr learned my addr 172.21.15.46:0/739480421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f97749e4700 1 --2- 172.21.15.46:0/739480421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edf9b14a0 0x559edfb37d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f976ffff700 1 --2- 172.21.15.46:0/739480421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfa9b160 0x559edfb36bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f97751e5700 1 -- 172.21.15.46:0/739480421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edf9b14a0 msgr2=0x559edfb37d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f97751e5700 1 --2- 172.21.15.46:0/739480421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edf9b14a0 0x559edfb37d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f97751e5700 1 -- 172.21.15.46:0/739480421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfa9b160 msgr2=0x559edfb36bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f97751e5700 1 --2- 172.21.15.46:0/739480421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfa9b160 0x559edfb36bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f97751e5700 1 -- 172.21.15.46:0/739480421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559edfb33a20 con 0x559edfb32ab0 2024-05-09T00:12:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f97751e5700 1 --2- 172.21.15.46:0/739480421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 0x559edfb3cf00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9768009140 tx=0x7f9768007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87fa1eea4c4e687 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.995+0000 7f975a7fc700 1 -- 172.21.15.46:0/739480421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768003a00 con 0x559edfb32ab0 2024-05-09T00:12:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.996+0000 7f975a7fc700 1 -- 172.21.15.46:0/739480421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9768003ba0 con 0x559edfb32ab0 2024-05-09T00:12:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.996+0000 7f975a7fc700 1 -- 172.21.15.46:0/739480421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97680058c0 con 0x559edfb32ab0 2024-05-09T00:12:46.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.996+0000 7f978565eb80 1 -- 172.21.15.46:0/739480421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 msgr2=0x559edfb3cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:46.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.996+0000 7f978565eb80 1 --2- 172.21.15.46:0/739480421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 0x559edfb3cf00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9768009140 tx=0x7f9768007b90 comp rx=0 tx=0).stop 2024-05-09T00:12:46.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.996+0000 7f978565eb80 1 -- 172.21.15.46:0/739480421 shutdown_connections 2024-05-09T00:12:46.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.996+0000 7f978565eb80 1 --2- 172.21.15.46:0/739480421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfb32ab0 0x559edfb3cf00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.996+0000 7f978565eb80 1 --2- 172.21.15.46:0/739480421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edf9b14a0 0x559edfb37d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.996+0000 7f978565eb80 1 --2- 172.21.15.46:0/739480421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfa9b160 0x559edfb36bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:46.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.996+0000 7f978565eb80 1 -- 172.21.15.46:0/739480421 >> 172.21.15.46:0/739480421 conn(0x559edfa116a0 msgr2=0x559edfb375f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.996+0000 7f978565eb80 1 -- 172.21.15.46:0/739480421 shutdown_connections 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.996+0000 7f978565eb80 1 -- 172.21.15.46:0/739480421 wait complete. 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.997+0000 7f978565eb80 1 -- start start 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.997+0000 7f978565eb80 1 Processor -- start 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.997+0000 7f978565eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edf9b14a0 0x559edfb532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.997+0000 7f978565eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfa9b160 0x559edfb537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.997+0000 7f978565eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfb32ab0 0x559edfb53d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.997+0000 7f978565eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97701aafd0 con 0x559edfa9b160 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.997+0000 7f978565eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f97701ac440 con 0x559edfb32ab0 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.997+0000 7f978565eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f97701ab6a0 con 0x559edf9b14a0 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f976ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfa9b160 0x559edfb537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f978565eb80 10 client.0 _ll_register_callbacks cb 0x559edf8c26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f976ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfa9b160 0x559edfb537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34000/0 (socket says 172.21.15.46:34000) 2024-05-09T00:12:46.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f976ffff700 1 -- 172.21.15.46:0/2599680566 learned_addr learned my addr 172.21.15.46:0/2599680566 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f97749e4700 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edf9b14a0 0x559edfb532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f97751e5700 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfb32ab0 0x559edfb53d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f976ffff700 1 -- 172.21.15.46:0/2599680566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edf9b14a0 msgr2=0x559edfb532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f976ffff700 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edf9b14a0 0x559edfb532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f976ffff700 1 -- 172.21.15.46:0/2599680566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfb32ab0 msgr2=0x559edfb53d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f976ffff700 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfb32ab0 0x559edfb53d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f976ffff700 1 -- 172.21.15.46:0/2599680566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559edfb436c0 con 0x559edfa9b160 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f97751e5700 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfb32ab0 0x559edfb53d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f97749e4700 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edf9b14a0 0x559edfb532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f976ffff700 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfa9b160 0x559edfb537e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f975c0a93d0 tx=0x7f975c0cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b11a98d9cf09b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f974ffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f974ffff700 1 -- 172.21.15.46:0/2599680566 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c0b2530 con 0x559edfa9b160 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f974ffff700 1 -- 172.21.15.46:0/2599680566 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f975c0a5cc0 con 0x559edfa9b160 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f978565eb80 1 -- 172.21.15.46:0/2599680566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9768003ba0 con 0x559edfa9b160 2024-05-09T00:12:47.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f974ffff700 1 -- 172.21.15.46:0/2599680566 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f975c0dfcc0 con 0x559edfa9b160 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f978565eb80 10 client.15267 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.998+0000 7f978565eb80 1 -- 172.21.15.46:0/2599680566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559edfb44980 con 0x559edfa9b160 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.999+0000 7f974ffff700 1 -- 172.21.15.46:0/2599680566 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9768003ba0 con 0x559edfa9b160 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.999+0000 7f974ffff700 1 -- 172.21.15.46:0/2599680566 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f975c0910b0 con 0x559edfa9b160 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.999+0000 7f978565eb80 10 client.15267 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.999+0000 7f978565eb80 10 client.15267 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.999+0000 7f978565eb80 20 client.15267 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.999+0000 7f978565eb80 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559edfb4c670 0x559edfbb8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.999+0000 7f978565eb80 4 client.15267 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.999+0000 7f978565eb80 1 -- 172.21.15.46:0/2599680566 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559edfa58a70 con 0x559edfb4c670 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:46.999+0000 7f97749e4700 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559edfb4c670 0x559edfbb8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f97749e4700 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559edfb4c670 0x559edfbb8e70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c5d5112f378946d server_cookie=7c15532d47bc1b2d in_seq=0 out_seq=0 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f974ffff700 10 client.15267 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f974ffff700 1 -- 172.21.15.46:0/2599680566 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559edfa58a70 con 0x559edfb4c670 2024-05-09T00:12:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f974ffff700 10 client.15267 handle_command_reply: tid=0 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f978565eb80 1 client.15267 shutdown 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f978565eb80 1 -- 172.21.15.46:0/2599680566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfa9b160 msgr2=0x559edfb537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f978565eb80 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfa9b160 0x559edfb537e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f975c0a93d0 tx=0x7f975c0cc040 comp rx=0 tx=0).stop 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f978565eb80 1 -- 172.21.15.46:0/2599680566 shutdown_connections 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f978565eb80 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559edfb4c670 0x559edfbb8e70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f978565eb80 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559edfb32ab0 0x559edfb53d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f978565eb80 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559edf9b14a0 0x559edfb532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f978565eb80 1 --2- 172.21.15.46:0/2599680566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559edfa9b160 0x559edfb537e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f978565eb80 1 -- 172.21.15.46:0/2599680566 >> 172.21.15.46:0/2599680566 conn(0x559edfa116a0 msgr2=0x559edfb36210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.000+0000 7f974ffff700 0 client.15267 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.001+0000 7f978565eb80 1 -- 172.21.15.46:0/2599680566 shutdown_connections 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.001+0000 7f978565eb80 1 -- 172.21.15.46:0/2599680566 wait complete. 2024-05-09T00:12:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.001+0000 7f978565eb80 20 client.15267 trim_cache size 0 max 16384 2024-05-09T00:12:47.003 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:12:47.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.001+0000 7f9776c48700 1 -- 172.21.15.46:0/1798724899 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97600b2880 msgr2=0x7f97600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:47.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.001+0000 7f9776c48700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97600b2880 0x7f97600b4d90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f97640099d0 tx=0x7f9764008040 comp rx=0 tx=0).stop 2024-05-09T00:12:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.001+0000 7f9776c48700 1 -- 172.21.15.46:0/1798724899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9770107df0 msgr2=0x7f977019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.001+0000 7f9776c48700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9770107df0 0x7f977019b3a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f975c00c8f0 tx=0x7f975c00cc00 comp rx=0 tx=0).stop 2024-05-09T00:12:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.001+0000 7f9776c48700 1 -- 172.21.15.46:0/1798724899 shutdown_connections 2024-05-09T00:12:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.001+0000 7f9776c48700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f97600b2880 0x7f97600b4d90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.001+0000 7f9776c48700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9770108740 0x7f977019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.001+0000 7f9776c48700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97701034c0 0x7f977019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.001+0000 7f9776c48700 1 --2- 172.21.15.46:0/1798724899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9770107df0 0x7f977019b3a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.002+0000 7f9776c48700 1 -- 172.21.15.46:0/1798724899 >> 172.21.15.46:0/1798724899 conn(0x7f97700fedc0 msgr2=0x7f9770107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.002+0000 7f9776c48700 1 -- 172.21.15.46:0/1798724899 shutdown_connections 2024-05-09T00:12:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:47.002+0000 7f9776c48700 1 -- 172.21.15.46:0/1798724899 wait complete. 2024-05-09T00:12:47.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:12:48.123 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:48.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:48 smithi046 ceph-mon[30151]: pgmap v419: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-09T00:12:48.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:48 smithi169 ceph-mon[34777]: pgmap v419: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-09T00:12:48.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:48 smithi142 ceph-mon[34533]: pgmap v419: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-09T00:12:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.853+0000 7f1db4f4e700 1 -- 172.21.15.46:0/300989969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1db0107df0 msgr2=0x7f1db0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.853+0000 7f1db4f4e700 1 --2- 172.21.15.46:0/300989969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1db0107df0 0x7f1db0108200 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1da40099c0 tx=0x7f1da4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.853+0000 7f1db4f4e700 1 -- 172.21.15.46:0/300989969 shutdown_connections 2024-05-09T00:12:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.853+0000 7f1db4f4e700 1 --2- 172.21.15.46:0/300989969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db0108740 0x7f1db01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.853+0000 7f1db4f4e700 1 --2- 172.21.15.46:0/300989969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1db01034c0 0x7f1db01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.853+0000 7f1db4f4e700 1 --2- 172.21.15.46:0/300989969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1db0107df0 0x7f1db0108200 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.853+0000 7f1db4f4e700 1 -- 172.21.15.46:0/300989969 >> 172.21.15.46:0/300989969 conn(0x7f1db00fedc0 msgr2=0x7f1db01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.853+0000 7f1db4f4e700 1 -- 172.21.15.46:0/300989969 shutdown_connections 2024-05-09T00:12:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.853+0000 7f1db4f4e700 1 -- 172.21.15.46:0/300989969 wait complete. 2024-05-09T00:12:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.854+0000 7f1db4f4e700 1 Processor -- start 2024-05-09T00:12:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.854+0000 7f1db4f4e700 1 -- start start 2024-05-09T00:12:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.854+0000 7f1db4f4e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1db01034c0 0x7f1db019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.854+0000 7f1db4f4e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db0107df0 0x7f1db019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1db4f4e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1db0108740 0x7f1db019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1db4f4e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01133a0 con 0x7f1db01034c0 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1db4f4e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1db0113220 con 0x7f1db0108740 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1db4f4e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1db0113520 con 0x7f1db0107df0 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1dae59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1db01034c0 0x7f1db019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1dae59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1db01034c0 0x7f1db019ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34008/0 (socket says 172.21.15.46:34008) 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1dae59c700 1 -- 172.21.15.46:0/3609595478 learned_addr learned my addr 172.21.15.46:0/3609595478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1dae59c700 1 -- 172.21.15.46:0/3609595478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db0107df0 msgr2=0x7f1db019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1dae59c700 1 --2- 172.21.15.46:0/3609595478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db0107df0 0x7f1db019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1dae59c700 1 -- 172.21.15.46:0/3609595478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1db0108740 msgr2=0x7f1db019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1dae59c700 1 --2- 172.21.15.46:0/3609595478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1db0108740 0x7f1db019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1dae59c700 1 -- 172.21.15.46:0/3609595478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db019fca0 con 0x7f1db01034c0 2024-05-09T00:12:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.855+0000 7f1dae59c700 1 --2- 172.21.15.46:0/3609595478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1db01034c0 0x7f1db019ae80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1da000c920 tx=0x7f1da000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.856+0000 7f1d9f7fe700 1 -- 172.21.15.46:0/3609595478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0007b00 con 0x7f1db01034c0 2024-05-09T00:12:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.856+0000 7f1db4f4e700 1 -- 172.21.15.46:0/3609595478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db019ff90 con 0x7f1db01034c0 2024-05-09T00:12:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.856+0000 7f1d9f7fe700 1 -- 172.21.15.46:0/3609595478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da0004ce0 con 0x7f1db01034c0 2024-05-09T00:12:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.856+0000 7f1d9f7fe700 1 -- 172.21.15.46:0/3609595478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0005650 con 0x7f1db01034c0 2024-05-09T00:12:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.856+0000 7f1db4f4e700 1 -- 172.21.15.46:0/3609595478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db010a070 con 0x7f1db01034c0 2024-05-09T00:12:49.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.860+0000 7f1d9f7fe700 1 -- 172.21.15.46:0/3609595478 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1da000f450 con 0x7f1db01034c0 2024-05-09T00:12:49.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.860+0000 7f1dc3964b80 1 Processor -- start 2024-05-09T00:12:49.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1dc3964b80 1 -- start start 2024-05-09T00:12:49.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1dc3964b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226acdd00 0x564226bd6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1dc3964b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 0x564226bd6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1dc3964b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226bd0400 0x564226bd0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1d9f7fe700 1 --2- 172.21.15.46:0/3609595478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1d980b2880 0x7f1d980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1dc3964b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564226af30a0 con 0x564226a534a0 2024-05-09T00:12:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1dc3964b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564226a989a0 con 0x564226bd0400 2024-05-09T00:12:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1dc3964b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564226aef060 con 0x564226acdd00 2024-05-09T00:12:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1dae59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 0x564226bd6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1daed9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226acdd00 0x564226bd6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1dadd9b700 1 --2- 172.21.15.46:0/3609595478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1d980b2880 0x7f1d980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1daed9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226acdd00 0x564226bd6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:42428/0 (socket says 172.21.15.46:42428) 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1daed9d700 1 -- 172.21.15.46:0/76069496 learned_addr learned my addr 172.21.15.46:0/76069496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.861+0000 7f1d9f7fe700 1 -- 172.21.15.46:0/3609595478 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1da00e21d0 con 0x7f1db01034c0 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.862+0000 7f1dae59c700 1 -- 172.21.15.46:0/76069496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226acdd00 msgr2=0x564226bd6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.862+0000 7f1dae59c700 1 --2- 172.21.15.46:0/76069496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226acdd00 0x564226bd6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.862+0000 7f1dae59c700 1 -- 172.21.15.46:0/76069496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226bd0400 msgr2=0x564226bd0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.862+0000 7f1dae59c700 1 --2- 172.21.15.46:0/76069496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226bd0400 0x564226bd0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.862+0000 7f1dae59c700 1 -- 172.21.15.46:0/76069496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564226a89500 con 0x564226a534a0 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.862+0000 7f1dae59c700 1 --2- 172.21.15.46:0/76069496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 0x564226bd6780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1da00a94f0 tx=0x7f1da0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d24e1cc0c36b33a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.862+0000 7f1dadd9b700 1 --2- 172.21.15.46:0/3609595478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1d980b2880 0x7f1d980b4d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1da40098e0 tx=0x7f1da4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.862+0000 7f1d9cff9700 1 -- 172.21.15.46:0/76069496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da001d040 con 0x564226a534a0 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.862+0000 7f1d9cff9700 1 -- 172.21.15.46:0/76069496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da00a9ef0 con 0x564226a534a0 2024-05-09T00:12:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.862+0000 7f1d9cff9700 1 -- 172.21.15.46:0/76069496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0007ca0 con 0x564226a534a0 2024-05-09T00:12:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.863+0000 7f1dc3964b80 1 -- 172.21.15.46:0/76069496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 msgr2=0x564226bd6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.863+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/76069496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 0x564226bd6780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1da00a94f0 tx=0x7f1da0012000 comp rx=0 tx=0).stop 2024-05-09T00:12:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.863+0000 7f1dc3964b80 1 -- 172.21.15.46:0/76069496 shutdown_connections 2024-05-09T00:12:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.863+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/76069496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226bd0400 0x564226bd0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.863+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/76069496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226acdd00 0x564226bd6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.863+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/76069496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 0x564226bd6780 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.863+0000 7f1dc3964b80 1 -- 172.21.15.46:0/76069496 >> 172.21.15.46:0/76069496 conn(0x564226a56f70 msgr2=0x564226bcb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.863+0000 7f1dc3964b80 1 -- 172.21.15.46:0/76069496 shutdown_connections 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.863+0000 7f1dc3964b80 1 -- 172.21.15.46:0/76069496 wait complete. 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1dc3964b80 1 -- start start 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1dc3964b80 1 Processor -- start 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1dc3964b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226a534a0 0x564226bda090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1dc3964b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226acdd00 0x564226bda5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1dc3964b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226bd0400 0x564226bdab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1dc3964b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564226bd1160 con 0x564226bd0400 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1dc3964b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564226a8db20 con 0x564226a534a0 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1dc3964b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564226bdd020 con 0x564226acdd00 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1dc3964b80 10 client.0 _ll_register_callbacks cb 0x5642269646c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1daed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226bd0400 0x564226bdab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1dae59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226a534a0 0x564226bda090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1daed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226bd0400 0x564226bdab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34024/0 (socket says 172.21.15.46:34024) 2024-05-09T00:12:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.864+0000 7f1daed9d700 1 -- 172.21.15.46:0/978864465 learned_addr learned my addr 172.21.15.46:0/978864465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1dadd9b700 1 --2- 172.21.15.46:0/978864465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226acdd00 0x564226bda5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1daed9d700 1 -- 172.21.15.46:0/978864465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226acdd00 msgr2=0x564226bda5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1daed9d700 1 --2- 172.21.15.46:0/978864465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226acdd00 0x564226bda5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1daed9d700 1 -- 172.21.15.46:0/978864465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226a534a0 msgr2=0x564226bda090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1daed9d700 1 --2- 172.21.15.46:0/978864465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226a534a0 0x564226bda090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1daed9d700 1 -- 172.21.15.46:0/978864465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56422696e490 con 0x564226bd0400 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1daed9d700 1 --2- 172.21.15.46:0/978864465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226bd0400 0x564226bdab10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1da800fcb0 tx=0x7f1da8007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1f82267ebac4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1d95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1d95ffb700 1 -- 172.21.15.46:0/978864465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8018030 con 0x564226bd0400 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1d95ffb700 1 -- 172.21.15.46:0/978864465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1da8010040 con 0x564226bd0400 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1dc3964b80 1 -- 172.21.15.46:0/978864465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da00a9ef0 con 0x564226bd0400 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1d95ffb700 1 -- 172.21.15.46:0/978864465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1da8003a00 con 0x564226bd0400 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1dc3964b80 10 client.15279 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.865+0000 7f1dc3964b80 1 -- 172.21.15.46:0/978864465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564226be7b00 con 0x564226bd0400 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.866+0000 7f1d95ffb700 1 -- 172.21.15.46:0/978864465 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da00a9ef0 con 0x564226bd0400 2024-05-09T00:12:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.866+0000 7f1d95ffb700 1 -- 172.21.15.46:0/978864465 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1da8004140 con 0x564226bd0400 2024-05-09T00:12:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.866+0000 7f1dc3964b80 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.866+0000 7f1dc3964b80 10 client.15279 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:12:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.866+0000 7f1dc3964b80 20 client.15279 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.866+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/978864465 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564226bea1e0 0x564226bec600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.866+0000 7f1dc3964b80 4 client.15279 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.866+0000 7f1dc3964b80 1 -- 172.21.15.46:0/978864465 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564226b00a80 con 0x564226bea1e0 2024-05-09T00:12:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.866+0000 7f1dae59c700 1 --2- 172.21.15.46:0/978864465 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564226bea1e0 0x564226bec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.867+0000 7f1d95ffb700 10 client.15279 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:12:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.867+0000 7f1dae59c700 1 --2- 172.21.15.46:0/978864465 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564226bea1e0 0x564226bec600 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b75b74b8cd2d31bf server_cookie=835a944b521390f5 in_seq=0 out_seq=0 2024-05-09T00:12:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1d95ffb700 1 -- 172.21.15.46:0/978864465 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564226b00a80 con 0x564226bea1e0 2024-05-09T00:12:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1d95ffb700 10 client.15279 handle_command_reply: tid=0 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1dc3964b80 1 client.15279 shutdown 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1dc3964b80 1 -- 172.21.15.46:0/978864465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226bd0400 msgr2=0x564226bdab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/978864465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226bd0400 0x564226bdab10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1da800fcb0 tx=0x7f1da8007ee0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1dc3964b80 1 -- 172.21.15.46:0/978864465 shutdown_connections 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/978864465 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564226bea1e0 0x564226bec600 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/978864465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226bd0400 0x564226bdab10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/978864465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226a534a0 0x564226bda090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1d95ffb700 0 client.15279 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/978864465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226acdd00 0x564226bda5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1dc3964b80 1 -- 172.21.15.46:0/978864465 >> 172.21.15.46:0/978864465 conn(0x564226a56f70 msgr2=0x564226bc4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1dc3964b80 1 -- 172.21.15.46:0/978864465 shutdown_connections 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1dc3964b80 1 -- 172.21.15.46:0/978864465 wait complete. 2024-05-09T00:12:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.868+0000 7f1dc3964b80 20 client.15279 trim_cache size 0 max 16384 2024-05-09T00:12:49.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.883+0000 7f1dc3964b80 1 Processor -- start 2024-05-09T00:12:49.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.883+0000 7f1dc3964b80 1 -- start start 2024-05-09T00:12:49.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dc3964b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226a534a0 0x564226bd5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dc3964b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226acdd00 0x564226bda950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dc3964b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226bd0400 0x564226be5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dc3964b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01b3820 con 0x564226acdd00 2024-05-09T00:12:49.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dc3964b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01b10b0 con 0x564226a534a0 2024-05-09T00:12:49.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dc3964b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01aef00 con 0x564226bd0400 2024-05-09T00:12:49.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dadd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226acdd00 0x564226bda950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dadd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226acdd00 0x564226bda950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34028/0 (socket says 172.21.15.46:34028) 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dadd9b700 1 -- 172.21.15.46:0/4043967949 learned_addr learned my addr 172.21.15.46:0/4043967949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dae59c700 1 --2- 172.21.15.46:0/4043967949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226a534a0 0x564226bd5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1daed9d700 1 --2- 172.21.15.46:0/4043967949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226bd0400 0x564226be5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dadd9b700 1 -- 172.21.15.46:0/4043967949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226bd0400 msgr2=0x564226be5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dadd9b700 1 --2- 172.21.15.46:0/4043967949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226bd0400 0x564226be5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dadd9b700 1 -- 172.21.15.46:0/4043967949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226a534a0 msgr2=0x564226bd5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dadd9b700 1 --2- 172.21.15.46:0/4043967949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226a534a0 0x564226bd5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.884+0000 7f1dadd9b700 1 -- 172.21.15.46:0/4043967949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564226bcbcf0 con 0x564226acdd00 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.885+0000 7f1dadd9b700 1 --2- 172.21.15.46:0/4043967949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226acdd00 0x564226bda950 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1da4005f50 tx=0x7f1da401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7676d99d0fefa5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.885+0000 7f1d97fff700 1 -- 172.21.15.46:0/4043967949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da40056b0 con 0x564226acdd00 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.885+0000 7f1d97fff700 1 -- 172.21.15.46:0/4043967949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da4004d70 con 0x564226acdd00 2024-05-09T00:12:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.885+0000 7f1d97fff700 1 -- 172.21.15.46:0/4043967949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da40238c0 con 0x564226acdd00 2024-05-09T00:12:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.886+0000 7f1dc3964b80 1 -- 172.21.15.46:0/4043967949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226acdd00 msgr2=0x564226bda950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.886+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/4043967949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226acdd00 0x564226bda950 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1da4005f50 tx=0x7f1da401d000 comp rx=0 tx=0).stop 2024-05-09T00:12:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.886+0000 7f1dc3964b80 1 -- 172.21.15.46:0/4043967949 shutdown_connections 2024-05-09T00:12:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.886+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/4043967949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226bd0400 0x564226be5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.886+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/4043967949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226a534a0 0x564226bd5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.886+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/4043967949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226acdd00 0x564226bda950 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.886+0000 7f1dc3964b80 1 -- 172.21.15.46:0/4043967949 >> 172.21.15.46:0/4043967949 conn(0x564226a56f70 msgr2=0x564226bd51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.886+0000 7f1dc3964b80 1 -- 172.21.15.46:0/4043967949 shutdown_connections 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.886+0000 7f1dc3964b80 1 -- 172.21.15.46:0/4043967949 wait complete. 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.886+0000 7f1dc3964b80 1 -- start start 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.886+0000 7f1dc3964b80 1 Processor -- start 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dc3964b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 0x564226be2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dc3964b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226acdd00 0x564226bf11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dc3964b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226bd0400 0x564226bf16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dc3964b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01af570 con 0x564226a534a0 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dc3964b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01b09e0 con 0x564226acdd00 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dc3964b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01afc40 con 0x564226bd0400 2024-05-09T00:12:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dae59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 0x564226be2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dae59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 0x564226be2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34038/0 (socket says 172.21.15.46:34038) 2024-05-09T00:12:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dae59c700 1 -- 172.21.15.46:0/1538570825 learned_addr learned my addr 172.21.15.46:0/1538570825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dc3964b80 10 client.0 _ll_register_callbacks cb 0x5642269646c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dadd9b700 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226acdd00 0x564226bf11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1daed9d700 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226bd0400 0x564226bf16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dae59c700 1 -- 172.21.15.46:0/1538570825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226bd0400 msgr2=0x564226bf16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dae59c700 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226bd0400 0x564226bf16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dae59c700 1 -- 172.21.15.46:0/1538570825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226acdd00 msgr2=0x564226bf11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dae59c700 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226acdd00 0x564226bf11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dae59c700 1 -- 172.21.15.46:0/1538570825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564226bf1c30 con 0x564226a534a0 2024-05-09T00:12:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1dadd9b700 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226acdd00 0x564226bf11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.887+0000 7f1daed9d700 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226bd0400 0x564226bf16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1dae59c700 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 0x564226be2430 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1da00cc670 tx=0x7f1da0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11530e398249dc8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1d95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1d95ffb700 1 -- 172.21.15.46:0/1538570825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da001d040 con 0x564226a534a0 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1d95ffb700 1 -- 172.21.15.46:0/1538570825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1da00a5de0 con 0x564226a534a0 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1dc3964b80 1 -- 172.21.15.46:0/1538570825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da4004d70 con 0x564226a534a0 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1d95ffb700 1 -- 172.21.15.46:0/1538570825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1da00a5a80 con 0x564226a534a0 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1dc3964b80 10 client.15285 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1dc3964b80 1 -- 172.21.15.46:0/1538570825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564226c54570 con 0x564226a534a0 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1d95ffb700 1 -- 172.21.15.46:0/1538570825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da4004d70 con 0x564226a534a0 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1d95ffb700 1 -- 172.21.15.46:0/1538570825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1da00a9a50 con 0x564226a534a0 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1dc3964b80 10 client.15285 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1dc3964b80 10 client.15285 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1dc3964b80 20 client.15285 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.888+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564226c54900 0x564226c56d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.889+0000 7f1dc3964b80 4 client.15285 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.889+0000 7f1dc3964b80 1 -- 172.21.15.46:0/1538570825 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564226afab20 con 0x564226c54900 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.889+0000 7f1dadd9b700 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564226c54900 0x564226c56d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.889+0000 7f1dadd9b700 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564226c54900 0x564226c56d20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=249a2752f6af7fd6 server_cookie=8aca60a22df52ebc in_seq=0 out_seq=0 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.889+0000 7f1d95ffb700 10 client.15285 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1d95ffb700 1 -- 172.21.15.46:0/1538570825 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564226afab20 con 0x564226c54900 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1d95ffb700 10 client.15285 handle_command_reply: tid=0 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1dc3964b80 1 client.15285 shutdown 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1dc3964b80 1 -- 172.21.15.46:0/1538570825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 msgr2=0x564226be2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 0x564226be2430 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1da00cc670 tx=0x7f1da0000bc0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1dc3964b80 1 -- 172.21.15.46:0/1538570825 shutdown_connections 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564226c54900 0x564226c56d20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564226bd0400 0x564226bf16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564226a534a0 0x564226be2430 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1dc3964b80 1 --2- 172.21.15.46:0/1538570825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564226acdd00 0x564226bf11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1dc3964b80 1 -- 172.21.15.46:0/1538570825 >> 172.21.15.46:0/1538570825 conn(0x564226a56f70 msgr2=0x564226bc4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1d95ffb700 0 client.15285 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:12:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1dc3964b80 1 -- 172.21.15.46:0/1538570825 shutdown_connections 2024-05-09T00:12:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1dc3964b80 1 -- 172.21.15.46:0/1538570825 wait complete. 2024-05-09T00:12:49.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.890+0000 7f1dc3964b80 20 client.15285 trim_cache size 0 max 16384 2024-05-09T00:12:49.892 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:12:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.891+0000 7f1db4f4e700 1 -- 172.21.15.46:0/3609595478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1d980b2880 msgr2=0x7f1d980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.891+0000 7f1db4f4e700 1 --2- 172.21.15.46:0/3609595478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1d980b2880 0x7f1d980b4d90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1da40098e0 tx=0x7f1da4000bc0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.891+0000 7f1db4f4e700 1 -- 172.21.15.46:0/3609595478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1db01034c0 msgr2=0x7f1db019ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.891+0000 7f1db4f4e700 1 --2- 172.21.15.46:0/3609595478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1db01034c0 0x7f1db019ae80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1da000c920 tx=0x7f1da000cce0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.891+0000 7f1db4f4e700 1 -- 172.21.15.46:0/3609595478 shutdown_connections 2024-05-09T00:12:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.891+0000 7f1db4f4e700 1 --2- 172.21.15.46:0/3609595478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1d980b2880 0x7f1d980b4d90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.891+0000 7f1db4f4e700 1 --2- 172.21.15.46:0/3609595478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1db0108740 0x7f1db019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.891+0000 7f1db4f4e700 1 --2- 172.21.15.46:0/3609595478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1db01034c0 0x7f1db019ae80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.891+0000 7f1db4f4e700 1 --2- 172.21.15.46:0/3609595478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db0107df0 0x7f1db019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.891+0000 7f1db4f4e700 1 -- 172.21.15.46:0/3609595478 >> 172.21.15.46:0/3609595478 conn(0x7f1db00fedc0 msgr2=0x7f1db01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.891+0000 7f1db4f4e700 1 -- 172.21.15.46:0/3609595478 shutdown_connections 2024-05-09T00:12:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:49.891+0000 7f1db4f4e700 1 -- 172.21.15.46:0/3609595478 wait complete. 2024-05-09T00:12:50.619 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:12:50.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:50 smithi046 ceph-mon[30151]: pgmap v420: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T00:12:50.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:50 smithi169 ceph-mon[34777]: pgmap v420: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T00:12:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:50.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:50 smithi142 ceph-mon[34533]: pgmap v420: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T00:12:52.745 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:52 smithi046 ceph-mon[30151]: pgmap v421: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-09T00:12:52.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.762+0000 7f66a71bb700 1 -- 172.21.15.46:0/484146845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a01034c0 msgr2=0x7f66a01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:52.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.762+0000 7f66a71bb700 1 --2- 172.21.15.46:0/484146845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a01034c0 0x7f66a01038d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f66940099c0 tx=0x7f6694009cd0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.762+0000 7f66a71bb700 1 -- 172.21.15.46:0/484146845 shutdown_connections 2024-05-09T00:12:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.762+0000 7f66a71bb700 1 --2- 172.21.15.46:0/484146845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66a0108740 0x7f66a01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.762+0000 7f66a71bb700 1 --2- 172.21.15.46:0/484146845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a01034c0 0x7f66a01038d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.762+0000 7f66a71bb700 1 --2- 172.21.15.46:0/484146845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0107df0 0x7f66a0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.762+0000 7f66a71bb700 1 -- 172.21.15.46:0/484146845 >> 172.21.15.46:0/484146845 conn(0x7f66a00fedc0 msgr2=0x7f66a01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.762+0000 7f66a71bb700 1 -- 172.21.15.46:0/484146845 shutdown_connections 2024-05-09T00:12:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.762+0000 7f66a71bb700 1 -- 172.21.15.46:0/484146845 wait complete. 2024-05-09T00:12:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f66a71bb700 1 Processor -- start 2024-05-09T00:12:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f66a71bb700 1 -- start start 2024-05-09T00:12:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f66a71bb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66a01034c0 0x7f66a00ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f66a71bb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0107df0 0x7f66a00ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f66a71bb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a0108740 0x7f66a00ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f66a71bb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01133e0 con 0x7f66a0108740 2024-05-09T00:12:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f66a71bb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0113260 con 0x7f66a01034c0 2024-05-09T00:12:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f66a71bb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0113560 con 0x7f66a0107df0 2024-05-09T00:12:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f66a5758700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a0108740 0x7f66a00ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f66a5758700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a0108740 0x7f66a00ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34050/0 (socket says 172.21.15.46:34050) 2024-05-09T00:12:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f66a5758700 1 -- 172.21.15.46:0/1966803545 learned_addr learned my addr 172.21.15.46:0/1966803545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f66a4f57700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66a01034c0 0x7f66a00ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.763+0000 7f669ffff700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0107df0 0x7f66a00ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.764+0000 7f66a5758700 1 -- 172.21.15.46:0/1966803545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0107df0 msgr2=0x7f66a00ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.764+0000 7f66a5758700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0107df0 0x7f66a00ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.764+0000 7f66a5758700 1 -- 172.21.15.46:0/1966803545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66a01034c0 msgr2=0x7f66a00ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.764+0000 7f66a5758700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66a01034c0 0x7f66a00ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.764+0000 7f66a5758700 1 -- 172.21.15.46:0/1966803545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a01a91c0 con 0x7f66a0108740 2024-05-09T00:12:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.764+0000 7f66a4f57700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66a01034c0 0x7f66a00ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.764+0000 7f66a5758700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a0108740 0x7f66a00ffd20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6698007ee0 tx=0x7f669800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.764+0000 7f669dffb700 1 -- 172.21.15.46:0/1966803545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669800dbd0 con 0x7f66a0108740 2024-05-09T00:12:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.764+0000 7f66a71bb700 1 -- 172.21.15.46:0/1966803545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a01a94b0 con 0x7f66a0108740 2024-05-09T00:12:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.764+0000 7f669dffb700 1 -- 172.21.15.46:0/1966803545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669800dd70 con 0x7f66a0108740 2024-05-09T00:12:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.764+0000 7f669dffb700 1 -- 172.21.15.46:0/1966803545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66980059b0 con 0x7f66a0108740 2024-05-09T00:12:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.764+0000 7f66a71bb700 1 -- 172.21.15.46:0/1966803545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a01a99f0 con 0x7f66a0108740 2024-05-09T00:12:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.767+0000 7f669dffb700 1 -- 172.21.15.46:0/1966803545 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f669801a020 con 0x7f66a0108740 2024-05-09T00:12:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.768+0000 7f669dffb700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f66900b2880 0x7f66900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.768+0000 7f66a4f57700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f66900b2880 0x7f66900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.768+0000 7f669dffb700 1 -- 172.21.15.46:0/1966803545 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f66980e1a30 con 0x7f66a0108740 2024-05-09T00:12:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.768+0000 7f66b5bd1b80 1 Processor -- start 2024-05-09T00:12:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f66b5bd1b80 1 -- start start 2024-05-09T00:12:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f66a4f57700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f66900b2880 0x7f66900b4d90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6694003760 tx=0x7f669401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f66b5bd1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba4114240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f66b5bd1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba4114780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f66b5bd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba410e400 0x562ba410e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f66b5bd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562ba40310a0 con 0x562ba410e400 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f66b5bd1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562ba3fd69a0 con 0x562ba3f914a0 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f66b5bd1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562ba402d060 con 0x562ba400bd00 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f66a4f57700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba4114240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f669ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba4114780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f66a4f57700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba4114240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:42496/0 (socket says 172.21.15.46:42496) 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f66a4f57700 1 -- 172.21.15.46:0/600176719 learned_addr learned my addr 172.21.15.46:0/600176719 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f669ffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba4114780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:59770/0 (socket says 172.21.15.46:59770) 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.769+0000 7f66a5758700 1 --2- 172.21.15.46:0/600176719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba410e400 0x562ba410e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.770+0000 7f66a4f57700 1 -- 172.21.15.46:0/600176719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 msgr2=0x562ba4114780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.770+0000 7f66a4f57700 1 --2- 172.21.15.46:0/600176719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba4114780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.770+0000 7f66a4f57700 1 -- 172.21.15.46:0/600176719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba410e400 msgr2=0x562ba410e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.770+0000 7f66a4f57700 1 --2- 172.21.15.46:0/600176719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba410e400 0x562ba410e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.770+0000 7f66a4f57700 1 -- 172.21.15.46:0/600176719 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ba3fc7500 con 0x562ba400bd00 2024-05-09T00:12:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.770+0000 7f66a4f57700 1 --2- 172.21.15.46:0/600176719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba4114240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6694015000 tx=0x7f669401e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd2c4301951f82d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.770+0000 7f668affd700 1 -- 172.21.15.46:0/600176719 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694003e50 con 0x562ba400bd00 2024-05-09T00:12:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.770+0000 7f668affd700 1 -- 172.21.15.46:0/600176719 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669400bc40 con 0x562ba400bd00 2024-05-09T00:12:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.770+0000 7f668affd700 1 -- 172.21.15.46:0/600176719 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669402fc00 con 0x562ba400bd00 2024-05-09T00:12:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.771+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/600176719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 msgr2=0x562ba4114240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.771+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/600176719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba4114240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6694015000 tx=0x7f669401e000 comp rx=0 tx=0).stop 2024-05-09T00:12:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.771+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/600176719 shutdown_connections 2024-05-09T00:12:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.771+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/600176719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba410e400 0x562ba410e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.771+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/600176719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba4114240 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.771+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/600176719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba4114780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.771+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/600176719 >> 172.21.15.46:0/600176719 conn(0x562ba3f94f70 msgr2=0x562ba4109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:52.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.771+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/600176719 shutdown_connections 2024-05-09T00:12:52.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.771+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/600176719 wait complete. 2024-05-09T00:12:52.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.772+0000 7f66b5bd1b80 1 -- start start 2024-05-09T00:12:52.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.772+0000 7f66b5bd1b80 1 Processor -- start 2024-05-09T00:12:52.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.772+0000 7f66b5bd1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba4117ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.772+0000 7f66b5bd1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba4118530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.772+0000 7f66b5bd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba410e400 0x562ba4118a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.772+0000 7f66b5bd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562ba410f100 con 0x562ba410e400 2024-05-09T00:12:52.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.772+0000 7f66b5bd1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562ba3fcbb20 con 0x562ba3f914a0 2024-05-09T00:12:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.772+0000 7f66b5bd1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562ba41241d0 con 0x562ba400bd00 2024-05-09T00:12:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f66b5bd1b80 10 client.0 _ll_register_callbacks cb 0x562ba3ea26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f66a5758700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba410e400 0x562ba4118a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f66a5758700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba410e400 0x562ba4118a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34072/0 (socket says 172.21.15.46:34072) 2024-05-09T00:12:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f66a5758700 1 -- 172.21.15.46:0/2863680239 learned_addr learned my addr 172.21.15.46:0/2863680239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f669ffff700 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba4118530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f66a4f57700 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba4117ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f66a5758700 1 -- 172.21.15.46:0/2863680239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 msgr2=0x562ba4118530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f66a5758700 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba4118530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f66a5758700 1 -- 172.21.15.46:0/2863680239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 msgr2=0x562ba4117ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f66a5758700 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba4117ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f66a5758700 1 -- 172.21.15.46:0/2863680239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ba3eac490 con 0x562ba410e400 2024-05-09T00:12:52.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f66a5758700 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba410e400 0x562ba4118a70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6698009510 tx=0x7f6698004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3ceaacc1b4c6c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:52.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f667ffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:52.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.773+0000 7f667ffff700 1 -- 172.21.15.46:0/2863680239 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66980a9050 con 0x562ba410e400 2024-05-09T00:12:52.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2863680239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669400bc40 con 0x562ba410e400 2024-05-09T00:12:52.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f667ffff700 1 -- 172.21.15.46:0/2863680239 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f66980a92b0 con 0x562ba410e400 2024-05-09T00:12:52.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f667ffff700 1 -- 172.21.15.46:0/2863680239 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66980ad400 con 0x562ba410e400 2024-05-09T00:12:52.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f66b5bd1b80 10 client.15297 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:52.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2863680239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562ba4125b80 con 0x562ba410e400 2024-05-09T00:12:52.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f667ffff700 1 -- 172.21.15.46:0/2863680239 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669400bc40 con 0x562ba410e400 2024-05-09T00:12:52.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f667ffff700 1 -- 172.21.15.46:0/2863680239 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f66980a49c0 con 0x562ba410e400 2024-05-09T00:12:52.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f66b5bd1b80 10 client.15297 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:52.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f66b5bd1b80 10 client.15297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:12:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f66b5bd1b80 20 client.15297 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562ba41281c0 0x562ba412a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f66b5bd1b80 4 client.15297 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.774+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2863680239 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ba403ea80 con 0x562ba41281c0 2024-05-09T00:12:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.775+0000 7f66a4f57700 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562ba41281c0 0x562ba412a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.775+0000 7f667ffff700 10 client.15297 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.775+0000 7f66a4f57700 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562ba41281c0 0x562ba412a5e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f4fc12abd0150f server_cookie=d4c1e07435e9e9e7 in_seq=0 out_seq=0 2024-05-09T00:12:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.776+0000 7f667ffff700 1 -- 172.21.15.46:0/2863680239 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562ba403ea80 con 0x562ba41281c0 2024-05-09T00:12:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.776+0000 7f667ffff700 10 client.15297 handle_command_reply: tid=0 2024-05-09T00:12:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.776+0000 7f66b5bd1b80 1 client.15297 shutdown 2024-05-09T00:12:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.776+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2863680239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba410e400 msgr2=0x562ba4118a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:52.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.776+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba410e400 0x562ba4118a70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6698009510 tx=0x7f6698004dd0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.776+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2863680239 shutdown_connections 2024-05-09T00:12:52.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.776+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562ba41281c0 0x562ba412a5e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.776+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba410e400 0x562ba4118a70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.776+0000 7f667ffff700 0 client.15297 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:52.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.776+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba4117ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.776+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2863680239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba4118530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.776+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2863680239 >> 172.21.15.46:0/2863680239 conn(0x562ba3f94f70 msgr2=0x562ba4102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:52.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.777+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2863680239 shutdown_connections 2024-05-09T00:12:52.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.777+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2863680239 wait complete. 2024-05-09T00:12:52.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.777+0000 7f66b5bd1b80 20 client.15297 trim_cache size 0 max 16384 2024-05-09T00:12:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.792+0000 7f66b5bd1b80 1 Processor -- start 2024-05-09T00:12:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.792+0000 7f66b5bd1b80 1 -- start start 2024-05-09T00:12:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.792+0000 7f66b5bd1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f94f70 0x562ba4103a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.792+0000 7f66b5bd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f914a0 0x562ba41137c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.792+0000 7f66b5bd1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba41188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.792+0000 7f66b5bd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01b36f0 con 0x562ba3f914a0 2024-05-09T00:12:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.792+0000 7f66b5bd1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01b0f80 con 0x562ba3f94f70 2024-05-09T00:12:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.792+0000 7f66b5bd1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01aedb0 con 0x562ba400bd00 2024-05-09T00:12:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f669ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f914a0 0x562ba41137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f669ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f914a0 0x562ba41137c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34084/0 (socket says 172.21.15.46:34084) 2024-05-09T00:12:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f669ffff700 1 -- 172.21.15.46:0/2307836785 learned_addr learned my addr 172.21.15.46:0/2307836785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f66a4f57700 1 --2- 172.21.15.46:0/2307836785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f94f70 0x562ba4103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f66a5758700 1 --2- 172.21.15.46:0/2307836785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba41188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f669ffff700 1 -- 172.21.15.46:0/2307836785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 msgr2=0x562ba41188b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f669ffff700 1 --2- 172.21.15.46:0/2307836785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba41188b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f669ffff700 1 -- 172.21.15.46:0/2307836785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f94f70 msgr2=0x562ba4103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f669ffff700 1 --2- 172.21.15.46:0/2307836785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f94f70 0x562ba4103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f669ffff700 1 -- 172.21.15.46:0/2307836785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ba41059d0 con 0x562ba3f914a0 2024-05-09T00:12:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f669ffff700 1 --2- 172.21.15.46:0/2307836785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f914a0 0x562ba41137c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f66a0076ab0 tx=0x7f668c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbae2fb6a2c7e1ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f668a7fc700 1 -- 172.21.15.46:0/2307836785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c012ba0 con 0x562ba3f914a0 2024-05-09T00:12:52.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f668a7fc700 1 -- 172.21.15.46:0/2307836785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668c0044d0 con 0x562ba3f914a0 2024-05-09T00:12:52.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.793+0000 7f668a7fc700 1 -- 172.21.15.46:0/2307836785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c00f440 con 0x562ba3f914a0 2024-05-09T00:12:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.794+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2307836785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f914a0 msgr2=0x562ba41137c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.794+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2307836785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f914a0 0x562ba41137c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f66a0076ab0 tx=0x7f668c009260 comp rx=0 tx=0).stop 2024-05-09T00:12:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.794+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2307836785 shutdown_connections 2024-05-09T00:12:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.794+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2307836785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba41188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.794+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2307836785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f94f70 0x562ba4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.794+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2307836785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f914a0 0x562ba41137c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.794+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2307836785 >> 172.21.15.46:0/2307836785 conn(0x562ba410eae0 msgr2=0x562ba4111c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.794+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2307836785 shutdown_connections 2024-05-09T00:12:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.794+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2307836785 wait complete. 2024-05-09T00:12:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.795+0000 7f66b5bd1b80 1 -- start start 2024-05-09T00:12:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.795+0000 7f66b5bd1b80 1 Processor -- start 2024-05-09T00:12:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66b5bd1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba41204b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66b5bd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f94f70 0x562ba411b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66b5bd1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba411bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66b5bd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01af440 con 0x562ba3f94f70 2024-05-09T00:12:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66b5bd1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01b08b0 con 0x562ba3f914a0 2024-05-09T00:12:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66b5bd1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01afb10 con 0x562ba400bd00 2024-05-09T00:12:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66b5bd1b80 10 client.0 _ll_register_callbacks cb 0x562ba3ea26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f669ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f94f70 0x562ba411b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f669ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f94f70 0x562ba411b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34098/0 (socket says 172.21.15.46:34098) 2024-05-09T00:12:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f669ffff700 1 -- 172.21.15.46:0/2082721338 learned_addr learned my addr 172.21.15.46:0/2082721338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66a4f57700 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba41204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66a5758700 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba411bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66a4f57700 1 -- 172.21.15.46:0/2082721338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 msgr2=0x562ba411bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66a4f57700 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba411bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66a4f57700 1 -- 172.21.15.46:0/2082721338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f94f70 msgr2=0x562ba411b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66a4f57700 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f94f70 0x562ba411b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f66a4f57700 1 -- 172.21.15.46:0/2082721338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ba411c340 con 0x562ba3f914a0 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.796+0000 7f669ffff700 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f94f70 0x562ba411b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.797+0000 7f66a4f57700 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba41204b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x562ba4117870 tx=0x7f6694009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2ca927af786a0da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.797+0000 7f667ffff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.797+0000 7f667ffff700 1 -- 172.21.15.46:0/2082721338 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669403f070 con 0x562ba3f914a0 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.797+0000 7f667ffff700 1 -- 172.21.15.46:0/2082721338 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f669400bba0 con 0x562ba3f914a0 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.797+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2082721338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f668c0044d0 con 0x562ba3f914a0 2024-05-09T00:12:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.797+0000 7f667ffff700 1 -- 172.21.15.46:0/2082721338 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f669404b050 con 0x562ba3f914a0 2024-05-09T00:12:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.797+0000 7f66b5bd1b80 10 client.25034 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.797+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2082721338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562ba419ae40 con 0x562ba3f914a0 2024-05-09T00:12:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.797+0000 7f667ffff700 1 -- 172.21.15.46:0/2082721338 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668c0044d0 con 0x562ba3f914a0 2024-05-09T00:12:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.798+0000 7f667ffff700 1 -- 172.21.15.46:0/2082721338 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6694022070 con 0x562ba3f914a0 2024-05-09T00:12:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.798+0000 7f66b5bd1b80 10 client.25034 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.798+0000 7f66b5bd1b80 10 client.25034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:12:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.798+0000 7f66b5bd1b80 20 client.25034 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.798+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562ba410e400 0x562ba419d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.798+0000 7f66b5bd1b80 4 client.25034 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:12:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.798+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2082721338 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ba4038b20 con 0x562ba410e400 2024-05-09T00:12:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.798+0000 7f669ffff700 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562ba410e400 0x562ba419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f669ffff700 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562ba410e400 0x562ba419d020 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933d40f2e7f381d1 server_cookie=42c61dd3ae8924ba in_seq=0 out_seq=0 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f667ffff700 10 client.25034 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f667ffff700 1 -- 172.21.15.46:0/2082721338 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ba4038b20 con 0x562ba410e400 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f667ffff700 10 client.25034 handle_command_reply: tid=0 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f66b5bd1b80 1 client.25034 shutdown 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2082721338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 msgr2=0x562ba41204b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba41204b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x562ba4117870 tx=0x7f6694009cd0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2082721338 shutdown_connections 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562ba410e400 0x562ba419d020 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562ba400bd00 0x562ba411bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562ba3f914a0 0x562ba41204b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f66b5bd1b80 1 --2- 172.21.15.46:0/2082721338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ba3f94f70 0x562ba411b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2082721338 >> 172.21.15.46:0/2082721338 conn(0x562ba410eae0 msgr2=0x562ba4103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f667ffff700 0 client.25034 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:52.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2082721338 shutdown_connections 2024-05-09T00:12:52.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f66b5bd1b80 1 -- 172.21.15.46:0/2082721338 wait complete. 2024-05-09T00:12:52.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.799+0000 7f66b5bd1b80 20 client.25034 trim_cache size 0 max 16384 2024-05-09T00:12:52.802 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:12:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.800+0000 7f66a71bb700 1 -- 172.21.15.46:0/1966803545 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f66900b2880 msgr2=0x7f66900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.800+0000 7f66a71bb700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f66900b2880 0x7f66900b4d90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6694003760 tx=0x7f669401a040 comp rx=0 tx=0).stop 2024-05-09T00:12:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.800+0000 7f66a71bb700 1 -- 172.21.15.46:0/1966803545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a0108740 msgr2=0x7f66a00ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.800+0000 7f66a71bb700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a0108740 0x7f66a00ffd20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6698007ee0 tx=0x7f669800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.800+0000 7f66a71bb700 1 -- 172.21.15.46:0/1966803545 shutdown_connections 2024-05-09T00:12:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.800+0000 7f66a71bb700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f66900b2880 0x7f66900b4d90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.800+0000 7f66a71bb700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a0108740 0x7f66a00ffd20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.800+0000 7f66a71bb700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66a01034c0 0x7f66a00ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.800+0000 7f66a71bb700 1 --2- 172.21.15.46:0/1966803545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0107df0 0x7f66a00ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:52.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.800+0000 7f66a71bb700 1 -- 172.21.15.46:0/1966803545 >> 172.21.15.46:0/1966803545 conn(0x7f66a00fedc0 msgr2=0x7f66a006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:52.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.800+0000 7f66a71bb700 1 -- 172.21.15.46:0/1966803545 shutdown_connections 2024-05-09T00:12:52.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:52.800+0000 7f66a71bb700 1 -- 172.21.15.46:0/1966803545 wait complete. 2024-05-09T00:12:52.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:52 smithi169 ceph-mon[34777]: pgmap v421: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-09T00:12:52.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:52 smithi142 ceph-mon[34533]: pgmap v421: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-09T00:12:54.599 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:12:54.599 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:12:54.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:54 smithi046 ceph-mon[30151]: pgmap v422: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T00:12:54.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:54 smithi169 ceph-mon[34777]: pgmap v422: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T00:12:54.870 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:54.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:54 smithi142 ceph-mon[34533]: pgmap v422: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T00:12:56.598 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:56 smithi046 ceph-mon[30151]: pgmap v423: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T00:12:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.616+0000 7f8c078f4700 1 -- 172.21.15.46:0/3609344335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c00107df0 msgr2=0x7f8c00108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.616+0000 7f8c078f4700 1 --2- 172.21.15.46:0/3609344335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c00107df0 0x7f8c00108200 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00099c0 tx=0x7f8bf0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.616+0000 7f8c078f4700 1 -- 172.21.15.46:0/3609344335 shutdown_connections 2024-05-09T00:12:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.616+0000 7f8c078f4700 1 --2- 172.21.15.46:0/3609344335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c00108740 0x7f8c001100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.616+0000 7f8c078f4700 1 --2- 172.21.15.46:0/3609344335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c001034c0 0x7f8c001038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.616+0000 7f8c078f4700 1 --2- 172.21.15.46:0/3609344335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c00107df0 0x7f8c00108200 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.616+0000 7f8c078f4700 1 -- 172.21.15.46:0/3609344335 >> 172.21.15.46:0/3609344335 conn(0x7f8c000fedc0 msgr2=0x7f8c001011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.616+0000 7f8c078f4700 1 -- 172.21.15.46:0/3609344335 shutdown_connections 2024-05-09T00:12:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.616+0000 7f8c078f4700 1 -- 172.21.15.46:0/3609344335 wait complete. 2024-05-09T00:12:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.617+0000 7f8c078f4700 1 Processor -- start 2024-05-09T00:12:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.617+0000 7f8c078f4700 1 -- start start 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.617+0000 7f8c078f4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c001034c0 0x7f8c0019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.617+0000 7f8c078f4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00107df0 0x7f8c0019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.617+0000 7f8c078f4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c00108740 0x7f8c00198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.617+0000 7f8c078f4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001122b0 con 0x7f8c001034c0 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.617+0000 7f8c078f4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8c00112130 con 0x7f8c00108740 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.617+0000 7f8c078f4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8c00112430 con 0x7f8c00107df0 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.617+0000 7f8c05690700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c001034c0 0x7f8c0019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.617+0000 7f8c05690700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c001034c0 0x7f8c0019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41728/0 (socket says 172.21.15.46:41728) 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.617+0000 7f8c05690700 1 -- 172.21.15.46:0/2443990279 learned_addr learned my addr 172.21.15.46:0/2443990279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.618+0000 7f8c04e8f700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00107df0 0x7f8c0019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.618+0000 7f8c05690700 1 -- 172.21.15.46:0/2443990279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00107df0 msgr2=0x7f8c0019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.618+0000 7f8c05690700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00107df0 0x7f8c0019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.618+0000 7f8c05690700 1 -- 172.21.15.46:0/2443990279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c00108740 msgr2=0x7f8c00198a90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:12:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.618+0000 7f8c05690700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c00108740 0x7f8c00198a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.618+0000 7f8c05690700 1 -- 172.21.15.46:0/2443990279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c00199140 con 0x7f8c001034c0 2024-05-09T00:12:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.618+0000 7f8c05690700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c001034c0 0x7f8c0019e870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00e9a0 tx=0x7f8bfc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.618+0000 7f8bf67fc700 1 -- 172.21.15.46:0/2443990279 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc00cbd0 con 0x7f8c001034c0 2024-05-09T00:12:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.618+0000 7f8c078f4700 1 -- 172.21.15.46:0/2443990279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c00199430 con 0x7f8c001034c0 2024-05-09T00:12:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.618+0000 7f8bf67fc700 1 -- 172.21.15.46:0/2443990279 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bfc00cd70 con 0x7f8c001034c0 2024-05-09T00:12:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.618+0000 7f8bf67fc700 1 -- 172.21.15.46:0/2443990279 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc018680 con 0x7f8c001034c0 2024-05-09T00:12:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.618+0000 7f8c078f4700 1 -- 172.21.15.46:0/2443990279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c001a5700 con 0x7f8c001034c0 2024-05-09T00:12:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.621+0000 7f8bf67fc700 1 -- 172.21.15.46:0/2443990279 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8bfc010970 con 0x7f8c001034c0 2024-05-09T00:12:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.622+0000 7f8bf67fc700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8bec0b6d10 0x7f8bec0b9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.622+0000 7f8bf67fc700 1 -- 172.21.15.46:0/2443990279 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8bfc0e2520 con 0x7f8c001034c0 2024-05-09T00:12:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.622+0000 7f8c04e8f700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8bec0b6d10 0x7f8bec0b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.622+0000 7f8c04e8f700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8bec0b6d10 0x7f8bec0b9220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0000c00 tx=0x7f8bf001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.623+0000 7f8c1630ab80 1 Processor -- start 2024-05-09T00:12:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.623+0000 7f8c1630ab80 1 -- start start 2024-05-09T00:12:56.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.623+0000 7f8c1630ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991e0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.623+0000 7f8c1630ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c09905d520 0x55c0991e0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.623+0000 7f8c1630ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 0x55c0991dab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.623+0000 7f8c1630ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c0990fd070 con 0x55c0991da720 2024-05-09T00:12:56.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.623+0000 7f8c1630ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c09909b870 con 0x55c0991471e0 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.623+0000 7f8c1630ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c0990f9030 con 0x55c09905d520 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.623+0000 7f8c05e91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 0x55c0991dab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.623+0000 7f8c05e91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 0x55c0991dab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41738/0 (socket says 172.21.15.46:41738) 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.623+0000 7f8c05e91700 1 -- 172.21.15.46:0/3571496976 learned_addr learned my addr 172.21.15.46:0/3571496976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.623+0000 7f8c04e8f700 1 --2- 172.21.15.46:0/3571496976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c09905d520 0x55c0991e0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.624+0000 7f8c05e91700 1 -- 172.21.15.46:0/3571496976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c09905d520 msgr2=0x55c0991e0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.624+0000 7f8c05690700 1 --2- 172.21.15.46:0/3571496976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991e0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.624+0000 7f8c05e91700 1 --2- 172.21.15.46:0/3571496976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c09905d520 0x55c0991e0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.624+0000 7f8c05e91700 1 -- 172.21.15.46:0/3571496976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 msgr2=0x55c0991e0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.624+0000 7f8c05e91700 1 --2- 172.21.15.46:0/3571496976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991e0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.624+0000 7f8c05e91700 1 -- 172.21.15.46:0/3571496976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0990933f0 con 0x55c0991da720 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.624+0000 7f8c05e91700 1 --2- 172.21.15.46:0/3571496976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 0x55c0991dab50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009d00 tx=0x7f8bf800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b524d1e6352207 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.624+0000 7f8beb7fe700 1 -- 172.21.15.46:0/3571496976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8009e50 con 0x55c0991da720 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.624+0000 7f8beb7fe700 1 -- 172.21.15.46:0/3571496976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf8017070 con 0x55c0991da720 2024-05-09T00:12:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.624+0000 7f8beb7fe700 1 -- 172.21.15.46:0/3571496976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8011390 con 0x55c0991da720 2024-05-09T00:12:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.625+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3571496976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 msgr2=0x55c0991dab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.625+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/3571496976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 0x55c0991dab50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009d00 tx=0x7f8bf800b330 comp rx=0 tx=0).stop 2024-05-09T00:12:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.625+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3571496976 shutdown_connections 2024-05-09T00:12:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.625+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/3571496976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 0x55c0991dab50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.625+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/3571496976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991e0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.625+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/3571496976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c09905d520 0x55c0991e0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.625+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3571496976 >> 172.21.15.46:0/3571496976 conn(0x55c0990bd720 msgr2=0x55c0991d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.625+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3571496976 shutdown_connections 2024-05-09T00:12:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.625+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3571496976 wait complete. 2024-05-09T00:12:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.626+0000 7f8c1630ab80 1 -- start start 2024-05-09T00:12:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.626+0000 7f8c1630ab80 1 Processor -- start 2024-05-09T00:12:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c1630ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c09905d520 0x55c0991e4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c1630ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991471e0 0x55c0991e4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c1630ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 0x55c0991e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c1630ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c098f78c20 con 0x55c0991da720 2024-05-09T00:12:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c1630ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c0990b71d0 con 0x55c09905d520 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c1630ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c0991e7250 con 0x55c0991471e0 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c05e91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 0x55c0991e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c05e91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 0x55c0991e4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41748/0 (socket says 172.21.15.46:41748) 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c05e91700 1 -- 172.21.15.46:0/4251651689 learned_addr learned my addr 172.21.15.46:0/4251651689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c1630ab80 10 client.0 _ll_register_callbacks cb 0x55c098f6e740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c04e8f700 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991471e0 0x55c0991e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c05690700 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c09905d520 0x55c0991e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c05e91700 1 -- 172.21.15.46:0/4251651689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991471e0 msgr2=0x55c0991e4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c05e91700 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991471e0 0x55c0991e4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c05e91700 1 -- 172.21.15.46:0/4251651689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c09905d520 msgr2=0x55c0991e4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c05e91700 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c09905d520 0x55c0991e4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c05e91700 1 -- 172.21.15.46:0/4251651689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c098f78510 con 0x55c0991da720 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8c05e91700 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 0x55c0991e4cd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8012a60 tx=0x7f8bf800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c24f736cd315be4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.627+0000 7f8be8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8be8ff9700 1 -- 172.21.15.46:0/4251651689 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8009e50 con 0x55c0991da720 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8be8ff9700 1 -- 172.21.15.46:0/4251651689 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8bf801b3f0 con 0x55c0991da720 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8c1630ab80 1 -- 172.21.15.46:0/4251651689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bf8017070 con 0x55c0991da720 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8be8ff9700 1 -- 172.21.15.46:0/4251651689 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bf8012840 con 0x55c0991da720 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8c1630ab80 10 client.15315 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8c1630ab80 1 -- 172.21.15.46:0/4251651689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c0991f1c20 con 0x55c0991da720 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8be8ff9700 1 -- 172.21.15.46:0/4251651689 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf8017070 con 0x55c0991da720 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8be8ff9700 1 -- 172.21.15.46:0/4251651689 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8bf8011070 con 0x55c0991da720 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8c1630ab80 10 client.15315 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8c1630ab80 10 client.15315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8c1630ab80 20 client.15315 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c0991f41b0 0x55c0991f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8c1630ab80 4 client.15315 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.628+0000 7f8c1630ab80 1 -- 172.21.15.46:0/4251651689 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c09910aa50 con 0x55c0991f41b0 2024-05-09T00:12:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.629+0000 7f8c05690700 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c0991f41b0 0x55c0991f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.629+0000 7f8be8ff9700 10 client.15315 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.629+0000 7f8c05690700 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c0991f41b0 0x55c0991f65d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf3d8861e496f765 server_cookie=4c0da3c176beaf0b in_seq=0 out_seq=0 2024-05-09T00:12:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.630+0000 7f8be8ff9700 1 -- 172.21.15.46:0/4251651689 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c09910aa50 con 0x55c0991f41b0 2024-05-09T00:12:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.630+0000 7f8be8ff9700 10 client.15315 handle_command_reply: tid=0 2024-05-09T00:12:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.630+0000 7f8c1630ab80 1 client.15315 shutdown 2024-05-09T00:12:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.630+0000 7f8c1630ab80 1 -- 172.21.15.46:0/4251651689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 msgr2=0x55c0991e4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.630+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 0x55c0991e4cd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8012a60 tx=0x7f8bf800be00 comp rx=0 tx=0).stop 2024-05-09T00:12:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.630+0000 7f8c1630ab80 1 -- 172.21.15.46:0/4251651689 shutdown_connections 2024-05-09T00:12:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.630+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c0991f41b0 0x55c0991f65d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.630+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0991da720 0x55c0991e4cd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.630+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c09905d520 0x55c0991e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.630+0000 7f8be8ff9700 0 client.15315 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.630+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/4251651689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991471e0 0x55c0991e4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.630+0000 7f8c1630ab80 1 -- 172.21.15.46:0/4251651689 >> 172.21.15.46:0/4251651689 conn(0x55c0990bd720 msgr2=0x55c0991d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.631+0000 7f8c1630ab80 1 -- 172.21.15.46:0/4251651689 shutdown_connections 2024-05-09T00:12:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.631+0000 7f8c1630ab80 1 -- 172.21.15.46:0/4251651689 wait complete. 2024-05-09T00:12:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.631+0000 7f8c1630ab80 20 client.15315 trim_cache size 0 max 16384 2024-05-09T00:12:56.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.652+0000 7f8c1630ab80 1 Processor -- start 2024-05-09T00:12:56.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c1630ab80 1 -- start start 2024-05-09T00:12:56.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c1630ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 0x55c0991e4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c1630ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991effb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c1630ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991da720 0x55c0991f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c1630ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001af570 con 0x55c09905d520 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c1630ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001ace00 con 0x55c0991471e0 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c1630ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001aac30 con 0x55c0991da720 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c05690700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 0x55c0991e4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c05690700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 0x55c0991e4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41756/0 (socket says 172.21.15.46:41756) 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c05690700 1 -- 172.21.15.46:0/2803498928 learned_addr learned my addr 172.21.15.46:0/2803498928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c05e91700 1 --2- 172.21.15.46:0/2803498928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991da720 0x55c0991f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c04e8f700 1 --2- 172.21.15.46:0/2803498928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c05690700 1 -- 172.21.15.46:0/2803498928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991da720 msgr2=0x55c0991f04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c05690700 1 --2- 172.21.15.46:0/2803498928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991da720 0x55c0991f04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c05690700 1 -- 172.21.15.46:0/2803498928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 msgr2=0x55c0991effb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.653+0000 7f8c05690700 1 --2- 172.21.15.46:0/2803498928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991effb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.654+0000 7f8c05690700 1 -- 172.21.15.46:0/2803498928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0991e27c0 con 0x55c09905d520 2024-05-09T00:12:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.654+0000 7f8c04e8f700 1 --2- 172.21.15.46:0/2803498928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.654+0000 7f8c05690700 1 --2- 172.21.15.46:0/2803498928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 0x55c0991e4c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00c5b0 tx=0x7f8bfc0dfef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b204d5646e9ae058 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.654+0000 7f8beaffd700 1 -- 172.21.15.46:0/2803498928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc0aa240 con 0x55c09905d520 2024-05-09T00:12:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.654+0000 7f8beaffd700 1 -- 172.21.15.46:0/2803498928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bfc0a9b40 con 0x55c09905d520 2024-05-09T00:12:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.654+0000 7f8beaffd700 1 -- 172.21.15.46:0/2803498928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc0e20b0 con 0x55c09905d520 2024-05-09T00:12:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.655+0000 7f8c1630ab80 1 -- 172.21.15.46:0/2803498928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 msgr2=0x55c0991e4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.655+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/2803498928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 0x55c0991e4c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00c5b0 tx=0x7f8bfc0dfef0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.655+0000 7f8c1630ab80 1 -- 172.21.15.46:0/2803498928 shutdown_connections 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.655+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/2803498928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991da720 0x55c0991f04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.655+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/2803498928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 0x55c0991e4c40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.655+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/2803498928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.655+0000 7f8c1630ab80 1 -- 172.21.15.46:0/2803498928 >> 172.21.15.46:0/2803498928 conn(0x55c0990bd720 msgr2=0x55c0991d9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.655+0000 7f8c1630ab80 1 -- 172.21.15.46:0/2803498928 shutdown_connections 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.655+0000 7f8c1630ab80 1 -- 172.21.15.46:0/2803498928 wait complete. 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c1630ab80 1 -- start start 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c1630ab80 1 Processor -- start 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c1630ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 0x55c0991ec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c1630ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c1630ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991da720 0x55c0991e9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c1630ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001ab2c0 con 0x55c09905d520 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c1630ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001ac730 con 0x55c0991471e0 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c1630ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001ab990 con 0x55c0991da720 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c1630ab80 10 client.0 _ll_register_callbacks cb 0x55c098f6e740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c05690700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 0x55c0991ec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c04e8f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c04e8f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991eca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:38276/0 (socket says 172.21.15.46:38276) 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.656+0000 7f8c04e8f700 1 -- 172.21.15.46:0/3217047103 learned_addr learned my addr 172.21.15.46:0/3217047103 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.657+0000 7f8c05e91700 1 --2- 172.21.15.46:0/3217047103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991da720 0x55c0991e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.657+0000 7f8c05690700 1 -- 172.21.15.46:0/3217047103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991da720 msgr2=0x55c0991e9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.657+0000 7f8c05690700 1 --2- 172.21.15.46:0/3217047103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991da720 0x55c0991e9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.657+0000 7f8c05690700 1 -- 172.21.15.46:0/3217047103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 msgr2=0x55c0991eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.657+0000 7f8c05690700 1 --2- 172.21.15.46:0/3217047103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.657+0000 7f8c05690700 1 -- 172.21.15.46:0/3217047103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0991e9ad0 con 0x55c09905d520 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.657+0000 7f8c05690700 1 --2- 172.21.15.46:0/3217047103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 0x55c0991ec4e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55c0991f6ab0 tx=0x7f8bfc0adef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0b41d269f2fd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.657+0000 7f8be8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.657+0000 7f8be8ff9700 1 -- 172.21.15.46:0/3217047103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc08e070 con 0x55c09905d520 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.657+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3217047103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bfc0a9b40 con 0x55c09905d520 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.657+0000 7f8be8ff9700 1 -- 172.21.15.46:0/3217047103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8bfc0e20f0 con 0x55c09905d520 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.657+0000 7f8be8ff9700 1 -- 172.21.15.46:0/3217047103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bfc0b29a0 con 0x55c09905d520 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.658+0000 7f8c1630ab80 10 client.15321 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.658+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3217047103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c09925f250 con 0x55c09905d520 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.658+0000 7f8be8ff9700 1 -- 172.21.15.46:0/3217047103 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bfc0a9b40 con 0x55c09905d520 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.658+0000 7f8be8ff9700 1 -- 172.21.15.46:0/3217047103 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8bfc0a54b0 con 0x55c09905d520 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.658+0000 7f8c1630ab80 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.658+0000 7f8c1630ab80 10 client.15321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.658+0000 7f8c1630ab80 20 client.15321 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.658+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/3217047103 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c09925f930 0x55c099261d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.658+0000 7f8c1630ab80 4 client.15321 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:12:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.658+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3217047103 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c099104af0 con 0x55c09925f930 2024-05-09T00:12:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.658+0000 7f8c04e8f700 1 --2- 172.21.15.46:0/3217047103 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c09925f930 0x55c099261d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.659+0000 7f8be8ff9700 10 client.15321 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.659+0000 7f8c04e8f700 1 --2- 172.21.15.46:0/3217047103 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c09925f930 0x55c099261d50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22433cfecfa68382 server_cookie=7bcf07ba3ace72a7 in_seq=0 out_seq=0 2024-05-09T00:12:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8be8ff9700 1 -- 172.21.15.46:0/3217047103 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c099104af0 con 0x55c09925f930 2024-05-09T00:12:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8be8ff9700 10 client.15321 handle_command_reply: tid=0 2024-05-09T00:12:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8c1630ab80 1 client.15321 shutdown 2024-05-09T00:12:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3217047103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 msgr2=0x55c0991ec4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/3217047103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 0x55c0991ec4e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55c0991f6ab0 tx=0x7f8bfc0adef0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3217047103 shutdown_connections 2024-05-09T00:12:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/3217047103 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c09925f930 0x55c099261d50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/3217047103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c0991da720 0x55c0991e9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/3217047103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c09905d520 0x55c0991ec4e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8c1630ab80 1 --2- 172.21.15.46:0/3217047103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c0991471e0 0x55c0991eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3217047103 >> 172.21.15.46:0/3217047103 conn(0x55c0990bd720 msgr2=0x55c0991dde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8be8ff9700 0 client.15321 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.660+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3217047103 shutdown_connections 2024-05-09T00:12:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.661+0000 7f8c1630ab80 1 -- 172.21.15.46:0/3217047103 wait complete. 2024-05-09T00:12:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.661+0000 7f8c1630ab80 20 client.15321 trim_cache size 0 max 16384 2024-05-09T00:12:56.662 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:12:56.662 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:12:56.662 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "203618dd-27e9-4384-8870-8378f00fc4d8", 2024-05-09T00:12:56.662 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:12:56.662 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:12:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.661+0000 7f8c078f4700 1 -- 172.21.15.46:0/2443990279 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8bec0b6d10 msgr2=0x7f8bec0b9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.661+0000 7f8c078f4700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8bec0b6d10 0x7f8bec0b9220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0000c00 tx=0x7f8bf001a040 comp rx=0 tx=0).stop 2024-05-09T00:12:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.661+0000 7f8c078f4700 1 -- 172.21.15.46:0/2443990279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c001034c0 msgr2=0x7f8c0019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.661+0000 7f8c078f4700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c001034c0 0x7f8c0019e870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00e9a0 tx=0x7f8bfc00ed60 comp rx=0 tx=0).stop 2024-05-09T00:12:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.661+0000 7f8c078f4700 1 -- 172.21.15.46:0/2443990279 shutdown_connections 2024-05-09T00:12:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.661+0000 7f8c078f4700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8bec0b6d10 0x7f8bec0b9220 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.661+0000 7f8c078f4700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c00108740 0x7f8c00198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.661+0000 7f8c078f4700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c001034c0 0x7f8c0019e870 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.661+0000 7f8c078f4700 1 --2- 172.21.15.46:0/2443990279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00107df0 0x7f8c0019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.661+0000 7f8c078f4700 1 -- 172.21.15.46:0/2443990279 >> 172.21.15.46:0/2443990279 conn(0x7f8c000fedc0 msgr2=0x7f8c001096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.662+0000 7f8c078f4700 1 -- 172.21.15.46:0/2443990279 shutdown_connections 2024-05-09T00:12:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:56.662+0000 7f8c078f4700 1 -- 172.21.15.46:0/2443990279 wait complete. 2024-05-09T00:12:56.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:56 smithi169 ceph-mon[34777]: pgmap v423: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T00:12:56.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:56 smithi142 ceph-mon[34533]: pgmap v423: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T00:12:57.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:12:57.777 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:12:57.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:57 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:12:57.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:57 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:12:57.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:57 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:12:57.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:57 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:12:57.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:57 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:12:57.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:57 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:12:57.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:57 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:12:57.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:57 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:12:57.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:57 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:12:57.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:57 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:12:57.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:57 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:12:57.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:57 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:12:57.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:57 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:12:57.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:57 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:12:57.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:57 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:12:58.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:12:58 smithi169 ceph-mon[34777]: pgmap v424: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:58.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:12:58 smithi142 ceph-mon[34533]: pgmap v424: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:58.947 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:12:58 smithi046 ceph-mon[30151]: pgmap v424: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:12:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.521+0000 7f6c41b93700 1 -- 172.21.15.46:0/1828302223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c108740 msgr2=0x7f6c3c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.521+0000 7f6c41b93700 1 --2- 172.21.15.46:0/1828302223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c108740 0x7f6c3c1100a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400b0d0 tx=0x7f6c3400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.521+0000 7f6c41b93700 1 -- 172.21.15.46:0/1828302223 shutdown_connections 2024-05-09T00:12:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.521+0000 7f6c41b93700 1 --2- 172.21.15.46:0/1828302223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c108740 0x7f6c3c1100a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.521+0000 7f6c41b93700 1 --2- 172.21.15.46:0/1828302223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c1034c0 0x7f6c3c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.521+0000 7f6c41b93700 1 --2- 172.21.15.46:0/1828302223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c3c107df0 0x7f6c3c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.521+0000 7f6c41b93700 1 -- 172.21.15.46:0/1828302223 >> 172.21.15.46:0/1828302223 conn(0x7f6c3c0fedc0 msgr2=0x7f6c3c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.521+0000 7f6c41b93700 1 -- 172.21.15.46:0/1828302223 shutdown_connections 2024-05-09T00:12:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.521+0000 7f6c41b93700 1 -- 172.21.15.46:0/1828302223 wait complete. 2024-05-09T00:12:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.522+0000 7f6c41b93700 1 Processor -- start 2024-05-09T00:12:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.522+0000 7f6c41b93700 1 -- start start 2024-05-09T00:12:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.522+0000 7f6c41b93700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c1034c0 0x7f6c3c19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.522+0000 7f6c41b93700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c107df0 0x7f6c3c19d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.522+0000 7f6c41b93700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c3c108740 0x7f6c3c1a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.522+0000 7f6c41b93700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c113370 con 0x7f6c3c107df0 2024-05-09T00:12:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.522+0000 7f6c41b93700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1131f0 con 0x7f6c3c108740 2024-05-09T00:12:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.522+0000 7f6c41b93700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1134f0 con 0x7f6c3c1034c0 2024-05-09T00:12:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.523+0000 7f6c3affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c107df0 0x7f6c3c19d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.523+0000 7f6c3affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c107df0 0x7f6c3c19d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41772/0 (socket says 172.21.15.46:41772) 2024-05-09T00:12:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.523+0000 7f6c3affd700 1 -- 172.21.15.46:0/3701740554 learned_addr learned my addr 172.21.15.46:0/3701740554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.523+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c1034c0 0x7f6c3c19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.523+0000 7f6c3bfff700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c3c108740 0x7f6c3c1a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.523+0000 7f6c3affd700 1 -- 172.21.15.46:0/3701740554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c1034c0 msgr2=0x7f6c3c19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.523+0000 7f6c3affd700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c1034c0 0x7f6c3c19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.523+0000 7f6c3affd700 1 -- 172.21.15.46:0/3701740554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c3c108740 msgr2=0x7f6c3c1a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.523+0000 7f6c3affd700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c3c108740 0x7f6c3c1a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.523+0000 7f6c3affd700 1 -- 172.21.15.46:0/3701740554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c1a1e60 con 0x7f6c3c107df0 2024-05-09T00:12:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.523+0000 7f6c3affd700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c107df0 0x7f6c3c19d530 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000d980 tx=0x7f6c3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.524+0000 7f6c38ff9700 1 -- 172.21.15.46:0/3701740554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30017070 con 0x7f6c3c107df0 2024-05-09T00:12:59.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.524+0000 7f6c41b93700 1 -- 172.21.15.46:0/3701740554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3c1a2150 con 0x7f6c3c107df0 2024-05-09T00:12:59.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.524+0000 7f6c41b93700 1 -- 172.21.15.46:0/3701740554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3c1a9a70 con 0x7f6c3c107df0 2024-05-09T00:12:59.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.524+0000 7f6c38ff9700 1 -- 172.21.15.46:0/3701740554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3000fd40 con 0x7f6c3c107df0 2024-05-09T00:12:59.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.524+0000 7f6c38ff9700 1 -- 172.21.15.46:0/3701740554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000e3b0 con 0x7f6c3c107df0 2024-05-09T00:12:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.528+0000 7f6c38ff9700 1 -- 172.21.15.46:0/3701740554 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6c3000e550 con 0x7f6c3c107df0 2024-05-09T00:12:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.528+0000 7f6c505a9b80 1 Processor -- start 2024-05-09T00:12:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c505a9b80 1 -- start start 2024-05-09T00:12:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c505a9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 0x55dee1a47240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c505a9b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee18c44a0 0x55dee1a47780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c505a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1a41400 0x55dee1a41810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c505a9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dee19640a0 con 0x55dee193ed00 2024-05-09T00:12:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c505a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dee19099a0 con 0x55dee1a41400 2024-05-09T00:12:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c505a9b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dee1960060 con 0x55dee18c44a0 2024-05-09T00:12:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c38ff9700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6c240b2900 0x7f6c240b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c3bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 0x55dee1a47240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c3bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 0x55dee1a47240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41786/0 (socket says 172.21.15.46:41786) 2024-05-09T00:12:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c3bfff700 1 -- 172.21.15.46:0/326970780 learned_addr learned my addr 172.21.15.46:0/326970780 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/326970780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee18c44a0 0x55dee1a47780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c3affd700 1 --2- 172.21.15.46:0/326970780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1a41400 0x55dee1a41810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c38ff9700 1 -- 172.21.15.46:0/3701740554 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6c300e1e40 con 0x7f6c3c107df0 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.529+0000 7f6c3bfff700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6c240b2900 0x7f6c240b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.530+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/326970780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1a41400 msgr2=0x55dee1a41810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.530+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/326970780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1a41400 0x55dee1a41810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.530+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/326970780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 msgr2=0x55dee1a47240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.530+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/326970780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 0x55dee1a47240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.530+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/326970780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee18fa500 con 0x55dee18c44a0 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.530+0000 7f6c3bfff700 1 --2- 172.21.15.46:0/326970780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 0x55dee1a47240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.530+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/326970780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee18c44a0 0x55dee1a47780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009cb0 tx=0x7f6c2c011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdc70499b2e4f25f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.530+0000 7f6c3bfff700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6c240b2900 0x7f6c240b4e10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009550 tx=0x7f6c3400b3e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.530+0000 7f6c21ffb700 1 -- 172.21.15.46:0/326970780 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c011d20 con 0x55dee18c44a0 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.530+0000 7f6c21ffb700 1 -- 172.21.15.46:0/326970780 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c2c004500 con 0x55dee18c44a0 2024-05-09T00:12:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.530+0000 7f6c21ffb700 1 -- 172.21.15.46:0/326970780 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c005310 con 0x55dee18c44a0 2024-05-09T00:12:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.531+0000 7f6c505a9b80 1 -- 172.21.15.46:0/326970780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee18c44a0 msgr2=0x55dee1a47780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.531+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/326970780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee18c44a0 0x55dee1a47780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009cb0 tx=0x7f6c2c011440 comp rx=0 tx=0).stop 2024-05-09T00:12:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.531+0000 7f6c505a9b80 1 -- 172.21.15.46:0/326970780 shutdown_connections 2024-05-09T00:12:59.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.531+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/326970780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1a41400 0x55dee1a41810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.531+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/326970780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 0x55dee1a47240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.531+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/326970780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee18c44a0 0x55dee1a47780 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.531+0000 7f6c505a9b80 1 -- 172.21.15.46:0/326970780 >> 172.21.15.46:0/326970780 conn(0x55dee18c7f70 msgr2=0x55dee1a3c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.531+0000 7f6c505a9b80 1 -- 172.21.15.46:0/326970780 shutdown_connections 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.531+0000 7f6c505a9b80 1 -- 172.21.15.46:0/326970780 wait complete. 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.532+0000 7f6c505a9b80 1 -- start start 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.532+0000 7f6c505a9b80 1 Processor -- start 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.532+0000 7f6c505a9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 0x55dee1a4af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.532+0000 7f6c505a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee193ed00 0x55dee1a4b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.532+0000 7f6c505a9b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee1a41400 0x55dee1a4b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.532+0000 7f6c505a9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dee1a42160 con 0x55dee18c44a0 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.532+0000 7f6c505a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dee18feb20 con 0x55dee193ed00 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.532+0000 7f6c505a9b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dee1a4e020 con 0x55dee1a41400 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c505a9b80 10 client.0 _ll_register_callbacks cb 0x55dee17d56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:59.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 0x55dee1a4af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 0x55dee1a4af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41792/0 (socket says 172.21.15.46:41792) 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/3967325240 learned_addr learned my addr 172.21.15.46:0/3967325240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c3affd700 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee193ed00 0x55dee1a4b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c3bfff700 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee1a41400 0x55dee1a4b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/3967325240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee1a41400 msgr2=0x55dee1a4b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee1a41400 0x55dee1a4b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/3967325240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee193ed00 msgr2=0x55dee1a4b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee193ed00 0x55dee1a4b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/3967325240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee17df490 con 0x55dee18c44a0 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c3bfff700 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee1a41400 0x55dee1a4b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 0x55dee1a4af60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c004d30 tx=0x7f6c2c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aabda22064ecd9d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.533+0000 7f6c1affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c1affd700 1 -- 172.21.15.46:0/3967325240 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c004750 con 0x55dee18c44a0 2024-05-09T00:12:59.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c1affd700 1 -- 172.21.15.46:0/3967325240 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6c2c0048f0 con 0x55dee18c44a0 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c505a9b80 1 -- 172.21.15.46:0/3967325240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c2c004500 con 0x55dee18c44a0 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c1affd700 1 -- 172.21.15.46:0/3967325240 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c2c004ef0 con 0x55dee18c44a0 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c505a9b80 10 client.15333 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c505a9b80 1 -- 172.21.15.46:0/3967325240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dee1a589c0 con 0x55dee18c44a0 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c1affd700 1 -- 172.21.15.46:0/3967325240 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c2c004500 con 0x55dee18c44a0 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c1affd700 1 -- 172.21.15.46:0/3967325240 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6c2c018070 con 0x55dee18c44a0 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c505a9b80 10 client.15333 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c505a9b80 10 client.15333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c505a9b80 20 client.15333 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55dee1a5aee0 0x55dee1a5d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c505a9b80 4 client.15333 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.534+0000 7f6c505a9b80 1 -- 172.21.15.46:0/3967325240 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dee1971a80 con 0x55dee1a5aee0 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.535+0000 7f6c3affd700 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55dee1a5aee0 0x55dee1a5d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.535+0000 7f6c1affd700 10 client.15333 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:59.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.535+0000 7f6c3affd700 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55dee1a5aee0 0x55dee1a5d300 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a99554136f330347 server_cookie=51587a2dca1a3556 in_seq=0 out_seq=0 2024-05-09T00:12:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.536+0000 7f6c1affd700 1 -- 172.21.15.46:0/3967325240 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dee1971a80 con 0x55dee1a5aee0 2024-05-09T00:12:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.536+0000 7f6c1affd700 10 client.15333 handle_command_reply: tid=0 2024-05-09T00:12:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.536+0000 7f6c505a9b80 1 client.15333 shutdown 2024-05-09T00:12:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.536+0000 7f6c505a9b80 1 -- 172.21.15.46:0/3967325240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 msgr2=0x55dee1a4af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.536+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 0x55dee1a4af60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c004d30 tx=0x7f6c2c011b20 comp rx=0 tx=0).stop 2024-05-09T00:12:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.536+0000 7f6c505a9b80 1 -- 172.21.15.46:0/3967325240 shutdown_connections 2024-05-09T00:12:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.536+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55dee1a5aee0 0x55dee1a5d300 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.536+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee1a41400 0x55dee1a4b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.536+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 0x55dee1a4af60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.536+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/3967325240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee193ed00 0x55dee1a4b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.536+0000 7f6c1affd700 0 client.15333 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:59.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.536+0000 7f6c505a9b80 1 -- 172.21.15.46:0/3967325240 >> 172.21.15.46:0/3967325240 conn(0x55dee18c7f70 msgr2=0x55dee1a35ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:59.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.537+0000 7f6c505a9b80 1 -- 172.21.15.46:0/3967325240 shutdown_connections 2024-05-09T00:12:59.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.537+0000 7f6c505a9b80 1 -- 172.21.15.46:0/3967325240 wait complete. 2024-05-09T00:12:59.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.537+0000 7f6c505a9b80 20 client.15333 trim_cache size 0 max 16384 2024-05-09T00:12:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.570+0000 7f6c505a9b80 1 Processor -- start 2024-05-09T00:12:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c505a9b80 1 -- start start 2024-05-09T00:12:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c505a9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 0x55dee1a3c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c505a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee193ed00 0x55dee1a4b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c505a9b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee1a41400 0x55dee1a36700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c505a9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2800fc80 con 0x55dee18c44a0 2024-05-09T00:12:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c505a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2800d510 con 0x55dee193ed00 2024-05-09T00:12:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c505a9b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2800b340 con 0x55dee1a41400 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 0x55dee1a3c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 0x55dee1a3c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41808/0 (socket says 172.21.15.46:41808) 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/430068472 learned_addr learned my addr 172.21.15.46:0/430068472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c3bfff700 1 --2- 172.21.15.46:0/430068472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee1a41400 0x55dee1a36700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c3affd700 1 --2- 172.21.15.46:0/430068472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee193ed00 0x55dee1a4b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/430068472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee1a41400 msgr2=0x55dee1a36700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/430068472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee1a41400 0x55dee1a36700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/430068472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee193ed00 msgr2=0x55dee1a4b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/430068472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee193ed00 0x55dee1a4b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.571+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/430068472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee1a389d0 con 0x55dee18c44a0 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.572+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/430068472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 0x55dee1a3c390 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c005e40 tx=0x7f6c2c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3446117ef735c4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.572+0000 7f6c217fa700 1 -- 172.21.15.46:0/430068472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c02b400 con 0x55dee18c44a0 2024-05-09T00:12:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.572+0000 7f6c217fa700 1 -- 172.21.15.46:0/430068472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c2c027070 con 0x55dee18c44a0 2024-05-09T00:12:59.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.572+0000 7f6c217fa700 1 -- 172.21.15.46:0/430068472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c004760 con 0x55dee18c44a0 2024-05-09T00:12:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.573+0000 7f6c505a9b80 1 -- 172.21.15.46:0/430068472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 msgr2=0x55dee1a3c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.573+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/430068472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 0x55dee1a3c390 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c005e40 tx=0x7f6c2c011c40 comp rx=0 tx=0).stop 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.573+0000 7f6c505a9b80 1 -- 172.21.15.46:0/430068472 shutdown_connections 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.573+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/430068472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee1a41400 0x55dee1a36700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.573+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/430068472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee18c44a0 0x55dee1a3c390 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.573+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/430068472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee193ed00 0x55dee1a4b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.573+0000 7f6c505a9b80 1 -- 172.21.15.46:0/430068472 >> 172.21.15.46:0/430068472 conn(0x55dee18c7f70 msgr2=0x55dee1a35ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.573+0000 7f6c505a9b80 1 -- 172.21.15.46:0/430068472 shutdown_connections 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.573+0000 7f6c505a9b80 1 -- 172.21.15.46:0/430068472 wait complete. 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.574+0000 7f6c505a9b80 1 -- start start 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.574+0000 7f6c505a9b80 1 Processor -- start 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.574+0000 7f6c505a9b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee18c44a0 0x55dee1a623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.574+0000 7f6c505a9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 0x55dee1a628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.574+0000 7f6c505a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1a41400 0x55dee1a62e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.574+0000 7f6c505a9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2800b9d0 con 0x55dee193ed00 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.574+0000 7f6c505a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2800ce40 con 0x55dee1a41400 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.574+0000 7f6c505a9b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2800c0a0 con 0x55dee18c44a0 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.574+0000 7f6c505a9b80 10 client.0 _ll_register_callbacks cb 0x55dee17d56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.574+0000 7f6c3affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 0x55dee1a628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.574+0000 7f6c3affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 0x55dee1a628f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41818/0 (socket says 172.21.15.46:41818) 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.574+0000 7f6c3affd700 1 -- 172.21.15.46:0/1231493863 learned_addr learned my addr 172.21.15.46:0/1231493863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee18c44a0 0x55dee1a623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c3bfff700 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1a41400 0x55dee1a62e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c3affd700 1 -- 172.21.15.46:0/1231493863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee18c44a0 msgr2=0x55dee1a623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c3affd700 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee18c44a0 0x55dee1a623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c3affd700 1 -- 172.21.15.46:0/1231493863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1a41400 msgr2=0x55dee1a62e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c3affd700 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1a41400 0x55dee1a62e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c3affd700 1 -- 172.21.15.46:0/1231493863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee1a5d7a0 con 0x55dee193ed00 2024-05-09T00:12:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c3bfff700 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1a41400 0x55dee1a62e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c3affd700 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 0x55dee1a628f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6c300a9160 tx=0x7f6c30017c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7706573ea67c378c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c1affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c1affd700 1 -- 172.21.15.46:0/1231493863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c300a9460 con 0x55dee193ed00 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c1affd700 1 -- 172.21.15.46:0/1231493863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6c300a5a50 con 0x55dee193ed00 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c505a9b80 1 -- 172.21.15.46:0/1231493863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c2c027070 con 0x55dee193ed00 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c1affd700 1 -- 172.21.15.46:0/1231493863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c300ad810 con 0x55dee193ed00 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c505a9b80 10 client.15339 fetch_fsmap learned FSMap version 26 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.575+0000 7f6c505a9b80 1 -- 172.21.15.46:0/1231493863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dee1a47af0 con 0x55dee193ed00 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.576+0000 7f6c1affd700 1 -- 172.21.15.46:0/1231493863 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c2c027070 con 0x55dee193ed00 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.576+0000 7f6c1affd700 1 -- 172.21.15.46:0/1231493863 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6c300b1e30 con 0x55dee193ed00 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.576+0000 7f6c505a9b80 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.576+0000 7f6c505a9b80 10 client.15339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.576+0000 7f6c505a9b80 20 client.15339 populate_metadata read hostname 'smithi046' 2024-05-09T00:12:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.576+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55dee1ac5d50 0x55dee1ac8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:12:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.576+0000 7f6c505a9b80 4 client.15339 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:12:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.576+0000 7f6c505a9b80 1 -- 172.21.15.46:0/1231493863 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dee196bb20 con 0x55dee1ac5d50 2024-05-09T00:12:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.576+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55dee1ac5d50 0x55dee1ac8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55dee1ac5d50 0x55dee1ac8170 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f1f79e81aa1090 server_cookie=7bdee8a85f7f4183 in_seq=0 out_seq=0 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c1affd700 10 client.15339 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c1affd700 1 -- 172.21.15.46:0/1231493863 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dee196bb20 con 0x55dee1ac5d50 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c1affd700 10 client.15339 handle_command_reply: tid=0 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c505a9b80 1 client.15339 shutdown 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c505a9b80 1 -- 172.21.15.46:0/1231493863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 msgr2=0x55dee1a628f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 0x55dee1a628f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6c300a9160 tx=0x7f6c30017c60 comp rx=0 tx=0).stop 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c505a9b80 1 -- 172.21.15.46:0/1231493863 shutdown_connections 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55dee1ac5d50 0x55dee1ac8170 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1a41400 0x55dee1a62e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dee18c44a0 0x55dee1a623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c505a9b80 1 --2- 172.21.15.46:0/1231493863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dee193ed00 0x55dee1a628f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.577+0000 7f6c505a9b80 1 -- 172.21.15.46:0/1231493863 >> 172.21.15.46:0/1231493863 conn(0x55dee18c7f70 msgr2=0x55dee1a45460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.578+0000 7f6c1affd700 0 client.15339 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:12:59.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.578+0000 7f6c505a9b80 1 -- 172.21.15.46:0/1231493863 shutdown_connections 2024-05-09T00:12:59.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.578+0000 7f6c505a9b80 1 -- 172.21.15.46:0/1231493863 wait complete. 2024-05-09T00:12:59.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.578+0000 7f6c505a9b80 20 client.15339 trim_cache size 0 max 16384 2024-05-09T00:12:59.580 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:12:59.580 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:12:59.580 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:12:59.580 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:12:59.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.578+0000 7f6c41b93700 1 -- 172.21.15.46:0/3701740554 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6c240b2900 msgr2=0x7f6c240b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.578+0000 7f6c41b93700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6c240b2900 0x7f6c240b4e10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009550 tx=0x7f6c3400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.578+0000 7f6c41b93700 1 -- 172.21.15.46:0/3701740554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c107df0 msgr2=0x7f6c3c19d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:12:59.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.578+0000 7f6c41b93700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c107df0 0x7f6c3c19d530 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000d980 tx=0x7f6c3000dc90 comp rx=0 tx=0).stop 2024-05-09T00:12:59.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.579+0000 7f6c41b93700 1 -- 172.21.15.46:0/3701740554 shutdown_connections 2024-05-09T00:12:59.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.579+0000 7f6c41b93700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6c240b2900 0x7f6c240b4e10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.579+0000 7f6c41b93700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c3c108740 0x7f6c3c1a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.579+0000 7f6c41b93700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c3c1034c0 0x7f6c3c19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.579+0000 7f6c41b93700 1 --2- 172.21.15.46:0/3701740554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c107df0 0x7f6c3c19d530 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:12:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.579+0000 7f6c41b93700 1 -- 172.21.15.46:0/3701740554 >> 172.21.15.46:0/3701740554 conn(0x7f6c3c0fedc0 msgr2=0x7f6c3c109700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:12:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.579+0000 7f6c41b93700 1 -- 172.21.15.46:0/3701740554 shutdown_connections 2024-05-09T00:12:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:12:59.579+0000 7f6c41b93700 1 -- 172.21.15.46:0/3701740554 wait complete. 2024-05-09T00:13:00.301 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:13:00.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:13:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:00.702 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:00 smithi046 ceph-mon[30151]: pgmap v425: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-09T00:13:00.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:00 smithi169 ceph-mon[34777]: pgmap v425: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-09T00:13:00.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:00 smithi142 ceph-mon[34533]: pgmap v425: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-09T00:13:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.525+0000 7f907eaec700 1 -- 172.21.15.46:0/342640442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104650 msgr2=0x7f9078104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.525+0000 7f907eaec700 1 --2- 172.21.15.46:0/342640442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104650 0x7f9078104a60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f906c0099c0 tx=0x7f906c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:13:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.525+0000 7f907eaec700 1 -- 172.21.15.46:0/342640442 shutdown_connections 2024-05-09T00:13:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.525+0000 7f907eaec700 1 --2- 172.21.15.46:0/342640442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9078109880 0x7f90781111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.525+0000 7f907eaec700 1 --2- 172.21.15.46:0/342640442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104650 0x7f9078104a60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.525+0000 7f907eaec700 1 --2- 172.21.15.46:0/342640442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9078108f30 0x7f9078109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.525+0000 7f907eaec700 1 -- 172.21.15.46:0/342640442 >> 172.21.15.46:0/342640442 conn(0x7f9078079f00 msgr2=0x7f907807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.525+0000 7f907eaec700 1 -- 172.21.15.46:0/342640442 shutdown_connections 2024-05-09T00:13:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.526+0000 7f907eaec700 1 -- 172.21.15.46:0/342640442 wait complete. 2024-05-09T00:13:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.527+0000 7f907eaec700 1 Processor -- start 2024-05-09T00:13:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.527+0000 7f907eaec700 1 -- start start 2024-05-09T00:13:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.527+0000 7f907eaec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104650 0x7f9078075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.527+0000 7f907eaec700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9078108f30 0x7f9078076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.527+0000 7f907eaec700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9078109880 0x7f9078076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.527+0000 7f907eaec700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90781144b0 con 0x7f9078104650 2024-05-09T00:13:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.527+0000 7f907eaec700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9078114330 con 0x7f9078108f30 2024-05-09T00:13:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.527+0000 7f907eaec700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9078114630 con 0x7f9078109880 2024-05-09T00:13:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.527+0000 7f907c888700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104650 0x7f9078075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.527+0000 7f907c888700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104650 0x7f9078075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41832/0 (socket says 172.21.15.46:41832) 2024-05-09T00:13:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.527+0000 7f907c888700 1 -- 172.21.15.46:0/3761509623 learned_addr learned my addr 172.21.15.46:0/3761509623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.527+0000 7f907d089700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9078109880 0x7f9078076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.528+0000 7f9077fff700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9078108f30 0x7f9078076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.528+0000 7f907c888700 1 -- 172.21.15.46:0/3761509623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9078109880 msgr2=0x7f9078076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.528+0000 7f907c888700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9078109880 0x7f9078076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.528+0000 7f907c888700 1 -- 172.21.15.46:0/3761509623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9078108f30 msgr2=0x7f9078076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.528+0000 7f907c888700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9078108f30 0x7f9078076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.528+0000 7f907c888700 1 -- 172.21.15.46:0/3761509623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90781a9200 con 0x7f9078104650 2024-05-09T00:13:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.528+0000 7f907c888700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104650 0x7f9078075ac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f906c006010 tx=0x7f906c0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.528+0000 7f9075ffb700 1 -- 172.21.15.46:0/3761509623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c026070 con 0x7f9078104650 2024-05-09T00:13:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.528+0000 7f907eaec700 1 -- 172.21.15.46:0/3761509623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90781a9490 con 0x7f9078104650 2024-05-09T00:13:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.528+0000 7f9075ffb700 1 -- 172.21.15.46:0/3761509623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906c00f460 con 0x7f9078104650 2024-05-09T00:13:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.528+0000 7f9075ffb700 1 -- 172.21.15.46:0/3761509623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c009e60 con 0x7f9078104650 2024-05-09T00:13:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.529+0000 7f907eaec700 1 -- 172.21.15.46:0/3761509623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90781a9940 con 0x7f9078104650 2024-05-09T00:13:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.530+0000 7f907eaec700 1 -- 172.21.15.46:0/3761509623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90780512c0 con 0x7f9078104650 2024-05-09T00:13:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.534+0000 7f9075ffb700 1 -- 172.21.15.46:0/3761509623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f906c029070 con 0x7f9078104650 2024-05-09T00:13:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.535+0000 7f9075ffb700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f90680b2980 0x7f90680b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.535+0000 7f9075ffb700 1 -- 172.21.15.46:0/3761509623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f906c0e2db0 con 0x7f9078104650 2024-05-09T00:13:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.535+0000 7f9077fff700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f90680b2980 0x7f90680b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.536+0000 7f9077fff700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f90680b2980 0x7f90680b4e90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f907807a430 tx=0x7f9064009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.540+0000 7f9075ffb700 1 -- 172.21.15.46:0/3761509623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f906c030080 con 0x7f9078104650 2024-05-09T00:13:02.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:02 smithi046 ceph-mon[30151]: pgmap v426: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:02.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:02 smithi169 ceph-mon[34777]: pgmap v426: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:02.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:02 smithi142 ceph-mon[34533]: pgmap v426: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:02.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.966+0000 7f907eaec700 1 -- 172.21.15.46:0/3761509623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f907802e5f0 con 0x7f9078104650 2024-05-09T00:13:02.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.967+0000 7f9075ffb700 1 -- 172.21.15.46:0/3761509623 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f906c013070 con 0x7f9078104650 2024-05-09T00:13:02.965 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:13:02.966 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:13:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.969+0000 7f907eaec700 1 -- 172.21.15.46:0/3761509623 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f90680b2980 msgr2=0x7f90680b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.969+0000 7f907eaec700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f90680b2980 0x7f90680b4e90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f907807a430 tx=0x7f9064009260 comp rx=0 tx=0).stop 2024-05-09T00:13:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.969+0000 7f907eaec700 1 -- 172.21.15.46:0/3761509623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104650 msgr2=0x7f9078075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.969+0000 7f907eaec700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104650 0x7f9078075ac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f906c006010 tx=0x7f906c0055f0 comp rx=0 tx=0).stop 2024-05-09T00:13:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.969+0000 7f907eaec700 1 -- 172.21.15.46:0/3761509623 shutdown_connections 2024-05-09T00:13:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.969+0000 7f907eaec700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f90680b2980 0x7f90680b4e90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.969+0000 7f907eaec700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9078109880 0x7f9078076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.969+0000 7f907eaec700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104650 0x7f9078075ac0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.969+0000 7f907eaec700 1 --2- 172.21.15.46:0/3761509623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9078108f30 0x7f9078076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.969+0000 7f907eaec700 1 -- 172.21.15.46:0/3761509623 >> 172.21.15.46:0/3761509623 conn(0x7f9078079f00 msgr2=0x7f9078102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.969+0000 7f907eaec700 1 -- 172.21.15.46:0/3761509623 shutdown_connections 2024-05-09T00:13:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:02.969+0000 7f907eaec700 1 -- 172.21.15.46:0/3761509623 wait complete. 2024-05-09T00:13:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:13:03.736 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:13:03.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:03 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3761509623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:13:03.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:03 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3761509623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:13:04.037 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:04.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:03 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3761509623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:13:04.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:04 smithi169 ceph-mon[34777]: pgmap v427: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:04.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:04 smithi142 ceph-mon[34533]: pgmap v427: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:05.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:04 smithi046 ceph-mon[30151]: pgmap v427: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.045+0000 7fc106372700 1 -- 172.21.15.46:0/372913625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc100103310 msgr2=0x7fc10010f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.045+0000 7fc106372700 1 --2- 172.21.15.46:0/372913625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc100103310 0x7fc10010f890 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800b0a0 tx=0x7fc0f800b3b0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.045+0000 7fc106372700 1 -- 172.21.15.46:0/372913625 shutdown_connections 2024-05-09T00:13:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.045+0000 7fc106372700 1 --2- 172.21.15.46:0/372913625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc100103310 0x7fc10010f890 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.045+0000 7fc106372700 1 --2- 172.21.15.46:0/372913625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1001089f0 0x7fc100108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.045+0000 7fc106372700 1 --2- 172.21.15.46:0/372913625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1001029c0 0x7fc100102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.045+0000 7fc106372700 1 -- 172.21.15.46:0/372913625 >> 172.21.15.46:0/372913625 conn(0x7fc1000fe4e0 msgr2=0x7fc100100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.046+0000 7fc106372700 1 -- 172.21.15.46:0/372913625 shutdown_connections 2024-05-09T00:13:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.046+0000 7fc106372700 1 -- 172.21.15.46:0/372913625 wait complete. 2024-05-09T00:13:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.047+0000 7fc106372700 1 Processor -- start 2024-05-09T00:13:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.047+0000 7fc106372700 1 -- start start 2024-05-09T00:13:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.047+0000 7fc106372700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1001029c0 0x7fc10019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.047+0000 7fc106372700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100103310 0x7fc10019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.047+0000 7fc106372700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1001089f0 0x7fc10019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.047+0000 7fc106372700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc100114520 con 0x7fc1001029c0 2024-05-09T00:13:06.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.047+0000 7fc106372700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001143a0 con 0x7fc100103310 2024-05-09T00:13:06.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.047+0000 7fc106372700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001146a0 con 0x7fc1001089f0 2024-05-09T00:13:06.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.048+0000 7fc0fffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1001029c0 0x7fc10019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.048+0000 7fc0fffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1001029c0 0x7fc10019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41842/0 (socket says 172.21.15.46:41842) 2024-05-09T00:13:06.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.048+0000 7fc0fffff700 1 -- 172.21.15.46:0/2556725942 learned_addr learned my addr 172.21.15.46:0/2556725942 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:06.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.048+0000 7fc10490f700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1001089f0 0x7fc10019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.048+0000 7fc0ff7fe700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100103310 0x7fc10019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.048+0000 7fc0fffff700 1 -- 172.21.15.46:0/2556725942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1001089f0 msgr2=0x7fc10019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:06.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.048+0000 7fc0fffff700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1001089f0 0x7fc10019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.048+0000 7fc0fffff700 1 -- 172.21.15.46:0/2556725942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100103310 msgr2=0x7fc10019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:06.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.048+0000 7fc0fffff700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100103310 0x7fc10019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.048+0000 7fc0fffff700 1 -- 172.21.15.46:0/2556725942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10019fce0 con 0x7fc1001029c0 2024-05-09T00:13:06.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.048+0000 7fc0fffff700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1001029c0 0x7fc10019ae10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000e970 tx=0x7fc0f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:06.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.049+0000 7fc0fd7fa700 1 -- 172.21.15.46:0/2556725942 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0004a30 con 0x7fc1001029c0 2024-05-09T00:13:06.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.049+0000 7fc0fd7fa700 1 -- 172.21.15.46:0/2556725942 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f000cc90 con 0x7fc1001029c0 2024-05-09T00:13:06.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.049+0000 7fc0fd7fa700 1 -- 172.21.15.46:0/2556725942 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0005610 con 0x7fc1001029c0 2024-05-09T00:13:06.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.049+0000 7fc106372700 1 -- 172.21.15.46:0/2556725942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10019ffd0 con 0x7fc1001029c0 2024-05-09T00:13:06.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.049+0000 7fc106372700 1 -- 172.21.15.46:0/2556725942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc100110f10 con 0x7fc1001029c0 2024-05-09T00:13:06.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.052+0000 7fc0fd7fa700 1 -- 172.21.15.46:0/2556725942 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc0f000c7e0 con 0x7fc1001029c0 2024-05-09T00:13:06.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.053+0000 7fc0fd7fa700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0e80b2880 0x7fc0e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.053+0000 7fc0ff7fe700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0e80b2880 0x7fc0e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.053+0000 7fc0fd7fa700 1 -- 172.21.15.46:0/2556725942 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc0f00e22f0 con 0x7fc1001029c0 2024-05-09T00:13:06.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.054+0000 7fc0ff7fe700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0e80b2880 0x7fc0e80b4d90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc100101f90 tx=0x7fc0f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:06.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.054+0000 7fc114d88b80 1 Processor -- start 2024-05-09T00:13:06.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.054+0000 7fc114d88b80 1 -- start start 2024-05-09T00:13:06.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc114d88b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 0x55777d662240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc114d88b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d4df4a0 0x55777d662780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc114d88b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d65c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc114d88b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55777d57f0a0 con 0x55777d559d00 2024-05-09T00:13:06.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc114d88b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55777d5249a0 con 0x55777d4df4a0 2024-05-09T00:13:06.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc114d88b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55777d57b060 con 0x55777d65c400 2024-05-09T00:13:06.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc0fffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 0x55777d662240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc10490f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d65c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc0ff7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d4df4a0 0x55777d662780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc0fffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 0x55777d662240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41844/0 (socket says 172.21.15.46:41844) 2024-05-09T00:13:06.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc0ff7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d4df4a0 0x55777d662780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:38398/0 (socket says 172.21.15.46:38398) 2024-05-09T00:13:06.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc0fffff700 1 -- 172.21.15.46:0/1907515471 learned_addr learned my addr 172.21.15.46:0/1907515471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:06.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc10490f700 1 -- 172.21.15.46:0/1907515471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d4df4a0 msgr2=0x55777d662780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:06.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.055+0000 7fc10490f700 1 --2- 172.21.15.46:0/1907515471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d4df4a0 0x55777d662780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.056+0000 7fc10490f700 1 -- 172.21.15.46:0/1907515471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 msgr2=0x55777d662240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.056+0000 7fc10490f700 1 --2- 172.21.15.46:0/1907515471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 0x55777d662240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.056+0000 7fc10490f700 1 -- 172.21.15.46:0/1907515471 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777d515500 con 0x55777d65c400 2024-05-09T00:13:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.056+0000 7fc10490f700 1 --2- 172.21.15.46:0/1907515471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d65c810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009fd0 tx=0x7fc0f8009d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=511620b2c3226684 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.056+0000 7fc0e67fc700 1 -- 172.21.15.46:0/1907515471 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8006020 con 0x55777d65c400 2024-05-09T00:13:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.056+0000 7fc0e67fc700 1 -- 172.21.15.46:0/1907515471 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f8003a00 con 0x55777d65c400 2024-05-09T00:13:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.056+0000 7fc0e67fc700 1 -- 172.21.15.46:0/1907515471 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8007ca0 con 0x55777d65c400 2024-05-09T00:13:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.057+0000 7fc114d88b80 1 -- 172.21.15.46:0/1907515471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 msgr2=0x55777d65c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.057+0000 7fc114d88b80 1 --2- 172.21.15.46:0/1907515471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d65c810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009fd0 tx=0x7fc0f8009d90 comp rx=0 tx=0).stop 2024-05-09T00:13:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.058+0000 7fc114d88b80 1 -- 172.21.15.46:0/1907515471 shutdown_connections 2024-05-09T00:13:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.058+0000 7fc114d88b80 1 --2- 172.21.15.46:0/1907515471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d65c810 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.058+0000 7fc114d88b80 1 --2- 172.21.15.46:0/1907515471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 0x55777d662240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.058+0000 7fc114d88b80 1 --2- 172.21.15.46:0/1907515471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d4df4a0 0x55777d662780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.058+0000 7fc114d88b80 1 -- 172.21.15.46:0/1907515471 >> 172.21.15.46:0/1907515471 conn(0x55777d4e2f70 msgr2=0x55777d657220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.058+0000 7fc114d88b80 1 -- 172.21.15.46:0/1907515471 shutdown_connections 2024-05-09T00:13:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.058+0000 7fc114d88b80 1 -- 172.21.15.46:0/1907515471 wait complete. 2024-05-09T00:13:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.059+0000 7fc114d88b80 1 -- start start 2024-05-09T00:13:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.059+0000 7fc114d88b80 1 Processor -- start 2024-05-09T00:13:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.059+0000 7fc114d88b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 0x55777d666020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc114d88b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d559d00 0x55777d666560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc114d88b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d666aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc114d88b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55777d65d100 con 0x55777d4df4a0 2024-05-09T00:13:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc114d88b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55777d519b20 con 0x55777d559d00 2024-05-09T00:13:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc114d88b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55777d6721d0 con 0x55777d65c400 2024-05-09T00:13:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc0fffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 0x55777d666020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc114d88b80 10 client.0 _ll_register_callbacks cb 0x55777d3f06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc0fffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 0x55777d666020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41850/0 (socket says 172.21.15.46:41850) 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc0fffff700 1 -- 172.21.15.46:0/3699868940 learned_addr learned my addr 172.21.15.46:0/3699868940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc0ff7fe700 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d559d00 0x55777d666560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc10490f700 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d666aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc0fffff700 1 -- 172.21.15.46:0/3699868940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 msgr2=0x55777d666aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc0fffff700 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d666aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc0fffff700 1 -- 172.21.15.46:0/3699868940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d559d00 msgr2=0x55777d666560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc0fffff700 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d559d00 0x55777d666560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc0fffff700 1 -- 172.21.15.46:0/3699868940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777d3fa490 con 0x55777d4df4a0 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc0fffff700 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 0x55777d666020 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009230 tx=0x7fc0f0004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabfd62ea611c9ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.060+0000 7fc0df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.061+0000 7fc0df7fe700 1 -- 172.21.15.46:0/3699868940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f00c2020 con 0x55777d4df4a0 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.061+0000 7fc0df7fe700 1 -- 172.21.15.46:0/3699868940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc0f00a9b70 con 0x55777d4df4a0 2024-05-09T00:13:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.061+0000 7fc114d88b80 1 -- 172.21.15.46:0/3699868940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f8003a00 con 0x55777d4df4a0 2024-05-09T00:13:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.061+0000 7fc0df7fe700 1 -- 172.21.15.46:0/3699868940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc0f000cad0 con 0x55777d4df4a0 2024-05-09T00:13:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.061+0000 7fc114d88b80 10 client.15357 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.061+0000 7fc114d88b80 1 -- 172.21.15.46:0/3699868940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55777d673b60 con 0x55777d4df4a0 2024-05-09T00:13:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.061+0000 7fc0df7fe700 1 -- 172.21.15.46:0/3699868940 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f8003a00 con 0x55777d4df4a0 2024-05-09T00:13:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.061+0000 7fc0df7fe700 1 -- 172.21.15.46:0/3699868940 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc0f00c7050 con 0x55777d4df4a0 2024-05-09T00:13:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.062+0000 7fc114d88b80 10 client.15357 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.062+0000 7fc114d88b80 10 client.15357 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:13:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.062+0000 7fc114d88b80 20 client.15357 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.062+0000 7fc114d88b80 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55777d6761a0 0x55777d6785e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.062+0000 7fc114d88b80 4 client.15357 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.062+0000 7fc114d88b80 1 -- 172.21.15.46:0/3699868940 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55777d58ca80 con 0x55777d6761a0 2024-05-09T00:13:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.062+0000 7fc0ff7fe700 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55777d6761a0 0x55777d6785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.063+0000 7fc0df7fe700 10 client.15357 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:13:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.063+0000 7fc0ff7fe700 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55777d6761a0 0x55777d6785e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7180b72e2cf339f7 server_cookie=2f97149f113a0713 in_seq=0 out_seq=0 2024-05-09T00:13:06.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.064+0000 7fc0df7fe700 1 -- 172.21.15.46:0/3699868940 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55777d58ca80 con 0x55777d6761a0 2024-05-09T00:13:06.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.064+0000 7fc0df7fe700 10 client.15357 handle_command_reply: tid=0 2024-05-09T00:13:06.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.064+0000 7fc114d88b80 1 client.15357 shutdown 2024-05-09T00:13:06.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.065+0000 7fc114d88b80 1 -- 172.21.15.46:0/3699868940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 msgr2=0x55777d666020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:06.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.065+0000 7fc114d88b80 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 0x55777d666020 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009230 tx=0x7fc0f0004bd0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.065+0000 7fc114d88b80 1 -- 172.21.15.46:0/3699868940 shutdown_connections 2024-05-09T00:13:06.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.065+0000 7fc114d88b80 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55777d6761a0 0x55777d6785e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.065+0000 7fc114d88b80 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d666aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.065+0000 7fc0df7fe700 0 client.15357 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:13:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.065+0000 7fc114d88b80 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 0x55777d666020 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.065+0000 7fc114d88b80 1 --2- 172.21.15.46:0/3699868940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d559d00 0x55777d666560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.065+0000 7fc114d88b80 1 -- 172.21.15.46:0/3699868940 >> 172.21.15.46:0/3699868940 conn(0x55777d4e2f70 msgr2=0x55777d650ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.065+0000 7fc114d88b80 1 -- 172.21.15.46:0/3699868940 shutdown_connections 2024-05-09T00:13:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.065+0000 7fc114d88b80 1 -- 172.21.15.46:0/3699868940 wait complete. 2024-05-09T00:13:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.065+0000 7fc114d88b80 20 client.15357 trim_cache size 0 max 16384 2024-05-09T00:13:06.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.092+0000 7fc114d88b80 1 Processor -- start 2024-05-09T00:13:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc114d88b80 1 -- start start 2024-05-09T00:13:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc114d88b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d4df4a0 0x55777d661820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc114d88b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 0x55777d6668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc114d88b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d67dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc114d88b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001b3680 con 0x55777d559d00 2024-05-09T00:13:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc114d88b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001b0f10 con 0x55777d4df4a0 2024-05-09T00:13:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc114d88b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001aed40 con 0x55777d65c400 2024-05-09T00:13:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc0ff7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 0x55777d6668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc0ff7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 0x55777d6668e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41856/0 (socket says 172.21.15.46:41856) 2024-05-09T00:13:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc0ff7fe700 1 -- 172.21.15.46:0/2060257846 learned_addr learned my addr 172.21.15.46:0/2060257846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc10490f700 1 --2- 172.21.15.46:0/2060257846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d67dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc0fffff700 1 --2- 172.21.15.46:0/2060257846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d4df4a0 0x55777d661820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc0ff7fe700 1 -- 172.21.15.46:0/2060257846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 msgr2=0x55777d67dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc0ff7fe700 1 --2- 172.21.15.46:0/2060257846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d67dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc0ff7fe700 1 -- 172.21.15.46:0/2060257846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d4df4a0 msgr2=0x55777d661820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc0ff7fe700 1 --2- 172.21.15.46:0/2060257846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d4df4a0 0x55777d661820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.093+0000 7fc0ff7fe700 1 -- 172.21.15.46:0/2060257846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777d6539d0 con 0x55777d559d00 2024-05-09T00:13:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.094+0000 7fc0fffff700 1 --2- 172.21.15.46:0/2060257846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d4df4a0 0x55777d661820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.094+0000 7fc0ff7fe700 1 --2- 172.21.15.46:0/2060257846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 0x55777d6668e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40097c0 tx=0x7fc0f4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da968ad13890c5c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.094+0000 7fc0e5ffb700 1 -- 172.21.15.46:0/2060257846 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4030070 con 0x55777d559d00 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.094+0000 7fc0e5ffb700 1 -- 172.21.15.46:0/2060257846 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f4003d70 con 0x55777d559d00 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.094+0000 7fc0e5ffb700 1 -- 172.21.15.46:0/2060257846 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f401d410 con 0x55777d559d00 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.095+0000 7fc114d88b80 1 -- 172.21.15.46:0/2060257846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 msgr2=0x55777d6668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.095+0000 7fc114d88b80 1 --2- 172.21.15.46:0/2060257846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 0x55777d6668e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40097c0 tx=0x7fc0f4014000 comp rx=0 tx=0).stop 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.096+0000 7fc114d88b80 1 -- 172.21.15.46:0/2060257846 shutdown_connections 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.096+0000 7fc114d88b80 1 --2- 172.21.15.46:0/2060257846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d67dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.096+0000 7fc114d88b80 1 --2- 172.21.15.46:0/2060257846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d4df4a0 0x55777d661820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.096+0000 7fc114d88b80 1 --2- 172.21.15.46:0/2060257846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d559d00 0x55777d6668e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.096+0000 7fc114d88b80 1 -- 172.21.15.46:0/2060257846 >> 172.21.15.46:0/2060257846 conn(0x55777d4e2f70 msgr2=0x55777d661110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.096+0000 7fc114d88b80 1 -- 172.21.15.46:0/2060257846 shutdown_connections 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.096+0000 7fc114d88b80 1 -- 172.21.15.46:0/2060257846 wait complete. 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.096+0000 7fc114d88b80 1 -- start start 2024-05-09T00:13:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.096+0000 7fc114d88b80 1 Processor -- start 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc114d88b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 0x55777d66e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc114d88b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d559d00 0x55777d66e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc114d88b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d67cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc114d88b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001af3d0 con 0x55777d4df4a0 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc114d88b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001b0840 con 0x55777d559d00 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc114d88b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001afaa0 con 0x55777d65c400 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc114d88b80 10 client.0 _ll_register_callbacks cb 0x55777d3f06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc0fffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 0x55777d66e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc0fffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 0x55777d66e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41870/0 (socket says 172.21.15.46:41870) 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc0fffff700 1 -- 172.21.15.46:0/2619965710 learned_addr learned my addr 172.21.15.46:0/2619965710 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc10490f700 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d67cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc0ff7fe700 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d559d00 0x55777d66e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc0fffff700 1 -- 172.21.15.46:0/2619965710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 msgr2=0x55777d67cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc0fffff700 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d67cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc0fffff700 1 -- 172.21.15.46:0/2619965710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d559d00 msgr2=0x55777d66e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.097+0000 7fc0fffff700 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d559d00 0x55777d66e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.098+0000 7fc0fffff700 1 -- 172.21.15.46:0/2619965710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777d67d4b0 con 0x55777d4df4a0 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.098+0000 7fc0fffff700 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 0x55777d66e130 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00a6b80 tx=0x7fc0f006f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30e7344a72d2ac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.098+0000 7fc0df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.098+0000 7fc0df7fe700 1 -- 172.21.15.46:0/2619965710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0089020 con 0x55777d4df4a0 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.098+0000 7fc0df7fe700 1 -- 172.21.15.46:0/2619965710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc0f00a5f00 con 0x55777d4df4a0 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.098+0000 7fc114d88b80 1 -- 172.21.15.46:0/2619965710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f4003d70 con 0x55777d4df4a0 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.098+0000 7fc0df7fe700 1 -- 172.21.15.46:0/2619965710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc0f00c2070 con 0x55777d4df4a0 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.098+0000 7fc114d88b80 10 client.15363 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.098+0000 7fc114d88b80 1 -- 172.21.15.46:0/2619965710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55777d6e02f0 con 0x55777d4df4a0 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.099+0000 7fc0df7fe700 1 -- 172.21.15.46:0/2619965710 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f4003d70 con 0x55777d4df4a0 2024-05-09T00:13:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.099+0000 7fc0df7fe700 1 -- 172.21.15.46:0/2619965710 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc0f00c7070 con 0x55777d4df4a0 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.099+0000 7fc114d88b80 10 client.15363 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.099+0000 7fc114d88b80 10 client.15363 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.099+0000 7fc114d88b80 20 client.15363 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.099+0000 7fc114d88b80 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55777d6761a0 0x55777d6e2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.099+0000 7fc114d88b80 4 client.15363 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.099+0000 7fc114d88b80 1 -- 172.21.15.46:0/2619965710 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55777d586b20 con 0x55777d6761a0 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.099+0000 7fc0ff7fe700 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55777d6761a0 0x55777d6e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.100+0000 7fc0ff7fe700 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55777d6761a0 0x55777d6e2820 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f35372f40a0aa7d server_cookie=3b419559b062fa19 in_seq=0 out_seq=0 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.100+0000 7fc0df7fe700 10 client.15363 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.100+0000 7fc0df7fe700 1 -- 172.21.15.46:0/2619965710 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55777d586b20 con 0x55777d6761a0 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.100+0000 7fc0df7fe700 10 client.15363 handle_command_reply: tid=0 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.100+0000 7fc114d88b80 1 client.15363 shutdown 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.101+0000 7fc114d88b80 1 -- 172.21.15.46:0/2619965710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 msgr2=0x55777d66e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.101+0000 7fc114d88b80 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 0x55777d66e130 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00a6b80 tx=0x7fc0f006f4b0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.101+0000 7fc114d88b80 1 -- 172.21.15.46:0/2619965710 shutdown_connections 2024-05-09T00:13:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.101+0000 7fc114d88b80 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55777d6761a0 0x55777d6e2820 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.101+0000 7fc114d88b80 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55777d65c400 0x55777d67cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.101+0000 7fc114d88b80 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777d4df4a0 0x55777d66e130 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.101+0000 7fc0df7fe700 0 client.15363 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:13:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.101+0000 7fc114d88b80 1 --2- 172.21.15.46:0/2619965710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55777d559d00 0x55777d66e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.101+0000 7fc114d88b80 1 -- 172.21.15.46:0/2619965710 >> 172.21.15.46:0/2619965710 conn(0x55777d4e2f70 msgr2=0x55777d650ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.101+0000 7fc114d88b80 1 -- 172.21.15.46:0/2619965710 shutdown_connections 2024-05-09T00:13:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.101+0000 7fc114d88b80 1 -- 172.21.15.46:0/2619965710 wait complete. 2024-05-09T00:13:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.101+0000 7fc114d88b80 20 client.15363 trim_cache size 0 max 16384 2024-05-09T00:13:06.107 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.102+0000 7fc106372700 1 -- 172.21.15.46:0/2556725942 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0e80b2880 msgr2=0x7fc0e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.102+0000 7fc106372700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0e80b2880 0x7fc0e80b4d90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc100101f90 tx=0x7fc0f4009260 comp rx=0 tx=0).stop 2024-05-09T00:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.102+0000 7fc106372700 1 -- 172.21.15.46:0/2556725942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1001029c0 msgr2=0x7fc10019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.102+0000 7fc106372700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1001029c0 0x7fc10019ae10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000e970 tx=0x7fc0f000ec80 comp rx=0 tx=0).stop 2024-05-09T00:13:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.102+0000 7fc106372700 1 -- 172.21.15.46:0/2556725942 shutdown_connections 2024-05-09T00:13:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.102+0000 7fc106372700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0e80b2880 0x7fc0e80b4d90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.102+0000 7fc106372700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1001089f0 0x7fc10019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.102+0000 7fc106372700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1001029c0 0x7fc10019ae10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.102+0000 7fc106372700 1 --2- 172.21.15.46:0/2556725942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100103310 0x7fc10019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.102+0000 7fc106372700 1 -- 172.21.15.46:0/2556725942 >> 172.21.15.46:0/2556725942 conn(0x7fc1000fe4e0 msgr2=0x7fc100106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.102+0000 7fc106372700 1 -- 172.21.15.46:0/2556725942 shutdown_connections 2024-05-09T00:13:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:06.102+0000 7fc106372700 1 -- 172.21.15.46:0/2556725942 wait complete. 2024-05-09T00:13:06.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:06 smithi046 ceph-mon[30151]: pgmap v428: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-05-09T00:13:06.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:06 smithi169 ceph-mon[34777]: pgmap v428: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-05-09T00:13:06.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:13:06.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:06 smithi142 ceph-mon[34533]: pgmap v428: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-05-09T00:13:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:08.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:08 smithi046 ceph-mon[30151]: pgmap v429: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:08.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:08 smithi169 ceph-mon[34777]: pgmap v429: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:08.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:08 smithi142 ceph-mon[34533]: pgmap v429: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.161+0000 7f1f66b7e700 1 -- 172.21.15.46:0/2687779172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f601029c0 msgr2=0x7f1f60102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.161+0000 7f1f66b7e700 1 --2- 172.21.15.46:0/2687779172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f601029c0 0x7f1f60102dd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00aaa0 tx=0x7f1f4c00adb0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.162+0000 7f1f66b7e700 1 -- 172.21.15.46:0/2687779172 shutdown_connections 2024-05-09T00:13:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.162+0000 7f1f66b7e700 1 --2- 172.21.15.46:0/2687779172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f60103310 0x7f1f6010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.162+0000 7f1f66b7e700 1 --2- 172.21.15.46:0/2687779172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f601089f0 0x7f1f60108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.162+0000 7f1f66b7e700 1 --2- 172.21.15.46:0/2687779172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f601029c0 0x7f1f60102dd0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.162+0000 7f1f66b7e700 1 -- 172.21.15.46:0/2687779172 >> 172.21.15.46:0/2687779172 conn(0x7f1f600fe6f0 msgr2=0x7f1f60100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.162+0000 7f1f66b7e700 1 -- 172.21.15.46:0/2687779172 shutdown_connections 2024-05-09T00:13:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.162+0000 7f1f66b7e700 1 -- 172.21.15.46:0/2687779172 wait complete. 2024-05-09T00:13:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.163+0000 7f1f66b7e700 1 Processor -- start 2024-05-09T00:13:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.163+0000 7f1f66b7e700 1 -- start start 2024-05-09T00:13:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.163+0000 7f1f66b7e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f601029c0 0x7f1f6019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.163+0000 7f1f66b7e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f60103310 0x7f1f6019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.163+0000 7f1f5ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f60103310 0x7f1f6019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.163+0000 7f1f5ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f60103310 0x7f1f6019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36538/0 (socket says 172.21.15.46:36538) 2024-05-09T00:13:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.163+0000 7f1f66b7e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f601089f0 0x7f1f60198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.163+0000 7f1f5ffff700 1 -- 172.21.15.46:0/4110455070 learned_addr learned my addr 172.21.15.46:0/4110455070 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.164+0000 7f1f6491a700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f601029c0 0x7f1f6019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.163+0000 7f1f66b7e700 1 -- 172.21.15.46:0/4110455070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f601122e0 con 0x7f1f60103310 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.164+0000 7f1f66b7e700 1 -- 172.21.15.46:0/4110455070 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1f60112160 con 0x7f1f601089f0 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.164+0000 7f1f66b7e700 1 -- 172.21.15.46:0/4110455070 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1f60112460 con 0x7f1f601029c0 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.164+0000 7f1f6511b700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f601089f0 0x7f1f60198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.164+0000 7f1f6511b700 1 -- 172.21.15.46:0/4110455070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f601029c0 msgr2=0x7f1f6019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.164+0000 7f1f6511b700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f601029c0 0x7f1f6019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.164+0000 7f1f6511b700 1 -- 172.21.15.46:0/4110455070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f60103310 msgr2=0x7f1f6019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.164+0000 7f1f6511b700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f60103310 0x7f1f6019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.164+0000 7f1f6511b700 1 -- 172.21.15.46:0/4110455070 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f60199130 con 0x7f1f601089f0 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.164+0000 7f1f5ffff700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f60103310 0x7f1f6019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.164+0000 7f1f6491a700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f601029c0 0x7f1f6019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.164+0000 7f1f6511b700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f601089f0 0x7f1f60198a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800ef30 tx=0x7f1f5800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.165+0000 7f1f5dffb700 1 -- 172.21.15.46:0/4110455070 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5801a030 con 0x7f1f601089f0 2024-05-09T00:13:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.165+0000 7f1f5dffb700 1 -- 172.21.15.46:0/4110455070 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f58009320 con 0x7f1f601089f0 2024-05-09T00:13:09.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.165+0000 7f1f66b7e700 1 -- 172.21.15.46:0/4110455070 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f60199420 con 0x7f1f601089f0 2024-05-09T00:13:09.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.165+0000 7f1f5dffb700 1 -- 172.21.15.46:0/4110455070 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800ce90 con 0x7f1f601089f0 2024-05-09T00:13:09.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.165+0000 7f1f66b7e700 1 -- 172.21.15.46:0/4110455070 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f601a56f0 con 0x7f1f601089f0 2024-05-09T00:13:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.169+0000 7f1f5dffb700 1 -- 172.21.15.46:0/4110455070 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1f5801d070 con 0x7f1f601089f0 2024-05-09T00:13:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.170+0000 7f1f5dffb700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1f500b2aa0 0x7f1f500b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.170+0000 7f1f5dffb700 1 -- 172.21.15.46:0/4110455070 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1f580e5b90 con 0x7f1f601089f0 2024-05-09T00:13:09.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.170+0000 7f1f6491a700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1f500b2aa0 0x7f1f500b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.171+0000 7f1f75594b80 1 Processor -- start 2024-05-09T00:13:09.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.171+0000 7f1f75594b80 1 -- start start 2024-05-09T00:13:09.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.171+0000 7f1f6491a700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1f500b2aa0 0x7f1f500b4fb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1f60199bc0 tx=0x7f1f54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:09.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.171+0000 7f1f75594b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf8219d00 0x55dbf8322240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.171+0000 7f1f75594b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf819f4a0 0x55dbf8322780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.171+0000 7f1f75594b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf831c400 0x55dbf831c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.171+0000 7f1f75594b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dbf823f0a0 con 0x55dbf831c400 2024-05-09T00:13:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.171+0000 7f1f75594b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dbf81e49a0 con 0x55dbf819f4a0 2024-05-09T00:13:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.171+0000 7f1f75594b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dbf823b060 con 0x55dbf8219d00 2024-05-09T00:13:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.171+0000 7f1f6491a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf8219d00 0x55dbf8322240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.171+0000 7f1f6491a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf8219d00 0x55dbf8322240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:54556/0 (socket says 172.21.15.46:54556) 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.171+0000 7f1f6491a700 1 -- 172.21.15.46:0/3324485756 learned_addr learned my addr 172.21.15.46:0/3324485756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.172+0000 7f1f5ffff700 1 --2- 172.21.15.46:0/3324485756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf819f4a0 0x55dbf8322780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.172+0000 7f1f6511b700 1 --2- 172.21.15.46:0/3324485756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf831c400 0x55dbf831c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.172+0000 7f1f6491a700 1 -- 172.21.15.46:0/3324485756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf819f4a0 msgr2=0x55dbf8322780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.172+0000 7f1f6491a700 1 --2- 172.21.15.46:0/3324485756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf819f4a0 0x55dbf8322780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.172+0000 7f1f6491a700 1 -- 172.21.15.46:0/3324485756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf831c400 msgr2=0x55dbf831c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.172+0000 7f1f6491a700 1 --2- 172.21.15.46:0/3324485756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf831c400 0x55dbf831c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.172+0000 7f1f6491a700 1 -- 172.21.15.46:0/3324485756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf81d5500 con 0x55dbf8219d00 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.172+0000 7f1f6491a700 1 --2- 172.21.15.46:0/3324485756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf8219d00 0x55dbf8322240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f54004430 tx=0x7f1f54014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8016f7a2c75e0b67 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.172+0000 7f1f6511b700 1 --2- 172.21.15.46:0/3324485756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf831c400 0x55dbf831c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.172+0000 7f1f4affd700 1 -- 172.21.15.46:0/3324485756 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54025bc0 con 0x55dbf8219d00 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.172+0000 7f1f4affd700 1 -- 172.21.15.46:0/3324485756 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f54029070 con 0x55dbf8219d00 2024-05-09T00:13:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.172+0000 7f1f5ffff700 1 --2- 172.21.15.46:0/3324485756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf819f4a0 0x55dbf8322780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.173+0000 7f1f4affd700 1 -- 172.21.15.46:0/3324485756 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400e3b0 con 0x55dbf8219d00 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.173+0000 7f1f75594b80 1 -- 172.21.15.46:0/3324485756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf8219d00 msgr2=0x55dbf8322240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.174+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3324485756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf8219d00 0x55dbf8322240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f54004430 tx=0x7f1f54014000 comp rx=0 tx=0).stop 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.174+0000 7f1f75594b80 1 -- 172.21.15.46:0/3324485756 shutdown_connections 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.174+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3324485756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf831c400 0x55dbf831c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.174+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3324485756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf8219d00 0x55dbf8322240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.174+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3324485756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf819f4a0 0x55dbf8322780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.174+0000 7f1f75594b80 1 -- 172.21.15.46:0/3324485756 >> 172.21.15.46:0/3324485756 conn(0x55dbf81a2f70 msgr2=0x55dbf8317220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.174+0000 7f1f75594b80 1 -- 172.21.15.46:0/3324485756 shutdown_connections 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.174+0000 7f1f75594b80 1 -- 172.21.15.46:0/3324485756 wait complete. 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.174+0000 7f1f75594b80 1 -- start start 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.174+0000 7f1f75594b80 1 Processor -- start 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.175+0000 7f1f75594b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf819f4a0 0x55dbf8326040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.175+0000 7f1f75594b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf8219d00 0x55dbf8326580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.175+0000 7f1f75594b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf831c400 0x55dbf8326ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.175+0000 7f1f75594b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dbf831d200 con 0x55dbf8219d00 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.175+0000 7f1f75594b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dbf81d9b20 con 0x55dbf819f4a0 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.175+0000 7f1f75594b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dbf8329020 con 0x55dbf831c400 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.175+0000 7f1f75594b80 10 client.0 _ll_register_callbacks cb 0x55dbf80b06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.175+0000 7f1f5ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf8219d00 0x55dbf8326580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.175+0000 7f1f5ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf8219d00 0x55dbf8326580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36564/0 (socket says 172.21.15.46:36564) 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.175+0000 7f1f5ffff700 1 -- 172.21.15.46:0/3185835112 learned_addr learned my addr 172.21.15.46:0/3185835112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.175+0000 7f1f6491a700 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf819f4a0 0x55dbf8326040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.176+0000 7f1f6511b700 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf831c400 0x55dbf8326ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.176+0000 7f1f6491a700 1 -- 172.21.15.46:0/3185835112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf831c400 msgr2=0x55dbf8326ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.176+0000 7f1f6491a700 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf831c400 0x55dbf8326ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.176+0000 7f1f6491a700 1 -- 172.21.15.46:0/3185835112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf8219d00 msgr2=0x55dbf8326580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.176+0000 7f1f6491a700 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf8219d00 0x55dbf8326580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.176+0000 7f1f6491a700 1 -- 172.21.15.46:0/3185835112 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf80ba490 con 0x55dbf819f4a0 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.176+0000 7f1f6491a700 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf819f4a0 0x55dbf8326040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1f54004400 tx=0x7f1f54025df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47f12e2cb6596ec1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.176+0000 7f1f3ffff700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.177+0000 7f1f3ffff700 1 -- 172.21.15.46:0/3185835112 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400e440 con 0x55dbf819f4a0 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.177+0000 7f1f75594b80 1 -- 172.21.15.46:0/3185835112 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f54029070 con 0x55dbf819f4a0 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.177+0000 7f1f3ffff700 1 -- 172.21.15.46:0/3185835112 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1f5401c3b0 con 0x55dbf819f4a0 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.177+0000 7f1f3ffff700 1 -- 172.21.15.46:0/3185835112 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1f5401c5b0 con 0x55dbf819f4a0 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.177+0000 7f1f75594b80 10 client.25094 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.177+0000 7f1f75594b80 1 -- 172.21.15.46:0/3185835112 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dbf8333a50 con 0x55dbf819f4a0 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.177+0000 7f1f3ffff700 1 -- 172.21.15.46:0/3185835112 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f54029070 con 0x55dbf819f4a0 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.178+0000 7f1f3ffff700 1 -- 172.21.15.46:0/3185835112 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1f54040050 con 0x55dbf819f4a0 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.178+0000 7f1f75594b80 10 client.25094 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.178+0000 7f1f75594b80 10 client.25094 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.178+0000 7f1f75594b80 20 client.25094 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.178+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55dbf8335f80 0x55dbf83383a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.178+0000 7f1f75594b80 4 client.25094 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.178+0000 7f1f75594b80 1 -- 172.21.15.46:0/3185835112 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbf824ca80 con 0x55dbf8335f80 2024-05-09T00:13:09.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.178+0000 7f1f5ffff700 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55dbf8335f80 0x55dbf83383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.179+0000 7f1f3ffff700 10 client.25094 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:13:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.179+0000 7f1f5ffff700 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55dbf8335f80 0x55dbf83383a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c65e71162b2e4e53 server_cookie=657a4e536b710ef9 in_seq=0 out_seq=0 2024-05-09T00:13:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.180+0000 7f1f3ffff700 1 -- 172.21.15.46:0/3185835112 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dbf824ca80 con 0x55dbf8335f80 2024-05-09T00:13:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.180+0000 7f1f3ffff700 10 client.25094 handle_command_reply: tid=0 2024-05-09T00:13:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.180+0000 7f1f75594b80 1 client.25094 shutdown 2024-05-09T00:13:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.180+0000 7f1f75594b80 1 -- 172.21.15.46:0/3185835112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf819f4a0 msgr2=0x55dbf8326040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.180+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf819f4a0 0x55dbf8326040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1f54004400 tx=0x7f1f54025df0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.180+0000 7f1f75594b80 1 -- 172.21.15.46:0/3185835112 shutdown_connections 2024-05-09T00:13:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.180+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55dbf8335f80 0x55dbf83383a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.180+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf831c400 0x55dbf8326ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.180+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf819f4a0 0x55dbf8326040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.180+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3185835112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf8219d00 0x55dbf8326580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.180+0000 7f1f75594b80 1 -- 172.21.15.46:0/3185835112 >> 172.21.15.46:0/3185835112 conn(0x55dbf81a2f70 msgr2=0x55dbf8310ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.180+0000 7f1f3ffff700 0 client.25094 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:13:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.181+0000 7f1f75594b80 1 -- 172.21.15.46:0/3185835112 shutdown_connections 2024-05-09T00:13:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.181+0000 7f1f75594b80 1 -- 172.21.15.46:0/3185835112 wait complete. 2024-05-09T00:13:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.181+0000 7f1f75594b80 20 client.25094 trim_cache size 0 max 16384 2024-05-09T00:13:09.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.211+0000 7f1f75594b80 1 Processor -- start 2024-05-09T00:13:09.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.211+0000 7f1f75594b80 1 -- start start 2024-05-09T00:13:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.211+0000 7f1f75594b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf819f4a0 0x55dbf8332280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.211+0000 7f1f75594b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf8219d00 0x55dbf8326900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.211+0000 7f1f75594b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf831c400 0x55dbf8311700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f75594b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f601af450 con 0x55dbf819f4a0 2024-05-09T00:13:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f6491a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf819f4a0 0x55dbf8332280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f6491a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf819f4a0 0x55dbf8332280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36568/0 (socket says 172.21.15.46:36568) 2024-05-09T00:13:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f6491a700 1 -- 172.21.15.46:0/3563815956 learned_addr learned my addr 172.21.15.46:0/3563815956 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f5ffff700 1 --2- 172.21.15.46:0/3563815956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf8219d00 0x55dbf8326900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f6511b700 1 --2- 172.21.15.46:0/3563815956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf831c400 0x55dbf8311700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f75594b80 1 -- 172.21.15.46:0/3563815956 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1f601acce0 con 0x55dbf8219d00 2024-05-09T00:13:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f75594b80 1 -- 172.21.15.46:0/3563815956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1f601aab10 con 0x55dbf831c400 2024-05-09T00:13:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f6491a700 1 -- 172.21.15.46:0/3563815956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf831c400 msgr2=0x55dbf8311700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f6491a700 1 --2- 172.21.15.46:0/3563815956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf831c400 0x55dbf8311700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f6491a700 1 -- 172.21.15.46:0/3563815956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf8219d00 msgr2=0x55dbf8326900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f6491a700 1 --2- 172.21.15.46:0/3563815956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf8219d00 0x55dbf8326900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f6491a700 1 -- 172.21.15.46:0/3563815956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf83139d0 con 0x55dbf819f4a0 2024-05-09T00:13:09.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.212+0000 7f1f5ffff700 1 --2- 172.21.15.46:0/3563815956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf8219d00 0x55dbf8326900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:09.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.213+0000 7f1f6491a700 1 --2- 172.21.15.46:0/3563815956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf819f4a0 0x55dbf8332280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1f54016040 tx=0x7f1f5401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109749b336723969 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:09.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.213+0000 7f1f4a7fc700 1 -- 172.21.15.46:0/3563815956 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54025e60 con 0x55dbf819f4a0 2024-05-09T00:13:09.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.213+0000 7f1f4a7fc700 1 -- 172.21.15.46:0/3563815956 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f54046020 con 0x55dbf819f4a0 2024-05-09T00:13:09.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.213+0000 7f1f4a7fc700 1 -- 172.21.15.46:0/3563815956 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400e6b0 con 0x55dbf819f4a0 2024-05-09T00:13:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.214+0000 7f1f75594b80 1 -- 172.21.15.46:0/3563815956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf819f4a0 msgr2=0x55dbf8332280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.214+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3563815956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf819f4a0 0x55dbf8332280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1f54016040 tx=0x7f1f5401c540 comp rx=0 tx=0).stop 2024-05-09T00:13:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.214+0000 7f1f75594b80 1 -- 172.21.15.46:0/3563815956 shutdown_connections 2024-05-09T00:13:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.214+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3563815956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf831c400 0x55dbf8311700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.214+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3563815956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf819f4a0 0x55dbf8332280 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.214+0000 7f1f75594b80 1 --2- 172.21.15.46:0/3563815956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf8219d00 0x55dbf8326900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.214+0000 7f1f75594b80 1 -- 172.21.15.46:0/3563815956 >> 172.21.15.46:0/3563815956 conn(0x55dbf81a2f70 msgr2=0x55dbf8310ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.215+0000 7f1f75594b80 1 -- 172.21.15.46:0/3563815956 shutdown_connections 2024-05-09T00:13:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.215+0000 7f1f75594b80 1 -- 172.21.15.46:0/3563815956 wait complete. 2024-05-09T00:13:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.215+0000 7f1f75594b80 1 -- start start 2024-05-09T00:13:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.215+0000 7f1f75594b80 1 Processor -- start 2024-05-09T00:13:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.216+0000 7f1f75594b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf819f4a0 0x55dbf8338640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.216+0000 7f1f75594b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf8219d00 0x55dbf8338b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.216+0000 7f1f6491a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf819f4a0 0x55dbf8338640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.216+0000 7f1f6491a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf819f4a0 0x55dbf8338640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:54572/0 (socket says 172.21.15.46:54572) 2024-05-09T00:13:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.216+0000 7f1f75594b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf831c400 0x55dbf839e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.216+0000 7f1f6491a700 1 -- 172.21.15.46:0/2427347952 learned_addr learned my addr 172.21.15.46:0/2427347952 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.216+0000 7f1f5ffff700 1 --2- 172.21.15.46:0/2427347952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf8219d00 0x55dbf8338b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.216+0000 7f1f75594b80 1 -- 172.21.15.46:0/2427347952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f601ab1a0 con 0x55dbf831c400 2024-05-09T00:13:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.216+0000 7f1f75594b80 1 -- 172.21.15.46:0/2427347952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1f601ac610 con 0x55dbf8219d00 2024-05-09T00:13:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.216+0000 7f1f75594b80 1 -- 172.21.15.46:0/2427347952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1f601ab870 con 0x55dbf819f4a0 2024-05-09T00:13:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.217+0000 7f1f75594b80 10 client.0 _ll_register_callbacks cb 0x55dbf80b06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.217+0000 7f1f5ffff700 1 -- 172.21.15.46:0/2427347952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf819f4a0 msgr2=0x55dbf8338640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.217+0000 7f1f5ffff700 1 --2- 172.21.15.46:0/2427347952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf819f4a0 0x55dbf8338640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.217+0000 7f1f5ffff700 1 -- 172.21.15.46:0/2427347952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf831c400 msgr2=0x55dbf839e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.217+0000 7f1f5ffff700 1 --2- 172.21.15.46:0/2427347952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf831c400 0x55dbf839e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.217+0000 7f1f5ffff700 1 -- 172.21.15.46:0/2427347952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf839ea00 con 0x55dbf8219d00 2024-05-09T00:13:09.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.217+0000 7f1f5ffff700 1 --2- 172.21.15.46:0/2427347952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf8219d00 0x55dbf8338b80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55dbf81d9b50 tx=0x7f1f4c0039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa05c07a174ca68 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.217+0000 7f1f3ffff700 10 client.25100 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.217+0000 7f1f75594b80 1 -- 172.21.15.46:0/2427347952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f54046020 con 0x55dbf8219d00 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.217+0000 7f1f3ffff700 1 -- 172.21.15.46:0/2427347952 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c01b040 con 0x55dbf8219d00 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.217+0000 7f1f3ffff700 1 -- 172.21.15.46:0/2427347952 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1f4c0052b0 con 0x55dbf8219d00 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.218+0000 7f1f3ffff700 1 -- 172.21.15.46:0/2427347952 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1f4c0054d0 con 0x55dbf8219d00 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.218+0000 7f1f75594b80 10 client.25100 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.218+0000 7f1f75594b80 1 -- 172.21.15.46:0/2427347952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55dbf839fa60 con 0x55dbf8219d00 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.218+0000 7f1f3ffff700 1 -- 172.21.15.46:0/2427347952 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1f4c02b020 con 0x55dbf8219d00 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.218+0000 7f1f75594b80 10 client.25100 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.218+0000 7f1f75594b80 10 client.25100 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.218+0000 7f1f75594b80 20 client.25100 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.218+0000 7f1f75594b80 1 --2- 172.21.15.46:0/2427347952 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55dbf83a0000 0x55dbf83a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.218+0000 7f1f75594b80 4 client.25100 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:13:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.218+0000 7f1f75594b80 1 -- 172.21.15.46:0/2427347952 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1f54046020 con 0x55dbf83a0000 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.219+0000 7f1f6491a700 1 --2- 172.21.15.46:0/2427347952 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55dbf83a0000 0x55dbf83a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.219+0000 7f1f6491a700 1 --2- 172.21.15.46:0/2427347952 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55dbf83a0000 0x55dbf83a24a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d874510a172be64 server_cookie=fd758783e8558e47 in_seq=0 out_seq=0 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.219+0000 7f1f3ffff700 10 client.25100 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.219+0000 7f1f3ffff700 1 -- 172.21.15.46:0/2427347952 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1f54046020 con 0x55dbf83a0000 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f3ffff700 10 client.25100 handle_command_reply: tid=0 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f75594b80 1 client.25100 shutdown 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f75594b80 1 -- 172.21.15.46:0/2427347952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf8219d00 msgr2=0x55dbf8338b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f75594b80 1 --2- 172.21.15.46:0/2427347952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf8219d00 0x55dbf8338b80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55dbf81d9b50 tx=0x7f1f4c0039c0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f75594b80 1 -- 172.21.15.46:0/2427347952 shutdown_connections 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f75594b80 1 --2- 172.21.15.46:0/2427347952 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55dbf83a0000 0x55dbf83a24a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f3ffff700 0 client.25100 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f75594b80 1 --2- 172.21.15.46:0/2427347952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbf831c400 0x55dbf839e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f75594b80 1 --2- 172.21.15.46:0/2427347952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dbf819f4a0 0x55dbf8338640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f75594b80 1 --2- 172.21.15.46:0/2427347952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dbf8219d00 0x55dbf8338b80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f75594b80 1 -- 172.21.15.46:0/2427347952 >> 172.21.15.46:0/2427347952 conn(0x55dbf81a2f70 msgr2=0x55dbf831b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f75594b80 1 -- 172.21.15.46:0/2427347952 shutdown_connections 2024-05-09T00:13:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f75594b80 1 -- 172.21.15.46:0/2427347952 wait complete. 2024-05-09T00:13:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.220+0000 7f1f75594b80 20 client.25100 trim_cache size 0 max 16384 2024-05-09T00:13:09.222 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:13:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.221+0000 7f1f66b7e700 1 -- 172.21.15.46:0/4110455070 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1f500b2aa0 msgr2=0x7f1f500b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.221+0000 7f1f66b7e700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1f500b2aa0 0x7f1f500b4fb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1f60199bc0 tx=0x7f1f54009260 comp rx=0 tx=0).stop 2024-05-09T00:13:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.221+0000 7f1f66b7e700 1 -- 172.21.15.46:0/4110455070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f601089f0 msgr2=0x7f1f60198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.221+0000 7f1f66b7e700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f601089f0 0x7f1f60198a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800ef30 tx=0x7f1f5800c3a0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.221+0000 7f1f66b7e700 1 -- 172.21.15.46:0/4110455070 shutdown_connections 2024-05-09T00:13:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.221+0000 7f1f66b7e700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1f500b2aa0 0x7f1f500b4fb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.221+0000 7f1f66b7e700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f601089f0 0x7f1f60198a80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.221+0000 7f1f66b7e700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f601029c0 0x7f1f6019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.221+0000 7f1f66b7e700 1 --2- 172.21.15.46:0/4110455070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f60103310 0x7f1f6019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.221+0000 7f1f66b7e700 1 -- 172.21.15.46:0/4110455070 >> 172.21.15.46:0/4110455070 conn(0x7f1f600fe6f0 msgr2=0x7f1f60107c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.221+0000 7f1f66b7e700 1 -- 172.21.15.46:0/4110455070 shutdown_connections 2024-05-09T00:13:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:09.222+0000 7f1f66b7e700 1 -- 172.21.15.46:0/4110455070 wait complete. 2024-05-09T00:13:10.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:10 smithi046 ceph-mon[30151]: pgmap v430: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-09T00:13:10.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:10 smithi169 ceph-mon[34777]: pgmap v430: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-09T00:13:10.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:10 smithi142 ceph-mon[34533]: pgmap v430: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-09T00:13:12.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:12 smithi169 ceph-mon[34777]: pgmap v431: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:13:12.940 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:12 smithi142 ceph-mon[34533]: pgmap v431: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:13:13.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:12 smithi046 ceph-mon[30151]: pgmap v431: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:13:13.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:13:13.537 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:14.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:14 smithi169 ceph-mon[34777]: pgmap v432: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:13:14.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:14 smithi142 ceph-mon[34533]: pgmap v432: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:13:15.024 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:14 smithi046 ceph-mon[30151]: pgmap v432: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:13:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.483+0000 7fbe72a43700 1 -- 172.21.15.46:0/226181653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe6c10e8e0 msgr2=0x7fbe6c110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.483+0000 7fbe72a43700 1 --2- 172.21.15.46:0/226181653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe6c10e8e0 0x7fbe6c110dc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400b0a0 tx=0x7fbe6400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.483+0000 7fbe72a43700 1 -- 172.21.15.46:0/226181653 shutdown_connections 2024-05-09T00:13:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.483+0000 7fbe72a43700 1 --2- 172.21.15.46:0/226181653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe6c10e8e0 0x7fbe6c110dc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.483+0000 7fbe72a43700 1 --2- 172.21.15.46:0/226181653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe6c105500 0x7fbe6c105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.483+0000 7fbe72a43700 1 --2- 172.21.15.46:0/226181653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c105e50 0x7fbe6c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.483+0000 7fbe72a43700 1 -- 172.21.15.46:0/226181653 >> 172.21.15.46:0/226181653 conn(0x7fbe6c0fedc0 msgr2=0x7fbe6c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.483+0000 7fbe72a43700 1 -- 172.21.15.46:0/226181653 shutdown_connections 2024-05-09T00:13:15.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.483+0000 7fbe72a43700 1 -- 172.21.15.46:0/226181653 wait complete. 2024-05-09T00:13:15.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.484+0000 7fbe72a43700 1 Processor -- start 2024-05-09T00:13:15.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.485+0000 7fbe72a43700 1 -- start start 2024-05-09T00:13:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.485+0000 7fbe72a43700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe6c105500 0x7fbe6c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.485+0000 7fbe72a43700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c105e50 0x7fbe6c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe72a43700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe6c10e8e0 0x7fbe6c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe72a43700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c114490 con 0x7fbe6c105500 2024-05-09T00:13:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe72a43700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c114310 con 0x7fbe6c10e8e0 2024-05-09T00:13:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe72a43700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c114610 con 0x7fbe6c105e50 2024-05-09T00:13:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe6bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe6c105500 0x7fbe6c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe6bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe6c105500 0x7fbe6c19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36598/0 (socket says 172.21.15.46:36598) 2024-05-09T00:13:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe6bfff700 1 -- 172.21.15.46:0/4223673024 learned_addr learned my addr 172.21.15.46:0/4223673024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe70fe0700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe6c10e8e0 0x7fbe6c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c105e50 0x7fbe6c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe6bfff700 1 -- 172.21.15.46:0/4223673024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c105e50 msgr2=0x7fbe6c19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c105e50 0x7fbe6c19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe6bfff700 1 -- 172.21.15.46:0/4223673024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe6c10e8e0 msgr2=0x7fbe6c19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe6c10e8e0 0x7fbe6c19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe6bfff700 1 -- 172.21.15.46:0/4223673024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c19fc50 con 0x7fbe6c105500 2024-05-09T00:13:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c105e50 0x7fbe6c19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.486+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe6c105500 0x7fbe6c19ad80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00e9a0 tx=0x7fbe5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.487+0000 7fbe697fa700 1 -- 172.21.15.46:0/4223673024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c00cbd0 con 0x7fbe6c105500 2024-05-09T00:13:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.487+0000 7fbe697fa700 1 -- 172.21.15.46:0/4223673024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe5c00cd70 con 0x7fbe6c105500 2024-05-09T00:13:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.487+0000 7fbe697fa700 1 -- 172.21.15.46:0/4223673024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c018680 con 0x7fbe6c105500 2024-05-09T00:13:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.487+0000 7fbe72a43700 1 -- 172.21.15.46:0/4223673024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6c19ff40 con 0x7fbe6c105500 2024-05-09T00:13:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.487+0000 7fbe72a43700 1 -- 172.21.15.46:0/4223673024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6c103120 con 0x7fbe6c105500 2024-05-09T00:13:15.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.490+0000 7fbe697fa700 1 -- 172.21.15.46:0/4223673024 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbe5c018820 con 0x7fbe6c105500 2024-05-09T00:13:15.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.491+0000 7fbe697fa700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbe540b2900 0x7fbe540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.492+0000 7fbe697fa700 1 -- 172.21.15.46:0/4223673024 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbe5c0e2980 con 0x7fbe6c105500 2024-05-09T00:13:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.492+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbe540b2900 0x7fbe540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.492+0000 7fbe81459b80 1 Processor -- start 2024-05-09T00:13:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.493+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbe540b2900 0x7fbe540b4e10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c102210 tx=0x7fbe60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.493+0000 7fbe81459b80 1 -- start start 2024-05-09T00:13:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.493+0000 7fbe81459b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 0x5611816d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.493+0000 7fbe81459b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816d5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.493+0000 7fbe81459b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe81459b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5611815f20a0 con 0x5611815ccd00 2024-05-09T00:13:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe6bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 0x5611816d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe6bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 0x5611816d5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36602/0 (socket says 172.21.15.46:36602) 2024-05-09T00:13:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe6bfff700 1 -- 172.21.15.46:0/2287233292 learned_addr learned my addr 172.21.15.46:0/2287233292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe81459b80 1 -- 172.21.15.46:0/2287233292 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5611815979a0 con 0x5611815524a0 2024-05-09T00:13:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe81459b80 1 -- 172.21.15.46:0/2287233292 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5611815ee060 con 0x5611816cf400 2024-05-09T00:13:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/2287233292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816d5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe70fe0700 1 --2- 172.21.15.46:0/2287233292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe6bfff700 1 -- 172.21.15.46:0/2287233292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 msgr2=0x5611816cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/2287233292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe6bfff700 1 -- 172.21.15.46:0/2287233292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 msgr2=0x5611816d5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/2287233292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816d5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe6bfff700 1 -- 172.21.15.46:0/2287233292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561181588500 con 0x5611815ccd00 2024-05-09T00:13:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.494+0000 7fbe70fe0700 1 --2- 172.21.15.46:0/2287233292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.495+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/2287233292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 0x5611816d5240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0aa970 tx=0x7fbe5c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121c2e6627e3efb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.495+0000 7fbe527fc700 1 -- 172.21.15.46:0/2287233292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0b3190 con 0x5611815ccd00 2024-05-09T00:13:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.495+0000 7fbe527fc700 1 -- 172.21.15.46:0/2287233292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe5c0aa4f0 con 0x5611815ccd00 2024-05-09T00:13:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.496+0000 7fbe527fc700 1 -- 172.21.15.46:0/2287233292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c018b10 con 0x5611815ccd00 2024-05-09T00:13:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.496+0000 7fbe81459b80 1 -- 172.21.15.46:0/2287233292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 msgr2=0x5611816d5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.496+0000 7fbe81459b80 1 --2- 172.21.15.46:0/2287233292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 0x5611816d5240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0aa970 tx=0x7fbe5c010cb0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.497+0000 7fbe81459b80 1 -- 172.21.15.46:0/2287233292 shutdown_connections 2024-05-09T00:13:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.497+0000 7fbe81459b80 1 --2- 172.21.15.46:0/2287233292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.497+0000 7fbe81459b80 1 --2- 172.21.15.46:0/2287233292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 0x5611816d5240 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.497+0000 7fbe81459b80 1 --2- 172.21.15.46:0/2287233292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816d5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.497+0000 7fbe81459b80 1 -- 172.21.15.46:0/2287233292 >> 172.21.15.46:0/2287233292 conn(0x561181555f70 msgr2=0x5611816ca220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.497+0000 7fbe81459b80 1 -- 172.21.15.46:0/2287233292 shutdown_connections 2024-05-09T00:13:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.497+0000 7fbe81459b80 1 -- 172.21.15.46:0/2287233292 wait complete. 2024-05-09T00:13:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.497+0000 7fbe81459b80 1 -- start start 2024-05-09T00:13:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.498+0000 7fbe81459b80 1 Processor -- start 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.498+0000 7fbe81459b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.498+0000 7fbe81459b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 0x5611816d96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.498+0000 7fbe81459b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816d9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.498+0000 7fbe6b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 0x5611816d96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.498+0000 7fbe6b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 0x5611816d96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36614/0 (socket says 172.21.15.46:36614) 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.498+0000 7fbe6b7fe700 1 -- 172.21.15.46:0/935318807 learned_addr learned my addr 172.21.15.46:0/935318807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.498+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.498+0000 7fbe81459b80 1 -- 172.21.15.46:0/935318807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5611816d01a0 con 0x5611815ccd00 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.498+0000 7fbe81459b80 1 -- 172.21.15.46:0/935318807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56118158cb20 con 0x5611815524a0 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.498+0000 7fbe81459b80 1 -- 172.21.15.46:0/935318807 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5611816dc020 con 0x5611816cf400 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.498+0000 7fbe70fe0700 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816d9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.499+0000 7fbe81459b80 10 client.0 _ll_register_callbacks cb 0x5611814636c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.499+0000 7fbe6bfff700 1 -- 172.21.15.46:0/935318807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 msgr2=0x5611816d9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.499+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816d9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.499+0000 7fbe6bfff700 1 -- 172.21.15.46:0/935318807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 msgr2=0x5611816d96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.499+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 0x5611816d96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.499+0000 7fbe6bfff700 1 -- 172.21.15.46:0/935318807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118146d490 con 0x5611815524a0 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.499+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 0x5611816d96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.499+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816d91a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c005830 tx=0x7fbe5c0a9ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d34af6a86fe6ef6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.499+0000 7fbe4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.499+0000 7fbe4b7fe700 1 -- 172.21.15.46:0/935318807 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0b3190 con 0x5611815524a0 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.500+0000 7fbe4b7fe700 1 -- 172.21.15.46:0/935318807 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbe5c018d20 con 0x5611815524a0 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.500+0000 7fbe81459b80 1 -- 172.21.15.46:0/935318807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5c0aa4f0 con 0x5611815524a0 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.500+0000 7fbe4b7fe700 1 -- 172.21.15.46:0/935318807 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe5c0aa9e0 con 0x5611815524a0 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.500+0000 7fbe81459b80 10 client.25112 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.500+0000 7fbe81459b80 1 -- 172.21.15.46:0/935318807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5611816ef2d0 con 0x5611815524a0 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.500+0000 7fbe4b7fe700 1 -- 172.21.15.46:0/935318807 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe5c0aa4f0 con 0x5611815524a0 2024-05-09T00:13:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.501+0000 7fbe4b7fe700 1 -- 172.21.15.46:0/935318807 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbe5c0a96b0 con 0x5611815524a0 2024-05-09T00:13:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.501+0000 7fbe81459b80 10 client.25112 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.501+0000 7fbe81459b80 10 client.25112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.501+0000 7fbe81459b80 20 client.25112 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.501+0000 7fbe81459b80 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5611816f17a0 0x5611816f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.501+0000 7fbe81459b80 4 client.25112 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.501+0000 7fbe81459b80 1 -- 172.21.15.46:0/935318807 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611815ffa80 con 0x5611816f17a0 2024-05-09T00:13:15.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.502+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5611816f17a0 0x5611816f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.502+0000 7fbe4b7fe700 10 client.25112 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:15.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.502+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5611816f17a0 0x5611816f3bc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=741f0252ef28e459 server_cookie=941c0b65a9f29f4e in_seq=0 out_seq=0 2024-05-09T00:13:15.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.503+0000 7fbe4b7fe700 1 -- 172.21.15.46:0/935318807 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5611815ffa80 con 0x5611816f17a0 2024-05-09T00:13:15.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.503+0000 7fbe4b7fe700 10 client.25112 handle_command_reply: tid=0 2024-05-09T00:13:15.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.503+0000 7fbe81459b80 1 client.25112 shutdown 2024-05-09T00:13:15.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.503+0000 7fbe81459b80 1 -- 172.21.15.46:0/935318807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 msgr2=0x5611816d91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:15.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.503+0000 7fbe81459b80 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816d91a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c005830 tx=0x7fbe5c0a9ef0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.504+0000 7fbe81459b80 1 -- 172.21.15.46:0/935318807 shutdown_connections 2024-05-09T00:13:15.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.504+0000 7fbe81459b80 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5611816f17a0 0x5611816f3bc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.504+0000 7fbe81459b80 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816d9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.504+0000 7fbe4b7fe700 0 client.25112 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:15.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.504+0000 7fbe81459b80 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816d91a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.504+0000 7fbe81459b80 1 --2- 172.21.15.46:0/935318807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815ccd00 0x5611816d96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.504+0000 7fbe81459b80 1 -- 172.21.15.46:0/935318807 >> 172.21.15.46:0/935318807 conn(0x561181555f70 msgr2=0x5611816c3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:15.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.504+0000 7fbe81459b80 1 -- 172.21.15.46:0/935318807 shutdown_connections 2024-05-09T00:13:15.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.504+0000 7fbe81459b80 1 -- 172.21.15.46:0/935318807 wait complete. 2024-05-09T00:13:15.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.504+0000 7fbe81459b80 20 client.25112 trim_cache size 0 max 16384 2024-05-09T00:13:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.529+0000 7fbe81459b80 1 Processor -- start 2024-05-09T00:13:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.529+0000 7fbe81459b80 1 -- start start 2024-05-09T00:13:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.530+0000 7fbe81459b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815524a0 0x5611816c4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.530+0000 7fbe81459b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815ccd00 0x5611816db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.530+0000 7fbe81459b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.530+0000 7fbe81459b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1af3f0 con 0x5611815524a0 2024-05-09T00:13:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.530+0000 7fbe81459b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1acc80 con 0x5611815ccd00 2024-05-09T00:13:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.530+0000 7fbe81459b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1aaad0 con 0x5611816cf400 2024-05-09T00:13:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.530+0000 7fbe6bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815524a0 0x5611816c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.530+0000 7fbe6bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815524a0 0x5611816c4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36630/0 (socket says 172.21.15.46:36630) 2024-05-09T00:13:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.530+0000 7fbe6bfff700 1 -- 172.21.15.46:0/3689078842 learned_addr learned my addr 172.21.15.46:0/3689078842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.530+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/3689078842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815ccd00 0x5611816db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.530+0000 7fbe70fe0700 1 --2- 172.21.15.46:0/3689078842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.531+0000 7fbe6bfff700 1 -- 172.21.15.46:0/3689078842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 msgr2=0x5611816cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.531+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/3689078842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.531+0000 7fbe6bfff700 1 -- 172.21.15.46:0/3689078842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815ccd00 msgr2=0x5611816db5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.531+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/3689078842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815ccd00 0x5611816db5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.531+0000 7fbe6bfff700 1 -- 172.21.15.46:0/3689078842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611816c3d40 con 0x5611815524a0 2024-05-09T00:13:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.531+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/3689078842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815ccd00 0x5611816db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:15.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.531+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/3689078842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815524a0 0x5611816c4660 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0aa6e0 tx=0x7fbe5c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f52d26acfc6ab7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:15.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.531+0000 7fbe51ffb700 1 -- 172.21.15.46:0/3689078842 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0c3070 con 0x5611815524a0 2024-05-09T00:13:15.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.531+0000 7fbe51ffb700 1 -- 172.21.15.46:0/3689078842 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe5c0b3190 con 0x5611815524a0 2024-05-09T00:13:15.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.531+0000 7fbe51ffb700 1 -- 172.21.15.46:0/3689078842 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0b3840 con 0x5611815524a0 2024-05-09T00:13:15.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.532+0000 7fbe81459b80 1 -- 172.21.15.46:0/3689078842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815524a0 msgr2=0x5611816c4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:15.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.532+0000 7fbe81459b80 1 --2- 172.21.15.46:0/3689078842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815524a0 0x5611816c4660 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0aa6e0 tx=0x7fbe5c010cb0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.532+0000 7fbe81459b80 1 -- 172.21.15.46:0/3689078842 shutdown_connections 2024-05-09T00:13:15.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.532+0000 7fbe81459b80 1 --2- 172.21.15.46:0/3689078842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611816cf400 0x5611816cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.532+0000 7fbe81459b80 1 --2- 172.21.15.46:0/3689078842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611815524a0 0x5611816c4660 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.532+0000 7fbe81459b80 1 --2- 172.21.15.46:0/3689078842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815ccd00 0x5611816db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.532+0000 7fbe81459b80 1 -- 172.21.15.46:0/3689078842 >> 172.21.15.46:0/3689078842 conn(0x561181555f70 msgr2=0x5611816d7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:15.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.532+0000 7fbe81459b80 1 -- 172.21.15.46:0/3689078842 shutdown_connections 2024-05-09T00:13:15.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.532+0000 7fbe81459b80 1 -- 172.21.15.46:0/3689078842 wait complete. 2024-05-09T00:13:15.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.533+0000 7fbe81459b80 1 -- start start 2024-05-09T00:13:15.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.533+0000 7fbe81459b80 1 Processor -- start 2024-05-09T00:13:15.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.533+0000 7fbe81459b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.533+0000 7fbe81459b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611815ccd00 0x5611816f6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.533+0000 7fbe81459b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611816cf400 0x561181759cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.533+0000 7fbe81459b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1ab140 con 0x5611816cf400 2024-05-09T00:13:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.533+0000 7fbe81459b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1ac5b0 con 0x5611815524a0 2024-05-09T00:13:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.533+0000 7fbe81459b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c1ab810 con 0x5611815ccd00 2024-05-09T00:13:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.533+0000 7fbe81459b80 10 client.0 _ll_register_callbacks cb 0x5611814636c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.534+0000 7fbe6bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.534+0000 7fbe6bfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:48194/0 (socket says 172.21.15.46:48194) 2024-05-09T00:13:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.534+0000 7fbe6bfff700 1 -- 172.21.15.46:0/2077577684 learned_addr learned my addr 172.21.15.46:0/2077577684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.534+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611815ccd00 0x5611816f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.534+0000 7fbe6bfff700 1 -- 172.21.15.46:0/2077577684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611815ccd00 msgr2=0x5611816f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.534+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611815ccd00 0x5611816f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.534+0000 7fbe6bfff700 1 -- 172.21.15.46:0/2077577684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611816cf400 msgr2=0x561181759cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.534+0000 7fbe70fe0700 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611816cf400 0x561181759cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.534+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611816cf400 0x561181759cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.534+0000 7fbe6bfff700 1 -- 172.21.15.46:0/2077577684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611816f6a20 con 0x5611815524a0 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.534+0000 7fbe70fe0700 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611816cf400 0x561181759cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.535+0000 7fbe6bfff700 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816f5be0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00e970 tx=0x7fbe5c0a9ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de7e4cfd71eb6e7d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.535+0000 7fbe4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.535+0000 7fbe4b7fe700 1 -- 172.21.15.46:0/2077577684 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0c3050 con 0x5611815524a0 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.535+0000 7fbe4b7fe700 1 -- 172.21.15.46:0/2077577684 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbe5c0a6350 con 0x5611815524a0 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.535+0000 7fbe81459b80 1 -- 172.21.15.46:0/2077577684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5c0b3190 con 0x5611815524a0 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.535+0000 7fbe4b7fe700 1 -- 172.21.15.46:0/2077577684 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe5c0aa150 con 0x5611815524a0 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.535+0000 7fbe81459b80 10 client.25118 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.535+0000 7fbe81459b80 1 -- 172.21.15.46:0/2077577684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5611816f6a20 con 0x5611815524a0 2024-05-09T00:13:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.535+0000 7fbe4b7fe700 1 -- 172.21.15.46:0/2077577684 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe5c0b3190 con 0x5611815524a0 2024-05-09T00:13:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.536+0000 7fbe4b7fe700 1 -- 172.21.15.46:0/2077577684 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbe5c0a6550 con 0x5611815524a0 2024-05-09T00:13:15.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.536+0000 7fbe81459b80 10 client.25118 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.536+0000 7fbe81459b80 10 client.25118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.536+0000 7fbe81459b80 20 client.25118 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.536+0000 7fbe81459b80 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56118175b630 0x56118175da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.536+0000 7fbe81459b80 4 client.25118 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:13:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.536+0000 7fbe81459b80 1 -- 172.21.15.46:0/2077577684 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611815f9b20 con 0x56118175b630 2024-05-09T00:13:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.537+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56118175b630 0x56118175da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.537+0000 7fbe4b7fe700 10 client.25118 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.537+0000 7fbe6b7fe700 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56118175b630 0x56118175da50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d495422a54044a55 server_cookie=68b67f10a7c7dfec in_seq=0 out_seq=0 2024-05-09T00:13:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe4b7fe700 1 -- 172.21.15.46:0/2077577684 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611815f9b20 con 0x56118175b630 2024-05-09T00:13:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe4b7fe700 10 client.25118 handle_command_reply: tid=0 2024-05-09T00:13:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe81459b80 1 client.25118 shutdown 2024-05-09T00:13:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe81459b80 1 -- 172.21.15.46:0/2077577684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 msgr2=0x5611816f5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe81459b80 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816f5be0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00e970 tx=0x7fbe5c0a9ef0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe81459b80 1 -- 172.21.15.46:0/2077577684 shutdown_connections 2024-05-09T00:13:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe81459b80 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56118175b630 0x56118175da50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe4b7fe700 0 client.25118 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe81459b80 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611816cf400 0x561181759cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe81459b80 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5611815524a0 0x5611816f5be0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe81459b80 1 --2- 172.21.15.46:0/2077577684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5611815ccd00 0x5611816f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe81459b80 1 -- 172.21.15.46:0/2077577684 >> 172.21.15.46:0/2077577684 conn(0x561181555f70 msgr2=0x5611816ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe81459b80 1 -- 172.21.15.46:0/2077577684 shutdown_connections 2024-05-09T00:13:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.538+0000 7fbe81459b80 1 -- 172.21.15.46:0/2077577684 wait complete. 2024-05-09T00:13:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.539+0000 7fbe81459b80 20 client.25118 trim_cache size 0 max 16384 2024-05-09T00:13:15.540 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:13:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.539+0000 7fbe72a43700 1 -- 172.21.15.46:0/4223673024 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbe540b2900 msgr2=0x7fbe540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.539+0000 7fbe72a43700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbe540b2900 0x7fbe540b4e10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c102210 tx=0x7fbe60009260 comp rx=0 tx=0).stop 2024-05-09T00:13:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.539+0000 7fbe72a43700 1 -- 172.21.15.46:0/4223673024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe6c105500 msgr2=0x7fbe6c19ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.539+0000 7fbe72a43700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe6c105500 0x7fbe6c19ad80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00e9a0 tx=0x7fbe5c00ecb0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.540+0000 7fbe72a43700 1 -- 172.21.15.46:0/4223673024 shutdown_connections 2024-05-09T00:13:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.540+0000 7fbe72a43700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbe540b2900 0x7fbe540b4e10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.540+0000 7fbe72a43700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe6c10e8e0 0x7fbe6c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.540+0000 7fbe72a43700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe6c105500 0x7fbe6c19ad80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.540+0000 7fbe72a43700 1 --2- 172.21.15.46:0/4223673024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c105e50 0x7fbe6c19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.540+0000 7fbe72a43700 1 -- 172.21.15.46:0/4223673024 >> 172.21.15.46:0/4223673024 conn(0x7fbe6c0fedc0 msgr2=0x7fbe6c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.540+0000 7fbe72a43700 1 -- 172.21.15.46:0/4223673024 shutdown_connections 2024-05-09T00:13:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:15.540+0000 7fbe72a43700 1 -- 172.21.15.46:0/4223673024 wait complete. 2024-05-09T00:13:16.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:16 smithi046 ceph-mon[30151]: pgmap v433: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:13:16.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:16 smithi169 ceph-mon[34777]: pgmap v433: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:13:16.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:16 smithi142 ceph-mon[34533]: pgmap v433: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T00:13:17.437 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:13:17.437 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:13:17.785 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:18.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:18 smithi169 ceph-mon[34777]: pgmap v434: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:13:18.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:18 smithi142 ceph-mon[34533]: pgmap v434: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:13:18.946 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:18 smithi046 ceph-mon[30151]: pgmap v434: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:13:19.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.723+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/1065362652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4108740 msgr2=0x7fd4d41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:19.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.723+0000 7fd4d8cb7700 1 --2- 172.21.15.46:0/1065362652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4108740 0x7fd4d41100a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00b0d0 tx=0x7fd4cc00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.723+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/1065362652 shutdown_connections 2024-05-09T00:13:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.723+0000 7fd4d8cb7700 1 --2- 172.21.15.46:0/1065362652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4108740 0x7fd4d41100a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.723+0000 7fd4d8cb7700 1 --2- 172.21.15.46:0/1065362652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d41034c0 0x7fd4d41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.723+0000 7fd4d8cb7700 1 --2- 172.21.15.46:0/1065362652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4d4107df0 0x7fd4d4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.723+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/1065362652 >> 172.21.15.46:0/1065362652 conn(0x7fd4d40fedc0 msgr2=0x7fd4d41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.723+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/1065362652 shutdown_connections 2024-05-09T00:13:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.724+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/1065362652 wait complete. 2024-05-09T00:13:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.725+0000 7fd4d8cb7700 1 Processor -- start 2024-05-09T00:13:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.725+0000 7fd4d8cb7700 1 -- start start 2024-05-09T00:13:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.725+0000 7fd4d8cb7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d41034c0 0x7fd4d419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.725+0000 7fd4d8cb7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4d4107df0 0x7fd4d419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.725+0000 7fd4d259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d41034c0 0x7fd4d419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.725+0000 7fd4d8cb7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d4108740 0x7fd4d4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.725+0000 7fd4d8cb7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4113370 con 0x7fd4d41034c0 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.725+0000 7fd4d8cb7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41131f0 con 0x7fd4d4107df0 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.725+0000 7fd4d8cb7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41134f0 con 0x7fd4d4108740 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.725+0000 7fd4d259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d41034c0 0x7fd4d419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55576/0 (socket says 172.21.15.46:55576) 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.725+0000 7fd4d259c700 1 -- 172.21.15.46:0/3023236969 learned_addr learned my addr 172.21.15.46:0/3023236969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.725+0000 7fd4cbfff700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4d4107df0 0x7fd4d419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.726+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d4108740 0x7fd4d4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.726+0000 7fd4d259c700 1 -- 172.21.15.46:0/3023236969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d4108740 msgr2=0x7fd4d4199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.726+0000 7fd4d259c700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d4108740 0x7fd4d4199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.726+0000 7fd4d259c700 1 -- 172.21.15.46:0/3023236969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4d4107df0 msgr2=0x7fd4d419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.726+0000 7fd4d259c700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4d4107df0 0x7fd4d419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.726+0000 7fd4d259c700 1 -- 172.21.15.46:0/3023236969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d419a160 con 0x7fd4d41034c0 2024-05-09T00:13:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.726+0000 7fd4cbfff700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4d4107df0 0x7fd4d419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.726+0000 7fd4d259c700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d41034c0 0x7fd4d419f890 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400d980 tx=0x7fd4c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.726+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d4108740 0x7fd4d4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.727+0000 7fd4cb7fe700 1 -- 172.21.15.46:0/3023236969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4016070 con 0x7fd4d41034c0 2024-05-09T00:13:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.727+0000 7fd4cb7fe700 1 -- 172.21.15.46:0/3023236969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c4004830 con 0x7fd4d41034c0 2024-05-09T00:13:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.727+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/3023236969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d419a450 con 0x7fd4d41034c0 2024-05-09T00:13:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.727+0000 7fd4cb7fe700 1 -- 172.21.15.46:0/3023236969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4005580 con 0x7fd4d41034c0 2024-05-09T00:13:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.727+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/3023236969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d419a8d0 con 0x7fd4d41034c0 2024-05-09T00:13:19.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.731+0000 7fd4cb7fe700 1 -- 172.21.15.46:0/3023236969 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd4c4004220 con 0x7fd4d41034c0 2024-05-09T00:13:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.732+0000 7fd4e76cdb80 1 Processor -- start 2024-05-09T00:13:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.732+0000 7fd4cb7fe700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4c00b29d0 0x7fd4c00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.732+0000 7fd4cb7fe700 1 -- 172.21.15.46:0/3023236969 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd4c40e1f20 con 0x7fd4d41034c0 2024-05-09T00:13:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.733+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4c00b29d0 0x7fd4c00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.733+0000 7fd4e76cdb80 1 -- start start 2024-05-09T00:13:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.733+0000 7fd4e76cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 0x55794c4792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.733+0000 7fd4e76cdb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c2f6520 0x55794c479800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.733+0000 7fd4e76cdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c473480 0x55794c473890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.733+0000 7fd4e76cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55794c396120 con 0x55794c370d80 2024-05-09T00:13:19.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.733+0000 7fd4e76cdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55794c33ba20 con 0x55794c473480 2024-05-09T00:13:19.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.733+0000 7fd4e76cdb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55794c3920e0 con 0x55794c2f6520 2024-05-09T00:13:19.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.733+0000 7fd4d259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 0x55794c4792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.733+0000 7fd4d259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 0x55794c4792c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55590/0 (socket says 172.21.15.46:55590) 2024-05-09T00:13:19.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.733+0000 7fd4d259c700 1 -- 172.21.15.46:0/3794935486 learned_addr learned my addr 172.21.15.46:0/3794935486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:19.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.734+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/3794935486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c473480 0x55794c473890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.734+0000 7fd4d259c700 1 -- 172.21.15.46:0/3794935486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c2f6520 msgr2=0x55794c479800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:19.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.734+0000 7fd4cbfff700 1 --2- 172.21.15.46:0/3794935486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c2f6520 0x55794c479800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.734+0000 7fd4d259c700 1 --2- 172.21.15.46:0/3794935486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c2f6520 0x55794c479800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.734+0000 7fd4d259c700 1 -- 172.21.15.46:0/3794935486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c473480 msgr2=0x55794c473890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.734+0000 7fd4d259c700 1 --2- 172.21.15.46:0/3794935486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c473480 0x55794c473890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.734+0000 7fd4d259c700 1 -- 172.21.15.46:0/3794935486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794c32c580 con 0x55794c370d80 2024-05-09T00:13:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.734+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/3794935486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c473480 0x55794c473890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.734+0000 7fd4cbfff700 1 --2- 172.21.15.46:0/3794935486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c2f6520 0x55794c479800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.734+0000 7fd4d259c700 1 --2- 172.21.15.46:0/3794935486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 0x55794c4792c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40a6770 tx=0x7fd4c4004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430c98dc53de0ef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.734+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4c00b29d0 0x7fd4c00b4ee0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009fd0 tx=0x7fd4cc0092e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.734+0000 7fd4c8ff9700 1 -- 172.21.15.46:0/3794935486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40c3070 con 0x55794c370d80 2024-05-09T00:13:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.735+0000 7fd4c8ff9700 1 -- 172.21.15.46:0/3794935486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c40a9500 con 0x55794c370d80 2024-05-09T00:13:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.735+0000 7fd4c8ff9700 1 -- 172.21.15.46:0/3794935486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40a9760 con 0x55794c370d80 2024-05-09T00:13:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.736+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/3794935486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 msgr2=0x55794c4792c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.736+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/3794935486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 0x55794c4792c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40a6770 tx=0x7fd4c4004c70 comp rx=0 tx=0).stop 2024-05-09T00:13:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.736+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/3794935486 shutdown_connections 2024-05-09T00:13:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.736+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/3794935486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c473480 0x55794c473890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.736+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/3794935486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 0x55794c4792c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.736+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/3794935486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c2f6520 0x55794c479800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.736+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/3794935486 >> 172.21.15.46:0/3794935486 conn(0x55794c2f9ff0 msgr2=0x55794c46e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.736+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/3794935486 shutdown_connections 2024-05-09T00:13:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.737+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/3794935486 wait complete. 2024-05-09T00:13:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.737+0000 7fd4e76cdb80 1 -- start start 2024-05-09T00:13:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.737+0000 7fd4e76cdb80 1 Processor -- start 2024-05-09T00:13:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4e76cdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c2f6520 0x55794c47d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4e76cdb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c370d80 0x55794c47d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4e76cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c473480 0x55794c47db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4e76cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55794c4741e0 con 0x55794c473480 2024-05-09T00:13:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4cbfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c370d80 0x55794c47d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4d2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c473480 0x55794c47db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4cbfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c370d80 0x55794c47d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:41798/0 (socket says 172.21.15.46:41798) 2024-05-09T00:13:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4cbfff700 1 -- 172.21.15.46:0/1375293490 learned_addr learned my addr 172.21.15.46:0/1375293490 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4d2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c473480 0x55794c47db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55592/0 (socket says 172.21.15.46:55592) 2024-05-09T00:13:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4d259c700 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c2f6520 0x55794c47d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/1375293490 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55794c330ba0 con 0x55794c2f6520 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/1375293490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55794c4800a0 con 0x55794c370d80 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.738+0000 7fd4e76cdb80 10 client.0 _ll_register_callbacks cb 0x55794c207740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.739+0000 7fd4d2d9d700 1 -- 172.21.15.46:0/1375293490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c370d80 msgr2=0x55794c47d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.739+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c370d80 0x55794c47d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.739+0000 7fd4d2d9d700 1 -- 172.21.15.46:0/1375293490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c2f6520 msgr2=0x55794c47d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.739+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c2f6520 0x55794c47d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.739+0000 7fd4d2d9d700 1 -- 172.21.15.46:0/1375293490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794c211510 con 0x55794c473480 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.739+0000 7fd4cbfff700 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c370d80 0x55794c47d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.739+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c473480 0x55794c47db50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00b770 tx=0x7fd4cc00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8954ce0f97f7d764 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.739+0000 7fd4b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.739+0000 7fd4b9ffb700 1 -- 172.21.15.46:0/1375293490 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc02d020 con 0x55794c473480 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/1375293490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c40a9500 con 0x55794c473480 2024-05-09T00:13:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4b9ffb700 1 -- 172.21.15.46:0/1375293490 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd4cc005030 con 0x55794c473480 2024-05-09T00:13:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4b9ffb700 1 -- 172.21.15.46:0/1375293490 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4cc005230 con 0x55794c473480 2024-05-09T00:13:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4e76cdb80 10 client.15402 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/1375293490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55794c48a9e0 con 0x55794c473480 2024-05-09T00:13:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4b9ffb700 1 -- 172.21.15.46:0/1375293490 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c40a9500 con 0x55794c473480 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4b9ffb700 1 -- 172.21.15.46:0/1375293490 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd4cc011070 con 0x55794c473480 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4e76cdb80 10 client.15402 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4e76cdb80 10 client.15402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4e76cdb80 20 client.15402 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55794c48d3f0 0x55794c48f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4e76cdb80 4 client.15402 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.740+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/1375293490 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55794c3a3b00 con 0x55794c48d3f0 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.741+0000 7fd4d259c700 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55794c48d3f0 0x55794c48f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.741+0000 7fd4b9ffb700 10 client.15402 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.741+0000 7fd4d259c700 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55794c48d3f0 0x55794c48f880 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7e6d20b6637f75d server_cookie=a5c69cb50d9ca28e in_seq=0 out_seq=0 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.742+0000 7fd4b9ffb700 1 -- 172.21.15.46:0/1375293490 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55794c3a3b00 con 0x55794c48d3f0 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.742+0000 7fd4b9ffb700 10 client.15402 handle_command_reply: tid=0 2024-05-09T00:13:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.742+0000 7fd4e76cdb80 1 client.15402 shutdown 2024-05-09T00:13:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.742+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/1375293490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c473480 msgr2=0x55794c47db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.742+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c473480 0x55794c47db50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00b770 tx=0x7fd4cc00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.742+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/1375293490 shutdown_connections 2024-05-09T00:13:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.742+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55794c48d3f0 0x55794c48f880 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.743+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c473480 0x55794c47db50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.743+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c2f6520 0x55794c47d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.743+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/1375293490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c370d80 0x55794c47d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.743+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/1375293490 >> 172.21.15.46:0/1375293490 conn(0x55794c2f9ff0 msgr2=0x55794c468070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.743+0000 7fd4b9ffb700 0 client.15402 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.743+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/1375293490 shutdown_connections 2024-05-09T00:13:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.743+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/1375293490 wait complete. 2024-05-09T00:13:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.743+0000 7fd4e76cdb80 20 client.15402 trim_cache size 0 max 16384 2024-05-09T00:13:19.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.764+0000 7fd4e76cdb80 1 Processor -- start 2024-05-09T00:13:19.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.764+0000 7fd4e76cdb80 1 -- start start 2024-05-09T00:13:19.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.764+0000 7fd4e76cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c2f6520 0x55794c47dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.764+0000 7fd4e76cdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c370d80 0x55794c47f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.764+0000 7fd4e76cdb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c473480 0x55794c495040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.764+0000 7fd4e76cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41b3770 con 0x55794c2f6520 2024-05-09T00:13:19.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.764+0000 7fd4e76cdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41b1000 con 0x55794c370d80 2024-05-09T00:13:19.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.764+0000 7fd4e76cdb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41aee30 con 0x55794c473480 2024-05-09T00:13:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.765+0000 7fd4cbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c370d80 0x55794c47f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.765+0000 7fd4d259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c2f6520 0x55794c47dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.765+0000 7fd4d2d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c473480 0x55794c495040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.765+0000 7fd4d259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c2f6520 0x55794c47dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55596/0 (socket says 172.21.15.46:55596) 2024-05-09T00:13:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.765+0000 7fd4d259c700 1 -- 172.21.15.46:0/194454161 learned_addr learned my addr 172.21.15.46:0/194454161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.765+0000 7fd4d2d9d700 1 -- 172.21.15.46:0/194454161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c370d80 msgr2=0x55794c47f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.765+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/194454161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c370d80 0x55794c47f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.765+0000 7fd4d2d9d700 1 -- 172.21.15.46:0/194454161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c2f6520 msgr2=0x55794c47dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.765+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/194454161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c2f6520 0x55794c47dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.765+0000 7fd4d2d9d700 1 -- 172.21.15.46:0/194454161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794c479b70 con 0x55794c473480 2024-05-09T00:13:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.765+0000 7fd4d259c700 1 --2- 172.21.15.46:0/194454161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c2f6520 0x55794c47dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.766+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/194454161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c473480 0x55794c495040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc01faf0 tx=0x7fd4cc0095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8841149cb05e620b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.766+0000 7fd4bbfff700 1 -- 172.21.15.46:0/194454161 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc02d070 con 0x55794c473480 2024-05-09T00:13:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.766+0000 7fd4bbfff700 1 -- 172.21.15.46:0/194454161 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4cc007c30 con 0x55794c473480 2024-05-09T00:13:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.766+0000 7fd4bbfff700 1 -- 172.21.15.46:0/194454161 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0053d0 con 0x55794c473480 2024-05-09T00:13:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.767+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/194454161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c473480 msgr2=0x55794c495040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.767+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/194454161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c473480 0x55794c495040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc01faf0 tx=0x7fd4cc0095d0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.767+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/194454161 shutdown_connections 2024-05-09T00:13:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.767+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/194454161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c473480 0x55794c495040 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.767+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/194454161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c2f6520 0x55794c47dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.767+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/194454161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c370d80 0x55794c47f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.767+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/194454161 >> 172.21.15.46:0/194454161 conn(0x55794c2f9ff0 msgr2=0x55794c4686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.767+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/194454161 shutdown_connections 2024-05-09T00:13:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.767+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/194454161 wait complete. 2024-05-09T00:13:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.768+0000 7fd4e76cdb80 1 -- start start 2024-05-09T00:13:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.768+0000 7fd4e76cdb80 1 Processor -- start 2024-05-09T00:13:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.768+0000 7fd4e76cdb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c2f6520 0x55794c4814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.768+0000 7fd4e76cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 0x55794c481a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.768+0000 7fd4cbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 0x55794c481a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.768+0000 7fd4e76cdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c473480 0x55794c4efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.768+0000 7fd4cbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 0x55794c481a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55608/0 (socket says 172.21.15.46:55608) 2024-05-09T00:13:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.768+0000 7fd4cbfff700 1 -- 172.21.15.46:0/2617706436 learned_addr learned my addr 172.21.15.46:0/2617706436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.768+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/2617706436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41af4c0 con 0x55794c370d80 2024-05-09T00:13:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.768+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/2617706436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41b0930 con 0x55794c473480 2024-05-09T00:13:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.768+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/2617706436 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41afb90 con 0x55794c2f6520 2024-05-09T00:13:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.768+0000 7fd4e76cdb80 10 client.0 _ll_register_callbacks cb 0x55794c46e900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.769+0000 7fd4d259c700 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c2f6520 0x55794c4814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.769+0000 7fd4d2d9d700 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c473480 0x55794c4efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.769+0000 7fd4d259c700 1 -- 172.21.15.46:0/2617706436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c473480 msgr2=0x55794c4efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.769+0000 7fd4d259c700 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c473480 0x55794c4efbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.769+0000 7fd4d259c700 1 -- 172.21.15.46:0/2617706436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 msgr2=0x55794c481a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.769+0000 7fd4d259c700 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 0x55794c481a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.769+0000 7fd4d259c700 1 -- 172.21.15.46:0/2617706436 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794c4f01f0 con 0x55794c2f6520 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.769+0000 7fd4cbfff700 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 0x55794c481a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.769+0000 7fd4d259c700 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c2f6520 0x55794c4814f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40a9200 tx=0x7fd4c4004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=981b0aa3ce4566f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.769+0000 7fd4b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.770+0000 7fd4b9ffb700 1 -- 172.21.15.46:0/2617706436 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40c3040 con 0x55794c2f6520 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.770+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/2617706436 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4cc007c30 con 0x55794c2f6520 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.770+0000 7fd4b9ffb700 1 -- 172.21.15.46:0/2617706436 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd4c40a5af0 con 0x55794c2f6520 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.770+0000 7fd4b9ffb700 1 -- 172.21.15.46:0/2617706436 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4c40a9500 con 0x55794c2f6520 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.770+0000 7fd4e76cdb80 10 client.25159 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.770+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/2617706436 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55794c4f8630 con 0x55794c2f6520 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.770+0000 7fd4b9ffb700 1 -- 172.21.15.46:0/2617706436 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c4088070 con 0x55794c2f6520 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.771+0000 7fd4b9ffb700 1 -- 172.21.15.46:0/2617706436 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd4c408b020 con 0x55794c2f6520 2024-05-09T00:13:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.771+0000 7fd4e76cdb80 10 client.25159 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.771+0000 7fd4e76cdb80 10 client.25159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.771+0000 7fd4e76cdb80 20 client.25159 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.771+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55794c4f8970 0x55794c4fad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.771+0000 7fd4e76cdb80 4 client.25159 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:13:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.771+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/2617706436 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fd4cc007c30 con 0x55794c4f8970 2024-05-09T00:13:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.771+0000 7fd4cbfff700 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55794c4f8970 0x55794c4fad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.772+0000 7fd4cbfff700 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55794c4f8970 0x55794c4fad90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d7c0b78dfbc98a6 server_cookie=249ebbe65009ab58 in_seq=0 out_seq=0 2024-05-09T00:13:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.772+0000 7fd4b9ffb700 10 client.25159 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.773+0000 7fd4b9ffb700 1 -- 172.21.15.46:0/2617706436 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fd4cc007c30 con 0x55794c4f8970 2024-05-09T00:13:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.773+0000 7fd4b9ffb700 10 client.25159 handle_command_reply: tid=0 2024-05-09T00:13:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.773+0000 7fd4e76cdb80 1 client.25159 shutdown 2024-05-09T00:13:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.773+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/2617706436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c2f6520 msgr2=0x55794c4814f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.773+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c2f6520 0x55794c4814f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40a9200 tx=0x7fd4c4004d00 comp rx=0 tx=0).stop 2024-05-09T00:13:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.773+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/2617706436 shutdown_connections 2024-05-09T00:13:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.773+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55794c4f8970 0x55794c4fad90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.774+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55794c473480 0x55794c4efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.774+0000 7fd4b9ffb700 0 client.25159 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.774+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55794c2f6520 0x55794c4814f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.774+0000 7fd4e76cdb80 1 --2- 172.21.15.46:0/2617706436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55794c370d80 0x55794c481a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.774+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/2617706436 >> 172.21.15.46:0/2617706436 conn(0x55794c2f9ff0 msgr2=0x55794c47d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.774+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/2617706436 shutdown_connections 2024-05-09T00:13:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.774+0000 7fd4e76cdb80 1 -- 172.21.15.46:0/2617706436 wait complete. 2024-05-09T00:13:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.774+0000 7fd4e76cdb80 20 client.25159 trim_cache size 0 max 16384 2024-05-09T00:13:19.775 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:13:19.775 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:13:19.775 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "381d5f61-e969-4576-a1fe-a621714b7381", 2024-05-09T00:13:19.775 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:13:19.775 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:13:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.775+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/3023236969 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4c00b29d0 msgr2=0x7fd4c00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.775+0000 7fd4d8cb7700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4c00b29d0 0x7fd4c00b4ee0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009fd0 tx=0x7fd4cc0092e0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.775+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/3023236969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d41034c0 msgr2=0x7fd4d419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.775+0000 7fd4d8cb7700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d41034c0 0x7fd4d419f890 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400d980 tx=0x7fd4c400dc90 comp rx=0 tx=0).stop 2024-05-09T00:13:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.776+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/3023236969 shutdown_connections 2024-05-09T00:13:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.776+0000 7fd4d8cb7700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4c00b29d0 0x7fd4c00b4ee0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.776+0000 7fd4d8cb7700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d4108740 0x7fd4d4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.776+0000 7fd4d8cb7700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d41034c0 0x7fd4d419f890 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.776+0000 7fd4d8cb7700 1 --2- 172.21.15.46:0/3023236969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4d4107df0 0x7fd4d419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.776+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/3023236969 >> 172.21.15.46:0/3023236969 conn(0x7fd4d40fedc0 msgr2=0x7fd4d4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.776+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/3023236969 shutdown_connections 2024-05-09T00:13:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:19.776+0000 7fd4d8cb7700 1 -- 172.21.15.46:0/3023236969 wait complete. 2024-05-09T00:13:20.617 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:13:20.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:20 smithi169 ceph-mon[34777]: pgmap v435: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:13:20.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:20 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:13:20.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:20 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:13:20.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:20 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:13:20.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:20 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:13:20.907 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:20.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:20 smithi142 ceph-mon[34533]: pgmap v435: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:13:20.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:20 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:13:20.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:20 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:13:20.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:20 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:13:20.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:20 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:13:21.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:20 smithi046 ceph-mon[30151]: pgmap v435: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T00:13:21.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:20 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:13:21.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:20 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:13:21.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:20 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:13:21.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:20 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:13:22.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:22 smithi046 ceph-mon[30151]: pgmap v436: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:13:22.708 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:22 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:13:22.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.791+0000 7f8efbb75700 1 -- 172.21.15.46:0/3813657696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef4108f00 msgr2=0x7f8ef4109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:22.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.791+0000 7f8efbb75700 1 --2- 172.21.15.46:0/3813657696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef4108f00 0x7f8ef4109310 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009a40 tx=0x7f8ee4009d50 comp rx=0 tx=0).stop 2024-05-09T00:13:22.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.791+0000 7f8efbb75700 1 -- 172.21.15.46:0/3813657696 shutdown_connections 2024-05-09T00:13:22.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.791+0000 7f8efbb75700 1 --2- 172.21.15.46:0/3813657696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef4109850 0x7f8ef41111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.791+0000 7f8efbb75700 1 --2- 172.21.15.46:0/3813657696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef41045d0 0x7f8ef41049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.791+0000 7f8efbb75700 1 --2- 172.21.15.46:0/3813657696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef4108f00 0x7f8ef4109310 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.791+0000 7f8efbb75700 1 -- 172.21.15.46:0/3813657696 >> 172.21.15.46:0/3813657696 conn(0x7f8ef40ffea0 msgr2=0x7f8ef41022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:22.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.792+0000 7f8efbb75700 1 -- 172.21.15.46:0/3813657696 shutdown_connections 2024-05-09T00:13:22.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.792+0000 7f8efbb75700 1 -- 172.21.15.46:0/3813657696 wait complete. 2024-05-09T00:13:22.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.793+0000 7f8efbb75700 1 Processor -- start 2024-05-09T00:13:22.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.793+0000 7f8efbb75700 1 -- start start 2024-05-09T00:13:22.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.794+0000 7f8efbb75700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef41045d0 0x7f8ef419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.794+0000 7f8efbb75700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef4108f00 0x7f8ef419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.794+0000 7f8efbb75700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef4109850 0x7f8ef4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.794+0000 7f8efbb75700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4112320 con 0x7f8ef4109850 2024-05-09T00:13:22.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.794+0000 7f8efbb75700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41121a0 con 0x7f8ef41045d0 2024-05-09T00:13:22.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.794+0000 7f8efbb75700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41124a0 con 0x7f8ef4108f00 2024-05-09T00:13:22.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.794+0000 7f8efa112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef4109850 0x7f8ef4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.794+0000 7f8efa112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef4109850 0x7f8ef4198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55630/0 (socket says 172.21.15.46:55630) 2024-05-09T00:13:22.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.794+0000 7f8efa112700 1 -- 172.21.15.46:0/3296981321 learned_addr learned my addr 172.21.15.46:0/3296981321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:22.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.794+0000 7f8ef9110700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef4108f00 0x7f8ef419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.794+0000 7f8ef9911700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef41045d0 0x7f8ef419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.795+0000 7f8efa112700 1 -- 172.21.15.46:0/3296981321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef4108f00 msgr2=0x7f8ef419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:22.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.795+0000 7f8efa112700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef4108f00 0x7f8ef419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.795+0000 7f8efa112700 1 -- 172.21.15.46:0/3296981321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef41045d0 msgr2=0x7f8ef419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:22.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.795+0000 7f8efa112700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef41045d0 0x7f8ef419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.795+0000 7f8efa112700 1 -- 172.21.15.46:0/3296981321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef4199150 con 0x7f8ef4109850 2024-05-09T00:13:22.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.795+0000 7f8ef9110700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef4108f00 0x7f8ef419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:22.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.795+0000 7f8efa112700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef4109850 0x7f8ef4198aa0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8eec007ee0 tx=0x7f8eec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:22.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.795+0000 7f8eeaffd700 1 -- 172.21.15.46:0/3296981321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec00ddd0 con 0x7f8ef4109850 2024-05-09T00:13:22.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.796+0000 7f8eeaffd700 1 -- 172.21.15.46:0/3296981321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eec004020 con 0x7f8ef4109850 2024-05-09T00:13:22.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.796+0000 7f8eeaffd700 1 -- 172.21.15.46:0/3296981321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec015660 con 0x7f8ef4109850 2024-05-09T00:13:22.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.796+0000 7f8efbb75700 1 -- 172.21.15.46:0/3296981321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef4199440 con 0x7f8ef4109850 2024-05-09T00:13:22.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.796+0000 7f8efbb75700 1 -- 172.21.15.46:0/3296981321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef4199840 con 0x7f8ef4109850 2024-05-09T00:13:22.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.801+0000 7f8eeaffd700 1 -- 172.21.15.46:0/3296981321 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8eec003a00 con 0x7f8ef4109850 2024-05-09T00:13:22.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.801+0000 7f8f0a58bb80 1 Processor -- start 2024-05-09T00:13:22.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.802+0000 7f8f0a58bb80 1 -- start start 2024-05-09T00:13:22.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.802+0000 7f8eeaffd700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8ee00b2900 0x7f8ee00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.802+0000 7f8f0a58bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40ded5d00 0x55e40dfde240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.802+0000 7f8f0a58bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40de5b4a0 0x55e40dfde780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.802+0000 7f8f0a58bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 0x55e40dfd8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8efa112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 0x55e40dfd8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8efa112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 0x55e40dfd8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55646/0 (socket says 172.21.15.46:55646) 2024-05-09T00:13:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8efa112700 1 -- 172.21.15.46:0/3690491756 learned_addr learned my addr 172.21.15.46:0/3690491756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/3690491756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e40defb0a0 con 0x55e40dfd8400 2024-05-09T00:13:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/3690491756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e40dea09a0 con 0x55e40ded5d00 2024-05-09T00:13:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/3690491756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e40def7060 con 0x55e40de5b4a0 2024-05-09T00:13:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8ef9911700 1 --2- 172.21.15.46:0/3690491756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40ded5d00 0x55e40dfde240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8ef9110700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8ee00b2900 0x7f8ee00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8eeaffd700 1 -- 172.21.15.46:0/3296981321 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8eec0e1bc0 con 0x7f8ef4109850 2024-05-09T00:13:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8ef9911700 1 -- 172.21.15.46:0/3690491756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40de5b4a0 msgr2=0x55e40dfde780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T00:13:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8ef9911700 1 --2- 172.21.15.46:0/3690491756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40de5b4a0 0x55e40dfde780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8ef9911700 1 -- 172.21.15.46:0/3690491756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 msgr2=0x55e40dfd8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8ef9911700 1 --2- 172.21.15.46:0/3690491756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 0x55e40dfd8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8ef9911700 1 -- 172.21.15.46:0/3690491756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e40de91500 con 0x55e40ded5d00 2024-05-09T00:13:22.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.803+0000 7f8efa112700 1 --2- 172.21.15.46:0/3690491756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 0x55e40dfd8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:22.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.804+0000 7f8ef9911700 1 --2- 172.21.15.46:0/3690491756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40ded5d00 0x55e40dfde240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0007bc0 tx=0x7f8ef0007ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9bb301459b91eb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:22.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.804+0000 7f8ef9110700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8ee00b2900 0x7f8ee00b4e10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4103520 tx=0x7f8ee4009690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:22.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.804+0000 7f8edffff700 1 -- 172.21.15.46:0/3690491756 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0010c40 con 0x55e40ded5d00 2024-05-09T00:13:22.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.804+0000 7f8edffff700 1 -- 172.21.15.46:0/3690491756 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef0010de0 con 0x55e40ded5d00 2024-05-09T00:13:22.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.805+0000 7f8edffff700 1 -- 172.21.15.46:0/3690491756 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0019750 con 0x55e40ded5d00 2024-05-09T00:13:22.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.806+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/3690491756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40ded5d00 msgr2=0x55e40dfde240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:22.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.806+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/3690491756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40ded5d00 0x55e40dfde240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0007bc0 tx=0x7f8ef0007ed0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.806+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/3690491756 shutdown_connections 2024-05-09T00:13:22.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.806+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/3690491756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 0x55e40dfd8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.806+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/3690491756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40ded5d00 0x55e40dfde240 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.806+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/3690491756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40de5b4a0 0x55e40dfde780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.806+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/3690491756 >> 172.21.15.46:0/3690491756 conn(0x55e40de5ef70 msgr2=0x55e40dfd3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:22.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.806+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/3690491756 shutdown_connections 2024-05-09T00:13:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.806+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/3690491756 wait complete. 2024-05-09T00:13:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.807+0000 7f8f0a58bb80 1 -- start start 2024-05-09T00:13:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.807+0000 7f8f0a58bb80 1 Processor -- start 2024-05-09T00:13:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8f0a58bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5b4a0 0x55e40dfe1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8f0a58bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40ded5d00 0x55e40dfe2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8f0a58bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 0x55e40dfe2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8f0a58bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e40dfd9160 con 0x55e40dfd8400 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8f0a58bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e40de95b20 con 0x55e40de5b4a0 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8efa112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 0x55e40dfe2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8efa112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 0x55e40dfe2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55660/0 (socket says 172.21.15.46:55660) 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8efa112700 1 -- 172.21.15.46:0/2852014335 learned_addr learned my addr 172.21.15.46:0/2852014335 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2852014335 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e40dfe5020 con 0x55e40ded5d00 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8ef9911700 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5b4a0 0x55e40dfe1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8f0a58bb80 10 client.0 _ll_register_callbacks cb 0x55e40dd6c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8ef9110700 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40ded5d00 0x55e40dfe2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8efa112700 1 -- 172.21.15.46:0/2852014335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40ded5d00 msgr2=0x55e40dfe2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8efa112700 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40ded5d00 0x55e40dfe2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8efa112700 1 -- 172.21.15.46:0/2852014335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5b4a0 msgr2=0x55e40dfe1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8efa112700 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5b4a0 0x55e40dfe1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.808+0000 7f8efa112700 1 -- 172.21.15.46:0/2852014335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40dd76490 con 0x55e40dfd8400 2024-05-09T00:13:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.809+0000 7f8ef9911700 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5b4a0 0x55e40dfe1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.809+0000 7f8efa112700 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 0x55e40dfe2a70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0a6450 tx=0x7f8eec0a99d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40c6dd1e7ccfa94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.809+0000 7f8edd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.809+0000 7f8edd7fa700 1 -- 172.21.15.46:0/2852014335 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec023040 con 0x55e40dfd8400 2024-05-09T00:13:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.809+0000 7f8edd7fa700 1 -- 172.21.15.46:0/2852014335 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8eec092160 con 0x55e40dfd8400 2024-05-09T00:13:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.809+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2852014335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef0010de0 con 0x55e40dfd8400 2024-05-09T00:13:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.810+0000 7f8edd7fa700 1 -- 172.21.15.46:0/2852014335 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8eec0b2040 con 0x55e40dfd8400 2024-05-09T00:13:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.810+0000 7f8f0a58bb80 10 client.15420 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.810+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2852014335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e40dfef9e0 con 0x55e40dfd8400 2024-05-09T00:13:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.810+0000 7f8edd7fa700 1 -- 172.21.15.46:0/2852014335 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef0010de0 con 0x55e40dfd8400 2024-05-09T00:13:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.810+0000 7f8edd7fa700 1 -- 172.21.15.46:0/2852014335 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8eec0ad1c0 con 0x55e40dfd8400 2024-05-09T00:13:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.810+0000 7f8f0a58bb80 10 client.15420 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.810+0000 7f8f0a58bb80 10 client.15420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.810+0000 7f8f0a58bb80 20 client.15420 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.810+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e40dff1f70 0x55e40dff4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.811+0000 7f8f0a58bb80 4 client.15420 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:22.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.811+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2852014335 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e40df08a80 con 0x55e40dff1f70 2024-05-09T00:13:22.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.811+0000 7f8ef9911700 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e40dff1f70 0x55e40dff4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.812+0000 7f8edd7fa700 10 client.15420 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:22.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.812+0000 7f8ef9911700 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e40dff1f70 0x55e40dff4390 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74ef7988586bbb26 server_cookie=c92fcf8c1f0a0153 in_seq=0 out_seq=0 2024-05-09T00:13:22.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.813+0000 7f8edd7fa700 1 -- 172.21.15.46:0/2852014335 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e40df08a80 con 0x55e40dff1f70 2024-05-09T00:13:22.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.813+0000 7f8edd7fa700 10 client.15420 handle_command_reply: tid=0 2024-05-09T00:13:22.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.813+0000 7f8f0a58bb80 1 client.15420 shutdown 2024-05-09T00:13:22.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.813+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2852014335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 msgr2=0x55e40dfe2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:22.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.813+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 0x55e40dfe2a70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0a6450 tx=0x7f8eec0a99d0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.813+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2852014335 shutdown_connections 2024-05-09T00:13:22.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.813+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e40dff1f70 0x55e40dff4390 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.813+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40dfd8400 0x55e40dfe2a70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.813+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5b4a0 0x55e40dfe1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.813+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/2852014335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40ded5d00 0x55e40dfe2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.813+0000 7f8edd7fa700 0 client.15420 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:22.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.813+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2852014335 >> 172.21.15.46:0/2852014335 conn(0x55e40de5ef70 msgr2=0x55e40dfccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:22.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.814+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2852014335 shutdown_connections 2024-05-09T00:13:22.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.814+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2852014335 wait complete. 2024-05-09T00:13:22.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.814+0000 7f8f0a58bb80 20 client.15420 trim_cache size 0 max 16384 2024-05-09T00:13:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.855+0000 7f8f0a58bb80 1 Processor -- start 2024-05-09T00:13:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.855+0000 7f8f0a58bb80 1 -- start start 2024-05-09T00:13:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.855+0000 7f8f0a58bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40de5ef70 0x55e40dfdfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.855+0000 7f8f0a58bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5b4a0 0x55e40dfe28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.855+0000 7f8f0a58bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40ded5d00 0x55e40dfcd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.855+0000 7f8f0a58bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41af410 con 0x55e40de5ef70 2024-05-09T00:13:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.855+0000 7f8f0a58bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41acca0 con 0x55e40de5b4a0 2024-05-09T00:13:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.855+0000 7f8f0a58bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41aaad0 con 0x55e40ded5d00 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.855+0000 7f8efa112700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40ded5d00 0x55e40dfcd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.855+0000 7f8efa112700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40ded5d00 0x55e40dfcd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:41852/0 (socket says 172.21.15.46:41852) 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.856+0000 7f8efa112700 1 -- 172.21.15.46:0/2509224257 learned_addr learned my addr 172.21.15.46:0/2509224257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.856+0000 7f8ef9911700 1 --2- 172.21.15.46:0/2509224257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40de5ef70 0x55e40dfdfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.856+0000 7f8ef9110700 1 --2- 172.21.15.46:0/2509224257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5b4a0 0x55e40dfe28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.856+0000 7f8efa112700 1 -- 172.21.15.46:0/2509224257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5b4a0 msgr2=0x55e40dfe28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.856+0000 7f8efa112700 1 --2- 172.21.15.46:0/2509224257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5b4a0 0x55e40dfe28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.856+0000 7f8efa112700 1 -- 172.21.15.46:0/2509224257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40de5ef70 msgr2=0x55e40dfdfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.856+0000 7f8efa112700 1 --2- 172.21.15.46:0/2509224257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40de5ef70 0x55e40dfdfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.856+0000 7f8efa112700 1 -- 172.21.15.46:0/2509224257 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e40dfd3750 con 0x55e40ded5d00 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.856+0000 7f8efa112700 1 --2- 172.21.15.46:0/2509224257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40ded5d00 0x55e40dfcd700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0a8ee0 tx=0x7f8eec0a90e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61ae67877b65f01c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.856+0000 7f8edf7fe700 1 -- 172.21.15.46:0/2509224257 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0a5470 con 0x55e40ded5d00 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.857+0000 7f8edf7fe700 1 -- 172.21.15.46:0/2509224257 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eec0ad160 con 0x55e40ded5d00 2024-05-09T00:13:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.857+0000 7f8edf7fe700 1 -- 172.21.15.46:0/2509224257 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0a4db0 con 0x55e40ded5d00 2024-05-09T00:13:22.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:22 smithi169 ceph-mon[34777]: pgmap v436: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:13:22.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:22 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:13:22.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.858+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2509224257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40ded5d00 msgr2=0x55e40dfcd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:22.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.858+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/2509224257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40ded5d00 0x55e40dfcd700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0a8ee0 tx=0x7f8eec0a90e0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.858+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2509224257 shutdown_connections 2024-05-09T00:13:22.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.858+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/2509224257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40ded5d00 0x55e40dfcd700 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.858+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/2509224257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40de5ef70 0x55e40dfdfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.858+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/2509224257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5b4a0 0x55e40dfe28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.858+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2509224257 >> 172.21.15.46:0/2509224257 conn(0x55e40dfd8ae0 msgr2=0x55e40dfccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.858+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2509224257 shutdown_connections 2024-05-09T00:13:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.858+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/2509224257 wait complete. 2024-05-09T00:13:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.859+0000 7f8f0a58bb80 1 -- start start 2024-05-09T00:13:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.859+0000 7f8f0a58bb80 1 Processor -- start 2024-05-09T00:13:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.860+0000 7f8f0a58bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40de5b4a0 0x55e40dfea020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.860+0000 7f8f0a58bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5ef70 0x55e40dfea560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.860+0000 7f8f0a58bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40ded5d00 0x55e40dfe6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.860+0000 7f8f0a58bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41ab160 con 0x55e40ded5d00 2024-05-09T00:13:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.860+0000 7f8f0a58bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41ac5d0 con 0x55e40de5ef70 2024-05-09T00:13:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.860+0000 7f8f0a58bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41ab830 con 0x55e40de5b4a0 2024-05-09T00:13:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.860+0000 7f8f0a58bb80 10 client.0 _ll_register_callbacks cb 0x55e40dd6c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.860+0000 7f8ef9911700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40de5b4a0 0x55e40dfea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.860+0000 7f8ef9911700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40de5b4a0 0x55e40dfea020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:41866/0 (socket says 172.21.15.46:41866) 2024-05-09T00:13:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.860+0000 7f8ef9911700 1 -- 172.21.15.46:0/4087223793 learned_addr learned my addr 172.21.15.46:0/4087223793 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.860+0000 7f8ef9110700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5ef70 0x55e40dfea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.860+0000 7f8efa112700 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40ded5d00 0x55e40dfe6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.861+0000 7f8ef9110700 1 -- 172.21.15.46:0/4087223793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40de5b4a0 msgr2=0x55e40dfea020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.861+0000 7f8ef9110700 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40de5b4a0 0x55e40dfea020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.861+0000 7f8ef9110700 1 -- 172.21.15.46:0/4087223793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40ded5d00 msgr2=0x55e40dfe6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.861+0000 7f8ef9110700 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40ded5d00 0x55e40dfe6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.861+0000 7f8ef9110700 1 -- 172.21.15.46:0/4087223793 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40dfe7530 con 0x55e40de5ef70 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.861+0000 7f8efa112700 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40ded5d00 0x55e40dfe6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.861+0000 7f8ef9110700 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5ef70 0x55e40dfea560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400b540 tx=0x7f8ee4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7786714858d69b4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.861+0000 7f8edd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.861+0000 7f8edd7fa700 1 -- 172.21.15.46:0/4087223793 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4034070 con 0x55e40de5ef70 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.862+0000 7f8edd7fa700 1 -- 172.21.15.46:0/4087223793 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8ee401e040 con 0x55e40de5ef70 2024-05-09T00:13:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.862+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/4087223793 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eec0ad160 con 0x55e40de5ef70 2024-05-09T00:13:22.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.862+0000 7f8edd7fa700 1 -- 172.21.15.46:0/4087223793 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8ee4005680 con 0x55e40de5ef70 2024-05-09T00:13:22.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.862+0000 7f8f0a58bb80 10 client.25148 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.862+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/4087223793 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e40e05cdc0 con 0x55e40de5ef70 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.862+0000 7f8edd7fa700 1 -- 172.21.15.46:0/4087223793 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eec0ad160 con 0x55e40de5ef70 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.863+0000 7f8edd7fa700 1 -- 172.21.15.46:0/4087223793 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8ee4004b00 con 0x55e40de5ef70 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.863+0000 7f8f0a58bb80 10 client.25148 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.863+0000 7f8f0a58bb80 10 client.25148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.863+0000 7f8f0a58bb80 20 client.25148 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.863+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e40dfd8400 0x55e40e05f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.863+0000 7f8f0a58bb80 4 client.25148 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.863+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/4087223793 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e40df02b20 con 0x55e40dfd8400 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.863+0000 7f8ef9911700 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e40dfd8400 0x55e40e05f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.863+0000 7f8edd7fa700 10 client.25148 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8ef9911700 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e40dfd8400 0x55e40e05f560 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5825039ceb8dc365 server_cookie=c1a7eb98e81c7ab6 in_seq=0 out_seq=0 2024-05-09T00:13:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8edd7fa700 1 -- 172.21.15.46:0/4087223793 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e40df02b20 con 0x55e40dfd8400 2024-05-09T00:13:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8edd7fa700 10 client.25148 handle_command_reply: tid=0 2024-05-09T00:13:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8f0a58bb80 1 client.25148 shutdown 2024-05-09T00:13:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/4087223793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5ef70 msgr2=0x55e40dfea560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5ef70 0x55e40dfea560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400b540 tx=0x7f8ee4009310 comp rx=0 tx=0).stop 2024-05-09T00:13:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/4087223793 shutdown_connections 2024-05-09T00:13:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e40dfd8400 0x55e40e05f560 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e40ded5d00 0x55e40dfe6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e40de5b4a0 0x55e40dfea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8edd7fa700 0 client.25148 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:22.865 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:13:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:13:22.866 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:13:22.866 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:13:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8f0a58bb80 1 --2- 172.21.15.46:0/4087223793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e40de5ef70 0x55e40dfea560 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.864+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/4087223793 >> 172.21.15.46:0/4087223793 conn(0x55e40dfd8ae0 msgr2=0x55e40dfdbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.865+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/4087223793 shutdown_connections 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.865+0000 7f8f0a58bb80 1 -- 172.21.15.46:0/4087223793 wait complete. 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.865+0000 7f8f0a58bb80 20 client.25148 trim_cache size 0 max 16384 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.865+0000 7f8efbb75700 1 -- 172.21.15.46:0/3296981321 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8ee00b2900 msgr2=0x7f8ee00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.865+0000 7f8efbb75700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8ee00b2900 0x7f8ee00b4e10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4103520 tx=0x7f8ee4009690 comp rx=0 tx=0).stop 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.865+0000 7f8efbb75700 1 -- 172.21.15.46:0/3296981321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef4109850 msgr2=0x7f8ef4198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.865+0000 7f8efbb75700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef4109850 0x7f8ef4198aa0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8eec007ee0 tx=0x7f8eec00d3b0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.866+0000 7f8efbb75700 1 -- 172.21.15.46:0/3296981321 shutdown_connections 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.866+0000 7f8efbb75700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8ee00b2900 0x7f8ee00b4e10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.866+0000 7f8efbb75700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef4109850 0x7f8ef4198aa0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.866+0000 7f8efbb75700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef41045d0 0x7f8ef419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.866+0000 7f8efbb75700 1 --2- 172.21.15.46:0/3296981321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ef4108f00 0x7f8ef419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.866+0000 7f8efbb75700 1 -- 172.21.15.46:0/3296981321 >> 172.21.15.46:0/3296981321 conn(0x7f8ef40ffea0 msgr2=0x7f8ef410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.866+0000 7f8efbb75700 1 -- 172.21.15.46:0/3296981321 shutdown_connections 2024-05-09T00:13:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:22.866+0000 7f8efbb75700 1 -- 172.21.15.46:0/3296981321 wait complete. 2024-05-09T00:13:22.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:22 smithi142 ceph-mon[34533]: pgmap v436: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T00:13:22.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:22 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:13:23.720 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:13:23.720 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:13:24.097 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:24.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:24 smithi046 ceph-mon[30151]: pgmap v437: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T00:13:24.853 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:24 smithi169 ceph-mon[34777]: pgmap v437: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T00:13:24.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:24 smithi142 ceph-mon[34533]: pgmap v437: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T00:13:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.031+0000 7f2c851eb700 1 -- 172.21.15.46:0/4101846943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c80105480 msgr2=0x7f2c80105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.031+0000 7f2c851eb700 1 --2- 172.21.15.46:0/4101846943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c80105480 0x7f2c80105890 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009990 tx=0x7f2c70009ca0 comp rx=0 tx=0).stop 2024-05-09T00:13:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.031+0000 7f2c851eb700 1 -- 172.21.15.46:0/4101846943 shutdown_connections 2024-05-09T00:13:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.032+0000 7f2c851eb700 1 --2- 172.21.15.46:0/4101846943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c8010e8f0 0x7f2c80110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.032+0000 7f2c851eb700 1 --2- 172.21.15.46:0/4101846943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c80105480 0x7f2c80105890 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.032+0000 7f2c851eb700 1 --2- 172.21.15.46:0/4101846943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c80105dd0 0x7f2c8010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.032+0000 7f2c851eb700 1 -- 172.21.15.46:0/4101846943 >> 172.21.15.46:0/4101846943 conn(0x7f2c800fef30 msgr2=0x7f2c80101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.032+0000 7f2c851eb700 1 -- 172.21.15.46:0/4101846943 shutdown_connections 2024-05-09T00:13:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.032+0000 7f2c851eb700 1 -- 172.21.15.46:0/4101846943 wait complete. 2024-05-09T00:13:26.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.034+0000 7f2c851eb700 1 Processor -- start 2024-05-09T00:13:26.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.034+0000 7f2c851eb700 1 -- start start 2024-05-09T00:13:26.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c851eb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c80105480 0x7f2c8019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:26.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c851eb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c80105dd0 0x7f2c8019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:26.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c851eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c8010e8f0 0x7f2c8019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c851eb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c80114510 con 0x7f2c8010e8f0 2024-05-09T00:13:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c851eb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2c80114390 con 0x7f2c80105dd0 2024-05-09T00:13:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c7f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c8010e8f0 0x7f2c8019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c7e59c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c80105dd0 0x7f2c8019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c7ed9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c80105480 0x7f2c8019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c7f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c8010e8f0 0x7f2c8019f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55704/0 (socket says 172.21.15.46:55704) 2024-05-09T00:13:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c7f59e700 1 -- 172.21.15.46:0/3540274006 learned_addr learned my addr 172.21.15.46:0/3540274006 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c851eb700 1 -- 172.21.15.46:0/3540274006 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2c80114690 con 0x7f2c80105480 2024-05-09T00:13:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c7ed9d700 1 -- 172.21.15.46:0/3540274006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c80105dd0 msgr2=0x7f2c8019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c7ed9d700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c80105dd0 0x7f2c8019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.035+0000 7f2c7ed9d700 1 -- 172.21.15.46:0/3540274006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c8010e8f0 msgr2=0x7f2c8019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.036+0000 7f2c7ed9d700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c8010e8f0 0x7f2c8019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.036+0000 7f2c7ed9d700 1 -- 172.21.15.46:0/3540274006 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8019fc30 con 0x7f2c80105480 2024-05-09T00:13:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.036+0000 7f2c7f59e700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c8010e8f0 0x7f2c8019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.036+0000 7f2c7ed9d700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c80105480 0x7f2c8019ad60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2c700098e0 tx=0x7f2c7000ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.036+0000 7f2c67fff700 1 -- 172.21.15.46:0/3540274006 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70026070 con 0x7f2c80105480 2024-05-09T00:13:26.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.036+0000 7f2c67fff700 1 -- 172.21.15.46:0/3540274006 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c700054e0 con 0x7f2c80105480 2024-05-09T00:13:26.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.036+0000 7f2c851eb700 1 -- 172.21.15.46:0/3540274006 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8019fec0 con 0x7f2c80105480 2024-05-09T00:13:26.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.036+0000 7f2c851eb700 1 -- 172.21.15.46:0/3540274006 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c801a7840 con 0x7f2c80105480 2024-05-09T00:13:26.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.036+0000 7f2c67fff700 1 -- 172.21.15.46:0/3540274006 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70021410 con 0x7f2c80105480 2024-05-09T00:13:26.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.039+0000 7f2c851eb700 1 -- 172.21.15.46:0/3540274006 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c8006c920 con 0x7f2c80105480 2024-05-09T00:13:26.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.041+0000 7f2c67fff700 1 -- 172.21.15.46:0/3540274006 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2c70029070 con 0x7f2c80105480 2024-05-09T00:13:26.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.042+0000 7f2c67fff700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2c680b2930 0x7f2c680b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:26.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.042+0000 7f2c67fff700 1 -- 172.21.15.46:0/3540274006 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f2c700e2c50 con 0x7f2c80105480 2024-05-09T00:13:26.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.042+0000 7f2c7e59c700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2c680b2930 0x7f2c680b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:26.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.043+0000 7f2c7e59c700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2c680b2930 0x7f2c680b4e40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2c800ff970 tx=0x7f2c74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:26.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.047+0000 7f2c67fff700 1 -- 172.21.15.46:0/3540274006 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c7005edf0 con 0x7f2c80105480 2024-05-09T00:13:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.446+0000 7f2c851eb700 1 -- 172.21.15.46:0/3540274006 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c8019c0c0 con 0x7f2c80105480 2024-05-09T00:13:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.447+0000 7f2c67fff700 1 -- 172.21.15.46:0/3540274006 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f2c70013070 con 0x7f2c80105480 2024-05-09T00:13:26.446 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:13:26.446 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:13:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.450+0000 7f2c851eb700 1 -- 172.21.15.46:0/3540274006 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2c680b2930 msgr2=0x7f2c680b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.450+0000 7f2c851eb700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2c680b2930 0x7f2c680b4e40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2c800ff970 tx=0x7f2c74009260 comp rx=0 tx=0).stop 2024-05-09T00:13:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.450+0000 7f2c851eb700 1 -- 172.21.15.46:0/3540274006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c80105480 msgr2=0x7f2c8019ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.450+0000 7f2c851eb700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c80105480 0x7f2c8019ad60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2c700098e0 tx=0x7f2c7000ff80 comp rx=0 tx=0).stop 2024-05-09T00:13:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.450+0000 7f2c851eb700 1 -- 172.21.15.46:0/3540274006 shutdown_connections 2024-05-09T00:13:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.450+0000 7f2c851eb700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2c680b2930 0x7f2c680b4e40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.450+0000 7f2c851eb700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c8010e8f0 0x7f2c8019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.450+0000 7f2c851eb700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c80105480 0x7f2c8019ad60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.450+0000 7f2c851eb700 1 --2- 172.21.15.46:0/3540274006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c80105dd0 0x7f2c8019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.450+0000 7f2c851eb700 1 -- 172.21.15.46:0/3540274006 >> 172.21.15.46:0/3540274006 conn(0x7f2c800fef30 msgr2=0x7f2c801082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.450+0000 7f2c851eb700 1 -- 172.21.15.46:0/3540274006 shutdown_connections 2024-05-09T00:13:26.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:26.450+0000 7f2c851eb700 1 -- 172.21.15.46:0/3540274006 wait complete. 2024-05-09T00:13:26.450 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:13:26.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:26 smithi046 ceph-mon[30151]: pgmap v438: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-05-09T00:13:26.786 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:26 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3540274006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:13:26.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:26 smithi142 ceph-mon[34533]: pgmap v438: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-05-09T00:13:26.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:26 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3540274006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:13:27.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:26 smithi169 ceph-mon[34777]: pgmap v438: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-05-09T00:13:27.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:26 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3540274006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:13:27.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:13:27.531 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:28.871 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:28 smithi046 ceph-mon[30151]: pgmap v439: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:28.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:28 smithi142 ceph-mon[34533]: pgmap v439: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:29.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:28 smithi169 ceph-mon[34777]: pgmap v439: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:29.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.519+0000 7fee9fa77700 1 -- 172.21.15.46:0/1639732356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981027e0 msgr2=0x7fee98102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:29.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.519+0000 7fee9fa77700 1 --2- 172.21.15.46:0/1639732356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981027e0 0x7fee98102bf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fee88009990 tx=0x7fee88009ca0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.520+0000 7fee9fa77700 1 -- 172.21.15.46:0/1639732356 shutdown_connections 2024-05-09T00:13:29.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.520+0000 7fee9fa77700 1 --2- 172.21.15.46:0/1639732356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee98103130 0x7fee98111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.520+0000 7fee9fa77700 1 --2- 172.21.15.46:0/1639732356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9810a3e0 0x7fee9810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.520+0000 7fee9fa77700 1 --2- 172.21.15.46:0/1639732356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981027e0 0x7fee98102bf0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.520+0000 7fee9fa77700 1 -- 172.21.15.46:0/1639732356 >> 172.21.15.46:0/1639732356 conn(0x7fee980fe6f0 msgr2=0x7fee98100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:29.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.520+0000 7fee9fa77700 1 -- 172.21.15.46:0/1639732356 shutdown_connections 2024-05-09T00:13:29.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.520+0000 7fee9fa77700 1 -- 172.21.15.46:0/1639732356 wait complete. 2024-05-09T00:13:29.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.521+0000 7fee9fa77700 1 Processor -- start 2024-05-09T00:13:29.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.521+0000 7fee9fa77700 1 -- start start 2024-05-09T00:13:29.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.521+0000 7fee9fa77700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981027e0 0x7fee9819d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.521+0000 7fee9fa77700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee98103130 0x7fee9819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.521+0000 7fee9fa77700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9810a3e0 0x7fee981a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.521+0000 7fee9fa77700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee98114460 con 0x7fee981027e0 2024-05-09T00:13:29.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.521+0000 7fee9fa77700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee981142e0 con 0x7fee98103130 2024-05-09T00:13:29.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.521+0000 7fee9fa77700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee981145e0 con 0x7fee9810a3e0 2024-05-09T00:13:29.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.521+0000 7fee9d813700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981027e0 0x7fee9819d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.522+0000 7fee9d813700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981027e0 0x7fee9819d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44322/0 (socket says 172.21.15.46:44322) 2024-05-09T00:13:29.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.522+0000 7fee9d012700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee98103130 0x7fee9819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.522+0000 7fee9d813700 1 -- 172.21.15.46:0/1412593885 learned_addr learned my addr 172.21.15.46:0/1412593885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:29.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.522+0000 7fee9e014700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9810a3e0 0x7fee981a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.522+0000 7fee9d012700 1 -- 172.21.15.46:0/1412593885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9810a3e0 msgr2=0x7fee981a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:29.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.522+0000 7fee9d012700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9810a3e0 0x7fee981a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.522+0000 7fee9d012700 1 -- 172.21.15.46:0/1412593885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981027e0 msgr2=0x7fee9819d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:29.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.522+0000 7fee9d012700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981027e0 0x7fee9819d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.522+0000 7fee9d012700 1 -- 172.21.15.46:0/1412593885 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee981a2000 con 0x7fee98103130 2024-05-09T00:13:29.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.522+0000 7fee9d813700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981027e0 0x7fee9819d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:29.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.523+0000 7fee9d012700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee98103130 0x7fee9819d6d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fee880098e0 tx=0x7fee880047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:29.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.523+0000 7fee8effd700 1 -- 172.21.15.46:0/1412593885 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88027070 con 0x7fee98103130 2024-05-09T00:13:29.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.523+0000 7fee8effd700 1 -- 172.21.15.46:0/1412593885 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee88003f20 con 0x7fee98103130 2024-05-09T00:13:29.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.523+0000 7fee9fa77700 1 -- 172.21.15.46:0/1412593885 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee981a2290 con 0x7fee98103130 2024-05-09T00:13:29.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.523+0000 7fee8effd700 1 -- 172.21.15.46:0/1412593885 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88022be0 con 0x7fee98103130 2024-05-09T00:13:29.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.523+0000 7fee9fa77700 1 -- 172.21.15.46:0/1412593885 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee981a2790 con 0x7fee98103130 2024-05-09T00:13:29.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.528+0000 7feeae48db80 1 Processor -- start 2024-05-09T00:13:29.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.528+0000 7feeae48db80 1 -- start start 2024-05-09T00:13:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.528+0000 7fee8effd700 1 -- 172.21.15.46:0/1412593885 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fee8802a070 con 0x7fee98103130 2024-05-09T00:13:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.528+0000 7feeae48db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 0x5567a4f704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.528+0000 7feeae48db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f70a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.528+0000 7feeae48db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4f6a6a0 0x5567a4f6aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.528+0000 7feeae48db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5567a4e8cff0 con 0x5567a4ed7160 2024-05-09T00:13:29.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.528+0000 7feeae48db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5567a4e2b7f0 con 0x5567a4ded4a0 2024-05-09T00:13:29.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.528+0000 7feeae48db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5567a4e88fb0 con 0x5567a4f6a6a0 2024-05-09T00:13:29.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.528+0000 7fee9e014700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 0x5567a4f704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.528+0000 7fee9e014700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 0x5567a4f704f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44334/0 (socket says 172.21.15.46:44334) 2024-05-09T00:13:29.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.528+0000 7fee9e014700 1 -- 172.21.15.46:0/3137324911 learned_addr learned my addr 172.21.15.46:0/3137324911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:29.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.529+0000 7fee9d813700 1 --2- 172.21.15.46:0/3137324911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f70a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.529+0000 7fee9d012700 1 --2- 172.21.15.46:0/3137324911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4f6a6a0 0x5567a4f6aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.529+0000 7fee9e014700 1 -- 172.21.15.46:0/3137324911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4f6a6a0 msgr2=0x5567a4f6aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.529+0000 7fee9e014700 1 --2- 172.21.15.46:0/3137324911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4f6a6a0 0x5567a4f6aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.529+0000 7fee9e014700 1 -- 172.21.15.46:0/3137324911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 msgr2=0x5567a4f70a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.529+0000 7fee9e014700 1 --2- 172.21.15.46:0/3137324911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f70a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.529+0000 7fee9e014700 1 -- 172.21.15.46:0/3137324911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567a4e23370 con 0x5567a4ed7160 2024-05-09T00:13:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.529+0000 7fee9d012700 1 --2- 172.21.15.46:0/3137324911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4f6a6a0 0x5567a4f6aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.529+0000 7fee9e014700 1 --2- 172.21.15.46:0/3137324911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 0x5567a4f704f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fee900098f0 tx=0x7fee90009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9c6a7619b4dd90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.530+0000 7fee83fff700 1 -- 172.21.15.46:0/3137324911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90004bb0 con 0x5567a4ed7160 2024-05-09T00:13:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.530+0000 7fee83fff700 1 -- 172.21.15.46:0/3137324911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee90004d50 con 0x5567a4ed7160 2024-05-09T00:13:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.530+0000 7fee83fff700 1 -- 172.21.15.46:0/3137324911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90015480 con 0x5567a4ed7160 2024-05-09T00:13:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.531+0000 7fee8effd700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fee840b2900 0x7fee840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.531+0000 7feeae48db80 1 -- 172.21.15.46:0/3137324911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 msgr2=0x5567a4f704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.531+0000 7feeae48db80 1 --2- 172.21.15.46:0/3137324911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 0x5567a4f704f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fee900098f0 tx=0x7fee90009c00 comp rx=0 tx=0).stop 2024-05-09T00:13:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.531+0000 7fee8effd700 1 -- 172.21.15.46:0/1412593885 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fee880e2fe0 con 0x7fee98103130 2024-05-09T00:13:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.531+0000 7feeae48db80 1 -- 172.21.15.46:0/3137324911 shutdown_connections 2024-05-09T00:13:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.531+0000 7fee9d813700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fee840b2900 0x7fee840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.531+0000 7feeae48db80 1 --2- 172.21.15.46:0/3137324911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4f6a6a0 0x5567a4f6aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.531+0000 7feeae48db80 1 --2- 172.21.15.46:0/3137324911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 0x5567a4f704f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.531+0000 7feeae48db80 1 --2- 172.21.15.46:0/3137324911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f70a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.531+0000 7feeae48db80 1 -- 172.21.15.46:0/3137324911 >> 172.21.15.46:0/3137324911 conn(0x5567a4e4d6a0 msgr2=0x5567a4f65400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.532+0000 7feeae48db80 1 -- 172.21.15.46:0/3137324911 shutdown_connections 2024-05-09T00:13:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.532+0000 7feeae48db80 1 -- 172.21.15.46:0/3137324911 wait complete. 2024-05-09T00:13:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.532+0000 7fee9d813700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fee840b2900 0x7fee840b4e10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5567a4f6ffb0 tx=0x7fee9400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.532+0000 7feeae48db80 1 -- start start 2024-05-09T00:13:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.532+0000 7feeae48db80 1 Processor -- start 2024-05-09T00:13:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.533+0000 7feeae48db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.533+0000 7feeae48db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 0x5567a4f746f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.533+0000 7feeae48db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4f6a6a0 0x5567a4f74c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.533+0000 7feeae48db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5567a4d08ba0 con 0x5567a4f6a6a0 2024-05-09T00:13:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.533+0000 7feeae48db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5567a4e47150 con 0x5567a4ded4a0 2024-05-09T00:13:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.533+0000 7feeae48db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5567a4f7f9e0 con 0x5567a4ed7160 2024-05-09T00:13:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.533+0000 7feeae48db80 10 client.0 _ll_register_callbacks cb 0x5567a4cfe6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.533+0000 7fee9d012700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 0x5567a4f746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.533+0000 7fee9d012700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 0x5567a4f746f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:45750/0 (socket says 172.21.15.46:45750) 2024-05-09T00:13:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.533+0000 7fee9d012700 1 -- 172.21.15.46:0/1970148089 learned_addr learned my addr 172.21.15.46:0/1970148089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.533+0000 7fee9e014700 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4f6a6a0 0x5567a4f74c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.534+0000 7fee9d813700 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.534+0000 7fee9d012700 1 -- 172.21.15.46:0/1970148089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 msgr2=0x5567a4f741b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.534+0000 7fee9d012700 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f741b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.534+0000 7fee9d012700 1 -- 172.21.15.46:0/1970148089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4f6a6a0 msgr2=0x5567a4f74c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.534+0000 7fee9d012700 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4f6a6a0 0x5567a4f74c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.534+0000 7fee9d012700 1 -- 172.21.15.46:0/1970148089 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a4d08490 con 0x5567a4ed7160 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.534+0000 7fee9d813700 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.534+0000 7fee9d012700 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 0x5567a4f746f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fee88000c00 tx=0x7fee880aabd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a700bf50281ff59e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.534+0000 7fee817fa700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.534+0000 7fee817fa700 1 -- 172.21.15.46:0/1970148089 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee880a6550 con 0x5567a4ed7160 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.534+0000 7fee817fa700 1 -- 172.21.15.46:0/1970148089 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fee88092110 con 0x5567a4ed7160 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.534+0000 7feeae48db80 1 -- 172.21.15.46:0/1970148089 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee90004d50 con 0x5567a4ed7160 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.535+0000 7fee817fa700 1 -- 172.21.15.46:0/1970148089 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee880a9e00 con 0x5567a4ed7160 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.535+0000 7feeae48db80 10 client.25189 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.535+0000 7feeae48db80 1 -- 172.21.15.46:0/1970148089 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567a4f8a220 con 0x5567a4ed7160 2024-05-09T00:13:29.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.535+0000 7fee817fa700 1 -- 172.21.15.46:0/1970148089 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee90004d50 con 0x5567a4ed7160 2024-05-09T00:13:29.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.535+0000 7fee817fa700 1 -- 172.21.15.46:0/1970148089 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fee880e50d0 con 0x5567a4ed7160 2024-05-09T00:13:29.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.536+0000 7feeae48db80 10 client.25189 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:29.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.536+0000 7feeae48db80 10 client.25189 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:13:29.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.536+0000 7feeae48db80 20 client.25189 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:29.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.536+0000 7feeae48db80 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5567a4f8ca10 0x5567a4f8ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.536+0000 7feeae48db80 4 client.25189 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:29.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.536+0000 7feeae48db80 1 -- 172.21.15.46:0/1970148089 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567a4e9a9d0 con 0x5567a4f8ca10 2024-05-09T00:13:29.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.536+0000 7fee9d813700 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5567a4f8ca10 0x5567a4f8ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.536+0000 7fee817fa700 10 client.25189 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:13:29.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.536+0000 7fee9d813700 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5567a4f8ca10 0x5567a4f8ee60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88a8acd4d1350aa5 server_cookie=9b0ec098ded461d2 in_seq=0 out_seq=0 2024-05-09T00:13:29.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.537+0000 7fee817fa700 1 -- 172.21.15.46:0/1970148089 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5567a4e9a9d0 con 0x5567a4f8ca10 2024-05-09T00:13:29.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.537+0000 7fee817fa700 10 client.25189 handle_command_reply: tid=0 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.537+0000 7feeae48db80 1 client.25189 shutdown 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.537+0000 7feeae48db80 1 -- 172.21.15.46:0/1970148089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 msgr2=0x5567a4f746f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.537+0000 7feeae48db80 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 0x5567a4f746f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fee88000c00 tx=0x7fee880aabd0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.538+0000 7feeae48db80 1 -- 172.21.15.46:0/1970148089 shutdown_connections 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.538+0000 7feeae48db80 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5567a4f8ca10 0x5567a4f8ee60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.538+0000 7feeae48db80 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4f6a6a0 0x5567a4f74c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.538+0000 7feeae48db80 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.538+0000 7feeae48db80 1 --2- 172.21.15.46:0/1970148089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 0x5567a4f746f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.538+0000 7feeae48db80 1 -- 172.21.15.46:0/1970148089 >> 172.21.15.46:0/1970148089 conn(0x5567a4e4d6a0 msgr2=0x5567a4f651c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.538+0000 7fee817fa700 0 client.25189 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.538+0000 7feeae48db80 1 -- 172.21.15.46:0/1970148089 shutdown_connections 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.538+0000 7feeae48db80 1 -- 172.21.15.46:0/1970148089 wait complete. 2024-05-09T00:13:29.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.538+0000 7feeae48db80 20 client.25189 trim_cache size 0 max 16384 2024-05-09T00:13:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.568+0000 7feeae48db80 1 Processor -- start 2024-05-09T00:13:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.569+0000 7feeae48db80 1 -- start start 2024-05-09T00:13:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.569+0000 7feeae48db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f6f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.569+0000 7feeae48db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 0x5567a4f888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.569+0000 7feeae48db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4f6a6a0 0x5567a4f94620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.569+0000 7feeae48db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee981af1a0 con 0x5567a4ed7160 2024-05-09T00:13:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.569+0000 7feeae48db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee981aca30 con 0x5567a4ded4a0 2024-05-09T00:13:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.569+0000 7feeae48db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee9810ef20 con 0x5567a4f6a6a0 2024-05-09T00:13:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.569+0000 7fee9d012700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 0x5567a4f888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.569+0000 7fee9d012700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 0x5567a4f888b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44350/0 (socket says 172.21.15.46:44350) 2024-05-09T00:13:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.569+0000 7fee9d012700 1 -- 172.21.15.46:0/486822018 learned_addr learned my addr 172.21.15.46:0/486822018 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.569+0000 7fee9e014700 1 --2- 172.21.15.46:0/486822018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4f6a6a0 0x5567a4f94620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.569+0000 7fee9d813700 1 --2- 172.21.15.46:0/486822018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f6f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.570+0000 7fee9e014700 1 -- 172.21.15.46:0/486822018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 msgr2=0x5567a4f6f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.570+0000 7fee9e014700 1 --2- 172.21.15.46:0/486822018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f6f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.570+0000 7fee9e014700 1 -- 172.21.15.46:0/486822018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 msgr2=0x5567a4f888b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.570+0000 7fee9e014700 1 --2- 172.21.15.46:0/486822018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 0x5567a4f888b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.570+0000 7fee9e014700 1 -- 172.21.15.46:0/486822018 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567a4f5ef60 con 0x5567a4f6a6a0 2024-05-09T00:13:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.570+0000 7fee9d012700 1 --2- 172.21.15.46:0/486822018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 0x5567a4f888b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.570+0000 7fee9e014700 1 --2- 172.21.15.46:0/486822018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4f6a6a0 0x5567a4f94620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee90000f80 tx=0x7fee90007a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13328d2a6ad9e321 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.570+0000 7fee837fe700 1 -- 172.21.15.46:0/486822018 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90007810 con 0x5567a4f6a6a0 2024-05-09T00:13:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.570+0000 7fee837fe700 1 -- 172.21.15.46:0/486822018 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee90023020 con 0x5567a4f6a6a0 2024-05-09T00:13:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.571+0000 7fee837fe700 1 -- 172.21.15.46:0/486822018 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90004d70 con 0x5567a4f6a6a0 2024-05-09T00:13:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.572+0000 7feeae48db80 1 -- 172.21.15.46:0/486822018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4f6a6a0 msgr2=0x5567a4f94620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.572+0000 7feeae48db80 1 --2- 172.21.15.46:0/486822018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4f6a6a0 0x5567a4f94620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee90000f80 tx=0x7fee90007a70 comp rx=0 tx=0).stop 2024-05-09T00:13:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.572+0000 7feeae48db80 1 -- 172.21.15.46:0/486822018 shutdown_connections 2024-05-09T00:13:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.572+0000 7feeae48db80 1 --2- 172.21.15.46:0/486822018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4f6a6a0 0x5567a4f94620 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.572+0000 7feeae48db80 1 --2- 172.21.15.46:0/486822018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4ded4a0 0x5567a4f6f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.572+0000 7feeae48db80 1 --2- 172.21.15.46:0/486822018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ed7160 0x5567a4f888b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.572+0000 7feeae48db80 1 -- 172.21.15.46:0/486822018 >> 172.21.15.46:0/486822018 conn(0x5567a4e4d6a0 msgr2=0x5567a4f6f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.572+0000 7feeae48db80 1 -- 172.21.15.46:0/486822018 shutdown_connections 2024-05-09T00:13:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.572+0000 7feeae48db80 1 -- 172.21.15.46:0/486822018 wait complete. 2024-05-09T00:13:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.572+0000 7feeae48db80 1 -- start start 2024-05-09T00:13:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.573+0000 7feeae48db80 1 Processor -- start 2024-05-09T00:13:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.573+0000 7feeae48db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ded4a0 0x5567a4f81c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.573+0000 7fee9d813700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ded4a0 0x5567a4f81c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.573+0000 7fee9d813700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ded4a0 0x5567a4f81c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44352/0 (socket says 172.21.15.46:44352) 2024-05-09T00:13:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.573+0000 7feeae48db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 0x5567a4f82140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.573+0000 7fee9d813700 1 -- 172.21.15.46:0/2639030496 learned_addr learned my addr 172.21.15.46:0/2639030496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.573+0000 7feeae48db80 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4f6a6a0 0x5567a4f82680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7feeae48db80 1 -- 172.21.15.46:0/2639030496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee981aaef0 con 0x5567a4ded4a0 2024-05-09T00:13:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7feeae48db80 1 -- 172.21.15.46:0/2639030496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee981ac360 con 0x5567a4f6a6a0 2024-05-09T00:13:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7feeae48db80 1 -- 172.21.15.46:0/2639030496 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee981ab5c0 con 0x5567a4ed7160 2024-05-09T00:13:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7fee9d012700 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 0x5567a4f82140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7fee9e014700 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4f6a6a0 0x5567a4f82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7fee9d813700 1 -- 172.21.15.46:0/2639030496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 msgr2=0x5567a4f82140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7fee9d813700 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 0x5567a4f82140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7fee9d813700 1 -- 172.21.15.46:0/2639030496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4f6a6a0 msgr2=0x5567a4f82680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7fee9d813700 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4f6a6a0 0x5567a4f82680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7fee9d813700 1 -- 172.21.15.46:0/2639030496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a4ff5200 con 0x5567a4ded4a0 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7fee9d012700 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 0x5567a4f82140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7feeae48db80 10 client.0 _ll_register_callbacks cb 0x5567a4cfe6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7feeae48db80 1 -- 172.21.15.46:0/2639030496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee90023020 con 0x5567a4ded4a0 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7fee9d813700 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ded4a0 0x5567a4f81c00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x5567a4f66ed0 tx=0x7fee94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bf924de940f810 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.574+0000 7fee817fa700 10 client.15450 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.575+0000 7fee817fa700 1 -- 172.21.15.46:0/2639030496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94004220 con 0x5567a4ded4a0 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.575+0000 7fee817fa700 1 -- 172.21.15.46:0/2639030496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fee94017070 con 0x5567a4ded4a0 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.575+0000 7fee817fa700 1 -- 172.21.15.46:0/2639030496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee94028240 con 0x5567a4ded4a0 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.575+0000 7feeae48db80 10 client.15450 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.575+0000 7feeae48db80 1 -- 172.21.15.46:0/2639030496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5567a4ff6390 con 0x5567a4ded4a0 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.575+0000 7fee817fa700 1 -- 172.21.15.46:0/2639030496 <== mon.0 v2:172.21.15.46:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fee94028ce0 con 0x5567a4ded4a0 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.575+0000 7feeae48db80 10 client.15450 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.575+0000 7feeae48db80 10 client.15450 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:13:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.575+0000 7feeae48db80 20 client.15450 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.575+0000 7feeae48db80 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5567a4ff6960 0x5567a4ff8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.575+0000 7feeae48db80 4 client.15450 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:13:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.575+0000 7feeae48db80 1 -- 172.21.15.46:0/2639030496 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567a4e94a70 con 0x5567a4ff6960 2024-05-09T00:13:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.576+0000 7fee9d012700 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5567a4ff6960 0x5567a4ff8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.576+0000 7fee9d012700 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5567a4ff6960 0x5567a4ff8e00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac0c1cf06e522cb0 server_cookie=edf37b9160a0f2c6 in_seq=0 out_seq=0 2024-05-09T00:13:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.576+0000 7fee817fa700 10 client.15450 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:13:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7fee817fa700 1 -- 172.21.15.46:0/2639030496 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567a4e94a70 con 0x5567a4ff6960 2024-05-09T00:13:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7fee817fa700 10 client.15450 handle_command_reply: tid=0 2024-05-09T00:13:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7feeae48db80 1 client.15450 shutdown 2024-05-09T00:13:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7feeae48db80 1 -- 172.21.15.46:0/2639030496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ded4a0 msgr2=0x5567a4f81c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7feeae48db80 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ded4a0 0x5567a4f81c00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x5567a4f66ed0 tx=0x7fee94004750 comp rx=0 tx=0).stop 2024-05-09T00:13:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7feeae48db80 1 -- 172.21.15.46:0/2639030496 shutdown_connections 2024-05-09T00:13:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7feeae48db80 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5567a4ff6960 0x5567a4ff8e00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7fee817fa700 0 client.15450 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:13:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7feeae48db80 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567a4f6a6a0 0x5567a4f82680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7feeae48db80 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a4ded4a0 0x5567a4f81c00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7feeae48db80 1 --2- 172.21.15.46:0/2639030496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5567a4ed7160 0x5567a4f82140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7feeae48db80 1 -- 172.21.15.46:0/2639030496 >> 172.21.15.46:0/2639030496 conn(0x5567a4e4d6a0 msgr2=0x5567a4f6de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7feeae48db80 1 -- 172.21.15.46:0/2639030496 shutdown_connections 2024-05-09T00:13:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7feeae48db80 1 -- 172.21.15.46:0/2639030496 wait complete. 2024-05-09T00:13:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.577+0000 7feeae48db80 20 client.15450 trim_cache size 0 max 16384 2024-05-09T00:13:29.578 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:13:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.578+0000 7fee9fa77700 1 -- 172.21.15.46:0/1412593885 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fee840b2900 msgr2=0x7fee840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.578+0000 7fee9fa77700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fee840b2900 0x7fee840b4e10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5567a4f6ffb0 tx=0x7fee9400b500 comp rx=0 tx=0).stop 2024-05-09T00:13:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.578+0000 7fee9fa77700 1 -- 172.21.15.46:0/1412593885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee98103130 msgr2=0x7fee9819d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.578+0000 7fee9fa77700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee98103130 0x7fee9819d6d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fee880098e0 tx=0x7fee880047b0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.578+0000 7fee9fa77700 1 -- 172.21.15.46:0/1412593885 shutdown_connections 2024-05-09T00:13:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.578+0000 7fee9fa77700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fee840b2900 0x7fee840b4e10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.578+0000 7fee9fa77700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9810a3e0 0x7fee981a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.578+0000 7fee9fa77700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981027e0 0x7fee9819d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.578+0000 7fee9fa77700 1 --2- 172.21.15.46:0/1412593885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee98103130 0x7fee9819d6d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.578+0000 7fee9fa77700 1 -- 172.21.15.46:0/1412593885 >> 172.21.15.46:0/1412593885 conn(0x7fee980fe6f0 msgr2=0x7fee981038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.579+0000 7fee9fa77700 1 -- 172.21.15.46:0/1412593885 shutdown_connections 2024-05-09T00:13:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:29.579+0000 7fee9fa77700 1 -- 172.21.15.46:0/1412593885 wait complete. 2024-05-09T00:13:30.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:13:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:30.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:30 smithi046 ceph-mon[30151]: pgmap v440: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-09T00:13:30.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:30 smithi142 ceph-mon[34533]: pgmap v440: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-09T00:13:31.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:30 smithi169 ceph-mon[34777]: pgmap v440: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-09T00:13:32.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:32 smithi046 ceph-mon[30151]: pgmap v441: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:32.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:32 smithi142 ceph-mon[34533]: pgmap v441: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.885+0000 7f696f7b2700 1 -- 172.21.15.46:0/773086632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968103290 msgr2=0x7f69681036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.885+0000 7f696f7b2700 1 --2- 172.21.15.46:0/773086632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968103290 0x7f69681036a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6964009990 tx=0x7f6964009ca0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.885+0000 7f696f7b2700 1 -- 172.21.15.46:0/773086632 shutdown_connections 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.885+0000 7f696f7b2700 1 --2- 172.21.15.46:0/773086632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681086e0 0x7f6968108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.885+0000 7f696f7b2700 1 --2- 172.21.15.46:0/773086632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968103290 0x7f69681036a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.885+0000 7f696f7b2700 1 --2- 172.21.15.46:0/773086632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6968103be0 0x7f6968108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.885+0000 7f696f7b2700 1 -- 172.21.15.46:0/773086632 >> 172.21.15.46:0/773086632 conn(0x7f6968078e50 msgr2=0x7f6968079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.885+0000 7f696f7b2700 1 -- 172.21.15.46:0/773086632 shutdown_connections 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.885+0000 7f696f7b2700 1 -- 172.21.15.46:0/773086632 wait complete. 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696f7b2700 1 Processor -- start 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696f7b2700 1 -- start start 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696f7b2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968103290 0x7f69681a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696f7b2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6968103be0 0x7f69681a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696f7b2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681086e0 0x7f69681a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696f7b2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f696811aac0 con 0x7f6968103290 2024-05-09T00:13:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696f7b2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f696811a940 con 0x7f69681086e0 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696f7b2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f696811ac40 con 0x7f6968103be0 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696d54e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968103290 0x7f69681a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696d54e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968103290 0x7f69681a1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44370/0 (socket says 172.21.15.46:44370) 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696d54e700 1 -- 172.21.15.46:0/1791353327 learned_addr learned my addr 172.21.15.46:0/1791353327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696cd4d700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6968103be0 0x7f69681a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696dd4f700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681086e0 0x7f69681a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696d54e700 1 -- 172.21.15.46:0/1791353327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6968103be0 msgr2=0x7f69681a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696d54e700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6968103be0 0x7f69681a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696d54e700 1 -- 172.21.15.46:0/1791353327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681086e0 msgr2=0x7f69681a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696d54e700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681086e0 0x7f69681a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.886+0000 7f696d54e700 1 -- 172.21.15.46:0/1791353327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69681a6340 con 0x7f6968103290 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.887+0000 7f696dd4f700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681086e0 0x7f69681a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.887+0000 7f696d54e700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968103290 0x7f69681a1470 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6964000c00 tx=0x7f69640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:32.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.887+0000 7f695e7fc700 1 -- 172.21.15.46:0/1791353327 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964027070 con 0x7f6968103290 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.887+0000 7f696f7b2700 1 -- 172.21.15.46:0/1791353327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69681a65d0 con 0x7f6968103290 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.887+0000 7f695e7fc700 1 -- 172.21.15.46:0/1791353327 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6964003fa0 con 0x7f6968103290 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.887+0000 7f696f7b2700 1 -- 172.21.15.46:0/1791353327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69681adf50 con 0x7f6968103290 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.887+0000 7f695e7fc700 1 -- 172.21.15.46:0/1791353327 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964022d70 con 0x7f6968103290 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.890+0000 7f695e7fc700 1 -- 172.21.15.46:0/1791353327 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f696402a070 con 0x7f6968103290 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.891+0000 7f695e7fc700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f69540b2880 0x7f69540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.891+0000 7f695e7fc700 1 -- 172.21.15.46:0/1791353327 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f69640e2fe0 con 0x7f6968103290 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.891+0000 7f696cd4d700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f69540b2880 0x7f69540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.891+0000 7f697e1c8b80 1 Processor -- start 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.891+0000 7f697e1c8b80 1 -- start start 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.891+0000 7f696cd4d700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f69540b2880 0x7f69540b4d90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6958005950 tx=0x7f69580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.891+0000 7f697e1c8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066ea13d00 0x56066eb1c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.891+0000 7f697e1c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066e9994a0 0x56066eb1c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.891+0000 7f697e1c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066eb16400 0x56066eb16810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.891+0000 7f697e1c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56066ea390a0 con 0x56066e9994a0 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.891+0000 7f697e1c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56066e9de9a0 con 0x56066eb16400 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.891+0000 7f697e1c8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56066ea35060 con 0x56066ea13d00 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f696cd4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066e9994a0 0x56066eb1c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f696cd4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066e9994a0 0x56066eb1c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44386/0 (socket says 172.21.15.46:44386) 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f696cd4d700 1 -- 172.21.15.46:0/2284219854 learned_addr learned my addr 172.21.15.46:0/2284219854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f696d54e700 1 --2- 172.21.15.46:0/2284219854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066ea13d00 0x56066eb1c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f696dd4f700 1 --2- 172.21.15.46:0/2284219854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066eb16400 0x56066eb16810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f696cd4d700 1 -- 172.21.15.46:0/2284219854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066ea13d00 msgr2=0x56066eb1c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f696cd4d700 1 --2- 172.21.15.46:0/2284219854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066ea13d00 0x56066eb1c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f696cd4d700 1 -- 172.21.15.46:0/2284219854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066eb16400 msgr2=0x56066eb16810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f696cd4d700 1 --2- 172.21.15.46:0/2284219854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066eb16400 0x56066eb16810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f696cd4d700 1 -- 172.21.15.46:0/2284219854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56066e9cf500 con 0x56066e9994a0 2024-05-09T00:13:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f696dd4f700 1 --2- 172.21.15.46:0/2284219854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066eb16400 0x56066eb16810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f696cd4d700 1 --2- 172.21.15.46:0/2284219854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066e9994a0 0x56066eb1c780 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f695800fd10 tx=0x7f6958012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2128bf7cefc4b4cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f69537fe700 1 -- 172.21.15.46:0/2284219854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958022070 con 0x56066e9994a0 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f69537fe700 1 -- 172.21.15.46:0/2284219854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6958011b80 con 0x56066e9994a0 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.892+0000 7f69537fe700 1 -- 172.21.15.46:0/2284219854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695801d5f0 con 0x56066e9994a0 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.893+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2284219854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066e9994a0 msgr2=0x56066eb1c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.893+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2284219854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066e9994a0 0x56066eb1c780 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f695800fd10 tx=0x7f6958012000 comp rx=0 tx=0).stop 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.893+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2284219854 shutdown_connections 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.893+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2284219854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066eb16400 0x56066eb16810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.893+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2284219854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066ea13d00 0x56066eb1c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.893+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2284219854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066e9994a0 0x56066eb1c780 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.893+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2284219854 >> 172.21.15.46:0/2284219854 conn(0x56066e99cf70 msgr2=0x56066eb11220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.893+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2284219854 shutdown_connections 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.893+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2284219854 wait complete. 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.894+0000 7f697e1c8b80 1 -- start start 2024-05-09T00:13:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.894+0000 7f697e1c8b80 1 Processor -- start 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.894+0000 7f697e1c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066e9994a0 0x56066eb1feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.894+0000 7f697e1c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 0x56066eb203f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.894+0000 7f697e1c8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066eb16400 0x56066eb20930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.894+0000 7f697e1c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56066eb17160 con 0x56066ea13d00 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.894+0000 7f697e1c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56066e9d3b20 con 0x56066e9994a0 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.894+0000 7f697e1c8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56066eb23020 con 0x56066eb16400 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f697e1c8b80 10 client.0 _ll_register_callbacks cb 0x56066e8aa6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f696cd4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 0x56066eb203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f696cd4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 0x56066eb203f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44398/0 (socket says 172.21.15.46:44398) 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f696cd4d700 1 -- 172.21.15.46:0/2603458488 learned_addr learned my addr 172.21.15.46:0/2603458488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f696d54e700 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066e9994a0 0x56066eb1feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f696dd4f700 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066eb16400 0x56066eb20930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f696cd4d700 1 -- 172.21.15.46:0/2603458488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066eb16400 msgr2=0x56066eb20930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f696cd4d700 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066eb16400 0x56066eb20930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f696cd4d700 1 -- 172.21.15.46:0/2603458488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066e9994a0 msgr2=0x56066eb1feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f696cd4d700 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066e9994a0 0x56066eb1feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f696cd4d700 1 -- 172.21.15.46:0/2603458488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56066e8b4490 con 0x56066ea13d00 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f696cd4d700 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 0x56066eb203f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f695801ecc0 tx=0x7f695801ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191a2984a4cf6ad9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f6950ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f6950ff9700 1 -- 172.21.15.46:0/2603458488 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958035060 con 0x56066ea13d00 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f6950ff9700 1 -- 172.21.15.46:0/2603458488 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f695801e7a0 con 0x56066ea13d00 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2603458488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6958011b80 con 0x56066ea13d00 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.895+0000 7f6950ff9700 1 -- 172.21.15.46:0/2603458488 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6958022070 con 0x56066ea13d00 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.896+0000 7f697e1c8b80 10 client.15462 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.896+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2603458488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56066eb2d8c0 con 0x56066ea13d00 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.896+0000 7f6950ff9700 1 -- 172.21.15.46:0/2603458488 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6958011b80 con 0x56066ea13d00 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.896+0000 7f6950ff9700 1 -- 172.21.15.46:0/2603458488 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f695803a070 con 0x56066ea13d00 2024-05-09T00:13:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.896+0000 7f697e1c8b80 10 client.15462 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.896+0000 7f697e1c8b80 10 client.15462 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.896+0000 7f697e1c8b80 20 client.15462 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.896+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56066eb2fe40 0x56066eb32260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.896+0000 7f697e1c8b80 4 client.15462 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.896+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2603458488 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56066ea46a80 con 0x56066eb2fe40 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.896+0000 7f696d54e700 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56066eb2fe40 0x56066eb32260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.897+0000 7f6950ff9700 10 client.15462 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.897+0000 7f696d54e700 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56066eb2fe40 0x56066eb32260 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55eb66be4ba08d4c server_cookie=27e9f8104396280 in_seq=0 out_seq=0 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f6950ff9700 1 -- 172.21.15.46:0/2603458488 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56066ea46a80 con 0x56066eb2fe40 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f6950ff9700 10 client.15462 handle_command_reply: tid=0 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f697e1c8b80 1 client.15462 shutdown 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2603458488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 msgr2=0x56066eb203f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 0x56066eb203f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f695801ecc0 tx=0x7f695801ea90 comp rx=0 tx=0).stop 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2603458488 shutdown_connections 2024-05-09T00:13:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56066eb2fe40 0x56066eb32260 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066eb16400 0x56066eb20930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066e9994a0 0x56066eb1feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f6950ff9700 0 client.15462 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2603458488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 0x56066eb203f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2603458488 >> 172.21.15.46:0/2603458488 conn(0x56066e99cf70 msgr2=0x56066eb0aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2603458488 shutdown_connections 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2603458488 wait complete. 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.898+0000 7f697e1c8b80 20 client.15462 trim_cache size 0 max 16384 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.914+0000 7f697e1c8b80 1 Processor -- start 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.914+0000 7f697e1c8b80 1 -- start start 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.914+0000 7f697e1c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066e9994a0 0x56066eb2c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.914+0000 7f697e1c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 0x56066eb1b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.914+0000 7f697e1c8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066eb16400 0x56066eb35280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.914+0000 7f697e1c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69681b7b60 con 0x56066ea13d00 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.914+0000 7f697e1c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f69681b53f0 con 0x56066e9994a0 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.914+0000 7f697e1c8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f69681b3220 con 0x56066eb16400 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f696cd4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 0x56066eb1b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f696cd4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 0x56066eb1b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44408/0 (socket says 172.21.15.46:44408) 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f696cd4d700 1 -- 172.21.15.46:0/2818688712 learned_addr learned my addr 172.21.15.46:0/2818688712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f696dd4f700 1 --2- 172.21.15.46:0/2818688712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066eb16400 0x56066eb35280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f696d54e700 1 --2- 172.21.15.46:0/2818688712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066e9994a0 0x56066eb2c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f696cd4d700 1 -- 172.21.15.46:0/2818688712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066eb16400 msgr2=0x56066eb35280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f696cd4d700 1 --2- 172.21.15.46:0/2818688712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066eb16400 0x56066eb35280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f696cd4d700 1 -- 172.21.15.46:0/2818688712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066e9994a0 msgr2=0x56066eb2c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f696cd4d700 1 --2- 172.21.15.46:0/2818688712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066e9994a0 0x56066eb2c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f696cd4d700 1 -- 172.21.15.46:0/2818688712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56066eb1caf0 con 0x56066ea13d00 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f696cd4d700 1 --2- 172.21.15.46:0/2818688712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 0x56066eb1b720 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f695800b960 tx=0x7f695800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6c307b54190b42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f6952ffd700 1 -- 172.21.15.46:0/2818688712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958040030 con 0x56066ea13d00 2024-05-09T00:13:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f6952ffd700 1 -- 172.21.15.46:0/2818688712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6958004bc0 con 0x56066ea13d00 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.915+0000 7f6952ffd700 1 -- 172.21.15.46:0/2818688712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958015070 con 0x56066ea13d00 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.916+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2818688712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 msgr2=0x56066eb1b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.916+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2818688712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 0x56066eb1b720 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f695800b960 tx=0x7f695800b8f0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.916+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2818688712 shutdown_connections 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.916+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2818688712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066eb16400 0x56066eb35280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.916+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2818688712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066e9994a0 0x56066eb2c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.916+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/2818688712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066ea13d00 0x56066eb1b720 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.916+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2818688712 >> 172.21.15.46:0/2818688712 conn(0x56066e99cf70 msgr2=0x56066eb1b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.916+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2818688712 shutdown_connections 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.916+0000 7f697e1c8b80 1 -- 172.21.15.46:0/2818688712 wait complete. 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.917+0000 7f697e1c8b80 1 -- start start 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.917+0000 7f697e1c8b80 1 Processor -- start 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.917+0000 7f697e1c8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066e9994a0 0x56066eb2b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.917+0000 7f697e1c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066ea13d00 0x56066eb2b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.917+0000 7f697e1c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066eb16400 0x56066eb98380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.917+0000 7f697e1c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69681b38b0 con 0x56066eb16400 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.917+0000 7f697e1c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f69681b4d20 con 0x56066ea13d00 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.917+0000 7f697e1c8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f69681b3f80 con 0x56066e9994a0 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.917+0000 7f697e1c8b80 10 client.0 _ll_register_callbacks cb 0x56066e8aa6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.917+0000 7f696dd4f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066eb16400 0x56066eb98380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.917+0000 7f696dd4f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066eb16400 0x56066eb98380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44410/0 (socket says 172.21.15.46:44410) 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.917+0000 7f696dd4f700 1 -- 172.21.15.46:0/4052414554 learned_addr learned my addr 172.21.15.46:0/4052414554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f696cd4d700 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066ea13d00 0x56066eb2b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f696d54e700 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066e9994a0 0x56066eb2b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f696dd4f700 1 -- 172.21.15.46:0/4052414554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066e9994a0 msgr2=0x56066eb2b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f696dd4f700 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066e9994a0 0x56066eb2b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f696dd4f700 1 -- 172.21.15.46:0/4052414554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066ea13d00 msgr2=0x56066eb2b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f696dd4f700 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066ea13d00 0x56066eb2b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f696dd4f700 1 -- 172.21.15.46:0/4052414554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56066eb988c0 con 0x56066eb16400 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f696dd4f700 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066eb16400 0x56066eb98380 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6960007b20 tx=0x7f6960007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8f98e1c6c1ad8e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f6950ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f6950ff9700 1 -- 172.21.15.46:0/4052414554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696001b030 con 0x56066eb16400 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f697e1c8b80 1 -- 172.21.15.46:0/4052414554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56066ea40b20 con 0x56066eb16400 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f6950ff9700 1 -- 172.21.15.46:0/4052414554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6960012040 con 0x56066eb16400 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f6950ff9700 1 -- 172.21.15.46:0/4052414554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f69600149c0 con 0x56066eb16400 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f697e1c8b80 10 client.15468 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f697e1c8b80 1 -- 172.21.15.46:0/4052414554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56066eb999e0 con 0x56066eb16400 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.918+0000 7f6950ff9700 1 -- 172.21.15.46:0/4052414554 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56066ea40b20 con 0x56066eb16400 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.919+0000 7f6950ff9700 1 -- 172.21.15.46:0/4052414554 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6960017c80 con 0x56066eb16400 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.919+0000 7f697e1c8b80 10 client.15468 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.919+0000 7f697e1c8b80 10 client.15468 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.919+0000 7f697e1c8b80 20 client.15468 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.919+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56066eb2fe40 0x56066eb9c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.919+0000 7f697e1c8b80 4 client.15468 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.919+0000 7f697e1c8b80 1 -- 172.21.15.46:0/4052414554 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56066e8b4970 con 0x56066eb2fe40 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.919+0000 7f696d54e700 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56066eb2fe40 0x56066eb9c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f696d54e700 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56066eb2fe40 0x56066eb9c150 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ebb4bbd39e84ba4 server_cookie=fdf5f57145b5ddf2 in_seq=0 out_seq=0 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f6950ff9700 10 client.15468 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f6950ff9700 1 -- 172.21.15.46:0/4052414554 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56066e8b4970 con 0x56066eb2fe40 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f6950ff9700 10 client.15468 handle_command_reply: tid=0 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f697e1c8b80 1 client.15468 shutdown 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f697e1c8b80 1 -- 172.21.15.46:0/4052414554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066eb16400 msgr2=0x56066eb98380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066eb16400 0x56066eb98380 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6960007b20 tx=0x7f6960007e30 comp rx=0 tx=0).stop 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f697e1c8b80 1 -- 172.21.15.46:0/4052414554 shutdown_connections 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56066eb2fe40 0x56066eb9c150 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56066eb16400 0x56066eb98380 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56066e9994a0 0x56066eb2b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f697e1c8b80 1 --2- 172.21.15.46:0/4052414554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56066ea13d00 0x56066eb2b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f697e1c8b80 1 -- 172.21.15.46:0/4052414554 >> 172.21.15.46:0/4052414554 conn(0x56066e99cf70 msgr2=0x56066eb1b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f6950ff9700 0 client.15468 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:13:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f697e1c8b80 1 -- 172.21.15.46:0/4052414554 shutdown_connections 2024-05-09T00:13:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f697e1c8b80 1 -- 172.21.15.46:0/4052414554 wait complete. 2024-05-09T00:13:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.920+0000 7f697e1c8b80 20 client.15468 trim_cache size 0 max 16384 2024-05-09T00:13:32.994 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:13:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.921+0000 7f696f7b2700 1 -- 172.21.15.46:0/1791353327 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f69540b2880 msgr2=0x7f69540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.921+0000 7f696f7b2700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f69540b2880 0x7f69540b4d90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6958005950 tx=0x7f69580058e0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.921+0000 7f696f7b2700 1 -- 172.21.15.46:0/1791353327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968103290 msgr2=0x7f69681a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.921+0000 7f696f7b2700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968103290 0x7f69681a1470 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6964000c00 tx=0x7f69640047b0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.921+0000 7f696f7b2700 1 -- 172.21.15.46:0/1791353327 shutdown_connections 2024-05-09T00:13:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.921+0000 7f696f7b2700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f69540b2880 0x7f69540b4d90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.921+0000 7f696f7b2700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69681086e0 0x7f69681a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.921+0000 7f696f7b2700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968103290 0x7f69681a1470 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.921+0000 7f696f7b2700 1 --2- 172.21.15.46:0/1791353327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6968103be0 0x7f69681a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.921+0000 7f696f7b2700 1 -- 172.21.15.46:0/1791353327 >> 172.21.15.46:0/1791353327 conn(0x7f6968078e50 msgr2=0x7f69681060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.921+0000 7f696f7b2700 1 -- 172.21.15.46:0/1791353327 shutdown_connections 2024-05-09T00:13:32.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:32.921+0000 7f696f7b2700 1 -- 172.21.15.46:0/1791353327 wait complete. 2024-05-09T00:13:33.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:32 smithi169 ceph-mon[34777]: pgmap v441: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:33.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:13:34.132 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:34.267 INFO:tasks.workunit.client.0.smithi046.stderr:Updating files: 87% (10757/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-09T00:13:34.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:34 smithi046 ceph-mon[30151]: pgmap v442: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:34.855 INFO:tasks.workunit.client.0.smithi046.stderr:Note: switching to 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'. 2024-05-09T00:13:34.855 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T00:13:34.855 INFO:tasks.workunit.client.0.smithi046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T00:13:34.856 INFO:tasks.workunit.client.0.smithi046.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T00:13:34.856 INFO:tasks.workunit.client.0.smithi046.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T00:13:34.856 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T00:13:34.856 INFO:tasks.workunit.client.0.smithi046.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T00:13:34.856 INFO:tasks.workunit.client.0.smithi046.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T00:13:34.856 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T00:13:34.856 INFO:tasks.workunit.client.0.smithi046.stderr: git switch -c 2024-05-09T00:13:34.856 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T00:13:34.857 INFO:tasks.workunit.client.0.smithi046.stderr:Or undo this operation with: 2024-05-09T00:13:34.857 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T00:13:34.857 INFO:tasks.workunit.client.0.smithi046.stderr: git switch - 2024-05-09T00:13:34.858 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T00:13:34.858 INFO:tasks.workunit.client.0.smithi046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T00:13:34.858 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T00:13:34.859 INFO:tasks.workunit.client.0.smithi046.stderr:HEAD is now at c9da4049f63 Merge PR #57081 into wip-pdonnell-testing-20240508.183908-debug 2024-05-09T00:13:34.862 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T00:13:34.904 INFO:tasks.workunit.client.0.smithi046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T00:13:34.907 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T00:13:34.907 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T00:13:34.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:34 smithi142 ceph-mon[34533]: pgmap v442: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:35.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:34 smithi169 ceph-mon[34777]: pgmap v442: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:13:36.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.143+0000 7f7e811de700 1 -- 172.21.15.46:0/119942243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7c10e8f0 msgr2=0x7f7e7c110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:36.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.143+0000 7f7e811de700 1 --2- 172.21.15.46:0/119942243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7c10e8f0 0x7f7e7c110dd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400b0a0 tx=0x7f7e7400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.143+0000 7f7e811de700 1 -- 172.21.15.46:0/119942243 shutdown_connections 2024-05-09T00:13:36.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.143+0000 7f7e811de700 1 --2- 172.21.15.46:0/119942243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7c10e8f0 0x7f7e7c110dd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.143+0000 7f7e811de700 1 --2- 172.21.15.46:0/119942243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c1054c0 0x7f7e7c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.143+0000 7f7e811de700 1 --2- 172.21.15.46:0/119942243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e7c105e10 0x7f7e7c10e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.143+0000 7f7e811de700 1 -- 172.21.15.46:0/119942243 >> 172.21.15.46:0/119942243 conn(0x7f7e7c0fedc0 msgr2=0x7f7e7c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:36.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.143+0000 7f7e811de700 1 -- 172.21.15.46:0/119942243 shutdown_connections 2024-05-09T00:13:36.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.143+0000 7f7e811de700 1 -- 172.21.15.46:0/119942243 wait complete. 2024-05-09T00:13:36.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.144+0000 7f7e811de700 1 Processor -- start 2024-05-09T00:13:36.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.144+0000 7f7e811de700 1 -- start start 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.144+0000 7f7e811de700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e7c1054c0 0x7f7e7c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.144+0000 7f7e811de700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7c105e10 0x7f7e7c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.144+0000 7f7e811de700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c10e8f0 0x7f7e7c19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.144+0000 7f7e811de700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c114450 con 0x7f7e7c105e10 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.144+0000 7f7e811de700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c1142d0 con 0x7f7e7c1054c0 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.144+0000 7f7e811de700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c1145d0 con 0x7f7e7c10e8f0 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.145+0000 7f7e7a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7c105e10 0x7f7e7c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.145+0000 7f7e7a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7c105e10 0x7f7e7c19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44424/0 (socket says 172.21.15.46:44424) 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.145+0000 7f7e7a59c700 1 -- 172.21.15.46:0/1311590137 learned_addr learned my addr 172.21.15.46:0/1311590137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.145+0000 7f7e7b59e700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c10e8f0 0x7f7e7c19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.145+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e7c1054c0 0x7f7e7c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.145+0000 7f7e7a59c700 1 -- 172.21.15.46:0/1311590137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c10e8f0 msgr2=0x7f7e7c19f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:36.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.145+0000 7f7e7a59c700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c10e8f0 0x7f7e7c19f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.145+0000 7f7e7a59c700 1 -- 172.21.15.46:0/1311590137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e7c1054c0 msgr2=0x7f7e7c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:36.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.145+0000 7f7e7a59c700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e7c1054c0 0x7f7e7c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.145+0000 7f7e7a59c700 1 -- 172.21.15.46:0/1311590137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7c19fc10 con 0x7f7e7c105e10 2024-05-09T00:13:36.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.145+0000 7f7e7a59c700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7c105e10 0x7f7e7c19b2e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000c920 tx=0x7f7e7000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:36.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.145+0000 7f7e63fff700 1 -- 172.21.15.46:0/1311590137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70004a30 con 0x7f7e7c105e10 2024-05-09T00:13:36.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.146+0000 7f7e811de700 1 -- 172.21.15.46:0/1311590137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7c19ff00 con 0x7f7e7c105e10 2024-05-09T00:13:36.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.146+0000 7f7e811de700 1 -- 172.21.15.46:0/1311590137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e7c1030e0 con 0x7f7e7c105e10 2024-05-09T00:13:36.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.146+0000 7f7e63fff700 1 -- 172.21.15.46:0/1311590137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e700054d0 con 0x7f7e7c105e10 2024-05-09T00:13:36.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.146+0000 7f7e63fff700 1 -- 172.21.15.46:0/1311590137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70007c70 con 0x7f7e7c105e10 2024-05-09T00:13:36.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.148+0000 7f7e63fff700 1 -- 172.21.15.46:0/1311590137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7e7000f450 con 0x7f7e7c105e10 2024-05-09T00:13:36.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.149+0000 7f7e63fff700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e640b2900 0x7f7e640b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.149+0000 7f7e8fbf4b80 1 Processor -- start 2024-05-09T00:13:36.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.149+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e640b2900 0x7f7e640b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.149+0000 7f7e8fbf4b80 1 -- start start 2024-05-09T00:13:36.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.149+0000 7f7e63fff700 1 -- 172.21.15.46:0/1311590137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7e700e1e80 con 0x7f7e7c105e10 2024-05-09T00:13:36.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e8fbf4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7632d00 0x5636b773b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e8fbf4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b75b84a0 0x5636b773b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e8fbf4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b7735400 0x5636b7735810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e8fbf4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5636b76580a0 con 0x5636b7632d00 2024-05-09T00:13:36.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e8fbf4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5636b75fd9a0 con 0x5636b75b84a0 2024-05-09T00:13:36.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e8fbf4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5636b7654060 con 0x5636b7735400 2024-05-09T00:13:36.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e7ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7632d00 0x5636b773b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e7ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7632d00 0x5636b773b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44426/0 (socket says 172.21.15.46:44426) 2024-05-09T00:13:36.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e7ad9d700 1 -- 172.21.15.46:0/716016766 learned_addr learned my addr 172.21.15.46:0/716016766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:36.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e640b2900 0x7f7e640b4e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c005950 tx=0x7f7e6c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:36.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e7b59e700 1 --2- 172.21.15.46:0/716016766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b7735400 0x5636b7735810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e7a59c700 1 --2- 172.21.15.46:0/716016766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b75b84a0 0x5636b773b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e7ad9d700 1 -- 172.21.15.46:0/716016766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b7735400 msgr2=0x5636b7735810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:36.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/716016766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b7735400 0x5636b7735810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e7ad9d700 1 -- 172.21.15.46:0/716016766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b75b84a0 msgr2=0x5636b773b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:36.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/716016766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b75b84a0 0x5636b773b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.150+0000 7f7e7ad9d700 1 -- 172.21.15.46:0/716016766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b75ee500 con 0x5636b7632d00 2024-05-09T00:13:36.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.151+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/716016766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7632d00 0x5636b773b240 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00fdf0 tx=0x7f7e6c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62edf0da5b4310f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.151+0000 7f7e617fa700 1 -- 172.21.15.46:0/716016766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c016070 con 0x5636b7632d00 2024-05-09T00:13:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.151+0000 7f7e617fa700 1 -- 172.21.15.46:0/716016766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c010d70 con 0x5636b7632d00 2024-05-09T00:13:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.151+0000 7f7e617fa700 1 -- 172.21.15.46:0/716016766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c01ce00 con 0x5636b7632d00 2024-05-09T00:13:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.152+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/716016766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7632d00 msgr2=0x5636b773b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.152+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/716016766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7632d00 0x5636b773b240 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00fdf0 tx=0x7f7e6c012000 comp rx=0 tx=0).stop 2024-05-09T00:13:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.152+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/716016766 shutdown_connections 2024-05-09T00:13:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.152+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/716016766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b7735400 0x5636b7735810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.152+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/716016766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7632d00 0x5636b773b240 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.152+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/716016766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b75b84a0 0x5636b773b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.152+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/716016766 >> 172.21.15.46:0/716016766 conn(0x5636b75bbf70 msgr2=0x5636b7730220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.152+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/716016766 shutdown_connections 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.152+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/716016766 wait complete. 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.152+0000 7f7e8fbf4b80 1 -- start start 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.152+0000 7f7e8fbf4b80 1 Processor -- start 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e8fbf4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b75b84a0 0x5636b773f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e8fbf4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b7632d00 0x5636b773f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e8fbf4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7735400 0x5636b773fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e8fbf4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5636b7736160 con 0x5636b7735400 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e8fbf4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5636b75f2b20 con 0x5636b75b84a0 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e8fbf4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5636b7742020 con 0x5636b7632d00 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e8fbf4b80 10 client.0 _ll_register_callbacks cb 0x5636b74c96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e7b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7735400 0x5636b773fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e7b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7735400 0x5636b773fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44428/0 (socket says 172.21.15.46:44428) 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e7b59e700 1 -- 172.21.15.46:0/3418796549 learned_addr learned my addr 172.21.15.46:0/3418796549 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b75b84a0 0x5636b773f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e7a59c700 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b7632d00 0x5636b773f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e7b59e700 1 -- 172.21.15.46:0/3418796549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b7632d00 msgr2=0x5636b773f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e7b59e700 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b7632d00 0x5636b773f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e7b59e700 1 -- 172.21.15.46:0/3418796549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b75b84a0 msgr2=0x5636b773f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e7b59e700 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b75b84a0 0x5636b773f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e7b59e700 1 -- 172.21.15.46:0/3418796549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b74d3490 con 0x5636b7735400 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e7b59e700 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7735400 0x5636b773fac0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7e74004cb0 tx=0x7f7e74004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1585e087bac4b0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.153+0000 7f7e5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e5eff5700 1 -- 172.21.15.46:0/3418796549 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e740079d0 con 0x5636b7735400 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e5eff5700 1 -- 172.21.15.46:0/3418796549 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7e74007b70 con 0x5636b7735400 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3418796549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e6c010d70 con 0x5636b7735400 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e5eff5700 1 -- 172.21.15.46:0/3418796549 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e74007d70 con 0x5636b7735400 2024-05-09T00:13:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e8fbf4b80 10 client.15480 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3418796549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636b774ca50 con 0x5636b7735400 2024-05-09T00:13:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e5eff5700 1 -- 172.21.15.46:0/3418796549 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c010d70 con 0x5636b7735400 2024-05-09T00:13:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e5eff5700 1 -- 172.21.15.46:0/3418796549 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7e74011070 con 0x5636b7735400 2024-05-09T00:13:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e8fbf4b80 10 client.15480 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e8fbf4b80 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e8fbf4b80 20 client.15480 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5636b774f0e0 0x5636b7751570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e8fbf4b80 4 client.15480 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.154+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3418796549 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636b7665a80 con 0x5636b774f0e0 2024-05-09T00:13:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.155+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5636b774f0e0 0x5636b7751570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.155+0000 7f7e5eff5700 10 client.15480 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.155+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5636b774f0e0 0x5636b7751570 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ede49f14c1b79e server_cookie=98edeccdf4f99b7 in_seq=0 out_seq=0 2024-05-09T00:13:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.156+0000 7f7e5eff5700 1 -- 172.21.15.46:0/3418796549 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5636b7665a80 con 0x5636b774f0e0 2024-05-09T00:13:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.156+0000 7f7e5eff5700 10 client.15480 handle_command_reply: tid=0 2024-05-09T00:13:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.156+0000 7f7e8fbf4b80 1 client.15480 shutdown 2024-05-09T00:13:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.157+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3418796549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7735400 msgr2=0x5636b773fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.157+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7735400 0x5636b773fac0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7e74004cb0 tx=0x7f7e74004ce0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.157+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3418796549 shutdown_connections 2024-05-09T00:13:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.157+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5636b774f0e0 0x5636b7751570 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.157+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b7735400 0x5636b773fac0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.157+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b75b84a0 0x5636b773f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.157+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/3418796549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b7632d00 0x5636b773f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.157+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3418796549 >> 172.21.15.46:0/3418796549 conn(0x5636b75bbf70 msgr2=0x5636b7729ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:36.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.157+0000 7f7e5eff5700 0 client.15480 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:36.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.157+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3418796549 shutdown_connections 2024-05-09T00:13:36.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.157+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3418796549 wait complete. 2024-05-09T00:13:36.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.157+0000 7f7e8fbf4b80 20 client.15480 trim_cache size 0 max 16384 2024-05-09T00:13:36.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.172+0000 7f7e8fbf4b80 1 Processor -- start 2024-05-09T00:13:36.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.172+0000 7f7e8fbf4b80 1 -- start start 2024-05-09T00:13:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.172+0000 7f7e8fbf4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75bbf70 0x5636b773a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e8fbf4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b75b84a0 0x5636b774adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e8fbf4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7632d00 0x5636b774b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e8fbf4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6800fcc0 con 0x5636b75bbf70 2024-05-09T00:13:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e8fbf4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6800d550 con 0x5636b7632d00 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e8fbf4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6800b380 con 0x5636b75b84a0 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75bbf70 0x5636b773a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75bbf70 0x5636b773a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44430/0 (socket says 172.21.15.46:44430) 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7ad9d700 1 -- 172.21.15.46:0/3039123287 learned_addr learned my addr 172.21.15.46:0/3039123287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7b59e700 1 --2- 172.21.15.46:0/3039123287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7632d00 0x5636b774b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7ad9d700 1 -- 172.21.15.46:0/3039123287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b75b84a0 msgr2=0x5636b774adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7a59c700 1 --2- 172.21.15.46:0/3039123287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b75b84a0 0x5636b774adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/3039123287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b75b84a0 0x5636b774adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7ad9d700 1 -- 172.21.15.46:0/3039123287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7632d00 msgr2=0x5636b774b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/3039123287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7632d00 0x5636b774b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7ad9d700 1 -- 172.21.15.46:0/3039123287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b77418e0 con 0x5636b75bbf70 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7b59e700 1 --2- 172.21.15.46:0/3039123287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7632d00 0x5636b774b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7a59c700 1 --2- 172.21.15.46:0/3039123287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b75b84a0 0x5636b774adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/3039123287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75bbf70 0x5636b773a840 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00fdc0 tx=0x7f7e6c005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebd052c9c4bf461 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:36.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.173+0000 7f7e60ff9700 1 -- 172.21.15.46:0/3039123287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c02a040 con 0x5636b75bbf70 2024-05-09T00:13:36.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.174+0000 7f7e60ff9700 1 -- 172.21.15.46:0/3039123287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c037070 con 0x5636b75bbf70 2024-05-09T00:13:36.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.174+0000 7f7e60ff9700 1 -- 172.21.15.46:0/3039123287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c016040 con 0x5636b75bbf70 2024-05-09T00:13:36.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.174+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3039123287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75bbf70 msgr2=0x5636b773a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:36.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.174+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/3039123287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75bbf70 0x5636b773a840 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00fdc0 tx=0x7f7e6c005df0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.174+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3039123287 shutdown_connections 2024-05-09T00:13:36.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.174+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/3039123287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7632d00 0x5636b774b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.174+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/3039123287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75bbf70 0x5636b773a840 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.174+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/3039123287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b75b84a0 0x5636b774adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.174+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3039123287 >> 172.21.15.46:0/3039123287 conn(0x5636b7730520 msgr2=0x5636b773a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:36.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.175+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3039123287 shutdown_connections 2024-05-09T00:13:36.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.175+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/3039123287 wait complete. 2024-05-09T00:13:36.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.175+0000 7f7e8fbf4b80 1 -- start start 2024-05-09T00:13:36.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.175+0000 7f7e8fbf4b80 1 Processor -- start 2024-05-09T00:13:36.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.175+0000 7f7e8fbf4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75b84a0 0x5636b77470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.175+0000 7f7e8fbf4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b75bbf70 0x5636b7747600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.175+0000 7f7e8fbf4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7632d00 0x5636b7755d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.175+0000 7f7e8fbf4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6800ba10 con 0x5636b75b84a0 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.175+0000 7f7e8fbf4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6800ce80 con 0x5636b7632d00 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.175+0000 7f7e8fbf4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6800c0e0 con 0x5636b75bbf70 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e8fbf4b80 10 client.0 _ll_register_callbacks cb 0x5636b74c96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e7ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75b84a0 0x5636b77470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e7ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75b84a0 0x5636b77470c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44446/0 (socket says 172.21.15.46:44446) 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e7ad9d700 1 -- 172.21.15.46:0/2474770667 learned_addr learned my addr 172.21.15.46:0/2474770667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e7a59c700 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b75bbf70 0x5636b7747600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e7b59e700 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7632d00 0x5636b7755d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e7ad9d700 1 -- 172.21.15.46:0/2474770667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b75bbf70 msgr2=0x5636b7747600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b75bbf70 0x5636b7747600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e7ad9d700 1 -- 172.21.15.46:0/2474770667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7632d00 msgr2=0x5636b7755d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:36.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7632d00 0x5636b7755d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e7ad9d700 1 -- 172.21.15.46:0/2474770667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b7757690 con 0x5636b75b84a0 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e7b59e700 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7632d00 0x5636b7755d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e7ad9d700 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75b84a0 0x5636b77470c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00a6b0 tx=0x7f7e6c010840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdf64aa47171d9a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e5eff5700 1 -- 172.21.15.46:0/2474770667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c03f020 con 0x5636b75b84a0 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e5eff5700 1 -- 172.21.15.46:0/2474770667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7e6c004c30 con 0x5636b75b84a0 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.176+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/2474770667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e6c037070 con 0x5636b75b84a0 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.177+0000 7f7e5eff5700 1 -- 172.21.15.46:0/2474770667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e6c016040 con 0x5636b75b84a0 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.177+0000 7f7e8fbf4b80 10 client.15486 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.177+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/2474770667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636b77b8c70 con 0x5636b75b84a0 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.177+0000 7f7e5eff5700 1 -- 172.21.15.46:0/2474770667 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c037070 con 0x5636b75b84a0 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.177+0000 7f7e5eff5700 1 -- 172.21.15.46:0/2474770667 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7e6c032310 con 0x5636b75b84a0 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.177+0000 7f7e8fbf4b80 10 client.15486 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:36.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.177+0000 7f7e8fbf4b80 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.177+0000 7f7e8fbf4b80 20 client.15486 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.177+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5636b7735400 0x5636b77bb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.177+0000 7f7e8fbf4b80 4 client.15486 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.177+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/2474770667 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636b765fb20 con 0x5636b7735400 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.177+0000 7f7e7a59c700 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5636b7735400 0x5636b77bb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e5eff5700 10 client.15486 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e7a59c700 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5636b7735400 0x5636b77bb210 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c56f85cf23d29ab server_cookie=bc3f2797bb870839 in_seq=0 out_seq=0 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e5eff5700 1 -- 172.21.15.46:0/2474770667 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636b765fb20 con 0x5636b7735400 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e5eff5700 10 client.15486 handle_command_reply: tid=0 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e8fbf4b80 1 client.15486 shutdown 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/2474770667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75b84a0 msgr2=0x5636b77470c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75b84a0 0x5636b77470c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00a6b0 tx=0x7f7e6c010840 comp rx=0 tx=0).stop 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/2474770667 shutdown_connections 2024-05-09T00:13:36.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5636b7735400 0x5636b77bb210 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7632d00 0x5636b7755d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636b75b84a0 0x5636b77470c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e5eff5700 0 client.15486 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:36.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e8fbf4b80 1 --2- 172.21.15.46:0/2474770667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636b75bbf70 0x5636b7747600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/2474770667 >> 172.21.15.46:0/2474770667 conn(0x5636b7730520 msgr2=0x5636b7729ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:36.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/2474770667 shutdown_connections 2024-05-09T00:13:36.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e8fbf4b80 1 -- 172.21.15.46:0/2474770667 wait complete. 2024-05-09T00:13:36.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.178+0000 7f7e8fbf4b80 20 client.15486 trim_cache size 0 max 16384 2024-05-09T00:13:36.182 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:13:36.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.179+0000 7f7e811de700 1 -- 172.21.15.46:0/1311590137 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e640b2900 msgr2=0x7f7e640b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:36.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.179+0000 7f7e811de700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e640b2900 0x7f7e640b4e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c005950 tx=0x7f7e6c00a300 comp rx=0 tx=0).stop 2024-05-09T00:13:36.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.179+0000 7f7e811de700 1 -- 172.21.15.46:0/1311590137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7c105e10 msgr2=0x7f7e7c19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:36.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.179+0000 7f7e811de700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7c105e10 0x7f7e7c19b2e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000c920 tx=0x7f7e7000cc30 comp rx=0 tx=0).stop 2024-05-09T00:13:36.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.179+0000 7f7e811de700 1 -- 172.21.15.46:0/1311590137 shutdown_connections 2024-05-09T00:13:36.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.179+0000 7f7e811de700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e640b2900 0x7f7e640b4e10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.179+0000 7f7e811de700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e7c10e8f0 0x7f7e7c19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.179+0000 7f7e811de700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e7c1054c0 0x7f7e7c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.179+0000 7f7e811de700 1 --2- 172.21.15.46:0/1311590137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7c105e10 0x7f7e7c19b2e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:36.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.179+0000 7f7e811de700 1 -- 172.21.15.46:0/1311590137 >> 172.21.15.46:0/1311590137 conn(0x7f7e7c0fedc0 msgr2=0x7f7e7c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:36.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.179+0000 7f7e811de700 1 -- 172.21.15.46:0/1311590137 shutdown_connections 2024-05-09T00:13:36.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:36.179+0000 7f7e811de700 1 -- 172.21.15.46:0/1311590137 wait complete. 2024-05-09T00:13:36.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:36 smithi142 ceph-mon[34533]: pgmap v443: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-05-09T00:13:36.984 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T00:13:37.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:36 smithi046 ceph-mon[30151]: pgmap v443: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-05-09T00:13:37.096 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T00:13:37.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:36 smithi169 ceph-mon[34777]: pgmap v443: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-05-09T00:13:37.139 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T00:13:37.140 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T00:13:37.141 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T00:13:37.181 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T00:13:37.185 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T00:13:37.185 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T00:13:37.243 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-05-09T00:13:37.244 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-05-09T00:13:37.244 DEBUG:teuthology.orchestra.run.smithi046:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c9da4049f638c0f84bdd9b4e33c42e86046ac554 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-05-09T00:13:37.305 INFO:tasks.workunit.client.0.smithi046.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-05-09T00:13:37.306 INFO:tasks.workunit.client.0.smithi046.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-05-09T00:13:37.306 INFO:tasks.workunit.client.0.smithi046.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-05-09T00:13:38.110 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:13:38.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:13:38.374 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:38.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:38 smithi046 ceph-mon[30151]: pgmap v444: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:13:38.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:38 smithi169 ceph-mon[34777]: pgmap v444: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:13:38.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:38 smithi142 ceph-mon[34533]: pgmap v444: 97 pgs: 97 active+clean; 657 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T00:13:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.551+0000 7fb4f18cd700 1 -- 172.21.15.46:0/661558291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4ec107df0 msgr2=0x7fb4ec108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.551+0000 7fb4f18cd700 1 --2- 172.21.15.46:0/661558291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4ec107df0 0x7fb4ec108200 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00099c0 tx=0x7fb4e0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.551+0000 7fb4f18cd700 1 -- 172.21.15.46:0/661558291 shutdown_connections 2024-05-09T00:13:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.551+0000 7fb4f18cd700 1 --2- 172.21.15.46:0/661558291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ec108740 0x7fb4ec1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.551+0000 7fb4f18cd700 1 --2- 172.21.15.46:0/661558291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4ec1034c0 0x7fb4ec1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.551+0000 7fb4f18cd700 1 --2- 172.21.15.46:0/661558291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4ec107df0 0x7fb4ec108200 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.551+0000 7fb4f18cd700 1 -- 172.21.15.46:0/661558291 >> 172.21.15.46:0/661558291 conn(0x7fb4ec0fedc0 msgr2=0x7fb4ec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.552+0000 7fb4f18cd700 1 -- 172.21.15.46:0/661558291 shutdown_connections 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.552+0000 7fb4f18cd700 1 -- 172.21.15.46:0/661558291 wait complete. 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.552+0000 7fb4f18cd700 1 Processor -- start 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.552+0000 7fb4f18cd700 1 -- start start 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4f18cd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4ec1034c0 0x7fb4ec19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4f18cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4ec107df0 0x7fb4ec19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4f18cd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ec108740 0x7fb4ec19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4f18cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec113350 con 0x7fb4ec107df0 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4f18cd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1131d0 con 0x7fb4ec108740 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4f18cd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1134d0 con 0x7fb4ec1034c0 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4ea7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4ec107df0 0x7fb4ec19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4ea7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4ec107df0 0x7fb4ec19c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39954/0 (socket says 172.21.15.46:39954) 2024-05-09T00:13:40.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/617982912 learned_addr learned my addr 172.21.15.46:0/617982912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:40.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4eb7fe700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ec108740 0x7fb4ec19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4eaffd700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4ec1034c0 0x7fb4ec19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/617982912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4ec1034c0 msgr2=0x7fb4ec19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:40.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4ec1034c0 0x7fb4ec19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/617982912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ec108740 msgr2=0x7fb4ec19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:40.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ec108740 0x7fb4ec19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.553+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/617982912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ec1a6f70 con 0x7fb4ec107df0 2024-05-09T00:13:40.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.554+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4ec107df0 0x7fb4ec19c1f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0006010 tx=0x7fb4e00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:40.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.554+0000 7fb4f08cb700 1 -- 172.21.15.46:0/617982912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0027070 con 0x7fb4ec107df0 2024-05-09T00:13:40.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.554+0000 7fb4f18cd700 1 -- 172.21.15.46:0/617982912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ec1a7200 con 0x7fb4ec107df0 2024-05-09T00:13:40.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.554+0000 7fb4f08cb700 1 -- 172.21.15.46:0/617982912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e0003f20 con 0x7fb4ec107df0 2024-05-09T00:13:40.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.554+0000 7fb4f08cb700 1 -- 172.21.15.46:0/617982912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0022c60 con 0x7fb4ec107df0 2024-05-09T00:13:40.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.554+0000 7fb4f18cd700 1 -- 172.21.15.46:0/617982912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ec1a7650 con 0x7fb4ec107df0 2024-05-09T00:13:40.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.558+0000 7fb4f08cb700 1 -- 172.21.15.46:0/617982912 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb4e002a070 con 0x7fb4ec107df0 2024-05-09T00:13:40.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb5002e3b80 1 Processor -- start 2024-05-09T00:13:40.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb5002e3b80 1 -- start start 2024-05-09T00:13:40.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb5002e3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4a02a1e0 0x562d4a0c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb5002e3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d49f40520 0x562d4a0c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb5002e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a0bd720 0x562d4a0bdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb4f08cb700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb4d40b2880 0x7fb4d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb5002e3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d49fe0070 con 0x562d49f40520 2024-05-09T00:13:40.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb5002e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562d49f7e870 con 0x562d4a0bd720 2024-05-09T00:13:40.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb5002e3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562d49fdc030 con 0x562d4a02a1e0 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb4eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d49f40520 0x562d4a0c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb4eb7fe700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb4d40b2880 0x7fb4d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb4eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d49f40520 0x562d4a0c3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39966/0 (socket says 172.21.15.46:39966) 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb4eaffd700 1 -- 172.21.15.46:0/1285774338 learned_addr learned my addr 172.21.15.46:0/1285774338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb4f08cb700 1 -- 172.21.15.46:0/617982912 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb4e0022e00 con 0x7fb4ec107df0 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb4eb7fe700 1 --2- 172.21.15.46:0/1285774338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4a02a1e0 0x562d4a0c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.559+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/1285774338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a0bd720 0x562d4a0bdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.560+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/1285774338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4a02a1e0 msgr2=0x562d4a0c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.560+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/1285774338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4a02a1e0 0x562d4a0c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.560+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/1285774338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d49f40520 msgr2=0x562d4a0c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.560+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/1285774338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d49f40520 0x562d4a0c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.560+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/1285774338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d49f763f0 con 0x562d4a0bd720 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.560+0000 7fb4eb7fe700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb4d40b2880 0x7fb4d40b4d90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec06c6f0 tx=0x7fb4e400a750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.560+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/1285774338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a0bd720 0x562d4a0bdb50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0003800 tx=0x7fb4e0022730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a698c6d255e9a6c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.560+0000 7fb4d1ffb700 1 -- 172.21.15.46:0/1285774338 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00aa190 con 0x562d4a0bd720 2024-05-09T00:13:40.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.560+0000 7fb4d1ffb700 1 -- 172.21.15.46:0/1285774338 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e00aad80 con 0x562d4a0bd720 2024-05-09T00:13:40.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.560+0000 7fb4d1ffb700 1 -- 172.21.15.46:0/1285774338 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00c6890 con 0x562d4a0bd720 2024-05-09T00:13:40.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.561+0000 7fb5002e3b80 1 -- 172.21.15.46:0/1285774338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a0bd720 msgr2=0x562d4a0bdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.561+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/1285774338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a0bd720 0x562d4a0bdb50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0003800 tx=0x7fb4e0022730 comp rx=0 tx=0).stop 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.561+0000 7fb5002e3b80 1 -- 172.21.15.46:0/1285774338 shutdown_connections 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.561+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/1285774338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a0bd720 0x562d4a0bdb50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.561+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/1285774338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4a02a1e0 0x562d4a0c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.561+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/1285774338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d49f40520 0x562d4a0c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.561+0000 7fb5002e3b80 1 -- 172.21.15.46:0/1285774338 >> 172.21.15.46:0/1285774338 conn(0x562d49fa0720 msgr2=0x562d4a0b8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.561+0000 7fb5002e3b80 1 -- 172.21.15.46:0/1285774338 shutdown_connections 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.561+0000 7fb5002e3b80 1 -- 172.21.15.46:0/1285774338 wait complete. 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb5002e3b80 1 -- start start 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb5002e3b80 1 Processor -- start 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb5002e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d49f40520 0x562d4a0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb5002e3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 0x562d4a0cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb5002e3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4a0bd720 0x562d4a0c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb5002e3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d49e5bc20 con 0x562d4a02a1e0 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb5002e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562d49f9a1d0 con 0x562d49f40520 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb5002e3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562d4a0be3e0 con 0x562d4a0bd720 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb5002e3b80 10 client.0 _ll_register_callbacks cb 0x562d49e51740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb4ea7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 0x562d4a0cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb4ea7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 0x562d4a0cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39978/0 (socket says 172.21.15.46:39978) 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/3437081345 learned_addr learned my addr 172.21.15.46:0/3437081345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.562+0000 7fb4eaffd700 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d49f40520 0x562d4a0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4eb7fe700 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4a0bd720 0x562d4a0c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/3437081345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4a0bd720 msgr2=0x562d4a0c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4a0bd720 0x562d4a0c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/3437081345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d49f40520 msgr2=0x562d4a0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d49f40520 0x562d4a0cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/3437081345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d49e5b510 con 0x562d4a02a1e0 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4eaffd700 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d49f40520 0x562d4a0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 0x562d4a0cd1f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00037d0 tx=0x7fb4e00226a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762bea95a1be112c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4caffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4caffd700 1 -- 172.21.15.46:0/3437081345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00aea30 con 0x562d4a02a1e0 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4caffd700 1 -- 172.21.15.46:0/3437081345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb4e00b2fc0 con 0x562d4a02a1e0 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3437081345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e00aad80 con 0x562d4a02a1e0 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4caffd700 1 -- 172.21.15.46:0/3437081345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb4e00b3450 con 0x562d4a02a1e0 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb5002e3b80 10 client.15498 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3437081345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d4a0d48d0 con 0x562d4a02a1e0 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.563+0000 7fb4caffd700 1 -- 172.21.15.46:0/3437081345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e00aad80 con 0x562d4a02a1e0 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.564+0000 7fb4caffd700 1 -- 172.21.15.46:0/3437081345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb4e00c6880 con 0x562d4a02a1e0 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.564+0000 7fb5002e3b80 10 client.15498 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.564+0000 7fb5002e3b80 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.564+0000 7fb5002e3b80 20 client.15498 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.564+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562d4a0d6bd0 0x562d4a0d8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.564+0000 7fb5002e3b80 4 client.15498 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.564+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3437081345 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d49feda50 con 0x562d4a0d6bd0 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.564+0000 7fb4eaffd700 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562d4a0d6bd0 0x562d4a0d8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.564+0000 7fb4eaffd700 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562d4a0d6bd0 0x562d4a0d8ff0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16b4faa82c950727 server_cookie=26894c5165be2d3d in_seq=0 out_seq=0 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.564+0000 7fb4caffd700 10 client.15498 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.565+0000 7fb4caffd700 1 -- 172.21.15.46:0/3437081345 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562d49feda50 con 0x562d4a0d6bd0 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.565+0000 7fb4caffd700 10 client.15498 handle_command_reply: tid=0 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.565+0000 7fb5002e3b80 1 client.15498 shutdown 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.565+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3437081345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 msgr2=0x562d4a0cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.565+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 0x562d4a0cd1f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00037d0 tx=0x7fb4e00226a0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.565+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3437081345 shutdown_connections 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.565+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562d4a0d6bd0 0x562d4a0d8ff0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.565+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4a0bd720 0x562d4a0c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.565+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d49f40520 0x562d4a0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.565+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/3437081345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 0x562d4a0cd1f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.565+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3437081345 >> 172.21.15.46:0/3437081345 conn(0x562d49fa0720 msgr2=0x562d4a0b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.565+0000 7fb4caffd700 0 client.15498 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.566+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3437081345 shutdown_connections 2024-05-09T00:13:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.566+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3437081345 wait complete. 2024-05-09T00:13:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.566+0000 7fb5002e3b80 20 client.15498 trim_cache size 0 max 16384 2024-05-09T00:13:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.586+0000 7fb5002e3b80 1 Processor -- start 2024-05-09T00:13:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.586+0000 7fb5002e3b80 1 -- start start 2024-05-09T00:13:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.586+0000 7fb5002e3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d49f40520 0x562d4a0c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb5002e3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 0x562d4a0c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb5002e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a0bd720 0x562d4a0de5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb5002e3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1b13d0 con 0x562d4a02a1e0 2024-05-09T00:13:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb5002e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1aec60 con 0x562d4a0bd720 2024-05-09T00:13:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb5002e3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1acab0 con 0x562d49f40520 2024-05-09T00:13:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4ea7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 0x562d4a0c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4ea7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 0x562d4a0c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39990/0 (socket says 172.21.15.46:39990) 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/3362167465 learned_addr learned my addr 172.21.15.46:0/3362167465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4eaffd700 1 --2- 172.21.15.46:0/3362167465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d49f40520 0x562d4a0c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4eb7fe700 1 --2- 172.21.15.46:0/3362167465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a0bd720 0x562d4a0de5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/3362167465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d49f40520 msgr2=0x562d4a0c2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/3362167465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d49f40520 0x562d4a0c2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/3362167465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a0bd720 msgr2=0x562d4a0de5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/3362167465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a0bd720 0x562d4a0de5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4ea7fc700 1 -- 172.21.15.46:0/3362167465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4a0b93a0 con 0x562d4a02a1e0 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4eb7fe700 1 --2- 172.21.15.46:0/3362167465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a0bd720 0x562d4a0de5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/3362167465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 0x562d4a0c6d20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00a78f0 tx=0x7fb4e00a6670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f184a614b158494c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4d17fa700 1 -- 172.21.15.46:0/3362167465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00aea30 con 0x562d4a02a1e0 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4d17fa700 1 -- 172.21.15.46:0/3362167465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e00a9a70 con 0x562d4a02a1e0 2024-05-09T00:13:40.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.587+0000 7fb4d17fa700 1 -- 172.21.15.46:0/3362167465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00afbc0 con 0x562d4a02a1e0 2024-05-09T00:13:40.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.588+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3362167465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 msgr2=0x562d4a0c6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:40.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.588+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/3362167465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 0x562d4a0c6d20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00a78f0 tx=0x7fb4e00a6670 comp rx=0 tx=0).stop 2024-05-09T00:13:40.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.588+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3362167465 shutdown_connections 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.588+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/3362167465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a0bd720 0x562d4a0de5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.588+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/3362167465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d49f40520 0x562d4a0c2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.588+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/3362167465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a02a1e0 0x562d4a0c6d20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.588+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3362167465 >> 172.21.15.46:0/3362167465 conn(0x562d49fa0720 msgr2=0x562d4a0c6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.588+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3362167465 shutdown_connections 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.588+0000 7fb5002e3b80 1 -- 172.21.15.46:0/3362167465 wait complete. 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb5002e3b80 1 -- start start 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb5002e3b80 1 Processor -- start 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb5002e3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d49f40520 0x562d4a0d1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb5002e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a02a1e0 0x562d4a0d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb5002e3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a0bd720 0x562d4a0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb5002e3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1ad120 con 0x562d4a0bd720 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb5002e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1ae590 con 0x562d4a02a1e0 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb5002e3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec1ad7f0 con 0x562d49f40520 2024-05-09T00:13:40.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb5002e3b80 10 client.0 _ll_register_callbacks cb 0x562d49e51740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb4eb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a0bd720 0x562d4a0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb4eb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a0bd720 0x562d4a0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39992/0 (socket says 172.21.15.46:39992) 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb4eb7fe700 1 -- 172.21.15.46:0/710609363 learned_addr learned my addr 172.21.15.46:0/710609363 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.589+0000 7fb4ea7fc700 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a02a1e0 0x562d4a0d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb4eaffd700 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d49f40520 0x562d4a0d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb4eb7fe700 1 -- 172.21.15.46:0/710609363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d49f40520 msgr2=0x562d4a0d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb4eb7fe700 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d49f40520 0x562d4a0d1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb4eb7fe700 1 -- 172.21.15.46:0/710609363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a02a1e0 msgr2=0x562d4a0d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb4eb7fe700 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a02a1e0 0x562d4a0d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb4eb7fe700 1 -- 172.21.15.46:0/710609363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4a0b9d80 con 0x562d4a0bd720 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb4eb7fe700 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a0bd720 0x562d4a0d2400 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4011b20 tx=0x7fb4e401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea7225702e42be5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb4caffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:40.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb4caffd700 1 -- 172.21.15.46:0/710609363 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4004a30 con 0x562d4a0bd720 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb4caffd700 1 -- 172.21.15.46:0/710609363 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb4e4004bd0 con 0x562d4a0bd720 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb5002e3b80 1 -- 172.21.15.46:0/710609363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e00a9a70 con 0x562d4a0bd720 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb4caffd700 1 -- 172.21.15.46:0/710609363 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb4e4004dd0 con 0x562d4a0bd720 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb5002e3b80 10 client.15504 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.590+0000 7fb5002e3b80 1 -- 172.21.15.46:0/710609363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d4a142140 con 0x562d4a0bd720 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.591+0000 7fb4caffd700 1 -- 172.21.15.46:0/710609363 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e00a9a70 con 0x562d4a0bd720 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.591+0000 7fb4caffd700 1 -- 172.21.15.46:0/710609363 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb4e401f070 con 0x562d4a0bd720 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.591+0000 7fb5002e3b80 10 client.15504 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.591+0000 7fb5002e3b80 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.591+0000 7fb5002e3b80 20 client.15504 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.591+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562d4a142880 0x562d4a144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.591+0000 7fb5002e3b80 4 client.15504 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.591+0000 7fb5002e3b80 1 -- 172.21.15.46:0/710609363 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562d49fe7af0 con 0x562d4a142880 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.591+0000 7fb4eaffd700 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562d4a142880 0x562d4a144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.592+0000 7fb4eaffd700 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562d4a142880 0x562d4a144ca0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11880fd1316cd90c server_cookie=55650f11e5edef65 in_seq=0 out_seq=0 2024-05-09T00:13:40.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.592+0000 7fb4caffd700 10 client.15504 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:40.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb4caffd700 1 -- 172.21.15.46:0/710609363 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562d49fe7af0 con 0x562d4a142880 2024-05-09T00:13:40.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb4caffd700 10 client.15504 handle_command_reply: tid=0 2024-05-09T00:13:40.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb5002e3b80 1 client.15504 shutdown 2024-05-09T00:13:40.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb5002e3b80 1 -- 172.21.15.46:0/710609363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a0bd720 msgr2=0x562d4a0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:40.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a0bd720 0x562d4a0d2400 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4011b20 tx=0x7fb4e401b000 comp rx=0 tx=0).stop 2024-05-09T00:13:40.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb5002e3b80 1 -- 172.21.15.46:0/710609363 shutdown_connections 2024-05-09T00:13:40.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562d4a142880 0x562d4a144ca0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4a0bd720 0x562d4a0d2400 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d49f40520 0x562d4a0d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb5002e3b80 1 --2- 172.21.15.46:0/710609363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d4a02a1e0 0x562d4a0d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb5002e3b80 1 -- 172.21.15.46:0/710609363 >> 172.21.15.46:0/710609363 conn(0x562d49fa0720 msgr2=0x562d4a0c1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb4caffd700 0 client.15504 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb5002e3b80 1 -- 172.21.15.46:0/710609363 shutdown_connections 2024-05-09T00:13:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb5002e3b80 1 -- 172.21.15.46:0/710609363 wait complete. 2024-05-09T00:13:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.593+0000 7fb5002e3b80 20 client.15504 trim_cache size 0 max 16384 2024-05-09T00:13:40.596 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:13:40.596 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:13:40.596 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "2cf711d0-76e4-4f82-9dbd-8af967227ea3", 2024-05-09T00:13:40.597 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:13:40.597 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:13:40.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.594+0000 7fb4f18cd700 1 -- 172.21.15.46:0/617982912 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb4d40b2880 msgr2=0x7fb4d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:40.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.594+0000 7fb4f18cd700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb4d40b2880 0x7fb4d40b4d90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec06c6f0 tx=0x7fb4e400a750 comp rx=0 tx=0).stop 2024-05-09T00:13:40.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.594+0000 7fb4f18cd700 1 -- 172.21.15.46:0/617982912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4ec107df0 msgr2=0x7fb4ec19c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.594+0000 7fb4f18cd700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4ec107df0 0x7fb4ec19c1f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0006010 tx=0x7fb4e00047b0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.594+0000 7fb4f18cd700 1 -- 172.21.15.46:0/617982912 shutdown_connections 2024-05-09T00:13:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.594+0000 7fb4f18cd700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb4d40b2880 0x7fb4d40b4d90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.594+0000 7fb4f18cd700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4ec108740 0x7fb4ec19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.594+0000 7fb4f18cd700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4ec1034c0 0x7fb4ec19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.594+0000 7fb4f18cd700 1 --2- 172.21.15.46:0/617982912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4ec107df0 0x7fb4ec19c1f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.594+0000 7fb4f18cd700 1 -- 172.21.15.46:0/617982912 >> 172.21.15.46:0/617982912 conn(0x7fb4ec0fedc0 msgr2=0x7fb4ec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.594+0000 7fb4f18cd700 1 -- 172.21.15.46:0/617982912 shutdown_connections 2024-05-09T00:13:40.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:40.594+0000 7fb4f18cd700 1 -- 172.21.15.46:0/617982912 wait complete. 2024-05-09T00:13:40.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:40 smithi046 ceph-mon[30151]: pgmap v445: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-09T00:13:40.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:40 smithi142 ceph-mon[34533]: pgmap v445: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-09T00:13:41.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:40 smithi169 ceph-mon[34777]: pgmap v445: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-09T00:13:41.414 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:13:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:41.721 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:41 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:13:41.721 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:41 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:13:41.721 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:41 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:13:41.721 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:41 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:13:41.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:41 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:13:41.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:41 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:13:41.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:41 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:13:41.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:41 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:13:42.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:41 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:13:42.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:41 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:13:42.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:41 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:13:42.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:41 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:13:42.919 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:42 smithi046 ceph-mon[30151]: pgmap v446: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-09T00:13:42.919 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:42 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:13:42.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:42 smithi142 ceph-mon[34533]: pgmap v446: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-09T00:13:42.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:42 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:13:43.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:42 smithi169 ceph-mon[34777]: pgmap v446: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-05-09T00:13:43.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:42 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:13:43.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.518+0000 7f9cc8184700 1 -- 172.21.15.46:0/3389321325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc0105e90 msgr2=0x7f9cc010e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:43.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.518+0000 7f9cc8184700 1 --2- 172.21.15.46:0/3389321325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc0105e90 0x7f9cc010e3e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009990 tx=0x7f9cb0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.518+0000 7f9cc8184700 1 -- 172.21.15.46:0/3389321325 shutdown_connections 2024-05-09T00:13:43.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.518+0000 7f9cc8184700 1 --2- 172.21.15.46:0/3389321325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cc010e920 0x7f9cc0110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.518+0000 7f9cc8184700 1 --2- 172.21.15.46:0/3389321325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc0105540 0x7f9cc0105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.518+0000 7f9cc8184700 1 --2- 172.21.15.46:0/3389321325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc0105e90 0x7f9cc010e3e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.518+0000 7f9cc8184700 1 -- 172.21.15.46:0/3389321325 >> 172.21.15.46:0/3389321325 conn(0x7f9cc0078e50 msgr2=0x7f9cc0079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:43.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.518+0000 7f9cc8184700 1 -- 172.21.15.46:0/3389321325 shutdown_connections 2024-05-09T00:13:43.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.518+0000 7f9cc8184700 1 -- 172.21.15.46:0/3389321325 wait complete. 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.519+0000 7f9cc8184700 1 Processor -- start 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.519+0000 7f9cc8184700 1 -- start start 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc8184700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc0105540 0x7f9cc019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc8184700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cc0105e90 0x7f9cc019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc8184700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc010e920 0x7f9cc019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc8184700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0114490 con 0x7f9cc010e920 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc8184700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0114310 con 0x7f9cc0105540 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc8184700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0114610 con 0x7f9cc0105e90 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc6721700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc010e920 0x7f9cc019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc6721700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc010e920 0x7f9cc019f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40006/0 (socket says 172.21.15.46:40006) 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc6721700 1 -- 172.21.15.46:0/1523503871 learned_addr learned my addr 172.21.15.46:0/1523503871 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc5f20700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc0105540 0x7f9cc019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc571f700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cc0105e90 0x7f9cc019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc6721700 1 -- 172.21.15.46:0/1523503871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cc0105e90 msgr2=0x7f9cc019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:43.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc6721700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cc0105e90 0x7f9cc019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc6721700 1 -- 172.21.15.46:0/1523503871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc0105540 msgr2=0x7f9cc019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:43.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc6721700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc0105540 0x7f9cc019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.520+0000 7f9cc6721700 1 -- 172.21.15.46:0/1523503871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc019fcf0 con 0x7f9cc010e920 2024-05-09T00:13:43.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.521+0000 7f9cc6721700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc010e920 0x7f9cc019f650 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800ef30 tx=0x7f9cb800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:43.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.521+0000 7f9cb6ffd700 1 -- 172.21.15.46:0/1523503871 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80092e0 con 0x7f9cc010e920 2024-05-09T00:13:43.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.521+0000 7f9cc8184700 1 -- 172.21.15.46:0/1523503871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc019ff80 con 0x7f9cc010e920 2024-05-09T00:13:43.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.521+0000 7f9cb6ffd700 1 -- 172.21.15.46:0/1523503871 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb8004bb0 con 0x7f9cc010e920 2024-05-09T00:13:43.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.521+0000 7f9cb6ffd700 1 -- 172.21.15.46:0/1523503871 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8004020 con 0x7f9cc010e920 2024-05-09T00:13:43.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.521+0000 7f9cc8184700 1 -- 172.21.15.46:0/1523503871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc01a0490 con 0x7f9cc010e920 2024-05-09T00:13:43.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.525+0000 7f9cb6ffd700 1 -- 172.21.15.46:0/1523503871 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9cb801b030 con 0x7f9cc010e920 2024-05-09T00:13:43.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.525+0000 7f9cb6ffd700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9cac0b2880 0x7f9cac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.526+0000 7f9cc5f20700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9cac0b2880 0x7f9cac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.526+0000 7f9cb6ffd700 1 -- 172.21.15.46:0/1523503871 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9cb80e1c30 con 0x7f9cc010e920 2024-05-09T00:13:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.526+0000 7f9cd6b9ab80 1 Processor -- start 2024-05-09T00:13:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.526+0000 7f9cd6b9ab80 1 -- start start 2024-05-09T00:13:43.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.526+0000 7f9cc5f20700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9cac0b2880 0x7f9cac0b4d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9cc006d130 tx=0x7f9cbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:43.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.526+0000 7f9cd6b9ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56485097dd00 0x564850a84ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.526+0000 7f9cd6b9ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648509034a0 0x564850a85030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.526+0000 7f9cd6b9ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564850a891e0 0x564850a895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.526+0000 7f9cd6b9ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5648509a30a0 con 0x5648509034a0 2024-05-09T00:13:43.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.526+0000 7f9cd6b9ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5648509489a0 con 0x564850a891e0 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.526+0000 7f9cd6b9ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56485099f060 con 0x56485097dd00 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.526+0000 7f9cc571f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648509034a0 0x564850a85030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cc571f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648509034a0 0x564850a85030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40008/0 (socket says 172.21.15.46:40008) 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cc571f700 1 -- 172.21.15.46:0/1526596698 learned_addr learned my addr 172.21.15.46:0/1526596698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cc6721700 1 --2- 172.21.15.46:0/1526596698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564850a891e0 0x564850a895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cc5f20700 1 --2- 172.21.15.46:0/1526596698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56485097dd00 0x564850a84ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cc571f700 1 -- 172.21.15.46:0/1526596698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56485097dd00 msgr2=0x564850a84ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cc571f700 1 --2- 172.21.15.46:0/1526596698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56485097dd00 0x564850a84ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cc571f700 1 -- 172.21.15.46:0/1526596698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564850a891e0 msgr2=0x564850a895f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cc571f700 1 --2- 172.21.15.46:0/1526596698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564850a891e0 0x564850a895f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cc571f700 1 -- 172.21.15.46:0/1526596698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564850939500 con 0x5648509034a0 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cc571f700 1 --2- 172.21.15.46:0/1526596698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648509034a0 0x564850a85030 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9cc006acf0 tx=0x7f9cb00096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fc13924ce2aea3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:43.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cabfff700 1 -- 172.21.15.46:0/1526596698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0027070 con 0x5648509034a0 2024-05-09T00:13:43.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cabfff700 1 -- 172.21.15.46:0/1526596698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb00056b0 con 0x5648509034a0 2024-05-09T00:13:43.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.527+0000 7f9cabfff700 1 -- 172.21.15.46:0/1526596698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0003bb0 con 0x5648509034a0 2024-05-09T00:13:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.528+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/1526596698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648509034a0 msgr2=0x564850a85030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.528+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/1526596698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648509034a0 0x564850a85030 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9cc006acf0 tx=0x7f9cb00096a0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.528+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/1526596698 shutdown_connections 2024-05-09T00:13:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.528+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/1526596698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564850a891e0 0x564850a895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.528+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/1526596698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56485097dd00 0x564850a84ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.528+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/1526596698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5648509034a0 0x564850a85030 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.528+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/1526596698 >> 172.21.15.46:0/1526596698 conn(0x564850906f70 msgr2=0x564850a7b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.528+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/1526596698 shutdown_connections 2024-05-09T00:13:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.528+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/1526596698 wait complete. 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.529+0000 7f9cd6b9ab80 1 -- start start 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.529+0000 7f9cd6b9ab80 1 Processor -- start 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cd6b9ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5648509034a0 0x564850a8e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cd6b9ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56485097dd00 0x564850a8e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cd6b9ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564850a891e0 0x564850a8ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cd6b9ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564850a89ea0 con 0x564850a891e0 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cd6b9ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56485093db20 con 0x56485097dd00 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cd6b9ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564850a8a020 con 0x5648509034a0 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cc6721700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564850a891e0 0x564850a8ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cd6b9ab80 10 client.0 _ll_register_callbacks cb 0x5648508146c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cc6721700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564850a891e0 0x564850a8ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40024/0 (socket says 172.21.15.46:40024) 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cc6721700 1 -- 172.21.15.46:0/4246129812 learned_addr learned my addr 172.21.15.46:0/4246129812 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cc571f700 1 --2- 172.21.15.46:0/4246129812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56485097dd00 0x564850a8e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cc6721700 1 -- 172.21.15.46:0/4246129812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5648509034a0 msgr2=0x564850a8e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cc6721700 1 --2- 172.21.15.46:0/4246129812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5648509034a0 0x564850a8e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cc6721700 1 -- 172.21.15.46:0/4246129812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56485097dd00 msgr2=0x564850a8e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cc6721700 1 --2- 172.21.15.46:0/4246129812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56485097dd00 0x564850a8e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cc6721700 1 -- 172.21.15.46:0/4246129812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56485081e490 con 0x564850a891e0 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9cc6721700 1 --2- 172.21.15.46:0/4246129812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564850a891e0 0x564850a8ee40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80a6780 tx=0x7f9cb8091330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4d11254a0ff1a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9ca97fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.530+0000 7f9ca97fa700 1 -- 172.21.15.46:0/4246129812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80afe60 con 0x564850a891e0 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9ca97fa700 1 -- 172.21.15.46:0/4246129812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9cb808fd70 con 0x564850a891e0 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/4246129812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb00056b0 con 0x564850a891e0 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9ca97fa700 1 -- 172.21.15.46:0/4246129812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9cb80a8770 con 0x564850a891e0 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9cd6b9ab80 10 client.15516 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/4246129812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564850a9bd70 con 0x564850a891e0 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9ca97fa700 1 -- 172.21.15.46:0/4246129812 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb00056b0 con 0x564850a891e0 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9ca97fa700 1 -- 172.21.15.46:0/4246129812 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9cb8003a00 con 0x564850a891e0 2024-05-09T00:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9cd6b9ab80 10 client.15516 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:43.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9cd6b9ab80 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:43.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9cd6b9ab80 20 client.15516 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:43.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/4246129812 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564850a9e300 0x564850aa0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9cd6b9ab80 4 client.15516 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:43.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.531+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/4246129812 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648509b0a80 con 0x564850a9e300 2024-05-09T00:13:43.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.532+0000 7f9cc5f20700 1 --2- 172.21.15.46:0/4246129812 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564850a9e300 0x564850aa0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.533+0000 7f9cc5f20700 1 --2- 172.21.15.46:0/4246129812 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564850a9e300 0x564850aa0780 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ecbd25093afbe5 server_cookie=d09d8d08ecb9c614 in_seq=0 out_seq=0 2024-05-09T00:13:43.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.533+0000 7f9ca97fa700 10 client.15516 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.533+0000 7f9ca97fa700 1 -- 172.21.15.46:0/4246129812 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5648509b0a80 con 0x564850a9e300 2024-05-09T00:13:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.533+0000 7f9ca97fa700 10 client.15516 handle_command_reply: tid=0 2024-05-09T00:13:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.533+0000 7f9cd6b9ab80 1 client.15516 shutdown 2024-05-09T00:13:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.533+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/4246129812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564850a891e0 msgr2=0x564850a8ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.533+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/4246129812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564850a891e0 0x564850a8ee40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80a6780 tx=0x7f9cb8091330 comp rx=0 tx=0).stop 2024-05-09T00:13:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.534+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/4246129812 shutdown_connections 2024-05-09T00:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.534+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/4246129812 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564850a9e300 0x564850aa0780 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.534+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/4246129812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564850a891e0 0x564850a8ee40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.534+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/4246129812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5648509034a0 0x564850a8e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.534+0000 7f9ca97fa700 0 client.15516 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.534+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/4246129812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56485097dd00 0x564850a8e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.534+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/4246129812 >> 172.21.15.46:0/4246129812 conn(0x564850906f70 msgr2=0x564850a74ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.534+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/4246129812 shutdown_connections 2024-05-09T00:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.534+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/4246129812 wait complete. 2024-05-09T00:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.534+0000 7f9cd6b9ab80 20 client.15516 trim_cache size 0 max 16384 2024-05-09T00:13:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.554+0000 7f9cd6b9ab80 1 Processor -- start 2024-05-09T00:13:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.554+0000 7f9cd6b9ab80 1 -- start start 2024-05-09T00:13:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cd6b9ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648509034a0 0x564850a759d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cd6b9ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 0x564850a7fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cd6b9ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564850a891e0 0x564850a88720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cd6b9ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01af280 con 0x56485097dd00 2024-05-09T00:13:43.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cd6b9ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01acb10 con 0x5648509034a0 2024-05-09T00:13:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cd6b9ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01aa980 con 0x564850a891e0 2024-05-09T00:13:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cc571f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 0x564850a7fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cc571f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 0x564850a7fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40038/0 (socket says 172.21.15.46:40038) 2024-05-09T00:13:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cc571f700 1 -- 172.21.15.46:0/2770477544 learned_addr learned my addr 172.21.15.46:0/2770477544 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cc5f20700 1 --2- 172.21.15.46:0/2770477544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648509034a0 0x564850a759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cc6721700 1 --2- 172.21.15.46:0/2770477544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564850a891e0 0x564850a88720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cc571f700 1 -- 172.21.15.46:0/2770477544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564850a891e0 msgr2=0x564850a88720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cc571f700 1 --2- 172.21.15.46:0/2770477544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564850a891e0 0x564850a88720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cc571f700 1 -- 172.21.15.46:0/2770477544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648509034a0 msgr2=0x564850a759d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cc571f700 1 --2- 172.21.15.46:0/2770477544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648509034a0 0x564850a759d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cc571f700 1 -- 172.21.15.46:0/2770477544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564850a7cee0 con 0x56485097dd00 2024-05-09T00:13:43.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.555+0000 7f9cc571f700 1 --2- 172.21.15.46:0/2770477544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 0x564850a7fe20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009990 tx=0x7f9cb0009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79eb47c6a3f3ebd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:43.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.556+0000 7f9cab7fe700 1 -- 172.21.15.46:0/2770477544 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0032070 con 0x56485097dd00 2024-05-09T00:13:43.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.556+0000 7f9cab7fe700 1 -- 172.21.15.46:0/2770477544 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb000b720 con 0x56485097dd00 2024-05-09T00:13:43.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.556+0000 7f9cab7fe700 1 -- 172.21.15.46:0/2770477544 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0027040 con 0x56485097dd00 2024-05-09T00:13:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.556+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/2770477544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 msgr2=0x564850a7fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.556+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/2770477544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 0x564850a7fe20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009990 tx=0x7f9cb0009710 comp rx=0 tx=0).stop 2024-05-09T00:13:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.556+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/2770477544 shutdown_connections 2024-05-09T00:13:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.556+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/2770477544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564850a891e0 0x564850a88720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.556+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/2770477544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648509034a0 0x564850a759d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.556+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/2770477544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 0x564850a7fe20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.556+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/2770477544 >> 172.21.15.46:0/2770477544 conn(0x564850906f70 msgr2=0x564850a88010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.556+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/2770477544 shutdown_connections 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.556+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/2770477544 wait complete. 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.557+0000 7f9cd6b9ab80 1 -- start start 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.557+0000 7f9cd6b9ab80 1 Processor -- start 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.557+0000 7f9cd6b9ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5648509034a0 0x564850a8b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.557+0000 7f9cd6b9ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 0x564850a8b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.557+0000 7f9cd6b9ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564850a891e0 0x564850a8be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.557+0000 7f9cd6b9ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01aafd0 con 0x56485097dd00 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.557+0000 7f9cd6b9ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01ac440 con 0x564850a891e0 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.557+0000 7f9cd6b9ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01ab6a0 con 0x5648509034a0 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.557+0000 7f9cd6b9ab80 10 client.0 _ll_register_callbacks cb 0x5648508146c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9cc571f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 0x564850a8b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9cc571f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 0x564850a8b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40054/0 (socket says 172.21.15.46:40054) 2024-05-09T00:13:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9cc571f700 1 -- 172.21.15.46:0/431559345 learned_addr learned my addr 172.21.15.46:0/431559345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9cc5f20700 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5648509034a0 0x564850a8b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9cc6721700 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564850a891e0 0x564850a8be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9cc571f700 1 -- 172.21.15.46:0/431559345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5648509034a0 msgr2=0x564850a8b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9cc571f700 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5648509034a0 0x564850a8b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9cc571f700 1 -- 172.21.15.46:0/431559345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564850a891e0 msgr2=0x564850a8be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9cc571f700 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564850a891e0 0x564850a8be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9cc571f700 1 -- 172.21.15.46:0/431559345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564850b07c20 con 0x56485097dd00 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9cc571f700 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 0x564850a8b8d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0025040 tx=0x7f9cb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642af8c5f1baea3a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9ca97fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9ca97fa700 1 -- 172.21.15.46:0/431559345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb003d060 con 0x56485097dd00 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9ca97fa700 1 -- 172.21.15.46:0/431559345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9cb000bad0 con 0x56485097dd00 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.558+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/431559345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb000b720 con 0x56485097dd00 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.559+0000 7f9ca97fa700 1 -- 172.21.15.46:0/431559345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9cb0032070 con 0x56485097dd00 2024-05-09T00:13:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.559+0000 7f9cd6b9ab80 10 client.15522 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.559+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/431559345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564850b08e50 con 0x56485097dd00 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.559+0000 7f9ca97fa700 1 -- 172.21.15.46:0/431559345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb000b720 con 0x56485097dd00 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.559+0000 7f9ca97fa700 1 -- 172.21.15.46:0/431559345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9cb0042070 con 0x56485097dd00 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.559+0000 7f9cd6b9ab80 10 client.15522 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.559+0000 7f9cd6b9ab80 10 client.15522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.559+0000 7f9cd6b9ab80 20 client.15522 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.559+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564850b09020 0x564850b0b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.559+0000 7f9cd6b9ab80 4 client.15522 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.559+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/431559345 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648509aab20 con 0x564850b09020 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.559+0000 7f9cc5f20700 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564850b09020 0x564850b0b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9cc5f20700 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564850b09020 0x564850b0b440 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f93d71fe7d349f2 server_cookie=b6a95d6bf7c60dc6 in_seq=0 out_seq=0 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9ca97fa700 10 client.15522 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9ca97fa700 1 -- 172.21.15.46:0/431559345 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5648509aab20 con 0x564850b09020 2024-05-09T00:13:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9ca97fa700 10 client.15522 handle_command_reply: tid=0 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9cd6b9ab80 1 client.15522 shutdown 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/431559345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 msgr2=0x564850a8b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 0x564850a8b8d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0025040 tx=0x7f9cb00057f0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/431559345 shutdown_connections 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564850b09020 0x564850b0b440 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564850a891e0 0x564850a8be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5648509034a0 0x564850a8b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9cd6b9ab80 1 --2- 172.21.15.46:0/431559345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485097dd00 0x564850a8b8d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9ca97fa700 0 client.15522 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/431559345 >> 172.21.15.46:0/431559345 conn(0x564850906f70 msgr2=0x564850a7fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/431559345 shutdown_connections 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.560+0000 7f9cd6b9ab80 1 -- 172.21.15.46:0/431559345 wait complete. 2024-05-09T00:13:43.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cd6b9ab80 20 client.15522 trim_cache size 0 max 16384 2024-05-09T00:13:43.563 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:13:43.563 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:13:43.563 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:13:43.563 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:13:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cc8184700 1 -- 172.21.15.46:0/1523503871 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9cac0b2880 msgr2=0x7f9cac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cc8184700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9cac0b2880 0x7f9cac0b4d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9cc006d130 tx=0x7f9cbc009260 comp rx=0 tx=0).stop 2024-05-09T00:13:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cc8184700 1 -- 172.21.15.46:0/1523503871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc010e920 msgr2=0x7f9cc019f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cc8184700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc010e920 0x7f9cc019f650 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800ef30 tx=0x7f9cb800c3a0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cc8184700 1 -- 172.21.15.46:0/1523503871 shutdown_connections 2024-05-09T00:13:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cc8184700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9cac0b2880 0x7f9cac0b4d90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cc8184700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc010e920 0x7f9cc019f650 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cc8184700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cc0105540 0x7f9cc019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cc8184700 1 --2- 172.21.15.46:0/1523503871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cc0105e90 0x7f9cc019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cc8184700 1 -- 172.21.15.46:0/1523503871 >> 172.21.15.46:0/1523503871 conn(0x7f9cc0078e50 msgr2=0x7f9cc0100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cc8184700 1 -- 172.21.15.46:0/1523503871 shutdown_connections 2024-05-09T00:13:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:43.561+0000 7f9cc8184700 1 -- 172.21.15.46:0/1523503871 wait complete. 2024-05-09T00:13:44.383 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:13:44.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:13:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:44.891 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:44 smithi046 ceph-mon[30151]: pgmap v447: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-05-09T00:13:44.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:44 smithi142 ceph-mon[34533]: pgmap v447: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-05-09T00:13:45.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:44 smithi169 ceph-mon[34777]: pgmap v447: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-05-09T00:13:45.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:45 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:13:45.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:45 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:13:46.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:45 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:13:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.403+0000 7f7aa42df700 1 -- 172.21.15.46:0/644835110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a9c107df0 msgr2=0x7f7a9c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.403+0000 7f7aa42df700 1 --2- 172.21.15.46:0/644835110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a9c107df0 0x7f7a9c108200 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7a980099c0 tx=0x7f7a98009cd0 comp rx=0 tx=0).stop 2024-05-09T00:13:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.403+0000 7f7aa42df700 1 -- 172.21.15.46:0/644835110 shutdown_connections 2024-05-09T00:13:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.403+0000 7f7aa42df700 1 --2- 172.21.15.46:0/644835110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c108740 0x7f7a9c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.403+0000 7f7aa42df700 1 --2- 172.21.15.46:0/644835110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c1034c0 0x7f7a9c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.403+0000 7f7aa42df700 1 --2- 172.21.15.46:0/644835110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a9c107df0 0x7f7a9c108200 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.403+0000 7f7aa42df700 1 -- 172.21.15.46:0/644835110 >> 172.21.15.46:0/644835110 conn(0x7f7a9c0fedf0 msgr2=0x7f7a9c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.403+0000 7f7aa42df700 1 -- 172.21.15.46:0/644835110 shutdown_connections 2024-05-09T00:13:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.403+0000 7f7aa42df700 1 -- 172.21.15.46:0/644835110 wait complete. 2024-05-09T00:13:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.404+0000 7f7aa42df700 1 Processor -- start 2024-05-09T00:13:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.404+0000 7f7aa42df700 1 -- start start 2024-05-09T00:13:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.404+0000 7f7aa42df700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a9c1034c0 0x7f7a9c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.404+0000 7f7aa42df700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c107df0 0x7f7a9c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.404+0000 7f7aa42df700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c108740 0x7f7a9c19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.404+0000 7f7aa42df700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c113350 con 0x7f7a9c1034c0 2024-05-09T00:13:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.404+0000 7f7aa42df700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c1131d0 con 0x7f7a9c108740 2024-05-09T00:13:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.404+0000 7f7aa42df700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c1134d0 con 0x7f7a9c107df0 2024-05-09T00:13:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.404+0000 7f7aa207b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a9c1034c0 0x7f7a9c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.404+0000 7f7aa207b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a9c1034c0 0x7f7a9c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34232/0 (socket says 172.21.15.46:34232) 2024-05-09T00:13:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.404+0000 7f7aa207b700 1 -- 172.21.15.46:0/2788753134 learned_addr learned my addr 172.21.15.46:0/2788753134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7aa187a700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c107df0 0x7f7a9c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7aa287c700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c108740 0x7f7a9c19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:46.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7aa207b700 1 -- 172.21.15.46:0/2788753134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c107df0 msgr2=0x7f7a9c19c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7aa207b700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c107df0 0x7f7a9c19c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7aa207b700 1 -- 172.21.15.46:0/2788753134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c108740 msgr2=0x7f7a9c19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7aa207b700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c108740 0x7f7a9c19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7aa207b700 1 -- 172.21.15.46:0/2788753134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9c1a7000 con 0x7f7a9c1034c0 2024-05-09T00:13:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7aa287c700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c108740 0x7f7a9c19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7aa207b700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a9c1034c0 0x7f7a9c19bcb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000e9a0 tx=0x7f7a9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7a8f7fe700 1 -- 172.21.15.46:0/2788753134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9000cbd0 con 0x7f7a9c1034c0 2024-05-09T00:13:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7a8f7fe700 1 -- 172.21.15.46:0/2788753134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a9000cd70 con 0x7f7a9c1034c0 2024-05-09T00:13:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7aa42df700 1 -- 172.21.15.46:0/2788753134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9c1a72f0 con 0x7f7a9c1034c0 2024-05-09T00:13:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7a8f7fe700 1 -- 172.21.15.46:0/2788753134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90018680 con 0x7f7a9c1034c0 2024-05-09T00:13:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.405+0000 7f7aa42df700 1 -- 172.21.15.46:0/2788753134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9c1a7800 con 0x7f7a9c1034c0 2024-05-09T00:13:46.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.406+0000 7f7aa42df700 1 -- 172.21.15.46:0/2788753134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a9c104970 con 0x7f7a9c1034c0 2024-05-09T00:13:46.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.408+0000 7f7a8f7fe700 1 -- 172.21.15.46:0/2788753134 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7a90010430 con 0x7f7a9c1034c0 2024-05-09T00:13:46.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.409+0000 7f7a8f7fe700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7a880b2880 0x7f7a880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:46.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.409+0000 7f7aa187a700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7a880b2880 0x7f7a880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:46.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.409+0000 7f7a8f7fe700 1 -- 172.21.15.46:0/2788753134 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7a900e2920 con 0x7f7a9c1034c0 2024-05-09T00:13:46.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.409+0000 7f7aa187a700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7a880b2880 0x7f7a880b4d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7a98000c00 tx=0x7f7a9801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.412+0000 7f7a8f7fe700 1 -- 172.21.15.46:0/2788753134 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a900aa640 con 0x7f7a9c1034c0 2024-05-09T00:13:46.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.680+0000 7f7aa42df700 1 -- 172.21.15.46:0/2788753134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a9c075fe0 con 0x7f7a9c1034c0 2024-05-09T00:13:46.679 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:46 smithi046 ceph-mon[30151]: pgmap v448: 97 pgs: 97 active+clean; 733 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-05-09T00:13:46.679 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:13:46.679 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:13:46.679 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:13:46.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.681+0000 7f7a8f7fe700 1 -- 172.21.15.46:0/2788753134 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f7a900a9f40 con 0x7f7a9c1034c0 2024-05-09T00:13:46.680 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:13:46.680 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:13:46.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.684+0000 7f7aa42df700 1 -- 172.21.15.46:0/2788753134 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7a880b2880 msgr2=0x7f7a880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:46.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.684+0000 7f7aa42df700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7a880b2880 0x7f7a880b4d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7a98000c00 tx=0x7f7a9801a040 comp rx=0 tx=0).stop 2024-05-09T00:13:46.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.684+0000 7f7aa42df700 1 -- 172.21.15.46:0/2788753134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a9c1034c0 msgr2=0x7f7a9c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:46.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.684+0000 7f7aa42df700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a9c1034c0 0x7f7a9c19bcb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000e9a0 tx=0x7f7a9000ecb0 comp rx=0 tx=0).stop 2024-05-09T00:13:46.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.684+0000 7f7aa42df700 1 -- 172.21.15.46:0/2788753134 shutdown_connections 2024-05-09T00:13:46.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.684+0000 7f7aa42df700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7a880b2880 0x7f7a880b4d90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:46.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.684+0000 7f7aa42df700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a9c108740 0x7f7a9c19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:46.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.684+0000 7f7aa42df700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a9c1034c0 0x7f7a9c19bcb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:46.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.684+0000 7f7aa42df700 1 --2- 172.21.15.46:0/2788753134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9c107df0 0x7f7a9c19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:46.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.684+0000 7f7aa42df700 1 -- 172.21.15.46:0/2788753134 >> 172.21.15.46:0/2788753134 conn(0x7f7a9c0fedf0 msgr2=0x7f7a9c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:46.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.684+0000 7f7aa42df700 1 -- 172.21.15.46:0/2788753134 shutdown_connections 2024-05-09T00:13:46.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:46.684+0000 7f7aa42df700 1 -- 172.21.15.46:0/2788753134 wait complete. 2024-05-09T00:13:46.684 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:13:46.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:46 smithi142 ceph-mon[34533]: pgmap v448: 97 pgs: 97 active+clean; 733 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-05-09T00:13:46.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:46 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:13:46.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:46 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:13:46.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:46 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:13:47.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:46 smithi169 ceph-mon[34777]: pgmap v448: 97 pgs: 97 active+clean; 733 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-05-09T00:13:47.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:46 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:13:47.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:46 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:13:47.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:46 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:13:47.360 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:13:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:47.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:47 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2788753134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:13:47.951 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:47 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2788753134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:13:48.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:47 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2788753134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:13:48.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:48 smithi046 ceph-mon[30151]: pgmap v449: 97 pgs: 97 active+clean; 733 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.8 KiB/s wr, 8 op/s 2024-05-09T00:13:48.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:48 smithi142 ceph-mon[34533]: pgmap v449: 97 pgs: 97 active+clean; 733 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.8 KiB/s wr, 8 op/s 2024-05-09T00:13:49.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:48 smithi169 ceph-mon[34777]: pgmap v449: 97 pgs: 97 active+clean; 733 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.8 KiB/s wr, 8 op/s 2024-05-09T00:13:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.358+0000 7fe230a54700 1 -- 172.21.15.46:0/3025893778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c107df0 msgr2=0x7fe22c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.358+0000 7fe230a54700 1 --2- 172.21.15.46:0/3025893778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c107df0 0x7fe22c108200 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe2140099c0 tx=0x7fe214009cd0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.358+0000 7fe230a54700 1 -- 172.21.15.46:0/3025893778 shutdown_connections 2024-05-09T00:13:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.358+0000 7fe230a54700 1 --2- 172.21.15.46:0/3025893778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108740 0x7fe22c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.358+0000 7fe230a54700 1 --2- 172.21.15.46:0/3025893778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe22c1034c0 0x7fe22c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.358+0000 7fe230a54700 1 --2- 172.21.15.46:0/3025893778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c107df0 0x7fe22c108200 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.358+0000 7fe230a54700 1 -- 172.21.15.46:0/3025893778 >> 172.21.15.46:0/3025893778 conn(0x7fe22c0fedf0 msgr2=0x7fe22c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.358+0000 7fe230a54700 1 -- 172.21.15.46:0/3025893778 shutdown_connections 2024-05-09T00:13:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.358+0000 7fe230a54700 1 -- 172.21.15.46:0/3025893778 wait complete. 2024-05-09T00:13:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.359+0000 7fe230a54700 1 Processor -- start 2024-05-09T00:13:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.359+0000 7fe230a54700 1 -- start start 2024-05-09T00:13:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe230a54700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe22c1034c0 0x7fe22c0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe230a54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c107df0 0x7fe22c0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe230a54700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108740 0x7fe22c0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe230a54700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c113350 con 0x7fe22c107df0 2024-05-09T00:13:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe230a54700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1131d0 con 0x7fe22c108740 2024-05-09T00:13:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe230a54700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1134d0 con 0x7fe22c1034c0 2024-05-09T00:13:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe221d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c107df0 0x7fe22c0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe221d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c107df0 0x7fe22c0ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34242/0 (socket says 172.21.15.46:34242) 2024-05-09T00:13:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe221d9b700 1 -- 172.21.15.46:0/1641534555 learned_addr learned my addr 172.21.15.46:0/1641534555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe22ad9d700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108740 0x7fe22c0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe22a59c700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe22c1034c0 0x7fe22c0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe221d9b700 1 -- 172.21.15.46:0/1641534555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe22c1034c0 msgr2=0x7fe22c0ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe221d9b700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe22c1034c0 0x7fe22c0ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe221d9b700 1 -- 172.21.15.46:0/1641534555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108740 msgr2=0x7fe22c0ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe221d9b700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108740 0x7fe22c0ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.360+0000 7fe221d9b700 1 -- 172.21.15.46:0/1641534555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c1a91d0 con 0x7fe22c107df0 2024-05-09T00:13:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.361+0000 7fe221d9b700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c107df0 0x7fe22c0ff810 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe214006010 tx=0x7fe2140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.361+0000 7fe223fff700 1 -- 172.21.15.46:0/1641534555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214027070 con 0x7fe22c107df0 2024-05-09T00:13:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.361+0000 7fe223fff700 1 -- 172.21.15.46:0/1641534555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe214003f20 con 0x7fe22c107df0 2024-05-09T00:13:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.361+0000 7fe230a54700 1 -- 172.21.15.46:0/1641534555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c1a9460 con 0x7fe22c107df0 2024-05-09T00:13:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.361+0000 7fe223fff700 1 -- 172.21.15.46:0/1641534555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214022c60 con 0x7fe22c107df0 2024-05-09T00:13:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.361+0000 7fe230a54700 1 -- 172.21.15.46:0/1641534555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c1a98b0 con 0x7fe22c107df0 2024-05-09T00:13:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.364+0000 7fe223fff700 1 -- 172.21.15.46:0/1641534555 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe21402a070 con 0x7fe22c107df0 2024-05-09T00:13:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.364+0000 7fe223fff700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe2180b2880 0x7fe2180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.365+0000 7fe22a59c700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe2180b2880 0x7fe2180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.365+0000 7fe223fff700 1 -- 172.21.15.46:0/1641534555 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe214022e00 con 0x7fe22c107df0 2024-05-09T00:13:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.365+0000 7fe23f46ab80 1 Processor -- start 2024-05-09T00:13:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.365+0000 7fe23f46ab80 1 -- start start 2024-05-09T00:13:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.365+0000 7fe22a59c700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe2180b2880 0x7fe2180b4d90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005950 tx=0x7fe21c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.365+0000 7fe23f46ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7033d00 0x55c6b713c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.365+0000 7fe23f46ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b713c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.365+0000 7fe23f46ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7136400 0x55c6b7136810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.365+0000 7fe23f46ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c6b70590a0 con 0x55c6b6fb94a0 2024-05-09T00:13:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.365+0000 7fe23f46ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c6b6ffe9a0 con 0x55c6b7136400 2024-05-09T00:13:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.365+0000 7fe23f46ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c6b7055060 con 0x55c6b7033d00 2024-05-09T00:13:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe221d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b713c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe221d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b713c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34248/0 (socket says 172.21.15.46:34248) 2024-05-09T00:13:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe221d9b700 1 -- 172.21.15.46:0/3056428040 learned_addr learned my addr 172.21.15.46:0/3056428040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe22ad9d700 1 --2- 172.21.15.46:0/3056428040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7136400 0x55c6b7136810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe22a59c700 1 --2- 172.21.15.46:0/3056428040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7033d00 0x55c6b713c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe221d9b700 1 -- 172.21.15.46:0/3056428040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7033d00 msgr2=0x55c6b713c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:49.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe221d9b700 1 --2- 172.21.15.46:0/3056428040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7033d00 0x55c6b713c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe221d9b700 1 -- 172.21.15.46:0/3056428040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7136400 msgr2=0x55c6b7136810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:49.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe221d9b700 1 --2- 172.21.15.46:0/3056428040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7136400 0x55c6b7136810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe221d9b700 1 -- 172.21.15.46:0/3056428040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6b6fef500 con 0x55c6b6fb94a0 2024-05-09T00:13:49.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe221d9b700 1 --2- 172.21.15.46:0/3056428040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b713c780 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe2140a6750 tx=0x7fe214009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1413fdac0898799 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:49.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe220d99700 1 -- 172.21.15.46:0/3056428040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2140aa6d0 con 0x55c6b6fb94a0 2024-05-09T00:13:49.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe220d99700 1 -- 172.21.15.46:0/3056428040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2140aea80 con 0x55c6b6fb94a0 2024-05-09T00:13:49.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.366+0000 7fe220d99700 1 -- 172.21.15.46:0/3056428040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2140aa930 con 0x55c6b6fb94a0 2024-05-09T00:13:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.367+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3056428040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 msgr2=0x55c6b713c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.367+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/3056428040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b713c780 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe2140a6750 tx=0x7fe214009310 comp rx=0 tx=0).stop 2024-05-09T00:13:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.367+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3056428040 shutdown_connections 2024-05-09T00:13:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.367+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/3056428040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7136400 0x55c6b7136810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.367+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/3056428040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7033d00 0x55c6b713c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.367+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/3056428040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b713c780 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.367+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3056428040 >> 172.21.15.46:0/3056428040 conn(0x55c6b6fbcf70 msgr2=0x55c6b7131220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.367+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3056428040 shutdown_connections 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.367+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3056428040 wait complete. 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe23f46ab80 1 -- start start 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe23f46ab80 1 Processor -- start 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe23f46ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b7145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe23f46ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7033d00 0x55c6b7145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe23f46ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7136400 0x55c6b713feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe23f46ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c6b70590a0 con 0x55c6b6fb94a0 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe23f46ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c6b6ff3b20 con 0x55c6b7033d00 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe23f46ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c6b7136fb0 con 0x55c6b7136400 2024-05-09T00:13:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe22a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b7145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe22a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b7145ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34258/0 (socket says 172.21.15.46:34258) 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe23f46ab80 10 client.0 _ll_register_callbacks cb 0x55c6b6eca6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe22a59c700 1 -- 172.21.15.46:0/3523798732 learned_addr learned my addr 172.21.15.46:0/3523798732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.368+0000 7fe221d9b700 1 --2- 172.21.15.46:0/3523798732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7033d00 0x55c6b7145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe22a59c700 1 -- 172.21.15.46:0/3523798732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7136400 msgr2=0x55c6b713feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe22a59c700 1 --2- 172.21.15.46:0/3523798732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7136400 0x55c6b713feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe22a59c700 1 -- 172.21.15.46:0/3523798732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7033d00 msgr2=0x55c6b7145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe22a59c700 1 --2- 172.21.15.46:0/3523798732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7033d00 0x55c6b7145ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe22a59c700 1 -- 172.21.15.46:0/3523798732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6b6ed4490 con 0x55c6b6fb94a0 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe22a59c700 1 --2- 172.21.15.46:0/3523798732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b7145ab0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe21c0116c0 tx=0x7fe21c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943afe67893ee402 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe211ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe211ffb700 1 -- 172.21.15.46:0/3523798732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c004a30 con 0x55c6b6fb94a0 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe211ffb700 1 -- 172.21.15.46:0/3523798732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe21c004bd0 con 0x55c6b6fb94a0 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3523798732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2140aea80 con 0x55c6b6fb94a0 2024-05-09T00:13:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe211ffb700 1 -- 172.21.15.46:0/3523798732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe21c004dd0 con 0x55c6b6fb94a0 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe23f46ab80 10 client.15540 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3523798732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c6b714d730 con 0x55c6b6fb94a0 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.369+0000 7fe211ffb700 1 -- 172.21.15.46:0/3523798732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2140aea80 con 0x55c6b6fb94a0 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.370+0000 7fe211ffb700 1 -- 172.21.15.46:0/3523798732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe21c01f970 con 0x55c6b6fb94a0 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.370+0000 7fe23f46ab80 10 client.15540 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.370+0000 7fe23f46ab80 10 client.15540 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.370+0000 7fe23f46ab80 20 client.15540 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.370+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/3523798732 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c6b714fd70 0x55c6b71521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.370+0000 7fe23f46ab80 4 client.15540 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.370+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3523798732 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6b7066a80 con 0x55c6b714fd70 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.370+0000 7fe221d9b700 1 --2- 172.21.15.46:0/3523798732 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c6b714fd70 0x55c6b71521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.371+0000 7fe221d9b700 1 --2- 172.21.15.46:0/3523798732 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c6b714fd70 0x55c6b71521f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fec2b83c48940e72 server_cookie=372a471d949d62e9 in_seq=0 out_seq=0 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.371+0000 7fe211ffb700 10 client.15540 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:13:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe211ffb700 1 -- 172.21.15.46:0/3523798732 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6b7066a80 con 0x55c6b714fd70 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe211ffb700 10 client.15540 handle_command_reply: tid=0 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe23f46ab80 1 client.15540 shutdown 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3523798732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 msgr2=0x55c6b7145ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/3523798732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b7145ab0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe21c0116c0 tx=0x7fe21c0047c0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3523798732 shutdown_connections 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/3523798732 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c6b714fd70 0x55c6b71521f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/3523798732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7136400 0x55c6b713feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/3523798732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b7145ab0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/3523798732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7033d00 0x55c6b7145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe211ffb700 0 client.15540 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3523798732 >> 172.21.15.46:0/3523798732 conn(0x55c6b6fbcf70 msgr2=0x55c6b712aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3523798732 shutdown_connections 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe23f46ab80 1 -- 172.21.15.46:0/3523798732 wait complete. 2024-05-09T00:13:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.372+0000 7fe23f46ab80 20 client.15540 trim_cache size 0 max 16384 2024-05-09T00:13:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.387+0000 7fe23f46ab80 1 Processor -- start 2024-05-09T00:13:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.387+0000 7fe23f46ab80 1 -- start start 2024-05-09T00:13:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.387+0000 7fe23f46ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b71579f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.387+0000 7fe23f46ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7033d00 0x55c6b7157f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.387+0000 7fe23f46ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7136400 0x55c6b713d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.387+0000 7fe23f46ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1b3600 con 0x55c6b6fb94a0 2024-05-09T00:13:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.387+0000 7fe23f46ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1b0e90 con 0x55c6b7033d00 2024-05-09T00:13:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.387+0000 7fe23f46ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1aecc0 con 0x55c6b7136400 2024-05-09T00:13:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe22a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b71579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe22a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b71579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34268/0 (socket says 172.21.15.46:34268) 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe22a59c700 1 -- 172.21.15.46:0/2525538874 learned_addr learned my addr 172.21.15.46:0/2525538874 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe22ad9d700 1 --2- 172.21.15.46:0/2525538874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7136400 0x55c6b713d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe221d9b700 1 --2- 172.21.15.46:0/2525538874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7033d00 0x55c6b7157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe22a59c700 1 -- 172.21.15.46:0/2525538874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7136400 msgr2=0x55c6b713d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe22a59c700 1 --2- 172.21.15.46:0/2525538874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7136400 0x55c6b713d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe22a59c700 1 -- 172.21.15.46:0/2525538874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7033d00 msgr2=0x55c6b7157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe22a59c700 1 --2- 172.21.15.46:0/2525538874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7033d00 0x55c6b7157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe22a59c700 1 -- 172.21.15.46:0/2525538874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6b712d9d0 con 0x55c6b6fb94a0 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe22ad9d700 1 --2- 172.21.15.46:0/2525538874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7136400 0x55c6b713d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe221d9b700 1 --2- 172.21.15.46:0/2525538874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7033d00 0x55c6b7157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe22a59c700 1 --2- 172.21.15.46:0/2525538874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b71579f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe21c000c00 tx=0x7fe21c010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b4b95ac1198a69 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe213fff700 1 -- 172.21.15.46:0/2525538874 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c040070 con 0x55c6b6fb94a0 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe213fff700 1 -- 172.21.15.46:0/2525538874 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe21c023070 con 0x55c6b6fb94a0 2024-05-09T00:13:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.388+0000 7fe213fff700 1 -- 172.21.15.46:0/2525538874 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c037040 con 0x55c6b6fb94a0 2024-05-09T00:13:49.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.389+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2525538874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 msgr2=0x55c6b71579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:49.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.389+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/2525538874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b71579f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe21c000c00 tx=0x7fe21c010f80 comp rx=0 tx=0).stop 2024-05-09T00:13:49.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.389+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2525538874 shutdown_connections 2024-05-09T00:13:49.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.389+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/2525538874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b7136400 0x55c6b713d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.389+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/2525538874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b6fb94a0 0x55c6b71579f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.389+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/2525538874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7033d00 0x55c6b7157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.389+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2525538874 >> 172.21.15.46:0/2525538874 conn(0x55c6b6fbcf70 msgr2=0x55c6b7130c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.389+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2525538874 shutdown_connections 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.389+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2525538874 wait complete. 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe23f46ab80 1 -- start start 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe23f46ab80 1 Processor -- start 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe23f46ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b6fb94a0 0x55c6b7149200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe23f46ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b7033d00 0x55c6b7149740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe23f46ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7136400 0x55c6b71b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe23f46ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1af350 con 0x55c6b7033d00 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe23f46ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1b07c0 con 0x55c6b7136400 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe23f46ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1afa20 con 0x55c6b6fb94a0 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe23f46ab80 10 client.0 _ll_register_callbacks cb 0x55c6b713f340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe221d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b7033d00 0x55c6b7149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe221d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b7033d00 0x55c6b7149740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34282/0 (socket says 172.21.15.46:34282) 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe221d9b700 1 -- 172.21.15.46:0/2907144678 learned_addr learned my addr 172.21.15.46:0/2907144678 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe22a59c700 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b6fb94a0 0x55c6b7149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe22ad9d700 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7136400 0x55c6b71b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe221d9b700 1 -- 172.21.15.46:0/2907144678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b6fb94a0 msgr2=0x55c6b7149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe221d9b700 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b6fb94a0 0x55c6b7149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe221d9b700 1 -- 172.21.15.46:0/2907144678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7136400 msgr2=0x55c6b71b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe221d9b700 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7136400 0x55c6b71b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe221d9b700 1 -- 172.21.15.46:0/2907144678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6b71b8ae0 con 0x55c6b7033d00 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.390+0000 7fe22ad9d700 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7136400 0x55c6b71b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe221d9b700 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b7033d00 0x55c6b7149740 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe214009990 tx=0x7fe214022730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a9d38432570f80 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe211ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe211ffb700 1 -- 172.21.15.46:0/2907144678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2140c3070 con 0x55c6b7033d00 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe211ffb700 1 -- 172.21.15.46:0/2907144678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe214022470 con 0x55c6b7033d00 2024-05-09T00:13:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2907144678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe21c023070 con 0x55c6b7033d00 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe211ffb700 1 -- 172.21.15.46:0/2907144678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe2140c9050 con 0x55c6b7033d00 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe23f46ab80 10 client.15546 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2907144678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c6b71b9c50 con 0x55c6b7033d00 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe211ffb700 1 -- 172.21.15.46:0/2907144678 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe21c023070 con 0x55c6b7033d00 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe211ffb700 1 -- 172.21.15.46:0/2907144678 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe2140aa9c0 con 0x55c6b7033d00 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe23f46ab80 10 client.15546 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe23f46ab80 10 client.15546 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe23f46ab80 20 client.15546 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c6b71ba220 0x55c6b71bc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe23f46ab80 4 client.15546 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.391+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2907144678 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6b7060b20 con 0x55c6b71ba220 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.392+0000 7fe22a59c700 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c6b71ba220 0x55c6b71bc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.392+0000 7fe22a59c700 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c6b71ba220 0x55c6b71bc640 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa09d7b6303d3837 server_cookie=9b1b251cb9d8fc07 in_seq=0 out_seq=0 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.392+0000 7fe211ffb700 10 client.15546 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.392+0000 7fe211ffb700 1 -- 172.21.15.46:0/2907144678 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6b7060b20 con 0x55c6b71ba220 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe211ffb700 10 client.15546 handle_command_reply: tid=0 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe23f46ab80 1 client.15546 shutdown 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2907144678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b7033d00 msgr2=0x55c6b7149740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b7033d00 0x55c6b7149740 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe214009990 tx=0x7fe214022730 comp rx=0 tx=0).stop 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2907144678 shutdown_connections 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c6b71ba220 0x55c6b71bc640 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6b7136400 0x55c6b71b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c6b6fb94a0 0x55c6b7149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe23f46ab80 1 --2- 172.21.15.46:0/2907144678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6b7033d00 0x55c6b7149740 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2907144678 >> 172.21.15.46:0/2907144678 conn(0x55c6b6fbcf70 msgr2=0x55c6b7139c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe211ffb700 0 client.15546 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2907144678 shutdown_connections 2024-05-09T00:13:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe23f46ab80 1 -- 172.21.15.46:0/2907144678 wait complete. 2024-05-09T00:13:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe23f46ab80 20 client.15546 trim_cache size 0 max 16384 2024-05-09T00:13:49.396 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:13:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe230a54700 1 -- 172.21.15.46:0/1641534555 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe2180b2880 msgr2=0x7fe2180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe230a54700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe2180b2880 0x7fe2180b4d90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005950 tx=0x7fe21c00b410 comp rx=0 tx=0).stop 2024-05-09T00:13:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe230a54700 1 -- 172.21.15.46:0/1641534555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c107df0 msgr2=0x7fe22c0ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.393+0000 7fe230a54700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c107df0 0x7fe22c0ff810 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe214006010 tx=0x7fe2140047b0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.394+0000 7fe230a54700 1 -- 172.21.15.46:0/1641534555 shutdown_connections 2024-05-09T00:13:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.394+0000 7fe230a54700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe2180b2880 0x7fe2180b4d90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.394+0000 7fe230a54700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22c108740 0x7fe22c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.394+0000 7fe230a54700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe22c1034c0 0x7fe22c0ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.394+0000 7fe230a54700 1 --2- 172.21.15.46:0/1641534555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c107df0 0x7fe22c0ff810 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.394+0000 7fe230a54700 1 -- 172.21.15.46:0/1641534555 >> 172.21.15.46:0/1641534555 conn(0x7fe22c0fedf0 msgr2=0x7fe22c075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.394+0000 7fe230a54700 1 -- 172.21.15.46:0/1641534555 shutdown_connections 2024-05-09T00:13:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:49.394+0000 7fe230a54700 1 -- 172.21.15.46:0/1641534555 wait complete. 2024-05-09T00:13:50.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:13:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:50.928 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:50 smithi046 ceph-mon[30151]: pgmap v450: 97 pgs: 97 active+clean; 789 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-05-09T00:13:50.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:50 smithi142 ceph-mon[34533]: pgmap v450: 97 pgs: 97 active+clean; 789 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-05-09T00:13:51.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:50 smithi169 ceph-mon[34777]: pgmap v450: 97 pgs: 97 active+clean; 789 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-05-09T00:13:52.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.266+0000 7f32380cf700 1 -- 172.21.15.46:0/3349257474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230108740 msgr2=0x7f32301100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:52.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.266+0000 7f32380cf700 1 --2- 172.21.15.46:0/3349257474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230108740 0x7f32301100a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f322800b0d0 tx=0x7f322800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.266+0000 7f32380cf700 1 -- 172.21.15.46:0/3349257474 shutdown_connections 2024-05-09T00:13:52.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.266+0000 7f32380cf700 1 --2- 172.21.15.46:0/3349257474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230108740 0x7f32301100a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.266+0000 7f32380cf700 1 --2- 172.21.15.46:0/3349257474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32301034c0 0x7f32301038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.266+0000 7f32380cf700 1 --2- 172.21.15.46:0/3349257474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3230107df0 0x7f3230108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.266+0000 7f32380cf700 1 -- 172.21.15.46:0/3349257474 >> 172.21.15.46:0/3349257474 conn(0x7f32300fedf0 msgr2=0x7f3230101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.266+0000 7f32380cf700 1 -- 172.21.15.46:0/3349257474 shutdown_connections 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.266+0000 7f32380cf700 1 -- 172.21.15.46:0/3349257474 wait complete. 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.267+0000 7f32380cf700 1 Processor -- start 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.267+0000 7f32380cf700 1 -- start start 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.267+0000 7f32380cf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32301034c0 0x7f323019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.267+0000 7f32380cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230107df0 0x7f323019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.267+0000 7f32380cf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3230108740 0x7f323019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.267+0000 7f32380cf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3230113370 con 0x7f3230107df0 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.267+0000 7f32380cf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32301131f0 con 0x7f3230108740 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.267+0000 7f32380cf700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32301134f0 con 0x7f32301034c0 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.267+0000 7f323566a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230107df0 0x7f323019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.267+0000 7f323566a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230107df0 0x7f323019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34294/0 (socket says 172.21.15.46:34294) 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.267+0000 7f323566a700 1 -- 172.21.15.46:0/3101470150 learned_addr learned my addr 172.21.15.46:0/3101470150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:52.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f323666c700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3230108740 0x7f323019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f3235e6b700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32301034c0 0x7f323019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f323566a700 1 -- 172.21.15.46:0/3101470150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32301034c0 msgr2=0x7f323019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f323566a700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32301034c0 0x7f323019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f323566a700 1 -- 172.21.15.46:0/3101470150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3230108740 msgr2=0x7f323019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f323566a700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3230108740 0x7f323019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f323566a700 1 -- 172.21.15.46:0/3101470150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32301a6fe0 con 0x7f3230107df0 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f3235e6b700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32301034c0 0x7f323019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f323566a700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230107df0 0x7f323019c1d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f322000d980 tx=0x7f322000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f3226ffd700 1 -- 172.21.15.46:0/3101470150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220017070 con 0x7f3230107df0 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f32380cf700 1 -- 172.21.15.46:0/3101470150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32301a72d0 con 0x7f3230107df0 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f3226ffd700 1 -- 172.21.15.46:0/3101470150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3220009de0 con 0x7f3230107df0 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f3226ffd700 1 -- 172.21.15.46:0/3101470150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322000f3c0 con 0x7f3230107df0 2024-05-09T00:13:52.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.268+0000 7f32380cf700 1 -- 172.21.15.46:0/3101470150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32301a7860 con 0x7f3230107df0 2024-05-09T00:13:52.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.271+0000 7f3226ffd700 1 -- 172.21.15.46:0/3101470150 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3220009930 con 0x7f3230107df0 2024-05-09T00:13:52.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.272+0000 7f3226ffd700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f321c0b2880 0x7f321c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.272+0000 7f3235e6b700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f321c0b2880 0x7f321c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.272+0000 7f3226ffd700 1 -- 172.21.15.46:0/3101470150 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f32200e1ca0 con 0x7f3230107df0 2024-05-09T00:13:52.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.272+0000 7f3246ae5b80 1 Processor -- start 2024-05-09T00:13:52.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.272+0000 7f3246ae5b80 1 -- start start 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f3246ae5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc23428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f3246ae5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2342df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f3235e6b700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f321c0b2880 0x7f321c0b4d90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f322c005950 tx=0x7f322c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f3246ae5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc233cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f3246ae5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55efc225aff0 con 0x55efc21bb4a0 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f3246ae5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55efc21f97f0 con 0x55efc22a5160 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f3246ae5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55efc2256fb0 con 0x55efc233cab0 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f323566a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2342df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f323566a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2342df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34298/0 (socket says 172.21.15.46:34298) 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f323566a700 1 -- 172.21.15.46:0/1803140879 learned_addr learned my addr 172.21.15.46:0/1803140879 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f3235e6b700 1 --2- 172.21.15.46:0/1803140879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc23428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f323666c700 1 --2- 172.21.15.46:0/1803140879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc233cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f323566a700 1 -- 172.21.15.46:0/1803140879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 msgr2=0x55efc233cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f323566a700 1 --2- 172.21.15.46:0/1803140879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc233cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f323566a700 1 -- 172.21.15.46:0/1803140879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 msgr2=0x55efc23428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f323566a700 1 --2- 172.21.15.46:0/1803140879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc23428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f323566a700 1 -- 172.21.15.46:0/1803140879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc21f1370 con 0x55efc21bb4a0 2024-05-09T00:13:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f323666c700 1 --2- 172.21.15.46:0/1803140879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc233cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f323566a700 1 --2- 172.21.15.46:0/1803140879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2342df0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f32200a8fc0 tx=0x7f32200a8dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33597f0da3e41f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.273+0000 7f321bfff700 1 -- 172.21.15.46:0/1803140879 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220020040 con 0x55efc21bb4a0 2024-05-09T00:13:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.274+0000 7f321bfff700 1 -- 172.21.15.46:0/1803140879 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32200a92c0 con 0x55efc21bb4a0 2024-05-09T00:13:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.274+0000 7f321bfff700 1 -- 172.21.15.46:0/1803140879 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32200b1ef0 con 0x55efc21bb4a0 2024-05-09T00:13:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.274+0000 7f3246ae5b80 1 -- 172.21.15.46:0/1803140879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 msgr2=0x55efc2342df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:52.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.274+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/1803140879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2342df0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f32200a8fc0 tx=0x7f32200a8dd0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.275+0000 7f3246ae5b80 1 -- 172.21.15.46:0/1803140879 shutdown_connections 2024-05-09T00:13:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.275+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/1803140879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc233cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.275+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/1803140879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc23428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.275+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/1803140879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2342df0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.275+0000 7f3246ae5b80 1 -- 172.21.15.46:0/1803140879 >> 172.21.15.46:0/1803140879 conn(0x55efc221b6a0 msgr2=0x55efc2333400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.275+0000 7f3246ae5b80 1 -- 172.21.15.46:0/1803140879 shutdown_connections 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.275+0000 7f3246ae5b80 1 -- 172.21.15.46:0/1803140879 wait complete. 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.275+0000 7f3246ae5b80 1 -- start start 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.275+0000 7f3246ae5b80 1 Processor -- start 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3246ae5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2346680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3246ae5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc2346bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3246ae5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc2347100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3246ae5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55efc20d6ba0 con 0x55efc21bb4a0 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3246ae5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55efc2215150 con 0x55efc22a5160 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3246ae5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55efc233d830 con 0x55efc233cab0 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3246ae5b80 10 client.0 _ll_register_callbacks cb 0x55efc20cc6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:52.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3235e6b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2346680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3235e6b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2346680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34304/0 (socket says 172.21.15.46:34304) 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3235e6b700 1 -- 172.21.15.46:0/3378372033 learned_addr learned my addr 172.21.15.46:0/3378372033 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f323666c700 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc2347100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f323566a700 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc2346bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3235e6b700 1 -- 172.21.15.46:0/3378372033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 msgr2=0x55efc2347100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3235e6b700 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc2347100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3235e6b700 1 -- 172.21.15.46:0/3378372033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 msgr2=0x55efc2346bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3235e6b700 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc2346bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f3235e6b700 1 -- 172.21.15.46:0/3378372033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc20d6490 con 0x55efc21bb4a0 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f323666c700 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc2347100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.276+0000 7f323566a700 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc2346bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.277+0000 7f3235e6b700 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2346680 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f322c00bfd0 tx=0x7f322c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120b5a8d069ea7eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:52.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.277+0000 7f32197fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.277+0000 7f32197fa700 1 -- 172.21.15.46:0/3378372033 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c024020 con 0x55efc21bb4a0 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.277+0000 7f32197fa700 1 -- 172.21.15.46:0/3378372033 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f322c003d10 con 0x55efc21bb4a0 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.277+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3378372033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32200a92c0 con 0x55efc21bb4a0 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.277+0000 7f32197fa700 1 -- 172.21.15.46:0/3378372033 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f322c020440 con 0x55efc21bb4a0 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.277+0000 7f3246ae5b80 10 client.15558 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.277+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3378372033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55efc2353e60 con 0x55efc21bb4a0 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.278+0000 7f32197fa700 1 -- 172.21.15.46:0/3378372033 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32200a92c0 con 0x55efc21bb4a0 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.278+0000 7f32197fa700 1 -- 172.21.15.46:0/3378372033 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f322c031020 con 0x55efc21bb4a0 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.278+0000 7f3246ae5b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.278+0000 7f3246ae5b80 10 client.15558 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.278+0000 7f3246ae5b80 20 client.15558 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.278+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55efc2356540 0x55efc23589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.278+0000 7f3246ae5b80 4 client.15558 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.278+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3378372033 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efc22689d0 con 0x55efc2356540 2024-05-09T00:13:52.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.278+0000 7f323566a700 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55efc2356540 0x55efc23589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.279+0000 7f323566a700 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55efc2356540 0x55efc23589c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4bab2735748b0a2e server_cookie=5e5eea8cd9775b28 in_seq=0 out_seq=0 2024-05-09T00:13:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.279+0000 7f32197fa700 10 client.15558 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:13:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.279+0000 7f32197fa700 1 -- 172.21.15.46:0/3378372033 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55efc22689d0 con 0x55efc2356540 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.279+0000 7f32197fa700 10 client.15558 handle_command_reply: tid=0 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.279+0000 7f3246ae5b80 1 client.15558 shutdown 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.280+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3378372033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 msgr2=0x55efc2346680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.280+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2346680 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f322c00bfd0 tx=0x7f322c004750 comp rx=0 tx=0).stop 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.280+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3378372033 shutdown_connections 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.280+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55efc2356540 0x55efc23589c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.280+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc2347100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.280+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2346680 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.280+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/3378372033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc2346bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.280+0000 7f32197fa700 0 client.15558 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.280+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3378372033 >> 172.21.15.46:0/3378372033 conn(0x55efc221b6a0 msgr2=0x55efc23331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.280+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3378372033 shutdown_connections 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.280+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3378372033 wait complete. 2024-05-09T00:13:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.280+0000 7f3246ae5b80 20 client.15558 trim_cache size 0 max 16384 2024-05-09T00:13:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.295+0000 7f3246ae5b80 1 Processor -- start 2024-05-09T00:13:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.295+0000 7f3246ae5b80 1 -- start start 2024-05-09T00:13:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3246ae5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2346f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3246ae5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc22a5160 0x55efc2345680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3246ae5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc233cab0 0x55efc235e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3246ae5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32301af490 con 0x55efc21bb4a0 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3246ae5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32301acd20 con 0x55efc233cab0 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3246ae5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32301aab50 con 0x55efc22a5160 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3235e6b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2346f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3235e6b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2346f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34318/0 (socket says 172.21.15.46:34318) 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3235e6b700 1 -- 172.21.15.46:0/478541950 learned_addr learned my addr 172.21.15.46:0/478541950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f323666c700 1 --2- 172.21.15.46:0/478541950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc233cab0 0x55efc235e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f323566a700 1 --2- 172.21.15.46:0/478541950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc22a5160 0x55efc2345680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3235e6b700 1 -- 172.21.15.46:0/478541950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc22a5160 msgr2=0x55efc2345680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3235e6b700 1 --2- 172.21.15.46:0/478541950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc22a5160 0x55efc2345680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3235e6b700 1 -- 172.21.15.46:0/478541950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc233cab0 msgr2=0x55efc235e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3235e6b700 1 --2- 172.21.15.46:0/478541950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc233cab0 0x55efc235e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3235e6b700 1 -- 172.21.15.46:0/478541950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc233dc00 con 0x55efc21bb4a0 2024-05-09T00:13:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f323666c700 1 --2- 172.21.15.46:0/478541950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc233cab0 0x55efc235e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:52.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f3235e6b700 1 --2- 172.21.15.46:0/478541950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2346f40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f322c011830 tx=0x7f322c011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3463c6fa33d3b66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:52.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.296+0000 7f321b7fe700 1 -- 172.21.15.46:0/478541950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c024070 con 0x55efc21bb4a0 2024-05-09T00:13:52.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.297+0000 7f321b7fe700 1 -- 172.21.15.46:0/478541950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f322c011c20 con 0x55efc21bb4a0 2024-05-09T00:13:52.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.297+0000 7f321b7fe700 1 -- 172.21.15.46:0/478541950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c0047c0 con 0x55efc21bb4a0 2024-05-09T00:13:52.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.297+0000 7f3246ae5b80 1 -- 172.21.15.46:0/478541950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 msgr2=0x55efc2346f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.297+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/478541950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2346f40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f322c011830 tx=0x7f322c011f80 comp rx=0 tx=0).stop 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.297+0000 7f3246ae5b80 1 -- 172.21.15.46:0/478541950 shutdown_connections 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.297+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/478541950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc233cab0 0x55efc235e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.297+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/478541950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc2346f40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.297+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/478541950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc22a5160 0x55efc2345680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.297+0000 7f3246ae5b80 1 -- 172.21.15.46:0/478541950 >> 172.21.15.46:0/478541950 conn(0x55efc221b6a0 msgr2=0x55efc2344f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.297+0000 7f3246ae5b80 1 -- 172.21.15.46:0/478541950 shutdown_connections 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.297+0000 7f3246ae5b80 1 -- 172.21.15.46:0/478541950 wait complete. 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.298+0000 7f3246ae5b80 1 -- start start 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.298+0000 7f3246ae5b80 1 Processor -- start 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.298+0000 7f3246ae5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc23b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.298+0000 7f3246ae5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc23b9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.298+0000 7f3246ae5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc2349ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.298+0000 7f3246ae5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32301ab1e0 con 0x55efc21bb4a0 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.298+0000 7f3246ae5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32301ac650 con 0x55efc22a5160 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.298+0000 7f3246ae5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32301ab8b0 con 0x55efc233cab0 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.298+0000 7f3246ae5b80 10 client.0 _ll_register_callbacks cb 0x55efc20cc6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f3235e6b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc23b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f3235e6b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc23b90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34322/0 (socket says 172.21.15.46:34322) 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f3235e6b700 1 -- 172.21.15.46:0/3870579093 learned_addr learned my addr 172.21.15.46:0/3870579093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f323666c700 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc2349ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f323566a700 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc23b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f3235e6b700 1 -- 172.21.15.46:0/3870579093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 msgr2=0x55efc2349ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f3235e6b700 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc2349ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f3235e6b700 1 -- 172.21.15.46:0/3870579093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 msgr2=0x55efc23b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f3235e6b700 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc23b9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f3235e6b700 1 -- 172.21.15.46:0/3870579093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc234a0f0 con 0x55efc21bb4a0 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f3235e6b700 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc23b90d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f322c00fa80 tx=0x7f322c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f8e31a602a4981 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f32197fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:52.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f32197fa700 1 -- 172.21.15.46:0/3870579093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c024040 con 0x55efc21bb4a0 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f32197fa700 1 -- 172.21.15.46:0/3870579093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f322c003b70 con 0x55efc21bb4a0 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.299+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3870579093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f322c011c20 con 0x55efc21bb4a0 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.300+0000 7f32197fa700 1 -- 172.21.15.46:0/3870579093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f322c003d70 con 0x55efc21bb4a0 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.300+0000 7f3246ae5b80 10 client.15564 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.300+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3870579093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55efc23c0750 con 0x55efc21bb4a0 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.300+0000 7f32197fa700 1 -- 172.21.15.46:0/3870579093 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f322c011c20 con 0x55efc21bb4a0 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.300+0000 7f32197fa700 1 -- 172.21.15.46:0/3870579093 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f322c044020 con 0x55efc21bb4a0 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.300+0000 7f3246ae5b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.300+0000 7f3246ae5b80 10 client.15564 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.300+0000 7f3246ae5b80 20 client.15564 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.300+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55efc23c0d20 0x55efc23c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.300+0000 7f3246ae5b80 4 client.15564 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:13:52.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.300+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3870579093 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efc2262a70 con 0x55efc23c0d20 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.300+0000 7f323566a700 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55efc23c0d20 0x55efc23c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f323566a700 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55efc23c0d20 0x55efc23c3140 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7446fa526e9e52dd server_cookie=d794823ecf9750d5 in_seq=0 out_seq=0 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f32197fa700 10 client.15564 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f32197fa700 1 -- 172.21.15.46:0/3870579093 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efc2262a70 con 0x55efc23c0d20 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f32197fa700 10 client.15564 handle_command_reply: tid=0 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f3246ae5b80 1 client.15564 shutdown 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3870579093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 msgr2=0x55efc23b90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc23b90d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f322c00fa80 tx=0x7f322c00fab0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3870579093 shutdown_connections 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55efc23c0d20 0x55efc23c3140 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55efc233cab0 0x55efc2349ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc21bb4a0 0x55efc23b90d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f3246ae5b80 1 --2- 172.21.15.46:0/3870579093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55efc22a5160 0x55efc23b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f32197fa700 0 client.15564 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:13:52.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3870579093 >> 172.21.15.46:0/3870579093 conn(0x55efc221b6a0 msgr2=0x55efc2337690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:52.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3870579093 shutdown_connections 2024-05-09T00:13:52.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f3246ae5b80 1 -- 172.21.15.46:0/3870579093 wait complete. 2024-05-09T00:13:52.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.301+0000 7f3246ae5b80 20 client.15564 trim_cache size 0 max 16384 2024-05-09T00:13:52.304 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:13:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.302+0000 7f32380cf700 1 -- 172.21.15.46:0/3101470150 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f321c0b2880 msgr2=0x7f321c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.302+0000 7f32380cf700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f321c0b2880 0x7f321c0b4d90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f322c005950 tx=0x7f322c0058e0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.302+0000 7f32380cf700 1 -- 172.21.15.46:0/3101470150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230107df0 msgr2=0x7f323019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.302+0000 7f32380cf700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230107df0 0x7f323019c1d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f322000d980 tx=0x7f322000dc90 comp rx=0 tx=0).stop 2024-05-09T00:13:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.302+0000 7f32380cf700 1 -- 172.21.15.46:0/3101470150 shutdown_connections 2024-05-09T00:13:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.302+0000 7f32380cf700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f321c0b2880 0x7f321c0b4d90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.302+0000 7f32380cf700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3230108740 0x7f323019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.302+0000 7f32380cf700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32301034c0 0x7f323019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.302+0000 7f32380cf700 1 --2- 172.21.15.46:0/3101470150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230107df0 0x7f323019c1d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.302+0000 7f32380cf700 1 -- 172.21.15.46:0/3101470150 >> 172.21.15.46:0/3101470150 conn(0x7f32300fedf0 msgr2=0x7f3230107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:52.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.302+0000 7f32380cf700 1 -- 172.21.15.46:0/3101470150 shutdown_connections 2024-05-09T00:13:52.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:52.302+0000 7f32380cf700 1 -- 172.21.15.46:0/3101470150 wait complete. 2024-05-09T00:13:52.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:52 smithi142 ceph-mon[34533]: pgmap v451: 97 pgs: 97 active+clean; 789 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:13:53.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:52 smithi046 ceph-mon[30151]: pgmap v451: 97 pgs: 97 active+clean; 789 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:13:53.091 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:13:53.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:52 smithi169 ceph-mon[34777]: pgmap v451: 97 pgs: 97 active+clean; 789 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:13:53.346 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:54.836 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:54 smithi046 ceph-mon[30151]: pgmap v452: 97 pgs: 97 active+clean; 789 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:13:54.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:54 smithi142 ceph-mon[34533]: pgmap v452: 97 pgs: 97 active+clean; 789 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:13:55.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:54 smithi169 ceph-mon[34777]: pgmap v452: 97 pgs: 97 active+clean; 789 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:13:55.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.199+0000 7f6ae1be6700 1 -- 172.21.15.46:0/1872867375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1034e0 msgr2=0x7f6adc1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:55.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.199+0000 7f6ae1be6700 1 --2- 172.21.15.46:0/1872867375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1034e0 0x7f6adc1038f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0099c0 tx=0x7f6acc009cd0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.199+0000 7f6ae1be6700 1 -- 172.21.15.46:0/1872867375 shutdown_connections 2024-05-09T00:13:55.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.199+0000 7f6ae1be6700 1 --2- 172.21.15.46:0/1872867375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6adc108760 0x7f6adc1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.199+0000 7f6ae1be6700 1 --2- 172.21.15.46:0/1872867375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1034e0 0x7f6adc1038f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.199+0000 7f6ae1be6700 1 --2- 172.21.15.46:0/1872867375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6adc107e10 0x7f6adc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.199+0000 7f6ae1be6700 1 -- 172.21.15.46:0/1872867375 >> 172.21.15.46:0/1872867375 conn(0x7f6adc0fef30 msgr2=0x7f6adc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.199+0000 7f6ae1be6700 1 -- 172.21.15.46:0/1872867375 shutdown_connections 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.199+0000 7f6ae1be6700 1 -- 172.21.15.46:0/1872867375 wait complete. 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.200+0000 7f6ae1be6700 1 Processor -- start 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.200+0000 7f6ae1be6700 1 -- start start 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.200+0000 7f6ae1be6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1034e0 0x7f6adc19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.200+0000 7f6ae1be6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6adc107e10 0x7f6adc19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.200+0000 7f6ae1be6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6adc108760 0x7f6adc1a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.200+0000 7f6ae1be6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc113390 con 0x7f6adc1034e0 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.200+0000 7f6ae1be6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc113210 con 0x7f6adc108760 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.200+0000 7f6ae1be6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc113510 con 0x7f6adc107e10 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.200+0000 7f6adb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1034e0 0x7f6adc19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.200+0000 7f6adb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1034e0 0x7f6adc19bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34348/0 (socket says 172.21.15.46:34348) 2024-05-09T00:13:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.200+0000 7f6adb7fe700 1 -- 172.21.15.46:0/3730107401 learned_addr learned my addr 172.21.15.46:0/3730107401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6adaffd700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6adc107e10 0x7f6adc19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6adbfff700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6adc108760 0x7f6adc1a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6adb7fe700 1 -- 172.21.15.46:0/3730107401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6adc107e10 msgr2=0x7f6adc19c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6adb7fe700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6adc107e10 0x7f6adc19c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6adb7fe700 1 -- 172.21.15.46:0/3730107401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6adc108760 msgr2=0x7f6adc1a6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6adb7fe700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6adc108760 0x7f6adc1a6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6adb7fe700 1 -- 172.21.15.46:0/3730107401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc19ca40 con 0x7f6adc1034e0 2024-05-09T00:13:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6adb7fe700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1034e0 0x7f6adc19bc50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00b580 tx=0x7f6acc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6ad8ff9700 1 -- 172.21.15.46:0/3730107401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc027070 con 0x7f6adc1034e0 2024-05-09T00:13:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6ad8ff9700 1 -- 172.21.15.46:0/3730107401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6acc003f20 con 0x7f6adc1034e0 2024-05-09T00:13:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6ae1be6700 1 -- 172.21.15.46:0/3730107401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6adc1a72d0 con 0x7f6adc1034e0 2024-05-09T00:13:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6ad8ff9700 1 -- 172.21.15.46:0/3730107401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc022d70 con 0x7f6adc1034e0 2024-05-09T00:13:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.201+0000 7f6ae1be6700 1 -- 172.21.15.46:0/3730107401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6adc1a7720 con 0x7f6adc1034e0 2024-05-09T00:13:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.204+0000 7f6ad8ff9700 1 -- 172.21.15.46:0/3730107401 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6acc00f460 con 0x7f6adc1034e0 2024-05-09T00:13:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.205+0000 7f6ad8ff9700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6ac40b2880 0x7f6ac40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.205+0000 7f6adaffd700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6ac40b2880 0x7f6ac40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.205+0000 7f6ad8ff9700 1 -- 172.21.15.46:0/3730107401 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6acc0e2ee0 con 0x7f6adc1034e0 2024-05-09T00:13:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.205+0000 7f6af05fcb80 1 Processor -- start 2024-05-09T00:13:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.205+0000 7f6adaffd700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6ac40b2880 0x7f6ac40b4d90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6adc101e60 tx=0x7f6ad000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.205+0000 7f6af05fcb80 1 -- start start 2024-05-09T00:13:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6af05fcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6af05fcb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c09e84a0 0x55a4c0b6ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6af05fcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b65ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6af05fcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0a87ff0 con 0x55a4c0b656a0 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6af05fcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0a267f0 con 0x55a4c0ad2160 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6af05fcb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0a83fb0 con 0x55a4c09e84a0 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b65ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b65ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34362/0 (socket says 172.21.15.46:34362) 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6adbfff700 1 -- 172.21.15.46:0/3238668170 learned_addr learned my addr 172.21.15.46:0/3238668170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6adb7fe700 1 --2- 172.21.15.46:0/3238668170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6adaffd700 1 --2- 172.21.15.46:0/3238668170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c09e84a0 0x55a4c0b6ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6adbfff700 1 -- 172.21.15.46:0/3238668170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c09e84a0 msgr2=0x55a4c0b6ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6adbfff700 1 --2- 172.21.15.46:0/3238668170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c09e84a0 0x55a4c0b6ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6adbfff700 1 -- 172.21.15.46:0/3238668170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c0ad2160 msgr2=0x55a4c0b6b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6adbfff700 1 --2- 172.21.15.46:0/3238668170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6adbfff700 1 -- 172.21.15.46:0/3238668170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4c0a1e370 con 0x55a4c0b656a0 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.206+0000 7f6adbfff700 1 --2- 172.21.15.46:0/3238668170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b65ad0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009d20 tx=0x7f6ad40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d5601cfddf59c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.207+0000 7f6ac1ffb700 1 -- 172.21.15.46:0/3238668170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4004bb0 con 0x55a4c0b656a0 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.207+0000 7f6ac1ffb700 1 -- 172.21.15.46:0/3238668170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ad4004d50 con 0x55a4c0b656a0 2024-05-09T00:13:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.207+0000 7f6ac1ffb700 1 -- 172.21.15.46:0/3238668170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4014670 con 0x55a4c0b656a0 2024-05-09T00:13:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.207+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3238668170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 msgr2=0x55a4c0b65ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.207+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/3238668170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b65ad0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009d20 tx=0x7f6ad40077e0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.208+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3238668170 shutdown_connections 2024-05-09T00:13:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.208+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/3238668170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b65ad0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.208+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/3238668170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.208+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/3238668170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c09e84a0 0x55a4c0b6ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.208+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3238668170 >> 172.21.15.46:0/3238668170 conn(0x55a4c0a486a0 msgr2=0x55a4c0b60400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.208+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3238668170 shutdown_connections 2024-05-09T00:13:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.208+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3238668170 wait complete. 2024-05-09T00:13:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.208+0000 7f6af05fcb80 1 -- start start 2024-05-09T00:13:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.208+0000 7f6af05fcb80 1 Processor -- start 2024-05-09T00:13:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6af05fcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 0x55a4c0b6f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6af05fcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6af05fcb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0b656a0 0x55a4c0b6fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6af05fcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0903ba0 con 0x55a4c0ad2160 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6af05fcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0a42150 con 0x55a4c09e84a0 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6af05fcb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a4c0b721d0 con 0x55a4c0b656a0 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6af05fcb80 10 client.0 _ll_register_callbacks cb 0x55a4c08f96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6adaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6adaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34366/0 (socket says 172.21.15.46:34366) 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6adaffd700 1 -- 172.21.15.46:0/3311455683 learned_addr learned my addr 172.21.15.46:0/3311455683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6adbfff700 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0b656a0 0x55a4c0b6fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6adb7fe700 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 0x55a4c0b6f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6adaffd700 1 -- 172.21.15.46:0/3311455683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0b656a0 msgr2=0x55a4c0b6fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6adaffd700 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0b656a0 0x55a4c0b6fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6adaffd700 1 -- 172.21.15.46:0/3311455683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 msgr2=0x55a4c0b6f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6adaffd700 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 0x55a4c0b6f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6adaffd700 1 -- 172.21.15.46:0/3311455683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4c0903490 con 0x55a4c0ad2160 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6adaffd700 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6f6e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00116c0 tx=0x7f6ad00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e9df62f6ed992b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6abaffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.209+0000 7f6abaffd700 1 -- 172.21.15.46:0/3311455683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0004a30 con 0x55a4c0ad2160 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6abaffd700 1 -- 172.21.15.46:0/3311455683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6ad0004bd0 con 0x55a4c0ad2160 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3311455683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad4004d50 con 0x55a4c0ad2160 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6abaffd700 1 -- 172.21.15.46:0/3311455683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ad0010e50 con 0x55a4c0ad2160 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6af05fcb80 10 client.15576 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3311455683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a4c0b7cc20 con 0x55a4c0ad2160 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6abaffd700 1 -- 172.21.15.46:0/3311455683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ad4004d50 con 0x55a4c0ad2160 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6abaffd700 1 -- 172.21.15.46:0/3311455683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6ad00050e0 con 0x55a4c0ad2160 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6af05fcb80 10 client.15576 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6af05fcb80 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6af05fcb80 20 client.15576 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a4c0b7f260 0x55a4c0b816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6af05fcb80 4 client.15576 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.210+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3311455683 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4c0a959d0 con 0x55a4c0b7f260 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.211+0000 7f6adb7fe700 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a4c0b7f260 0x55a4c0b816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.211+0000 7f6abaffd700 10 client.15576 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.211+0000 7f6adb7fe700 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a4c0b7f260 0x55a4c0b816c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=448b354a8c83d3f1 server_cookie=356c96d0829103de in_seq=0 out_seq=0 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6abaffd700 1 -- 172.21.15.46:0/3311455683 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a4c0a959d0 con 0x55a4c0b7f260 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6abaffd700 10 client.15576 handle_command_reply: tid=0 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6af05fcb80 1 client.15576 shutdown 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3311455683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0ad2160 msgr2=0x55a4c0b6f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6f6e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00116c0 tx=0x7f6ad00047c0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3311455683 shutdown_connections 2024-05-09T00:13:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a4c0b7f260 0x55a4c0b816c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0b656a0 0x55a4c0b6fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 0x55a4c0b6f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/3311455683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6f6e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3311455683 >> 172.21.15.46:0/3311455683 conn(0x55a4c0a486a0 msgr2=0x55a4c0b601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6abaffd700 0 client.15576 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3311455683 shutdown_connections 2024-05-09T00:13:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6af05fcb80 1 -- 172.21.15.46:0/3311455683 wait complete. 2024-05-09T00:13:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.212+0000 7f6af05fcb80 20 client.15576 trim_cache size 0 max 16384 2024-05-09T00:13:55.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.227+0000 7f6af05fcb80 1 Processor -- start 2024-05-09T00:13:55.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.227+0000 7f6af05fcb80 1 -- start start 2024-05-09T00:13:55.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6af05fcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 0x55a4c0b69860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6af05fcb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6af05fcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b6f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6af05fcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc1b13c0 con 0x55a4c0b656a0 2024-05-09T00:13:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6af05fcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc1aec50 con 0x55a4c09e84a0 2024-05-09T00:13:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6af05fcb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc1acaa0 con 0x55a4c0ad2160 2024-05-09T00:13:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b6f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b6f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34380/0 (socket says 172.21.15.46:34380) 2024-05-09T00:13:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6adbfff700 1 -- 172.21.15.46:0/4243707447 learned_addr learned my addr 172.21.15.46:0/4243707447 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6adaffd700 1 --2- 172.21.15.46:0/4243707447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6adb7fe700 1 --2- 172.21.15.46:0/4243707447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 0x55a4c0b69860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6adbfff700 1 -- 172.21.15.46:0/4243707447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0ad2160 msgr2=0x55a4c0b6a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6adbfff700 1 --2- 172.21.15.46:0/4243707447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6adbfff700 1 -- 172.21.15.46:0/4243707447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 msgr2=0x55a4c0b69860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6adbfff700 1 --2- 172.21.15.46:0/4243707447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 0x55a4c0b69860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6adbfff700 1 -- 172.21.15.46:0/4243707447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4c0b608f0 con 0x55a4c0b656a0 2024-05-09T00:13:55.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.228+0000 7f6adbfff700 1 --2- 172.21.15.46:0/4243707447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b6f160 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009d20 tx=0x7f6ad40092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248157a43d3cea26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:55.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.229+0000 7f6ac17fa700 1 -- 172.21.15.46:0/4243707447 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad401e420 con 0x55a4c0b656a0 2024-05-09T00:13:55.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.229+0000 7f6ac17fa700 1 -- 172.21.15.46:0/4243707447 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ad401ea70 con 0x55a4c0b656a0 2024-05-09T00:13:55.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.229+0000 7f6ac17fa700 1 -- 172.21.15.46:0/4243707447 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4014a50 con 0x55a4c0b656a0 2024-05-09T00:13:55.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.229+0000 7f6af05fcb80 1 -- 172.21.15.46:0/4243707447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 msgr2=0x55a4c0b6f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:55.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.229+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/4243707447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b6f160 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009d20 tx=0x7f6ad40092b0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.230+0000 7f6af05fcb80 1 -- 172.21.15.46:0/4243707447 shutdown_connections 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.230+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/4243707447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b6f160 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.230+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/4243707447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 0x55a4c0b69860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.230+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/4243707447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0ad2160 0x55a4c0b6a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.230+0000 7f6af05fcb80 1 -- 172.21.15.46:0/4243707447 >> 172.21.15.46:0/4243707447 conn(0x55a4c0a486a0 msgr2=0x55a4c0b6a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.230+0000 7f6af05fcb80 1 -- 172.21.15.46:0/4243707447 shutdown_connections 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.230+0000 7f6af05fcb80 1 -- 172.21.15.46:0/4243707447 wait complete. 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.230+0000 7f6af05fcb80 1 -- start start 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.230+0000 7f6af05fcb80 1 Processor -- start 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6af05fcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 0x55a4c0b86170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6af05fcb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0ad2160 0x55a4c0b866b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6af05fcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b76390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6af05fcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc1ad110 con 0x55a4c0b656a0 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6af05fcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc1ae580 con 0x55a4c09e84a0 2024-05-09T00:13:55.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6af05fcb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc1ad7e0 con 0x55a4c0ad2160 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6af05fcb80 10 client.0 _ll_register_callbacks cb 0x55a4c08f96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b76390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6adaffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0ad2160 0x55a4c0b866b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6adb7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 0x55a4c0b86170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b76390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34388/0 (socket says 172.21.15.46:34388) 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6adbfff700 1 -- 172.21.15.46:0/1079453353 learned_addr learned my addr 172.21.15.46:0/1079453353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6adbfff700 1 -- 172.21.15.46:0/1079453353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0ad2160 msgr2=0x55a4c0b866b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6adbfff700 1 --2- 172.21.15.46:0/1079453353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0ad2160 0x55a4c0b866b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6adbfff700 1 -- 172.21.15.46:0/1079453353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 msgr2=0x55a4c0b86170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6adbfff700 1 --2- 172.21.15.46:0/1079453353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 0x55a4c0b86170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6adbfff700 1 -- 172.21.15.46:0/1079453353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4c0b768d0 con 0x55a4c0b656a0 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6adaffd700 1 --2- 172.21.15.46:0/1079453353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0ad2160 0x55a4c0b866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6adbfff700 1 --2- 172.21.15.46:0/1079453353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b76390 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009cf0 tx=0x7f6ad4004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8b3a8315335424 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:55.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.231+0000 7f6abaffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6abaffd700 1 -- 172.21.15.46:0/1079453353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad401e830 con 0x55a4c0b656a0 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6abaffd700 1 -- 172.21.15.46:0/1079453353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6ad4014690 con 0x55a4c0b656a0 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6af05fcb80 1 -- 172.21.15.46:0/1079453353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad401ea70 con 0x55a4c0b656a0 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6abaffd700 1 -- 172.21.15.46:0/1079453353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ad4014480 con 0x55a4c0b656a0 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6af05fcb80 10 client.15582 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6af05fcb80 1 -- 172.21.15.46:0/1079453353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a4c0b77b30 con 0x55a4c0b656a0 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6abaffd700 1 -- 172.21.15.46:0/1079453353 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ad401ea70 con 0x55a4c0b656a0 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6abaffd700 1 -- 172.21.15.46:0/1079453353 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6ad40338f0 con 0x55a4c0b656a0 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6af05fcb80 10 client.15582 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6af05fcb80 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6af05fcb80 20 client.15582 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/1079453353 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a4c0be9a70 0x55a4c0bebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6af05fcb80 4 client.15582 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.232+0000 7f6af05fcb80 1 -- 172.21.15.46:0/1079453353 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a4c0a8fa70 con 0x55a4c0be9a70 2024-05-09T00:13:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.233+0000 7f6adb7fe700 1 --2- 172.21.15.46:0/1079453353 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a4c0be9a70 0x55a4c0bebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:55.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.233+0000 7f6abaffd700 10 client.15582 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:55.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.233+0000 7f6adb7fe700 1 --2- 172.21.15.46:0/1079453353 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a4c0be9a70 0x55a4c0bebe90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5f5409898f57d6a server_cookie=468134147f00da58 in_seq=0 out_seq=0 2024-05-09T00:13:55.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.233+0000 7f6abaffd700 1 -- 172.21.15.46:0/1079453353 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a4c0a8fa70 con 0x55a4c0be9a70 2024-05-09T00:13:55.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.233+0000 7f6abaffd700 10 client.15582 handle_command_reply: tid=0 2024-05-09T00:13:55.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.233+0000 7f6af05fcb80 1 client.15582 shutdown 2024-05-09T00:13:55.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6af05fcb80 1 -- 172.21.15.46:0/1079453353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 msgr2=0x55a4c0b76390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:55.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/1079453353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b76390 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009cf0 tx=0x7f6ad4004ce0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6af05fcb80 1 -- 172.21.15.46:0/1079453353 shutdown_connections 2024-05-09T00:13:55.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/1079453353 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a4c0be9a70 0x55a4c0bebe90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6abaffd700 0 client.15582 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:55.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/1079453353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c0b656a0 0x55a4c0b76390 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/1079453353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4c09e84a0 0x55a4c0b86170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6af05fcb80 1 --2- 172.21.15.46:0/1079453353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4c0ad2160 0x55a4c0b866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6af05fcb80 1 -- 172.21.15.46:0/1079453353 >> 172.21.15.46:0/1079453353 conn(0x55a4c0a486a0 msgr2=0x55a4c0b68e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:55.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6af05fcb80 1 -- 172.21.15.46:0/1079453353 shutdown_connections 2024-05-09T00:13:55.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6af05fcb80 1 -- 172.21.15.46:0/1079453353 wait complete. 2024-05-09T00:13:55.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6af05fcb80 20 client.15582 trim_cache size 0 max 16384 2024-05-09T00:13:55.236 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:13:55.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6ae1be6700 1 -- 172.21.15.46:0/3730107401 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6ac40b2880 msgr2=0x7f6ac40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:55.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6ae1be6700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6ac40b2880 0x7f6ac40b4d90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6adc101e60 tx=0x7f6ad000b410 comp rx=0 tx=0).stop 2024-05-09T00:13:55.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6ae1be6700 1 -- 172.21.15.46:0/3730107401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1034e0 msgr2=0x7f6adc19bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:55.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6ae1be6700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1034e0 0x7f6adc19bc50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00b580 tx=0x7f6acc0047b0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6ae1be6700 1 -- 172.21.15.46:0/3730107401 shutdown_connections 2024-05-09T00:13:55.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6ae1be6700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6ac40b2880 0x7f6ac40b4d90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6ae1be6700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6adc108760 0x7f6adc1a6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6ae1be6700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1034e0 0x7f6adc19bc50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6ae1be6700 1 --2- 172.21.15.46:0/3730107401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6adc107e10 0x7f6adc19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:55.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.234+0000 7f6ae1be6700 1 -- 172.21.15.46:0/3730107401 >> 172.21.15.46:0/3730107401 conn(0x7f6adc0fef30 msgr2=0x7f6adc1063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:55.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.235+0000 7f6ae1be6700 1 -- 172.21.15.46:0/3730107401 shutdown_connections 2024-05-09T00:13:55.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:55.235+0000 7f6ae1be6700 1 -- 172.21.15.46:0/3730107401 wait complete. 2024-05-09T00:13:56.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:56 smithi046 ceph-mon[30151]: pgmap v453: 97 pgs: 97 active+clean; 844 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:13:56.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:56 smithi142 ceph-mon[34533]: pgmap v453: 97 pgs: 97 active+clean; 844 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:13:57.000 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:13:57.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:13:57.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:56 smithi169 ceph-mon[34777]: pgmap v453: 97 pgs: 97 active+clean; 844 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:13:57.267 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:13:58.705 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:13:58 smithi046 ceph-mon[30151]: pgmap v454: 97 pgs: 97 active+clean; 844 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:13:58.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:13:58 smithi142 ceph-mon[34533]: pgmap v454: 97 pgs: 97 active+clean; 844 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:13:59.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.093+0000 7f00d4f03700 1 -- 172.21.15.46:0/518379925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d0108740 msgr2=0x7f00d01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:59.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.093+0000 7f00d4f03700 1 --2- 172.21.15.46:0/518379925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d0108740 0x7f00d01100a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f00c800b0d0 tx=0x7f00c800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.093+0000 7f00d4f03700 1 -- 172.21.15.46:0/518379925 shutdown_connections 2024-05-09T00:13:59.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.093+0000 7f00d4f03700 1 --2- 172.21.15.46:0/518379925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d0108740 0x7f00d01100a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.093+0000 7f00d4f03700 1 --2- 172.21.15.46:0/518379925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00d01034c0 0x7f00d01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.093+0000 7f00d4f03700 1 --2- 172.21.15.46:0/518379925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d0107df0 0x7f00d0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.093+0000 7f00d4f03700 1 -- 172.21.15.46:0/518379925 >> 172.21.15.46:0/518379925 conn(0x7f00d00fedf0 msgr2=0x7f00d0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.093+0000 7f00d4f03700 1 -- 172.21.15.46:0/518379925 shutdown_connections 2024-05-09T00:13:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.093+0000 7f00d4f03700 1 -- 172.21.15.46:0/518379925 wait complete. 2024-05-09T00:13:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.094+0000 7f00d4f03700 1 Processor -- start 2024-05-09T00:13:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.094+0000 7f00d4f03700 1 -- start start 2024-05-09T00:13:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.094+0000 7f00d4f03700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d01034c0 0x7f00d019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.094+0000 7f00d4f03700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00d0107df0 0x7f00d019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.094+0000 7f00d4f03700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d0108740 0x7f00d019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.094+0000 7f00d4f03700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f00d0113370 con 0x7f00d01034c0 2024-05-09T00:13:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.094+0000 7f00d4f03700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01131f0 con 0x7f00d0107df0 2024-05-09T00:13:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.094+0000 7f00d4f03700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01134f0 con 0x7f00d0108740 2024-05-09T00:13:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.094+0000 7f00ce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d01034c0 0x7f00d019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.094+0000 7f00ce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d01034c0 0x7f00d019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49476/0 (socket says 172.21.15.46:49476) 2024-05-09T00:13:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.094+0000 7f00ce59c700 1 -- 172.21.15.46:0/4256841292 learned_addr learned my addr 172.21.15.46:0/4256841292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.095+0000 7f00ce59c700 1 -- 172.21.15.46:0/4256841292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d0108740 msgr2=0x7f00d019f5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:13:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.095+0000 7f00ce59c700 1 --2- 172.21.15.46:0/4256841292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d0108740 0x7f00d019f5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.095+0000 7f00ce59c700 1 -- 172.21.15.46:0/4256841292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00d0107df0 msgr2=0x7f00d019b330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T00:13:59.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.095+0000 7f00ce59c700 1 --2- 172.21.15.46:0/4256841292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00d0107df0 0x7f00d019b330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.095+0000 7f00ce59c700 1 -- 172.21.15.46:0/4256841292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d019fbb0 con 0x7f00d01034c0 2024-05-09T00:13:59.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.095+0000 7f00ce59c700 1 --2- 172.21.15.46:0/4256841292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d01034c0 0x7f00d019ad90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c9b0 tx=0x7f00c000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:59.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.095+0000 7f00bf7fe700 1 -- 172.21.15.46:0/4256841292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0007b00 con 0x7f00d01034c0 2024-05-09T00:13:59.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.095+0000 7f00d4f03700 1 -- 172.21.15.46:0/4256841292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d019fea0 con 0x7f00d01034c0 2024-05-09T00:13:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.095+0000 7f00bf7fe700 1 -- 172.21.15.46:0/4256841292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c0007ca0 con 0x7f00d01034c0 2024-05-09T00:13:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.095+0000 7f00bf7fe700 1 -- 172.21.15.46:0/4256841292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0018730 con 0x7f00d01034c0 2024-05-09T00:13:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.095+0000 7f00d4f03700 1 -- 172.21.15.46:0/4256841292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d01a03e0 con 0x7f00d01034c0 2024-05-09T00:13:59.109 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:13:58 smithi169 ceph-mon[34777]: pgmap v454: 97 pgs: 97 active+clean; 844 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:13:59.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.098+0000 7f00bf7fe700 1 -- 172.21.15.46:0/4256841292 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f00c00188d0 con 0x7f00d01034c0 2024-05-09T00:13:59.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.099+0000 7f00bf7fe700 1 --2- 172.21.15.46:0/4256841292 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f00b80b2880 0x7f00b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.099+0000 7f00e3919b80 1 Processor -- start 2024-05-09T00:13:59.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.099+0000 7f00cdd9b700 1 --2- 172.21.15.46:0/4256841292 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f00b80b2880 0x7f00b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.099+0000 7f00bf7fe700 1 -- 172.21.15.46:0/4256841292 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f00c00e22f0 con 0x7f00d01034c0 2024-05-09T00:13:59.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.099+0000 7f00e3919b80 1 -- start start 2024-05-09T00:13:59.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.099+0000 7f00e3919b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee90d80 0x56353ef992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00e3919b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 0x56353ef99800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00e3919b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ef93480 0x56353ef93890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00e3919b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56353eeb6120 con 0x56353ee16520 2024-05-09T00:13:59.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00e3919b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56353ee5ba20 con 0x56353ef93480 2024-05-09T00:13:59.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00e3919b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56353eeb20e0 con 0x56353ee90d80 2024-05-09T00:13:59.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00cdd9b700 1 --2- 172.21.15.46:0/4256841292 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f00b80b2880 0x7f00b80b4d90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00d0109890 tx=0x7f00c400b370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00cdd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 0x56353ef99800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00cdd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 0x56353ef99800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49490/0 (socket says 172.21.15.46:49490) 2024-05-09T00:13:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00cdd9b700 1 -- 172.21.15.46:0/3141449827 learned_addr learned my addr 172.21.15.46:0/3141449827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00ce59c700 1 --2- 172.21.15.46:0/3141449827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee90d80 0x56353ef992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00ced9d700 1 --2- 172.21.15.46:0/3141449827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ef93480 0x56353ef93890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00cdd9b700 1 -- 172.21.15.46:0/3141449827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee90d80 msgr2=0x56353ef992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00cdd9b700 1 --2- 172.21.15.46:0/3141449827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee90d80 0x56353ef992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00cdd9b700 1 -- 172.21.15.46:0/3141449827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ef93480 msgr2=0x56353ef93890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00cdd9b700 1 --2- 172.21.15.46:0/3141449827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ef93480 0x56353ef93890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00cdd9b700 1 -- 172.21.15.46:0/3141449827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56353ee4c580 con 0x56353ee16520 2024-05-09T00:13:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00cdd9b700 1 --2- 172.21.15.46:0/3141449827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 0x56353ef99800 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f00c400db30 tx=0x7f00c4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a078e81141b679b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00bcff9700 1 -- 172.21.15.46:0/3141449827 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4010af0 con 0x56353ee16520 2024-05-09T00:13:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00bcff9700 1 -- 172.21.15.46:0/3141449827 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c4022070 con 0x56353ee16520 2024-05-09T00:13:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.100+0000 7f00bcff9700 1 -- 172.21.15.46:0/3141449827 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c400f3c0 con 0x56353ee16520 2024-05-09T00:13:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.101+0000 7f00e3919b80 1 -- 172.21.15.46:0/3141449827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 msgr2=0x56353ef99800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.101+0000 7f00e3919b80 1 --2- 172.21.15.46:0/3141449827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 0x56353ef99800 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f00c400db30 tx=0x7f00c4010000 comp rx=0 tx=0).stop 2024-05-09T00:13:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.101+0000 7f00e3919b80 1 -- 172.21.15.46:0/3141449827 shutdown_connections 2024-05-09T00:13:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.101+0000 7f00e3919b80 1 --2- 172.21.15.46:0/3141449827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ef93480 0x56353ef93890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.101+0000 7f00e3919b80 1 --2- 172.21.15.46:0/3141449827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee90d80 0x56353ef992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.101+0000 7f00e3919b80 1 --2- 172.21.15.46:0/3141449827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 0x56353ef99800 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.101+0000 7f00e3919b80 1 -- 172.21.15.46:0/3141449827 >> 172.21.15.46:0/3141449827 conn(0x56353ee19ff0 msgr2=0x56353ef8e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.102+0000 7f00e3919b80 1 -- 172.21.15.46:0/3141449827 shutdown_connections 2024-05-09T00:13:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.102+0000 7f00e3919b80 1 -- 172.21.15.46:0/3141449827 wait complete. 2024-05-09T00:13:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.102+0000 7f00e3919b80 1 -- start start 2024-05-09T00:13:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.102+0000 7f00e3919b80 1 Processor -- start 2024-05-09T00:13:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.102+0000 7f00e3919b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee16520 0x56353ef9d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.102+0000 7f00e3919b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee90d80 0x56353ef9d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.102+0000 7f00e3919b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 0x56353ef9dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.102+0000 7f00e3919b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56353ef94180 con 0x56353ef93480 2024-05-09T00:13:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.102+0000 7f00e3919b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56353ee50ba0 con 0x56353ee16520 2024-05-09T00:13:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.102+0000 7f00e3919b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56353efa9250 con 0x56353ee90d80 2024-05-09T00:13:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.102+0000 7f00e3919b80 10 client.0 _ll_register_callbacks cb 0x56353ed27740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00ced9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 0x56353ef9dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00ced9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 0x56353ef9dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49494/0 (socket says 172.21.15.46:49494) 2024-05-09T00:13:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00ced9d700 1 -- 172.21.15.46:0/1433327446 learned_addr learned my addr 172.21.15.46:0/1433327446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00ce59c700 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee16520 0x56353ef9d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00cdd9b700 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee90d80 0x56353ef9d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00ced9d700 1 -- 172.21.15.46:0/1433327446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee90d80 msgr2=0x56353ef9d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00ced9d700 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee90d80 0x56353ef9d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00ced9d700 1 -- 172.21.15.46:0/1433327446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee16520 msgr2=0x56353ef9d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00ced9d700 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee16520 0x56353ef9d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00ced9d700 1 -- 172.21.15.46:0/1433327446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56353ed31510 con 0x56353ef93480 2024-05-09T00:13:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00ced9d700 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 0x56353ef9dad0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f00c8009e20 tx=0x7f00c8004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fe56380ba5567d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00b5ffb700 1 -- 172.21.15.46:0/1433327446 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8020030 con 0x56353ef93480 2024-05-09T00:13:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00b5ffb700 1 -- 172.21.15.46:0/1433327446 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f00c80075c0 con 0x56353ef93480 2024-05-09T00:13:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00e3919b80 1 -- 172.21.15.46:0/1433327446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c4022070 con 0x56353ef93480 2024-05-09T00:13:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00b5ffb700 1 -- 172.21.15.46:0/1433327446 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00c80077c0 con 0x56353ef93480 2024-05-09T00:13:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00e3919b80 10 client.15594 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.103+0000 7f00e3919b80 1 -- 172.21.15.46:0/1433327446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56353efaab40 con 0x56353ef93480 2024-05-09T00:13:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.104+0000 7f00b5ffb700 1 -- 172.21.15.46:0/1433327446 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c4022070 con 0x56353ef93480 2024-05-09T00:13:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.104+0000 7f00b5ffb700 1 -- 172.21.15.46:0/1433327446 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f00c8011070 con 0x56353ef93480 2024-05-09T00:13:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.104+0000 7f00e3919b80 10 client.15594 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.104+0000 7f00e3919b80 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.104+0000 7f00e3919b80 20 client.15594 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.104+0000 7f00e3919b80 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56353efad0d0 0x56353efaf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.104+0000 7f00e3919b80 4 client.15594 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:13:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.104+0000 7f00e3919b80 1 -- 172.21.15.46:0/1433327446 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56353eec3b00 con 0x56353efad0d0 2024-05-09T00:13:59.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.104+0000 7f00ce59c700 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56353efad0d0 0x56353efaf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.105+0000 7f00ce59c700 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56353efad0d0 0x56353efaf530 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5a8320bf43e592d server_cookie=b3dfd80c0a455d2a in_seq=0 out_seq=0 2024-05-09T00:13:59.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.105+0000 7f00b5ffb700 10 client.15594 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:59.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.105+0000 7f00b5ffb700 1 -- 172.21.15.46:0/1433327446 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56353eec3b00 con 0x56353efad0d0 2024-05-09T00:13:59.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.105+0000 7f00b5ffb700 10 client.15594 handle_command_reply: tid=0 2024-05-09T00:13:59.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00e3919b80 1 client.15594 shutdown 2024-05-09T00:13:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00e3919b80 1 -- 172.21.15.46:0/1433327446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 msgr2=0x56353ef9dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00e3919b80 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 0x56353ef9dad0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f00c8009e20 tx=0x7f00c8004a10 comp rx=0 tx=0).stop 2024-05-09T00:13:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00e3919b80 1 -- 172.21.15.46:0/1433327446 shutdown_connections 2024-05-09T00:13:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00e3919b80 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56353efad0d0 0x56353efaf530 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00e3919b80 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 0x56353ef9dad0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00e3919b80 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee16520 0x56353ef9d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00b5ffb700 0 client.15594 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00e3919b80 1 --2- 172.21.15.46:0/1433327446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee90d80 0x56353ef9d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00e3919b80 1 -- 172.21.15.46:0/1433327446 >> 172.21.15.46:0/1433327446 conn(0x56353ee19ff0 msgr2=0x56353ef88070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00e3919b80 1 -- 172.21.15.46:0/1433327446 shutdown_connections 2024-05-09T00:13:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00e3919b80 1 -- 172.21.15.46:0/1433327446 wait complete. 2024-05-09T00:13:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.106+0000 7f00e3919b80 20 client.15594 trim_cache size 0 max 16384 2024-05-09T00:13:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.126+0000 7f00e3919b80 1 Processor -- start 2024-05-09T00:13:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.126+0000 7f00e3919b80 1 -- start start 2024-05-09T00:13:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00e3919b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee16520 0x56353ef92ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00e3919b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee90d80 0x56353ef9c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00e3919b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 0x56353ef9d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00e3919b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01b1470 con 0x56353ef93480 2024-05-09T00:13:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00e3919b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01aed00 con 0x56353ee90d80 2024-05-09T00:13:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00e3919b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01acb30 con 0x56353ee16520 2024-05-09T00:13:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00ced9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 0x56353ef9d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00ced9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 0x56353ef9d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49504/0 (socket says 172.21.15.46:49504) 2024-05-09T00:13:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00ced9d700 1 -- 172.21.15.46:0/289058707 learned_addr learned my addr 172.21.15.46:0/289058707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00ce59c700 1 --2- 172.21.15.46:0/289058707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee16520 0x56353ef92ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00cdd9b700 1 --2- 172.21.15.46:0/289058707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee90d80 0x56353ef9c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00ced9d700 1 -- 172.21.15.46:0/289058707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee16520 msgr2=0x56353ef92ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00ced9d700 1 --2- 172.21.15.46:0/289058707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee16520 0x56353ef92ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00ced9d700 1 -- 172.21.15.46:0/289058707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee90d80 msgr2=0x56353ef9c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00ced9d700 1 --2- 172.21.15.46:0/289058707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee90d80 0x56353ef9c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00ced9d700 1 -- 172.21.15.46:0/289058707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56353ef99b70 con 0x56353ef93480 2024-05-09T00:13:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00cdd9b700 1 --2- 172.21.15.46:0/289058707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee90d80 0x56353ef9c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:13:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.127+0000 7f00ced9d700 1 --2- 172.21.15.46:0/289058707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 0x56353ef9d010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f00c801ca00 tx=0x7f00c800b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118e30df561cd2c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.128+0000 7f00b7fff700 1 -- 172.21.15.46:0/289058707 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8020070 con 0x56353ef93480 2024-05-09T00:13:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.128+0000 7f00b7fff700 1 -- 172.21.15.46:0/289058707 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c801c680 con 0x56353ef93480 2024-05-09T00:13:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.128+0000 7f00b7fff700 1 -- 172.21.15.46:0/289058707 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8025410 con 0x56353ef93480 2024-05-09T00:13:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.128+0000 7f00e3919b80 1 -- 172.21.15.46:0/289058707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 msgr2=0x56353ef9d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.128+0000 7f00e3919b80 1 --2- 172.21.15.46:0/289058707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 0x56353ef9d010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f00c801ca00 tx=0x7f00c800b800 comp rx=0 tx=0).stop 2024-05-09T00:13:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.128+0000 7f00e3919b80 1 -- 172.21.15.46:0/289058707 shutdown_connections 2024-05-09T00:13:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.128+0000 7f00e3919b80 1 --2- 172.21.15.46:0/289058707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ef93480 0x56353ef9d010 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.128+0000 7f00e3919b80 1 --2- 172.21.15.46:0/289058707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ee16520 0x56353ef92ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.128+0000 7f00e3919b80 1 --2- 172.21.15.46:0/289058707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee90d80 0x56353ef9c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.128+0000 7f00e3919b80 1 -- 172.21.15.46:0/289058707 >> 172.21.15.46:0/289058707 conn(0x56353ee19ff0 msgr2=0x56353ef886e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.128+0000 7f00e3919b80 1 -- 172.21.15.46:0/289058707 shutdown_connections 2024-05-09T00:13:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.128+0000 7f00e3919b80 1 -- 172.21.15.46:0/289058707 wait complete. 2024-05-09T00:13:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.129+0000 7f00e3919b80 1 -- start start 2024-05-09T00:13:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.129+0000 7f00e3919b80 1 Processor -- start 2024-05-09T00:13:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.129+0000 7f00e3919b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 0x56353efa53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00e3919b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee90d80 0x56353efa1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00e3919b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ef93480 0x56353efa1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00e3919b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01ad1c0 con 0x56353ee16520 2024-05-09T00:13:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00e3919b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01ae630 con 0x56353ee90d80 2024-05-09T00:13:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00e3919b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01ad890 con 0x56353ef93480 2024-05-09T00:13:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00e3919b80 10 client.0 _ll_register_callbacks cb 0x56353ef93e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:13:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00ce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 0x56353efa53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00ce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 0x56353efa53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49508/0 (socket says 172.21.15.46:49508) 2024-05-09T00:13:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00ce59c700 1 -- 172.21.15.46:0/1889702912 learned_addr learned my addr 172.21.15.46:0/1889702912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:13:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00ced9d700 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ef93480 0x56353efa1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00cdd9b700 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee90d80 0x56353efa1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00ce59c700 1 -- 172.21.15.46:0/1889702912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ef93480 msgr2=0x56353efa1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00ce59c700 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ef93480 0x56353efa1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00ce59c700 1 -- 172.21.15.46:0/1889702912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee90d80 msgr2=0x56353efa1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:13:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00ce59c700 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee90d80 0x56353efa1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00ce59c700 1 -- 172.21.15.46:0/1889702912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56353efa21a0 con 0x56353ee16520 2024-05-09T00:13:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00cdd9b700 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee90d80 0x56353efa1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:13:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00ce59c700 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 0x56353efa53b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f00c000afd0 tx=0x7f00c00b2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebc1c08c4ba2dc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:13:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.130+0000 7f00b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:13:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00b5ffb700 1 -- 172.21.15.46:0/1889702912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c006f060 con 0x56353ee16520 2024-05-09T00:13:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00b5ffb700 1 -- 172.21.15.46:0/1889702912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f00c00a5ba0 con 0x56353ee16520 2024-05-09T00:13:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00e3919b80 1 -- 172.21.15.46:0/1889702912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c801c680 con 0x56353ee16520 2024-05-09T00:13:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00b5ffb700 1 -- 172.21.15.46:0/1889702912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00c00c2070 con 0x56353ee16520 2024-05-09T00:13:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00e3919b80 10 client.15600 fetch_fsmap learned FSMap version 26 2024-05-09T00:13:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00e3919b80 1 -- 172.21.15.46:0/1889702912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56353f0184b0 con 0x56353ee16520 2024-05-09T00:13:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00b5ffb700 1 -- 172.21.15.46:0/1889702912 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c801c680 con 0x56353ee16520 2024-05-09T00:13:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00b5ffb700 1 -- 172.21.15.46:0/1889702912 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f00c00c9070 con 0x56353ee16520 2024-05-09T00:13:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00e3919b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:13:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00e3919b80 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:13:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00e3919b80 20 client.15600 populate_metadata read hostname 'smithi046' 2024-05-09T00:13:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00e3919b80 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56353f018b60 0x56353f01af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:13:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00e3919b80 4 client.15600 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:13:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00e3919b80 1 -- 172.21.15.46:0/1889702912 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56353eebdba0 con 0x56353f018b60 2024-05-09T00:13:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.131+0000 7f00cdd9b700 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56353f018b60 0x56353f01af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:13:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.132+0000 7f00cdd9b700 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56353f018b60 0x56353f01af80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82aec967b7988002 server_cookie=1942e4890848862a in_seq=0 out_seq=0 2024-05-09T00:13:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.132+0000 7f00b5ffb700 10 client.15600 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.133+0000 7f00b5ffb700 1 -- 172.21.15.46:0/1889702912 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56353eebdba0 con 0x56353f018b60 2024-05-09T00:13:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.133+0000 7f00b5ffb700 10 client.15600 handle_command_reply: tid=0 2024-05-09T00:13:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.133+0000 7f00e3919b80 1 client.15600 shutdown 2024-05-09T00:13:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.133+0000 7f00e3919b80 1 -- 172.21.15.46:0/1889702912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 msgr2=0x56353efa53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.133+0000 7f00e3919b80 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 0x56353efa53b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f00c000afd0 tx=0x7f00c00b2040 comp rx=0 tx=0).stop 2024-05-09T00:13:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.133+0000 7f00e3919b80 1 -- 172.21.15.46:0/1889702912 shutdown_connections 2024-05-09T00:13:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.133+0000 7f00e3919b80 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56353f018b60 0x56353f01af80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.133+0000 7f00e3919b80 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56353ef93480 0x56353efa1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.133+0000 7f00e3919b80 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56353ee16520 0x56353efa53b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.133+0000 7f00b5ffb700 0 client.15600 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:13:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.133+0000 7f00e3919b80 1 --2- 172.21.15.46:0/1889702912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56353ee90d80 0x56353efa1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.133+0000 7f00e3919b80 1 -- 172.21.15.46:0/1889702912 >> 172.21.15.46:0/1889702912 conn(0x56353ee19ff0 msgr2=0x56353ef9d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00e3919b80 1 -- 172.21.15.46:0/1889702912 shutdown_connections 2024-05-09T00:13:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00e3919b80 1 -- 172.21.15.46:0/1889702912 wait complete. 2024-05-09T00:13:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00e3919b80 20 client.15600 trim_cache size 0 max 16384 2024-05-09T00:13:59.155 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:13:59.155 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:13:59.155 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "70a4fdba-f22a-432a-8fe5-c22d2944ab57", 2024-05-09T00:13:59.155 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:13:59.155 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:13:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00d4f03700 1 -- 172.21.15.46:0/4256841292 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f00b80b2880 msgr2=0x7f00b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00d4f03700 1 --2- 172.21.15.46:0/4256841292 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f00b80b2880 0x7f00b80b4d90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00d0109890 tx=0x7f00c400b370 comp rx=0 tx=0).stop 2024-05-09T00:13:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00d4f03700 1 -- 172.21.15.46:0/4256841292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d01034c0 msgr2=0x7f00d019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:13:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00d4f03700 1 --2- 172.21.15.46:0/4256841292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d01034c0 0x7f00d019ad90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c9b0 tx=0x7f00c000cd70 comp rx=0 tx=0).stop 2024-05-09T00:13:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00d4f03700 1 -- 172.21.15.46:0/4256841292 shutdown_connections 2024-05-09T00:13:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00d4f03700 1 --2- 172.21.15.46:0/4256841292 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f00b80b2880 0x7f00b80b4d90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00d4f03700 1 --2- 172.21.15.46:0/4256841292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00d0108740 0x7f00d019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00d4f03700 1 --2- 172.21.15.46:0/4256841292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00d01034c0 0x7f00d019ad90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00d4f03700 1 --2- 172.21.15.46:0/4256841292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00d0107df0 0x7f00d019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:13:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00d4f03700 1 -- 172.21.15.46:0/4256841292 >> 172.21.15.46:0/4256841292 conn(0x7f00d00fedf0 msgr2=0x7f00d0075c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:13:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00d4f03700 1 -- 172.21.15.46:0/4256841292 shutdown_connections 2024-05-09T00:13:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:13:59.134+0000 7f00d4f03700 1 -- 172.21.15.46:0/4256841292 wait complete. 2024-05-09T00:13:59.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:14:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:00.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:00 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:14:00.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:00 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:00.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:00 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:14:00.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:00 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:00.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:00 smithi046 ceph-mon[30151]: pgmap v455: 97 pgs: 97 active+clean; 904 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:00.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:00 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:14:00.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:00 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:00.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:00 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:14:00.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:00 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:00.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:00 smithi142 ceph-mon[34533]: pgmap v455: 97 pgs: 97 active+clean; 904 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:01.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:00 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:14:01.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:00 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:01.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:00 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:14:01.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:00 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:01.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:00 smithi169 ceph-mon[34777]: pgmap v455: 97 pgs: 97 active+clean; 904 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:01.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.988+0000 7f1d20426700 1 -- 172.21.15.46:0/2365323227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d18108f00 msgr2=0x7f1d18109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:01.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.988+0000 7f1d20426700 1 --2- 172.21.15.46:0/2365323227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d18108f00 0x7f1d18109310 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1d080099c0 tx=0x7f1d08009cd0 comp rx=0 tx=0).stop 2024-05-09T00:14:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.988+0000 7f1d20426700 1 -- 172.21.15.46:0/2365323227 shutdown_connections 2024-05-09T00:14:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.988+0000 7f1d20426700 1 --2- 172.21.15.46:0/2365323227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d18109850 0x7f1d181111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.988+0000 7f1d20426700 1 --2- 172.21.15.46:0/2365323227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d181045d0 0x7f1d181049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.988+0000 7f1d20426700 1 --2- 172.21.15.46:0/2365323227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d18108f00 0x7f1d18109310 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.988+0000 7f1d20426700 1 -- 172.21.15.46:0/2365323227 >> 172.21.15.46:0/2365323227 conn(0x7f1d180ffea0 msgr2=0x7f1d181022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.988+0000 7f1d20426700 1 -- 172.21.15.46:0/2365323227 shutdown_connections 2024-05-09T00:14:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.988+0000 7f1d20426700 1 -- 172.21.15.46:0/2365323227 wait complete. 2024-05-09T00:14:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.989+0000 7f1d20426700 1 Processor -- start 2024-05-09T00:14:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.989+0000 7f1d20426700 1 -- start start 2024-05-09T00:14:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d20426700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d181045d0 0x7f1d1819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d20426700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d18108f00 0x7f1d1819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d20426700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d18109850 0x7f1d1819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d20426700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d18114460 con 0x7f1d181045d0 2024-05-09T00:14:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d20426700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181142e0 con 0x7f1d18109850 2024-05-09T00:14:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d20426700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181145e0 con 0x7f1d18108f00 2024-05-09T00:14:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d1e1c2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d181045d0 0x7f1d1819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d1e1c2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d181045d0 0x7f1d1819ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49524/0 (socket says 172.21.15.46:49524) 2024-05-09T00:14:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d1e1c2700 1 -- 172.21.15.46:0/3647824997 learned_addr learned my addr 172.21.15.46:0/3647824997 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:01.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d1d9c1700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d18108f00 0x7f1d1819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d1e1c2700 1 -- 172.21.15.46:0/3647824997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d18108f00 msgr2=0x7f1d1819b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d18108f00 0x7f1d1819b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d1e9c3700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d18109850 0x7f1d1819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d1e1c2700 1 -- 172.21.15.46:0/3647824997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d18109850 msgr2=0x7f1d1819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d18109850 0x7f1d1819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d1e1c2700 1 -- 172.21.15.46:0/3647824997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1819fd60 con 0x7f1d181045d0 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d1e9c3700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d18109850 0x7f1d1819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d181045d0 0x7f1d1819ae90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c920 tx=0x7f1d1400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.990+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/3647824997 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14007b00 con 0x7f1d181045d0 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.991+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/3647824997 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d14004ce0 con 0x7f1d181045d0 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.991+0000 7f1d20426700 1 -- 172.21.15.46:0/3647824997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d181a0050 con 0x7f1d181045d0 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.991+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/3647824997 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14005650 con 0x7f1d181045d0 2024-05-09T00:14:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.991+0000 7f1d20426700 1 -- 172.21.15.46:0/3647824997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d181a7970 con 0x7f1d181045d0 2024-05-09T00:14:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.993+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/3647824997 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1d1400f450 con 0x7f1d181045d0 2024-05-09T00:14:01.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.995+0000 7f1d0f7fe700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1d040b2880 0x7f1d040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:01.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.995+0000 7f1d1d9c1700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1d040b2880 0x7f1d040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:01.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.995+0000 7f1d2ee3cb80 1 Processor -- start 2024-05-09T00:14:01.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.995+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/3647824997 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1d140e1f40 con 0x7f1d181045d0 2024-05-09T00:14:01.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.995+0000 7f1d2ee3cb80 1 -- start start 2024-05-09T00:14:01.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.995+0000 7f1d1d9c1700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1d040b2880 0x7f1d040b4d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1d08000c00 tx=0x7f1d0801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:01.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.995+0000 7f1d2ee3cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef080ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:01.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d2ee3cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef06894a0 0x563ef080b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d2ee3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 0x563ef080f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d2ee3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563ef0728ff0 con 0x563ef080f400 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d2ee3cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ef06c77f0 con 0x563ef0773160 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d2ee3cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563ef0724fb0 con 0x563ef06894a0 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d1e9c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 0x563ef080f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d1e9c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 0x563ef080f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49526/0 (socket says 172.21.15.46:49526) 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d1e9c3700 1 -- 172.21.15.46:0/2809987050 learned_addr learned my addr 172.21.15.46:0/2809987050 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/2809987050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef080ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d1d9c1700 1 --2- 172.21.15.46:0/2809987050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef06894a0 0x563ef080b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d1e9c3700 1 -- 172.21.15.46:0/2809987050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef06894a0 msgr2=0x563ef080b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d1e9c3700 1 --2- 172.21.15.46:0/2809987050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef06894a0 0x563ef080b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d1e9c3700 1 -- 172.21.15.46:0/2809987050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 msgr2=0x563ef080ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d1e9c3700 1 --2- 172.21.15.46:0/2809987050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef080ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d1e9c3700 1 -- 172.21.15.46:0/2809987050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ef06bf370 con 0x563ef080f400 2024-05-09T00:14:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/2809987050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef080ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:01.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d1e9c3700 1 --2- 172.21.15.46:0/2809987050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 0x563ef080f810 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009cd0 tx=0x7f1d10007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae8aa4bf667d3e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:01.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.996+0000 7f1d0cd47700 1 -- 172.21.15.46:0/2809987050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10019070 con 0x563ef080f400 2024-05-09T00:14:01.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.997+0000 7f1d0cd47700 1 -- 172.21.15.46:0/2809987050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d10007e40 con 0x563ef080f400 2024-05-09T00:14:01.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.997+0000 7f1d0cd47700 1 -- 172.21.15.46:0/2809987050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10013970 con 0x563ef080f400 2024-05-09T00:14:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.997+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2809987050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 msgr2=0x563ef080f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.997+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/2809987050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 0x563ef080f810 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009cd0 tx=0x7f1d10007500 comp rx=0 tx=0).stop 2024-05-09T00:14:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.997+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2809987050 shutdown_connections 2024-05-09T00:14:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.997+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/2809987050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 0x563ef080f810 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.997+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/2809987050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef080ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.997+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/2809987050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef06894a0 0x563ef080b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.998+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2809987050 >> 172.21.15.46:0/2809987050 conn(0x563ef06e96a0 msgr2=0x563ef0801400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.998+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2809987050 shutdown_connections 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.998+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2809987050 wait complete. 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.998+0000 7f1d2ee3cb80 1 -- start start 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.998+0000 7f1d2ee3cb80 1 Processor -- start 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d2ee3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 0x563ef08145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d2ee3cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef0773160 0x563ef0814b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d2ee3cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef080f400 0x563ef0815070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d2ee3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563ef05a4ba0 con 0x563ef06894a0 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d2ee3cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ef06e3150 con 0x563ef080f400 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d2ee3cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563ef08101b0 con 0x563ef0773160 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d2ee3cb80 10 client.0 _ll_register_callbacks cb 0x563ef059a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d1e1c2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 0x563ef08145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d1e1c2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 0x563ef08145f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49538/0 (socket says 172.21.15.46:49538) 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d1e1c2700 1 -- 172.21.15.46:0/2864162721 learned_addr learned my addr 172.21.15.46:0/2864162721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d1d9c1700 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef0773160 0x563ef0814b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d1e1c2700 1 -- 172.21.15.46:0/2864162721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef0773160 msgr2=0x563ef0814b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef0773160 0x563ef0814b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d1e9c3700 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef080f400 0x563ef0815070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d1e1c2700 1 -- 172.21.15.46:0/2864162721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef080f400 msgr2=0x563ef0815070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef080f400 0x563ef0815070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d1e1c2700 1 -- 172.21.15.46:0/2864162721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ef05a4490 con 0x563ef06894a0 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d1e9c3700 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef080f400 0x563ef0815070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 0x563ef08145f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1d140a9220 tx=0x7f1d140a9100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8700994e54a19091 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d01ffb700 1 -- 172.21.15.46:0/2864162721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1401d030 con 0x563ef06894a0 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:01.999+0000 7f1d01ffb700 1 -- 172.21.15.46:0/2864162721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1d14007d30 con 0x563ef06894a0 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2864162721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563ef07369d0 con 0x563ef06894a0 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d01ffb700 1 -- 172.21.15.46:0/2864162721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d140ad8d0 con 0x563ef06894a0 2024-05-09T00:14:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d2ee3cb80 10 client.15612 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2864162721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ef0821ed0 con 0x563ef06894a0 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d01ffb700 1 -- 172.21.15.46:0/2864162721 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563ef07369d0 con 0x563ef06894a0 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d01ffb700 1 -- 172.21.15.46:0/2864162721 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1d140a81f0 con 0x563ef06894a0 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d2ee3cb80 10 client.15612 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d2ee3cb80 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d2ee3cb80 20 client.15612 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ef0824510 0x563ef0826990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d2ee3cb80 4 client.15612 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2864162721 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ef0730a70 con 0x563ef0824510 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.000+0000 7f1d1d9c1700 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ef0824510 0x563ef0826990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.001+0000 7f1d01ffb700 10 client.15612 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.001+0000 7f1d1d9c1700 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ef0824510 0x563ef0826990 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916bcd78b3506f0a server_cookie=5ca105b12c76df6c in_seq=0 out_seq=0 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d01ffb700 1 -- 172.21.15.46:0/2864162721 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ef0730a70 con 0x563ef0824510 2024-05-09T00:14:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d01ffb700 10 client.15612 handle_command_reply: tid=0 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d2ee3cb80 1 client.15612 shutdown 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2864162721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 msgr2=0x563ef08145f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 0x563ef08145f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1d140a9220 tx=0x7f1d140a9100 comp rx=0 tx=0).stop 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2864162721 shutdown_connections 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ef0824510 0x563ef0826990 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef080f400 0x563ef0815070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 0x563ef08145f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d01ffb700 0 client.15612 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/2864162721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef0773160 0x563ef0814b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2864162721 >> 172.21.15.46:0/2864162721 conn(0x563ef06e96a0 msgr2=0x563ef08011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2864162721 shutdown_connections 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/2864162721 wait complete. 2024-05-09T00:14:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.002+0000 7f1d2ee3cb80 20 client.15612 trim_cache size 0 max 16384 2024-05-09T00:14:02.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d2ee3cb80 1 Processor -- start 2024-05-09T00:14:02.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d2ee3cb80 1 -- start start 2024-05-09T00:14:02.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d2ee3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 0x563ef0813910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:02.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d2ee3cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef0814eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:02.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d2ee3cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef080f400 0x563ef0805da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:02.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d2ee3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181af420 con 0x563ef06894a0 2024-05-09T00:14:02.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d2ee3cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181accb0 con 0x563ef0773160 2024-05-09T00:14:02.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d2ee3cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181aab00 con 0x563ef080f400 2024-05-09T00:14:02.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d1e1c2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 0x563ef0813910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:02.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d1e1c2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 0x563ef0813910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49552/0 (socket says 172.21.15.46:49552) 2024-05-09T00:14:02.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d1e1c2700 1 -- 172.21.15.46:0/3431220976 learned_addr learned my addr 172.21.15.46:0/3431220976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:02.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d1d9c1700 1 --2- 172.21.15.46:0/3431220976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef0814eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d1e9c3700 1 --2- 172.21.15.46:0/3431220976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef080f400 0x563ef0805da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d1e1c2700 1 -- 172.21.15.46:0/3431220976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef080f400 msgr2=0x563ef0805da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/3431220976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef080f400 0x563ef0805da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d1e1c2700 1 -- 172.21.15.46:0/3431220976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 msgr2=0x563ef0814eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/3431220976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef0814eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.023+0000 7f1d1e1c2700 1 -- 172.21.15.46:0/3431220976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ef0812610 con 0x563ef06894a0 2024-05-09T00:14:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.024+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/3431220976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 0x563ef0813910 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1d140a6790 tx=0x7f1d140a9d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7f8afc6cfdbf52 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.024+0000 7f1d03fff700 1 -- 172.21.15.46:0/3431220976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1401d070 con 0x563ef06894a0 2024-05-09T00:14:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.024+0000 7f1d03fff700 1 -- 172.21.15.46:0/3431220976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d140c7070 con 0x563ef06894a0 2024-05-09T00:14:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.024+0000 7f1d03fff700 1 -- 172.21.15.46:0/3431220976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d140a4e90 con 0x563ef06894a0 2024-05-09T00:14:02.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/3431220976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 msgr2=0x563ef0813910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:02.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/3431220976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 0x563ef0813910 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1d140a6790 tx=0x7f1d140a9d90 comp rx=0 tx=0).stop 2024-05-09T00:14:02.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/3431220976 shutdown_connections 2024-05-09T00:14:02.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/3431220976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef080f400 0x563ef0805da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/3431220976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef06894a0 0x563ef0813910 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/3431220976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef0814eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/3431220976 >> 172.21.15.46:0/3431220976 conn(0x563ef06e96a0 msgr2=0x563ef0805690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/3431220976 shutdown_connections 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/3431220976 wait complete. 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 -- start start 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 Processor -- start 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef06894a0 0x563ef080d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef080d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 0x563ef081ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181ab170 con 0x563ef080f400 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181ac5e0 con 0x563ef0773160 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181ab840 con 0x563ef06894a0 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.025+0000 7f1d2ee3cb80 10 client.0 _ll_register_callbacks cb 0x563ef059a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d1e9c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 0x563ef081ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d1e9c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 0x563ef081ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49556/0 (socket says 172.21.15.46:49556) 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d1e9c3700 1 -- 172.21.15.46:0/54916544 learned_addr learned my addr 172.21.15.46:0/54916544 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef06894a0 0x563ef080d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d1d9c1700 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef080d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d1e9c3700 1 -- 172.21.15.46:0/54916544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef06894a0 msgr2=0x563ef080d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d1e9c3700 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef06894a0 0x563ef080d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d1e9c3700 1 -- 172.21.15.46:0/54916544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 msgr2=0x563ef080d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d1e9c3700 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef080d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d1e9c3700 1 -- 172.21.15.46:0/54916544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ef080dec0 con 0x563ef080f400 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef06894a0 0x563ef080d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d1e9c3700 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 0x563ef081ae40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1d10000f80 tx=0x7f1d100146e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120b85381c807e59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d01ffb700 1 -- 172.21.15.46:0/54916544 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10022070 con 0x563ef080f400 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d01ffb700 1 -- 172.21.15.46:0/54916544 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1d10014b00 con 0x563ef080f400 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.026+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/54916544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d140c7070 con 0x563ef080f400 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.027+0000 7f1d01ffb700 1 -- 172.21.15.46:0/54916544 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d1002e050 con 0x563ef080f400 2024-05-09T00:14:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.027+0000 7f1d2ee3cb80 10 client.15618 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.027+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/54916544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ef088f2e0 con 0x563ef080f400 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.027+0000 7f1d01ffb700 1 -- 172.21.15.46:0/54916544 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d140c7070 con 0x563ef080f400 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.027+0000 7f1d01ffb700 1 -- 172.21.15.46:0/54916544 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1d10005820 con 0x563ef080f400 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.027+0000 7f1d2ee3cb80 10 client.15618 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.027+0000 7f1d2ee3cb80 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.027+0000 7f1d2ee3cb80 20 client.15618 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.027+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ef088f800 0x563ef0891c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.027+0000 7f1d2ee3cb80 4 client.15618 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.027+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/54916544 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ef05a4970 con 0x563ef088f800 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.027+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ef088f800 0x563ef0891c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d1e1c2700 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ef088f800 0x563ef0891c20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b65f7ace9373f373 server_cookie=9b8d3b1a3c141043 in_seq=0 out_seq=0 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d01ffb700 10 client.15618 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d01ffb700 1 -- 172.21.15.46:0/54916544 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563ef05a4970 con 0x563ef088f800 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d01ffb700 10 client.15618 handle_command_reply: tid=0 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d2ee3cb80 1 client.15618 shutdown 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/54916544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 msgr2=0x563ef081ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 0x563ef081ae40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1d10000f80 tx=0x7f1d100146e0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/54916544 shutdown_connections 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ef088f800 0x563ef0891c20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ef080f400 0x563ef081ae40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ef06894a0 0x563ef080d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d2ee3cb80 1 --2- 172.21.15.46:0/54916544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ef0773160 0x563ef080d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d01ffb700 0 client.15618 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/54916544 >> 172.21.15.46:0/54916544 conn(0x563ef06e96a0 msgr2=0x563ef0805e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/54916544 shutdown_connections 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d2ee3cb80 1 -- 172.21.15.46:0/54916544 wait complete. 2024-05-09T00:14:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.028+0000 7f1d2ee3cb80 20 client.15618 trim_cache size 0 max 16384 2024-05-09T00:14:02.032 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:14:02.032 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:14:02.032 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:14:02.032 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:14:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.029+0000 7f1d20426700 1 -- 172.21.15.46:0/3647824997 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1d040b2880 msgr2=0x7f1d040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.029+0000 7f1d20426700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1d040b2880 0x7f1d040b4d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1d08000c00 tx=0x7f1d0801a040 comp rx=0 tx=0).stop 2024-05-09T00:14:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.029+0000 7f1d20426700 1 -- 172.21.15.46:0/3647824997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d181045d0 msgr2=0x7f1d1819ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.029+0000 7f1d20426700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d181045d0 0x7f1d1819ae90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c920 tx=0x7f1d1400cce0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.029+0000 7f1d20426700 1 -- 172.21.15.46:0/3647824997 shutdown_connections 2024-05-09T00:14:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.029+0000 7f1d20426700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1d040b2880 0x7f1d040b4d90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.029+0000 7f1d20426700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d18109850 0x7f1d1819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.029+0000 7f1d20426700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d181045d0 0x7f1d1819ae90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.029+0000 7f1d20426700 1 --2- 172.21.15.46:0/3647824997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d18108f00 0x7f1d1819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.029+0000 7f1d20426700 1 -- 172.21.15.46:0/3647824997 >> 172.21.15.46:0/3647824997 conn(0x7f1d180ffea0 msgr2=0x7f1d1810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.029+0000 7f1d20426700 1 -- 172.21.15.46:0/3647824997 shutdown_connections 2024-05-09T00:14:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:02.029+0000 7f1d20426700 1 -- 172.21.15.46:0/3647824997 wait complete. 2024-05-09T00:14:02.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:02 smithi046 ceph-mon[30151]: pgmap v456: 97 pgs: 97 active+clean; 904 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:14:02.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:02 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:14:02.862 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:14:02.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:14:02.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:02 smithi142 ceph-mon[34533]: pgmap v456: 97 pgs: 97 active+clean; 904 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:14:02.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:02 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:14:03.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:02 smithi169 ceph-mon[34777]: pgmap v456: 97 pgs: 97 active+clean; 904 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:14:03.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:02 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:14:03.121 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:04.901 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:04 smithi046 ceph-mon[30151]: pgmap v457: 97 pgs: 97 active+clean; 904 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:04.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:04 smithi142 ceph-mon[34533]: pgmap v457: 97 pgs: 97 active+clean; 904 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:05.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:04 smithi169 ceph-mon[34777]: pgmap v457: 97 pgs: 97 active+clean; 904 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.903+0000 7f6161459700 1 -- 172.21.15.46:0/1944771525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f615c104610 msgr2=0x7f615c104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.903+0000 7f6161459700 1 --2- 172.21.15.46:0/1944771525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f615c104610 0x7f615c104a20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f614c00b0d0 tx=0x7f614c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:14:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.903+0000 7f6161459700 1 -- 172.21.15.46:0/1944771525 shutdown_connections 2024-05-09T00:14:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.903+0000 7f6161459700 1 --2- 172.21.15.46:0/1944771525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f615c109890 0x7f615c1111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.903+0000 7f6161459700 1 --2- 172.21.15.46:0/1944771525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f615c104610 0x7f615c104a20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.903+0000 7f6161459700 1 --2- 172.21.15.46:0/1944771525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f615c108f40 0x7f615c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.903+0000 7f6161459700 1 -- 172.21.15.46:0/1944771525 >> 172.21.15.46:0/1944771525 conn(0x7f615c07af70 msgr2=0x7f615c07b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.903+0000 7f6161459700 1 -- 172.21.15.46:0/1944771525 shutdown_connections 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.903+0000 7f6161459700 1 -- 172.21.15.46:0/1944771525 wait complete. 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f6161459700 1 Processor -- start 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f6161459700 1 -- start start 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f6161459700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f615c104610 0x7f615c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f6161459700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f615c108f40 0x7f615c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f6161459700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f615c109890 0x7f615c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f6161459700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1144c0 con 0x7f615c109890 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f6161459700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f615c114340 con 0x7f615c104610 2024-05-09T00:14:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f6161459700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f615c114640 con 0x7f615c108f40 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f615b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f615c109890 0x7f615c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f615b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f615c109890 0x7f615c076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49580/0 (socket says 172.21.15.46:49580) 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f615b7fe700 1 -- 172.21.15.46:0/498517878 learned_addr learned my addr 172.21.15.46:0/498517878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f615a7fc700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f615c108f40 0x7f615c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.904+0000 7f615affd700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f615c104610 0x7f615c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.905+0000 7f615b7fe700 1 -- 172.21.15.46:0/498517878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f615c108f40 msgr2=0x7f615c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.905+0000 7f615b7fe700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f615c108f40 0x7f615c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.905+0000 7f615b7fe700 1 -- 172.21.15.46:0/498517878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f615c104610 msgr2=0x7f615c075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.905+0000 7f615b7fe700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f615c104610 0x7f615c075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.905+0000 7f615b7fe700 1 -- 172.21.15.46:0/498517878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615c1a91f0 con 0x7f615c109890 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.905+0000 7f615b7fe700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f615c109890 0x7f615c076540 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f615400d980 tx=0x7f615400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.905+0000 7f6153fff700 1 -- 172.21.15.46:0/498517878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154016070 con 0x7f615c109890 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.905+0000 7f6161459700 1 -- 172.21.15.46:0/498517878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615c1a94e0 con 0x7f615c109890 2024-05-09T00:14:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.905+0000 7f6153fff700 1 -- 172.21.15.46:0/498517878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6154004830 con 0x7f615c109890 2024-05-09T00:14:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.905+0000 7f6153fff700 1 -- 172.21.15.46:0/498517878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61540054f0 con 0x7f615c109890 2024-05-09T00:14:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.905+0000 7f6161459700 1 -- 172.21.15.46:0/498517878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615c1a9a20 con 0x7f615c109890 2024-05-09T00:14:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.906+0000 7f6161459700 1 -- 172.21.15.46:0/498517878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f615c0512c0 con 0x7f615c109890 2024-05-09T00:14:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.908+0000 7f6153fff700 1 -- 172.21.15.46:0/498517878 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f61540049e0 con 0x7f615c109890 2024-05-09T00:14:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.909+0000 7f6153fff700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61480b2900 0x7f61480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.909+0000 7f615affd700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61480b2900 0x7f61480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.909+0000 7f6153fff700 1 -- 172.21.15.46:0/498517878 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f61540e2370 con 0x7f615c109890 2024-05-09T00:14:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.909+0000 7f615affd700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61480b2900 0x7f61480b4e10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f614c009900 tx=0x7f614c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:04.913+0000 7f6153fff700 1 -- 172.21.15.46:0/498517878 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f615401f020 con 0x7f615c109890 2024-05-09T00:14:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.172+0000 7f6161459700 1 -- 172.21.15.46:0/498517878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f615c02e7b0 con 0x7f615c109890 2024-05-09T00:14:05.401 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:14:05.402 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:14:05.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.173+0000 7f6153fff700 1 -- 172.21.15.46:0/498517878 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f61540aa090 con 0x7f615c109890 2024-05-09T00:14:05.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.175+0000 7f6161459700 1 -- 172.21.15.46:0/498517878 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61480b2900 msgr2=0x7f61480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:05.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.175+0000 7f6161459700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61480b2900 0x7f61480b4e10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f614c009900 tx=0x7f614c00b040 comp rx=0 tx=0).stop 2024-05-09T00:14:05.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.175+0000 7f6161459700 1 -- 172.21.15.46:0/498517878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f615c109890 msgr2=0x7f615c076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:05.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.175+0000 7f6161459700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f615c109890 0x7f615c076540 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f615400d980 tx=0x7f615400dc90 comp rx=0 tx=0).stop 2024-05-09T00:14:05.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.176+0000 7f6161459700 1 -- 172.21.15.46:0/498517878 shutdown_connections 2024-05-09T00:14:05.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.176+0000 7f6161459700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61480b2900 0x7f61480b4e10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:05.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.176+0000 7f6161459700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f615c109890 0x7f615c076540 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.176+0000 7f6161459700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f615c104610 0x7f615c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.176+0000 7f6161459700 1 --2- 172.21.15.46:0/498517878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f615c108f40 0x7f615c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.176+0000 7f6161459700 1 -- 172.21.15.46:0/498517878 >> 172.21.15.46:0/498517878 conn(0x7f615c07af70 msgr2=0x7f615c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.176+0000 7f6161459700 1 -- 172.21.15.46:0/498517878 shutdown_connections 2024-05-09T00:14:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:05.176+0000 7f6161459700 1 -- 172.21.15.46:0/498517878 wait complete. 2024-05-09T00:14:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:14:05.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:05 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/498517878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:14:06.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:05 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/498517878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:14:06.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:05 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/498517878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:14:06.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:06 smithi142 ceph-mon[34533]: pgmap v458: 97 pgs: 97 active+clean; 961 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:07.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:06 smithi046 ceph-mon[30151]: pgmap v458: 97 pgs: 97 active+clean; 961 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:07.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:06 smithi169 ceph-mon[34777]: pgmap v458: 97 pgs: 97 active+clean; 961 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:07.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:14:08.015 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:08.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:08 smithi046 ceph-mon[30151]: pgmap v459: 97 pgs: 97 active+clean; 961 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:08.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:08 smithi142 ceph-mon[34533]: pgmap v459: 97 pgs: 97 active+clean; 961 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:09.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:08 smithi169 ceph-mon[34777]: pgmap v459: 97 pgs: 97 active+clean; 961 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.726+0000 7ff843860700 1 -- 172.21.15.46:0/3137463647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c109870 msgr2=0x7ff83c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.726+0000 7ff843860700 1 --2- 172.21.15.46:0/3137463647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c109870 0x7ff83c1111d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff83400b0d0 tx=0x7ff83400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.726+0000 7ff843860700 1 -- 172.21.15.46:0/3137463647 shutdown_connections 2024-05-09T00:14:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.726+0000 7ff843860700 1 --2- 172.21.15.46:0/3137463647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c109870 0x7ff83c1111d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.726+0000 7ff843860700 1 --2- 172.21.15.46:0/3137463647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff83c1045f0 0x7ff83c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.726+0000 7ff843860700 1 --2- 172.21.15.46:0/3137463647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c108f20 0x7ff83c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.726+0000 7ff843860700 1 -- 172.21.15.46:0/3137463647 >> 172.21.15.46:0/3137463647 conn(0x7ff83c100040 msgr2=0x7ff83c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.726+0000 7ff843860700 1 -- 172.21.15.46:0/3137463647 shutdown_connections 2024-05-09T00:14:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.726+0000 7ff843860700 1 -- 172.21.15.46:0/3137463647 wait complete. 2024-05-09T00:14:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.727+0000 7ff843860700 1 Processor -- start 2024-05-09T00:14:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.727+0000 7ff843860700 1 -- start start 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.727+0000 7ff843860700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c1045f0 0x7ff83c19d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff843860700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff83c108f20 0x7ff83c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff843860700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c109870 0x7ff83c1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff843860700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1144f0 con 0x7ff83c1045f0 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff843860700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c114370 con 0x7ff83c109870 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff843860700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c114670 con 0x7ff83c108f20 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff8415fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c1045f0 0x7ff83c19d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff8415fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c1045f0 0x7ff83c19d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33014/0 (socket says 172.21.15.46:33014) 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff8415fc700 1 -- 172.21.15.46:0/715127229 learned_addr learned my addr 172.21.15.46:0/715127229 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff840dfb700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff83c108f20 0x7ff83c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff841dfd700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c109870 0x7ff83c1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff8415fc700 1 -- 172.21.15.46:0/715127229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff83c108f20 msgr2=0x7ff83c19d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff8415fc700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff83c108f20 0x7ff83c19d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff8415fc700 1 -- 172.21.15.46:0/715127229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c109870 msgr2=0x7ff83c1a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff8415fc700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c109870 0x7ff83c1a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff8415fc700 1 -- 172.21.15.46:0/715127229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c1a2010 con 0x7ff83c1045f0 2024-05-09T00:14:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff8415fc700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c1045f0 0x7ff83c19d140 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff83800e9a0 tx=0x7ff83800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff8327fc700 1 -- 172.21.15.46:0/715127229 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83800ca10 con 0x7ff83c1045f0 2024-05-09T00:14:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff843860700 1 -- 172.21.15.46:0/715127229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83c1a2300 con 0x7ff83c1045f0 2024-05-09T00:14:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff8327fc700 1 -- 172.21.15.46:0/715127229 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff83800cbb0 con 0x7ff83c1045f0 2024-05-09T00:14:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.728+0000 7ff8327fc700 1 -- 172.21.15.46:0/715127229 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838018680 con 0x7ff83c1045f0 2024-05-09T00:14:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.729+0000 7ff843860700 1 -- 172.21.15.46:0/715127229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83c1a9c20 con 0x7ff83c1045f0 2024-05-09T00:14:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.731+0000 7ff8327fc700 1 -- 172.21.15.46:0/715127229 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff838018820 con 0x7ff83c1045f0 2024-05-09T00:14:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.732+0000 7ff8327fc700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff8280b2880 0x7ff8280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.732+0000 7ff8327fc700 1 -- 172.21.15.46:0/715127229 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff8380e2460 con 0x7ff83c1045f0 2024-05-09T00:14:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.732+0000 7ff840dfb700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff8280b2880 0x7ff8280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.732+0000 7ff852276b80 1 Processor -- start 2024-05-09T00:14:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff852276b80 1 -- start start 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff840dfb700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff8280b2880 0x7ff8280b4d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff83c110370 tx=0x7ff82c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff852276b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd56cd00 0x5642cd675240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff852276b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd4f24a0 0x5642cd675780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff852276b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd66f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff852276b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5642cd5920a0 con 0x5642cd66f400 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff852276b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5642cd5379a0 con 0x5642cd56cd00 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff852276b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5642cd58e060 con 0x5642cd4f24a0 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff841dfd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd66f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff841dfd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd66f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33022/0 (socket says 172.21.15.46:33022) 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff841dfd700 1 -- 172.21.15.46:0/2842250104 learned_addr learned my addr 172.21.15.46:0/2842250104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff8415fc700 1 --2- 172.21.15.46:0/2842250104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd56cd00 0x5642cd675240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff840dfb700 1 --2- 172.21.15.46:0/2842250104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd4f24a0 0x5642cd675780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff841dfd700 1 -- 172.21.15.46:0/2842250104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd4f24a0 msgr2=0x5642cd675780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff841dfd700 1 --2- 172.21.15.46:0/2842250104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd4f24a0 0x5642cd675780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff841dfd700 1 -- 172.21.15.46:0/2842250104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd56cd00 msgr2=0x5642cd675240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff841dfd700 1 --2- 172.21.15.46:0/2842250104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd56cd00 0x5642cd675240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff841dfd700 1 -- 172.21.15.46:0/2842250104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cd528500 con 0x5642cd66f400 2024-05-09T00:14:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff840dfb700 1 --2- 172.21.15.46:0/2842250104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd4f24a0 0x5642cd675780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.733+0000 7ff8415fc700 1 --2- 172.21.15.46:0/2842250104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd56cd00 0x5642cd675240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.734+0000 7ff841dfd700 1 --2- 172.21.15.46:0/2842250104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd66f810 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff8340098c0 tx=0x7ff834009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f67964115d5554d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.734+0000 7ff8277fe700 1 -- 172.21.15.46:0/2842250104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83400baa0 con 0x5642cd66f400 2024-05-09T00:14:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.734+0000 7ff8277fe700 1 -- 172.21.15.46:0/2842250104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff834020070 con 0x5642cd66f400 2024-05-09T00:14:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.734+0000 7ff8277fe700 1 -- 172.21.15.46:0/2842250104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834005390 con 0x5642cd66f400 2024-05-09T00:14:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.735+0000 7ff852276b80 1 -- 172.21.15.46:0/2842250104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 msgr2=0x5642cd66f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.735+0000 7ff852276b80 1 --2- 172.21.15.46:0/2842250104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd66f810 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff8340098c0 tx=0x7ff834009b80 comp rx=0 tx=0).stop 2024-05-09T00:14:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.735+0000 7ff852276b80 1 -- 172.21.15.46:0/2842250104 shutdown_connections 2024-05-09T00:14:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.735+0000 7ff852276b80 1 --2- 172.21.15.46:0/2842250104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd66f810 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.735+0000 7ff852276b80 1 --2- 172.21.15.46:0/2842250104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd56cd00 0x5642cd675240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.735+0000 7ff852276b80 1 --2- 172.21.15.46:0/2842250104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd4f24a0 0x5642cd675780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.735+0000 7ff852276b80 1 -- 172.21.15.46:0/2842250104 >> 172.21.15.46:0/2842250104 conn(0x5642cd4f5f70 msgr2=0x5642cd66a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.735+0000 7ff852276b80 1 -- 172.21.15.46:0/2842250104 shutdown_connections 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.735+0000 7ff852276b80 1 -- 172.21.15.46:0/2842250104 wait complete. 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.735+0000 7ff852276b80 1 -- start start 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.735+0000 7ff852276b80 1 Processor -- start 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff852276b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd6790d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff852276b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd56cd00 0x5642cd679610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff852276b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd66f400 0x5642cd679b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff852276b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5642cd670100 con 0x5642cd56cd00 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff852276b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5642cd52cb20 con 0x5642cd4f24a0 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff852276b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5642cd685250 con 0x5642cd66f400 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff852276b80 10 client.0 _ll_register_callbacks cb 0x5642cd4036c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff840dfb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd56cd00 0x5642cd679610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff840dfb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd56cd00 0x5642cd679610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33036/0 (socket says 172.21.15.46:33036) 2024-05-09T00:14:09.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff840dfb700 1 -- 172.21.15.46:0/28587714 learned_addr learned my addr 172.21.15.46:0/28587714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff841dfd700 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd66f400 0x5642cd679b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff8415fc700 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd6790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff840dfb700 1 -- 172.21.15.46:0/28587714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd66f400 msgr2=0x5642cd679b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff840dfb700 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd66f400 0x5642cd679b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff840dfb700 1 -- 172.21.15.46:0/28587714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 msgr2=0x5642cd6790d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff840dfb700 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd6790d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.736+0000 7ff840dfb700 1 -- 172.21.15.46:0/28587714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cd40d490 con 0x5642cd56cd00 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff840dfb700 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd56cd00 0x5642cd679610 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff82c01e890 tx=0x7ff82c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3eacf868a158c4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff824ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff824ff9700 1 -- 172.21.15.46:0/28587714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c004030 con 0x5642cd56cd00 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff852276b80 1 -- 172.21.15.46:0/28587714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff834020070 con 0x5642cd56cd00 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff824ff9700 1 -- 172.21.15.46:0/28587714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff82c0041d0 con 0x5642cd56cd00 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff824ff9700 1 -- 172.21.15.46:0/28587714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff82c0043d0 con 0x5642cd56cd00 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff852276b80 10 client.15636 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff852276b80 1 -- 172.21.15.46:0/28587714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5642cd686ca0 con 0x5642cd56cd00 2024-05-09T00:14:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff824ff9700 1 -- 172.21.15.46:0/28587714 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff834020070 con 0x5642cd56cd00 2024-05-09T00:14:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff824ff9700 1 -- 172.21.15.46:0/28587714 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff82c00e7a0 con 0x5642cd56cd00 2024-05-09T00:14:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff852276b80 10 client.15636 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff852276b80 10 client.15636 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:14:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff852276b80 20 client.15636 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff852276b80 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5642cd689330 0x5642cd68b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff852276b80 4 client.15636 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:14:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.737+0000 7ff852276b80 1 -- 172.21.15.46:0/28587714 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642cd59fa80 con 0x5642cd689330 2024-05-09T00:14:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.738+0000 7ff8415fc700 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5642cd689330 0x5642cd68b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.738+0000 7ff8415fc700 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5642cd689330 0x5642cd68b7b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=180df2461702f99 server_cookie=6662bca51f14a7f6 in_seq=0 out_seq=0 2024-05-09T00:14:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.738+0000 7ff824ff9700 10 client.15636 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:14:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.739+0000 7ff824ff9700 1 -- 172.21.15.46:0/28587714 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5642cd59fa80 con 0x5642cd689330 2024-05-09T00:14:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.739+0000 7ff824ff9700 10 client.15636 handle_command_reply: tid=0 2024-05-09T00:14:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.739+0000 7ff852276b80 1 client.15636 shutdown 2024-05-09T00:14:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.739+0000 7ff852276b80 1 -- 172.21.15.46:0/28587714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd56cd00 msgr2=0x5642cd679610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.739+0000 7ff852276b80 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd56cd00 0x5642cd679610 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff82c01e890 tx=0x7ff82c0047c0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.739+0000 7ff852276b80 1 -- 172.21.15.46:0/28587714 shutdown_connections 2024-05-09T00:14:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.739+0000 7ff852276b80 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5642cd689330 0x5642cd68b7b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.739+0000 7ff852276b80 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd66f400 0x5642cd679b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.739+0000 7ff852276b80 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd6790d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.739+0000 7ff824ff9700 0 client.15636 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:14:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.739+0000 7ff852276b80 1 --2- 172.21.15.46:0/28587714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd56cd00 0x5642cd679610 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.739+0000 7ff852276b80 1 -- 172.21.15.46:0/28587714 >> 172.21.15.46:0/28587714 conn(0x5642cd4f5f70 msgr2=0x5642cd663ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.740+0000 7ff852276b80 1 -- 172.21.15.46:0/28587714 shutdown_connections 2024-05-09T00:14:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.740+0000 7ff852276b80 1 -- 172.21.15.46:0/28587714 wait complete. 2024-05-09T00:14:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.740+0000 7ff852276b80 20 client.15636 trim_cache size 0 max 16384 2024-05-09T00:14:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff852276b80 1 Processor -- start 2024-05-09T00:14:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff852276b80 1 -- start start 2024-05-09T00:14:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff852276b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd663ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff852276b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd56cd00 0x5642cd664530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff852276b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd66e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff852276b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1b36c0 con 0x5642cd66f400 2024-05-09T00:14:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff852276b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1b0f50 con 0x5642cd4f24a0 2024-05-09T00:14:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff852276b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1aed80 con 0x5642cd56cd00 2024-05-09T00:14:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff841dfd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd66e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff841dfd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd66e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33044/0 (socket says 172.21.15.46:33044) 2024-05-09T00:14:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff841dfd700 1 -- 172.21.15.46:0/101665233 learned_addr learned my addr 172.21.15.46:0/101665233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff840dfb700 1 --2- 172.21.15.46:0/101665233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd56cd00 0x5642cd664530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.755+0000 7ff8415fc700 1 --2- 172.21.15.46:0/101665233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd663ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.756+0000 7ff841dfd700 1 -- 172.21.15.46:0/101665233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd56cd00 msgr2=0x5642cd664530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.756+0000 7ff841dfd700 1 --2- 172.21.15.46:0/101665233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd56cd00 0x5642cd664530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.756+0000 7ff841dfd700 1 -- 172.21.15.46:0/101665233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 msgr2=0x5642cd663ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.756+0000 7ff841dfd700 1 --2- 172.21.15.46:0/101665233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd663ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.756+0000 7ff841dfd700 1 -- 172.21.15.46:0/101665233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cd6669d0 con 0x5642cd66f400 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.756+0000 7ff840dfb700 1 --2- 172.21.15.46:0/101665233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd56cd00 0x5642cd664530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.756+0000 7ff841dfd700 1 --2- 172.21.15.46:0/101665233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd66e3f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff834009fd0 tx=0x7ff834009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834deef2d5cf7bd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.756+0000 7ff826ffd700 1 -- 172.21.15.46:0/101665233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83400bca0 con 0x5642cd66f400 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.756+0000 7ff826ffd700 1 -- 172.21.15.46:0/101665233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff834020070 con 0x5642cd66f400 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.756+0000 7ff826ffd700 1 -- 172.21.15.46:0/101665233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834007dc0 con 0x5642cd66f400 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.757+0000 7ff852276b80 1 -- 172.21.15.46:0/101665233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 msgr2=0x5642cd66e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.757+0000 7ff852276b80 1 --2- 172.21.15.46:0/101665233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd66e3f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff834009fd0 tx=0x7ff834009470 comp rx=0 tx=0).stop 2024-05-09T00:14:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.757+0000 7ff852276b80 1 -- 172.21.15.46:0/101665233 shutdown_connections 2024-05-09T00:14:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.757+0000 7ff852276b80 1 --2- 172.21.15.46:0/101665233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd66e3f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.757+0000 7ff852276b80 1 --2- 172.21.15.46:0/101665233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd663ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.757+0000 7ff852276b80 1 --2- 172.21.15.46:0/101665233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd56cd00 0x5642cd664530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.757+0000 7ff852276b80 1 -- 172.21.15.46:0/101665233 >> 172.21.15.46:0/101665233 conn(0x5642cd4f5f70 msgr2=0x5642cd6779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.757+0000 7ff852276b80 1 -- 172.21.15.46:0/101665233 shutdown_connections 2024-05-09T00:14:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.757+0000 7ff852276b80 1 -- 172.21.15.46:0/101665233 wait complete. 2024-05-09T00:14:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.757+0000 7ff852276b80 1 -- start start 2024-05-09T00:14:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.757+0000 7ff852276b80 1 Processor -- start 2024-05-09T00:14:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff852276b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd681670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff852276b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd56cd00 0x5642cd690580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff852276b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd690ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff852276b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1af410 con 0x5642cd66f400 2024-05-09T00:14:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff852276b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1b0880 con 0x5642cd4f24a0 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff852276b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1afae0 con 0x5642cd56cd00 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff852276b80 10 client.0 _ll_register_callbacks cb 0x5642cd66fdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff841dfd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd690ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff841dfd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd690ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33054/0 (socket says 172.21.15.46:33054) 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff841dfd700 1 -- 172.21.15.46:0/2707418595 learned_addr learned my addr 172.21.15.46:0/2707418595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff8415fc700 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd681670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff840dfb700 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd56cd00 0x5642cd690580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff841dfd700 1 -- 172.21.15.46:0/2707418595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd56cd00 msgr2=0x5642cd690580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff841dfd700 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd56cd00 0x5642cd690580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff841dfd700 1 -- 172.21.15.46:0/2707418595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 msgr2=0x5642cd681670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff841dfd700 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd681670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff841dfd700 1 -- 172.21.15.46:0/2707418595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cd6911f0 con 0x5642cd66f400 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff8415fc700 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd681670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff841dfd700 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd690ac0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff834009ca0 tx=0x7ff834003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3da04612d57b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:09.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.758+0000 7ff824ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff824ff9700 1 -- 172.21.15.46:0/2707418595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8340079d0 con 0x5642cd66f400 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff824ff9700 1 -- 172.21.15.46:0/2707418595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff834007b70 con 0x5642cd66f400 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff852276b80 1 -- 172.21.15.46:0/2707418595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff834020070 con 0x5642cd66f400 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff824ff9700 1 -- 172.21.15.46:0/2707418595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff834007d70 con 0x5642cd66f400 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff852276b80 10 client.15642 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff852276b80 1 -- 172.21.15.46:0/2707418595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5642cd6f3a00 con 0x5642cd66f400 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff824ff9700 1 -- 172.21.15.46:0/2707418595 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff834020070 con 0x5642cd66f400 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff824ff9700 1 -- 172.21.15.46:0/2707418595 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff834011070 con 0x5642cd66f400 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff852276b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff852276b80 10 client.15642 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff852276b80 20 client.15642 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff852276b80 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5642cd6f3bd0 0x5642cd6f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff852276b80 4 client.15642 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:14:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.759+0000 7ff852276b80 1 -- 172.21.15.46:0/2707418595 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642cd599b20 con 0x5642cd6f3bd0 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.760+0000 7ff8415fc700 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5642cd6f3bd0 0x5642cd6f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.760+0000 7ff824ff9700 10 client.15642 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.760+0000 7ff8415fc700 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5642cd6f3bd0 0x5642cd6f5ff0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d063fb8be6a2c0b7 server_cookie=e39b4256bdd72c1b in_seq=0 out_seq=0 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.760+0000 7ff824ff9700 1 -- 172.21.15.46:0/2707418595 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642cd599b20 con 0x5642cd6f3bd0 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.760+0000 7ff824ff9700 10 client.15642 handle_command_reply: tid=0 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.760+0000 7ff852276b80 1 client.15642 shutdown 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.760+0000 7ff852276b80 1 -- 172.21.15.46:0/2707418595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 msgr2=0x5642cd690ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.760+0000 7ff852276b80 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd690ac0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff834009ca0 tx=0x7ff834003d60 comp rx=0 tx=0).stop 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff852276b80 1 -- 172.21.15.46:0/2707418595 shutdown_connections 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff852276b80 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5642cd6f3bd0 0x5642cd6f5ff0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff852276b80 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5642cd66f400 0x5642cd690ac0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff852276b80 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642cd4f24a0 0x5642cd681670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff852276b80 1 --2- 172.21.15.46:0/2707418595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642cd56cd00 0x5642cd690580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff852276b80 1 -- 172.21.15.46:0/2707418595 >> 172.21.15.46:0/2707418595 conn(0x5642cd4f5f70 msgr2=0x5642cd663ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff824ff9700 0 client.15642 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:14:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff852276b80 1 -- 172.21.15.46:0/2707418595 shutdown_connections 2024-05-09T00:14:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff852276b80 1 -- 172.21.15.46:0/2707418595 wait complete. 2024-05-09T00:14:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff852276b80 20 client.15642 trim_cache size 0 max 16384 2024-05-09T00:14:09.764 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:14:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff843860700 1 -- 172.21.15.46:0/715127229 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff8280b2880 msgr2=0x7ff8280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff843860700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff8280b2880 0x7ff8280b4d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff83c110370 tx=0x7ff82c009260 comp rx=0 tx=0).stop 2024-05-09T00:14:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff843860700 1 -- 172.21.15.46:0/715127229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c1045f0 msgr2=0x7ff83c19d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.761+0000 7ff843860700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c1045f0 0x7ff83c19d140 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff83800e9a0 tx=0x7ff83800ecb0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.762+0000 7ff843860700 1 -- 172.21.15.46:0/715127229 shutdown_connections 2024-05-09T00:14:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.762+0000 7ff843860700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff8280b2880 0x7ff8280b4d90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.762+0000 7ff843860700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c109870 0x7ff83c1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.762+0000 7ff843860700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff83c1045f0 0x7ff83c19d140 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.762+0000 7ff843860700 1 --2- 172.21.15.46:0/715127229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff83c108f20 0x7ff83c19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.762+0000 7ff843860700 1 -- 172.21.15.46:0/715127229 >> 172.21.15.46:0/715127229 conn(0x7ff83c100040 msgr2=0x7ff83c110e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.762+0000 7ff843860700 1 -- 172.21.15.46:0/715127229 shutdown_connections 2024-05-09T00:14:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:09.762+0000 7ff843860700 1 -- 172.21.15.46:0/715127229 wait complete. 2024-05-09T00:14:10.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:14:10.848 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:10.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:10 smithi142 ceph-mon[34533]: pgmap v460: 97 pgs: 97 active+clean; 1022 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:14:10.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:10 smithi046 ceph-mon[30151]: pgmap v460: 97 pgs: 97 active+clean; 1022 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:14:11.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:10 smithi169 ceph-mon[34777]: pgmap v460: 97 pgs: 97 active+clean; 1022 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:14:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.588+0000 7f790c909700 1 -- 172.21.15.46:0/2027282375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79081034c0 msgr2=0x7f79081038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.588+0000 7f790c909700 1 --2- 172.21.15.46:0/2027282375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79081034c0 0x7f79081038d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f78f80099c0 tx=0x7f78f8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.588+0000 7f790c909700 1 -- 172.21.15.46:0/2027282375 shutdown_connections 2024-05-09T00:14:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.588+0000 7f790c909700 1 --2- 172.21.15.46:0/2027282375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7908108740 0x7f79081100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.588+0000 7f790c909700 1 --2- 172.21.15.46:0/2027282375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79081034c0 0x7f79081038d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.588+0000 7f790c909700 1 --2- 172.21.15.46:0/2027282375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7908107df0 0x7f7908108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.588+0000 7f790c909700 1 -- 172.21.15.46:0/2027282375 >> 172.21.15.46:0/2027282375 conn(0x7f79080fedc0 msgr2=0x7f79081011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.588+0000 7f790c909700 1 -- 172.21.15.46:0/2027282375 shutdown_connections 2024-05-09T00:14:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.588+0000 7f790c909700 1 -- 172.21.15.46:0/2027282375 wait complete. 2024-05-09T00:14:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f790c909700 1 Processor -- start 2024-05-09T00:14:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f790c909700 1 -- start start 2024-05-09T00:14:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f790c909700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79081034c0 0x7f790819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f790c909700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7908107df0 0x7f790819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f790c909700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7908108740 0x7f79081a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f790c909700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79081144c0 con 0x7f7908107df0 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f790c909700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7908114340 con 0x7f79081034c0 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f790c909700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7908114640 con 0x7f7908108740 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f7905d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7908107df0 0x7f790819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f7905d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7908107df0 0x7f790819d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33074/0 (socket says 172.21.15.46:33074) 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f7905d9b700 1 -- 172.21.15.46:0/3931030313 learned_addr learned my addr 172.21.15.46:0/3931030313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f790659c700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79081034c0 0x7f790819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.589+0000 7f7906d9d700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7908108740 0x7f79081a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.590+0000 7f7905d9b700 1 -- 172.21.15.46:0/3931030313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7908108740 msgr2=0x7f79081a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.590+0000 7f7905d9b700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7908108740 0x7f79081a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.590+0000 7f7905d9b700 1 -- 172.21.15.46:0/3931030313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79081034c0 msgr2=0x7f790819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.590+0000 7f7905d9b700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79081034c0 0x7f790819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.590+0000 7f7905d9b700 1 -- 172.21.15.46:0/3931030313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79081a1ee0 con 0x7f7908107df0 2024-05-09T00:14:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.590+0000 7f7905d9b700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7908107df0 0x7f790819d5b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00d950 tx=0x7f78fc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.590+0000 7f78f77fe700 1 -- 172.21.15.46:0/3931030313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc016070 con 0x7f7908107df0 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.590+0000 7f790c909700 1 -- 172.21.15.46:0/3931030313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79081a21d0 con 0x7f7908107df0 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.590+0000 7f78f77fe700 1 -- 172.21.15.46:0/3931030313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78fc004830 con 0x7f7908107df0 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.590+0000 7f78f77fe700 1 -- 172.21.15.46:0/3931030313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc005520 con 0x7f7908107df0 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.590+0000 7f790c909700 1 -- 172.21.15.46:0/3931030313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79081a9af0 con 0x7f7908107df0 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.593+0000 7f78f77fe700 1 -- 172.21.15.46:0/3931030313 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f78fc004220 con 0x7f7908107df0 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.593+0000 7f78f77fe700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f78f00b2880 0x7f78f00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.594+0000 7f790659c700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f78f00b2880 0x7f78f00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.594+0000 7f78f77fe700 1 -- 172.21.15.46:0/3931030313 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f78fc0e1ed0 con 0x7f7908107df0 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.594+0000 7f791b31fb80 1 Processor -- start 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.594+0000 7f791b31fb80 1 -- start start 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.594+0000 7f790659c700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f78f00b2880 0x7f78f00b4d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009990 tx=0x7f78f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.594+0000 7f791b31fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b18cad00 0x5593b19d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.594+0000 7f791b31fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 0x5593b19d3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.594+0000 7f791b31fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b19cd400 0x5593b19cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.594+0000 7f791b31fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5593b18f00a0 con 0x5593b18504a0 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.594+0000 7f791b31fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5593b18959a0 con 0x5593b18cad00 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.594+0000 7f791b31fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5593b18ec060 con 0x5593b19cd400 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f7905d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 0x5593b19d3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f7905d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 0x5593b19d3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33080/0 (socket says 172.21.15.46:33080) 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f7905d9b700 1 -- 172.21.15.46:0/120420848 learned_addr learned my addr 172.21.15.46:0/120420848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f790659c700 1 --2- 172.21.15.46:0/120420848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b18cad00 0x5593b19d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f7906d9d700 1 --2- 172.21.15.46:0/120420848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b19cd400 0x5593b19cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f7905d9b700 1 -- 172.21.15.46:0/120420848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b19cd400 msgr2=0x5593b19cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f7905d9b700 1 --2- 172.21.15.46:0/120420848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b19cd400 0x5593b19cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f7905d9b700 1 -- 172.21.15.46:0/120420848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b18cad00 msgr2=0x5593b19d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f7905d9b700 1 --2- 172.21.15.46:0/120420848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b18cad00 0x5593b19d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f7905d9b700 1 -- 172.21.15.46:0/120420848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593b1886500 con 0x5593b18504a0 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f7905d9b700 1 --2- 172.21.15.46:0/120420848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 0x5593b19d3780 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0a5570 tx=0x7f78fc0a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a96a05ec3a1bea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:12.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f78f4ff9700 1 -- 172.21.15.46:0/120420848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0a9bf0 con 0x5593b18504a0 2024-05-09T00:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f78f4ff9700 1 -- 172.21.15.46:0/120420848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78fc0910d0 con 0x5593b18504a0 2024-05-09T00:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.595+0000 7f78f4ff9700 1 -- 172.21.15.46:0/120420848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0049e0 con 0x5593b18504a0 2024-05-09T00:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.596+0000 7f791b31fb80 1 -- 172.21.15.46:0/120420848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 msgr2=0x5593b19d3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.596+0000 7f791b31fb80 1 --2- 172.21.15.46:0/120420848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 0x5593b19d3780 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0a5570 tx=0x7f78fc0a9000 comp rx=0 tx=0).stop 2024-05-09T00:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.596+0000 7f791b31fb80 1 -- 172.21.15.46:0/120420848 shutdown_connections 2024-05-09T00:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.596+0000 7f791b31fb80 1 --2- 172.21.15.46:0/120420848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b19cd400 0x5593b19cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.596+0000 7f791b31fb80 1 --2- 172.21.15.46:0/120420848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b18cad00 0x5593b19d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.596+0000 7f791b31fb80 1 --2- 172.21.15.46:0/120420848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 0x5593b19d3780 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.596+0000 7f791b31fb80 1 -- 172.21.15.46:0/120420848 >> 172.21.15.46:0/120420848 conn(0x5593b1853f70 msgr2=0x5593b19c8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.596+0000 7f791b31fb80 1 -- 172.21.15.46:0/120420848 shutdown_connections 2024-05-09T00:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.596+0000 7f791b31fb80 1 -- 172.21.15.46:0/120420848 wait complete. 2024-05-09T00:14:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f791b31fb80 1 -- start start 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f791b31fb80 1 Processor -- start 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f791b31fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18504a0 0x5593b19d7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f791b31fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18cad00 0x5593b19d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f791b31fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b19cd400 0x5593b19d7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f791b31fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5593b19ce160 con 0x5593b18cad00 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f791b31fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5593b188ab20 con 0x5593b19cd400 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f791b31fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5593b19da020 con 0x5593b18504a0 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f791b31fb80 10 client.0 _ll_register_callbacks cb 0x5593b17616c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f7905d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18cad00 0x5593b19d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f7905d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18cad00 0x5593b19d75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33084/0 (socket says 172.21.15.46:33084) 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f7905d9b700 1 -- 172.21.15.46:0/231792533 learned_addr learned my addr 172.21.15.46:0/231792533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f790659c700 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18504a0 0x5593b19d7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.597+0000 7f7906d9d700 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b19cd400 0x5593b19d7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f7905d9b700 1 -- 172.21.15.46:0/231792533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18504a0 msgr2=0x5593b19d7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f7905d9b700 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18504a0 0x5593b19d7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f7905d9b700 1 -- 172.21.15.46:0/231792533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b19cd400 msgr2=0x5593b19d7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f7905d9b700 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b19cd400 0x5593b19d7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f7905d9b700 1 -- 172.21.15.46:0/231792533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593b176b490 con 0x5593b18cad00 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f7905d9b700 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18cad00 0x5593b19d75c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f78fc005cd0 tx=0x7f78fc0a53d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a3abf0c45044a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f78edffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f78edffb700 1 -- 172.21.15.46:0/231792533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc06d060 con 0x5593b18cad00 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f78edffb700 1 -- 172.21.15.46:0/231792533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f78fc0049e0 con 0x5593b18cad00 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f791b31fb80 1 -- 172.21.15.46:0/231792533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78fc0910d0 con 0x5593b18cad00 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f78edffb700 1 -- 172.21.15.46:0/231792533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f78fc0a9bf0 con 0x5593b18cad00 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f791b31fb80 10 client.15654 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f791b31fb80 1 -- 172.21.15.46:0/231792533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5593b19e4a60 con 0x5593b18cad00 2024-05-09T00:14:12.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.598+0000 7f78edffb700 1 -- 172.21.15.46:0/231792533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78fc0910d0 con 0x5593b18cad00 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.599+0000 7f78edffb700 1 -- 172.21.15.46:0/231792533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f78fc0a4e60 con 0x5593b18cad00 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.599+0000 7f791b31fb80 10 client.15654 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.599+0000 7f791b31fb80 10 client.15654 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.599+0000 7f791b31fb80 20 client.15654 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.599+0000 7f791b31fb80 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5593b19e6f30 0x5593b19e9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.599+0000 7f791b31fb80 4 client.15654 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.599+0000 7f791b31fb80 1 -- 172.21.15.46:0/231792533 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593b18fda80 con 0x5593b19e6f30 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.599+0000 7f790659c700 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5593b19e6f30 0x5593b19e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.599+0000 7f78edffb700 10 client.15654 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.599+0000 7f790659c700 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5593b19e6f30 0x5593b19e9350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8b3c344e0e58b98 server_cookie=45e8d6c55b363240 in_seq=0 out_seq=0 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.600+0000 7f78edffb700 1 -- 172.21.15.46:0/231792533 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5593b18fda80 con 0x5593b19e6f30 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.600+0000 7f78edffb700 10 client.15654 handle_command_reply: tid=0 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.600+0000 7f791b31fb80 1 client.15654 shutdown 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.600+0000 7f791b31fb80 1 -- 172.21.15.46:0/231792533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18cad00 msgr2=0x5593b19d75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.600+0000 7f791b31fb80 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18cad00 0x5593b19d75c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f78fc005cd0 tx=0x7f78fc0a53d0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.600+0000 7f791b31fb80 1 -- 172.21.15.46:0/231792533 shutdown_connections 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.600+0000 7f791b31fb80 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5593b19e6f30 0x5593b19e9350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.600+0000 7f791b31fb80 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b19cd400 0x5593b19d7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.600+0000 7f791b31fb80 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18504a0 0x5593b19d7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.600+0000 7f78edffb700 0 client.15654 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.600+0000 7f791b31fb80 1 --2- 172.21.15.46:0/231792533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18cad00 0x5593b19d75c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.600+0000 7f791b31fb80 1 -- 172.21.15.46:0/231792533 >> 172.21.15.46:0/231792533 conn(0x5593b1853f70 msgr2=0x5593b19c1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.601+0000 7f791b31fb80 1 -- 172.21.15.46:0/231792533 shutdown_connections 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.601+0000 7f791b31fb80 1 -- 172.21.15.46:0/231792533 wait complete. 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.601+0000 7f791b31fb80 20 client.15654 trim_cache size 0 max 16384 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f791b31fb80 1 Processor -- start 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f791b31fb80 1 -- start start 2024-05-09T00:14:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f791b31fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 0x5593b19d4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f791b31fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18cad00 0x5593b19cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f791b31fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b19cd400 0x5593b19d7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f791b31fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79081b3830 con 0x5593b18504a0 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f791b31fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79081b10c0 con 0x5593b19cd400 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f791b31fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79081aeef0 con 0x5593b18cad00 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f790659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 0x5593b19d4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f790659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 0x5593b19d4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33100/0 (socket says 172.21.15.46:33100) 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f790659c700 1 -- 172.21.15.46:0/1707985838 learned_addr learned my addr 172.21.15.46:0/1707985838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f7905d9b700 1 --2- 172.21.15.46:0/1707985838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18cad00 0x5593b19cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f790659c700 1 -- 172.21.15.46:0/1707985838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18cad00 msgr2=0x5593b19cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f790659c700 1 --2- 172.21.15.46:0/1707985838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18cad00 0x5593b19cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f7906d9d700 1 --2- 172.21.15.46:0/1707985838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b19cd400 0x5593b19d7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f790659c700 1 -- 172.21.15.46:0/1707985838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b19cd400 msgr2=0x5593b19d7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f790659c700 1 --2- 172.21.15.46:0/1707985838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b19cd400 0x5593b19d7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f790659c700 1 -- 172.21.15.46:0/1707985838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593b19c49d0 con 0x5593b18504a0 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.616+0000 7f7905d9b700 1 --2- 172.21.15.46:0/1707985838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18cad00 0x5593b19cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.617+0000 7f790659c700 1 --2- 172.21.15.46:0/1707985838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 0x5593b19d4cf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f78f800b580 tx=0x7f78f801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e99aa653abfcf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.617+0000 7f78effff700 1 -- 172.21.15.46:0/1707985838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8005620 con 0x5593b18504a0 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.617+0000 7f78effff700 1 -- 172.21.15.46:0/1707985838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f803a070 con 0x5593b18504a0 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.617+0000 7f78effff700 1 -- 172.21.15.46:0/1707985838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f800f460 con 0x5593b18504a0 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.617+0000 7f791b31fb80 1 -- 172.21.15.46:0/1707985838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 msgr2=0x5593b19d4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.617+0000 7f791b31fb80 1 --2- 172.21.15.46:0/1707985838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 0x5593b19d4cf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f78f800b580 tx=0x7f78f801a050 comp rx=0 tx=0).stop 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 -- 172.21.15.46:0/1707985838 shutdown_connections 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 --2- 172.21.15.46:0/1707985838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b19cd400 0x5593b19d7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 --2- 172.21.15.46:0/1707985838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b18504a0 0x5593b19d4cf0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 --2- 172.21.15.46:0/1707985838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18cad00 0x5593b19cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 -- 172.21.15.46:0/1707985838 >> 172.21.15.46:0/1707985838 conn(0x5593b1853f70 msgr2=0x5593b19c2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 -- 172.21.15.46:0/1707985838 shutdown_connections 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 -- 172.21.15.46:0/1707985838 wait complete. 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 -- start start 2024-05-09T00:14:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 Processor -- start 2024-05-09T00:14:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b18504a0 0x5593b19e9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18cad00 0x5593b19ea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b19cd400 0x5593b19ea900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79081af580 con 0x5593b19cd400 2024-05-09T00:14:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79081b09f0 con 0x5593b18504a0 2024-05-09T00:14:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.618+0000 7f791b31fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79081afc50 con 0x5593b18cad00 2024-05-09T00:14:12.813 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:14:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f791b31fb80 10 client.0 _ll_register_callbacks cb 0x5593b19d2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f7906d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b19cd400 0x5593b19ea900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f7906d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b19cd400 0x5593b19ea900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33108/0 (socket says 172.21.15.46:33108) 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f7906d9d700 1 -- 172.21.15.46:0/2692440643 learned_addr learned my addr 172.21.15.46:0/2692440643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f7905d9b700 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18cad00 0x5593b19ea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f790659c700 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b18504a0 0x5593b19e9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f7906d9d700 1 -- 172.21.15.46:0/2692440643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18cad00 msgr2=0x5593b19ea3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f7906d9d700 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18cad00 0x5593b19ea3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f7906d9d700 1 -- 172.21.15.46:0/2692440643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b18504a0 msgr2=0x5593b19e9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f7906d9d700 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b18504a0 0x5593b19e9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f7906d9d700 1 -- 172.21.15.46:0/2692440643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593b19c1cc0 con 0x5593b19cd400 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f7906d9d700 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b19cd400 0x5593b19ea900 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7900007cd0 tx=0x7f7900007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7190ae935d29addf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f78edffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f78edffb700 1 -- 172.21.15.46:0/2692440643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900017a20 con 0x5593b19cd400 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f78edffb700 1 -- 172.21.15.46:0/2692440643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7900017bc0 con 0x5593b19cd400 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.619+0000 7f791b31fb80 1 -- 172.21.15.46:0/2692440643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78f803a070 con 0x5593b19cd400 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.620+0000 7f78edffb700 1 -- 172.21.15.46:0/2692440643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7900017dc0 con 0x5593b19cd400 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.620+0000 7f791b31fb80 10 client.15660 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.620+0000 7f791b31fb80 1 -- 172.21.15.46:0/2692440643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5593b19ce600 con 0x5593b19cd400 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.620+0000 7f78edffb700 1 -- 172.21.15.46:0/2692440643 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f803a070 con 0x5593b19cd400 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.620+0000 7f78edffb700 1 -- 172.21.15.46:0/2692440643 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7900018680 con 0x5593b19cd400 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.620+0000 7f791b31fb80 10 client.15660 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.620+0000 7f791b31fb80 10 client.15660 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.620+0000 7f791b31fb80 20 client.15660 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.620+0000 7f791b31fb80 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5593b1a518d0 0x5593b1a53cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.620+0000 7f791b31fb80 4 client.15660 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.620+0000 7f791b31fb80 1 -- 172.21.15.46:0/2692440643 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593b18f7b20 con 0x5593b1a518d0 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.620+0000 7f790659c700 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5593b1a518d0 0x5593b1a53cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f790659c700 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5593b1a518d0 0x5593b1a53cf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ecab72dc256f58a server_cookie=4fbfb50ca7111383 in_seq=0 out_seq=0 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f78edffb700 10 client.15660 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f78edffb700 1 -- 172.21.15.46:0/2692440643 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593b18f7b20 con 0x5593b1a518d0 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f78edffb700 10 client.15660 handle_command_reply: tid=0 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f791b31fb80 1 client.15660 shutdown 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f791b31fb80 1 -- 172.21.15.46:0/2692440643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b19cd400 msgr2=0x5593b19ea900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f791b31fb80 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b19cd400 0x5593b19ea900 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7900007cd0 tx=0x7f7900007db0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f791b31fb80 1 -- 172.21.15.46:0/2692440643 shutdown_connections 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f791b31fb80 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5593b1a518d0 0x5593b1a53cf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f78edffb700 0 client.15660 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f791b31fb80 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593b19cd400 0x5593b19ea900 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f791b31fb80 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593b18504a0 0x5593b19e9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f791b31fb80 1 --2- 172.21.15.46:0/2692440643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593b18cad00 0x5593b19ea3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f791b31fb80 1 -- 172.21.15.46:0/2692440643 >> 172.21.15.46:0/2692440643 conn(0x5593b1853f70 msgr2=0x5593b19d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f791b31fb80 1 -- 172.21.15.46:0/2692440643 shutdown_connections 2024-05-09T00:14:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f791b31fb80 1 -- 172.21.15.46:0/2692440643 wait complete. 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.621+0000 7f791b31fb80 20 client.15660 trim_cache size 0 max 16384 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.622+0000 7f790c909700 1 -- 172.21.15.46:0/3931030313 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f78f00b2880 msgr2=0x7f78f00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.622+0000 7f790c909700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f78f00b2880 0x7f78f00b4d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009990 tx=0x7f78f8000bc0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.622+0000 7f790c909700 1 -- 172.21.15.46:0/3931030313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7908107df0 msgr2=0x7f790819d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.622+0000 7f790c909700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7908107df0 0x7f790819d5b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00d950 tx=0x7f78fc00dc60 comp rx=0 tx=0).stop 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.622+0000 7f790c909700 1 -- 172.21.15.46:0/3931030313 shutdown_connections 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.622+0000 7f790c909700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f78f00b2880 0x7f78f00b4d90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.622+0000 7f790c909700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7908108740 0x7f79081a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.622+0000 7f790c909700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79081034c0 0x7f790819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.622+0000 7f790c909700 1 --2- 172.21.15.46:0/3931030313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7908107df0 0x7f790819d5b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.622+0000 7f790c909700 1 -- 172.21.15.46:0/3931030313 >> 172.21.15.46:0/3931030313 conn(0x7f79080fedc0 msgr2=0x7f7908107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.622+0000 7f790c909700 1 -- 172.21.15.46:0/3931030313 shutdown_connections 2024-05-09T00:14:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:12.622+0000 7f790c909700 1 -- 172.21.15.46:0/3931030313 wait complete. 2024-05-09T00:14:12.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:12 smithi142 ceph-mon[34533]: pgmap v461: 97 pgs: 97 active+clean; 1022 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:13.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:12 smithi046 ceph-mon[30151]: pgmap v461: 97 pgs: 97 active+clean; 1022 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:13.103 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:12 smithi169 ceph-mon[34777]: pgmap v461: 97 pgs: 97 active+clean; 1022 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:13.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:14:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:14.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:14 smithi142 ceph-mon[34533]: pgmap v462: 97 pgs: 97 active+clean; 1022 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:15.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:14 smithi046 ceph-mon[30151]: pgmap v462: 97 pgs: 97 active+clean; 1022 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:15.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:14 smithi169 ceph-mon[34777]: pgmap v462: 97 pgs: 97 active+clean; 1022 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.708+0000 7fef4d3b8700 1 -- 172.21.15.46:0/2640542760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef48109890 msgr2=0x7fef481111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.708+0000 7fef4d3b8700 1 --2- 172.21.15.46:0/2640542760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef48109890 0x7fef481111f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fef4000b0d0 tx=0x7fef4000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.708+0000 7fef4d3b8700 1 -- 172.21.15.46:0/2640542760 shutdown_connections 2024-05-09T00:14:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.708+0000 7fef4d3b8700 1 --2- 172.21.15.46:0/2640542760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef48109890 0x7fef481111f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.708+0000 7fef4d3b8700 1 --2- 172.21.15.46:0/2640542760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef48102b20 0x7fef48102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.708+0000 7fef4d3b8700 1 --2- 172.21.15.46:0/2640542760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef48108f40 0x7fef48109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.708+0000 7fef4d3b8700 1 -- 172.21.15.46:0/2640542760 >> 172.21.15.46:0/2640542760 conn(0x7fef480fe520 msgr2=0x7fef48100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.708+0000 7fef4d3b8700 1 -- 172.21.15.46:0/2640542760 shutdown_connections 2024-05-09T00:14:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.708+0000 7fef4d3b8700 1 -- 172.21.15.46:0/2640542760 wait complete. 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.709+0000 7fef4d3b8700 1 Processor -- start 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.709+0000 7fef4d3b8700 1 -- start start 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.709+0000 7fef4d3b8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef48102b20 0x7fef4819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.709+0000 7fef4d3b8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef48108f40 0x7fef4819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.709+0000 7fef4d3b8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef48109890 0x7fef4819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.709+0000 7fef4d3b8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef481144d0 con 0x7fef48102b20 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.709+0000 7fef4d3b8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef48114350 con 0x7fef48108f40 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.709+0000 7fef4d3b8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef48114650 con 0x7fef48109890 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.709+0000 7fef46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef48102b20 0x7fef4819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.709+0000 7fef46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef48102b20 0x7fef4819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33132/0 (socket says 172.21.15.46:33132) 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef46ffd700 1 -- 172.21.15.46:0/4020730264 learned_addr learned my addr 172.21.15.46:0/4020730264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef467fc700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef48108f40 0x7fef4819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef477fe700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef48109890 0x7fef4819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef46ffd700 1 -- 172.21.15.46:0/4020730264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef48109890 msgr2=0x7fef4819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef46ffd700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef48109890 0x7fef4819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef46ffd700 1 -- 172.21.15.46:0/4020730264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef48108f40 msgr2=0x7fef4819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef46ffd700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef48108f40 0x7fef4819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef46ffd700 1 -- 172.21.15.46:0/4020730264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4819fbc0 con 0x7fef48102b20 2024-05-09T00:14:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef467fc700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef48108f40 0x7fef4819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:14:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef46ffd700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef48102b20 0x7fef4819ada0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fef3800e970 tx=0x7fef3800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef2ffff700 1 -- 172.21.15.46:0/4020730264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3800cbd0 con 0x7fef48102b20 2024-05-09T00:14:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef4d3b8700 1 -- 172.21.15.46:0/4020730264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4819feb0 con 0x7fef48102b20 2024-05-09T00:14:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef2ffff700 1 -- 172.21.15.46:0/4020730264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef380044d0 con 0x7fef48102b20 2024-05-09T00:14:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef2ffff700 1 -- 172.21.15.46:0/4020730264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38003df0 con 0x7fef48102b20 2024-05-09T00:14:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.710+0000 7fef4d3b8700 1 -- 172.21.15.46:0/4020730264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef481a7880 con 0x7fef48102b20 2024-05-09T00:14:15.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.713+0000 7fef2ffff700 1 -- 172.21.15.46:0/4020730264 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fef38018430 con 0x7fef48102b20 2024-05-09T00:14:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.714+0000 7fef2ffff700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef300b2880 0x7fef300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.714+0000 7fef5bdceb80 1 Processor -- start 2024-05-09T00:14:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.714+0000 7fef5bdceb80 1 -- start start 2024-05-09T00:14:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.714+0000 7fef2ffff700 1 -- 172.21.15.46:0/4020730264 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fef380e24e0 con 0x7fef48102b20 2024-05-09T00:14:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.714+0000 7fef467fc700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef300b2880 0x7fef300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef5bdceb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c574085160 0x55c57411e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef5bdceb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c573f9b4a0 0x55c57411ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef5bdceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5741186a0 0x55c574118ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef5bdceb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c57403aff0 con 0x55c574085160 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef5bdceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c573fd97f0 con 0x55c5741186a0 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef5bdceb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c574036fb0 con 0x55c573f9b4a0 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c574085160 0x55c57411e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c574085160 0x55c57411e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33146/0 (socket says 172.21.15.46:33146) 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef46ffd700 1 -- 172.21.15.46:0/851122312 learned_addr learned my addr 172.21.15.46:0/851122312 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef467fc700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef300b2880 0x7fef300b4d90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef48101fd0 tx=0x7fef3c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef467fc700 1 --2- 172.21.15.46:0/851122312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c573f9b4a0 0x55c57411ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef46ffd700 1 -- 172.21.15.46:0/851122312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c573f9b4a0 msgr2=0x55c57411ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef46ffd700 1 --2- 172.21.15.46:0/851122312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c573f9b4a0 0x55c57411ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef46ffd700 1 -- 172.21.15.46:0/851122312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5741186a0 msgr2=0x55c574118ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef46ffd700 1 --2- 172.21.15.46:0/851122312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5741186a0 0x55c574118ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef46ffd700 1 -- 172.21.15.46:0/851122312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c573fd1370 con 0x55c574085160 2024-05-09T00:14:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef467fc700 1 --2- 172.21.15.46:0/851122312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c573f9b4a0 0x55c57411ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:14:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef46ffd700 1 --2- 172.21.15.46:0/851122312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c574085160 0x55c57411e4f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fef380a5b80 tx=0x7fef38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0418b347d41049 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef2d7fa700 1 -- 172.21.15.46:0/851122312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef380b22b0 con 0x55c574085160 2024-05-09T00:14:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef2d7fa700 1 -- 172.21.15.46:0/851122312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef3800ce00 con 0x55c574085160 2024-05-09T00:14:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.715+0000 7fef2d7fa700 1 -- 172.21.15.46:0/851122312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef380e2340 con 0x55c574085160 2024-05-09T00:14:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.716+0000 7fef5bdceb80 1 -- 172.21.15.46:0/851122312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c574085160 msgr2=0x55c57411e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.716+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/851122312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c574085160 0x55c57411e4f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fef380a5b80 tx=0x7fef38009840 comp rx=0 tx=0).stop 2024-05-09T00:14:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.716+0000 7fef5bdceb80 1 -- 172.21.15.46:0/851122312 shutdown_connections 2024-05-09T00:14:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.716+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/851122312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5741186a0 0x55c574118ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.716+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/851122312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c574085160 0x55c57411e4f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.716+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/851122312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c573f9b4a0 0x55c57411ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.716+0000 7fef5bdceb80 1 -- 172.21.15.46:0/851122312 >> 172.21.15.46:0/851122312 conn(0x55c573ffb6a0 msgr2=0x55c574113400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.716+0000 7fef5bdceb80 1 -- 172.21.15.46:0/851122312 shutdown_connections 2024-05-09T00:14:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.716+0000 7fef5bdceb80 1 -- 172.21.15.46:0/851122312 wait complete. 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.717+0000 7fef5bdceb80 1 -- start start 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.717+0000 7fef5bdceb80 1 Processor -- start 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.717+0000 7fef5bdceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c573f9b4a0 0x55c574122120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.717+0000 7fef5bdceb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c574085160 0x55c574122660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.717+0000 7fef5bdceb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 0x55c574122ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.717+0000 7fef5bdceb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c573eb6ba0 con 0x55c5741186a0 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.717+0000 7fef5bdceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c573ff5150 con 0x55c573f9b4a0 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.717+0000 7fef5bdceb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c5741251d0 con 0x55c574085160 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.717+0000 7fef5bdceb80 10 client.0 _ll_register_callbacks cb 0x55c573eac6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.717+0000 7fef477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 0x55c574122ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.717+0000 7fef477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 0x55c574122ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33154/0 (socket says 172.21.15.46:33154) 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.717+0000 7fef477fe700 1 -- 172.21.15.46:0/1115243309 learned_addr learned my addr 172.21.15.46:0/1115243309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef46ffd700 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c573f9b4a0 0x55c574122120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef467fc700 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c574085160 0x55c574122660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef477fe700 1 -- 172.21.15.46:0/1115243309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c574085160 msgr2=0x55c574122660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef477fe700 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c574085160 0x55c574122660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef477fe700 1 -- 172.21.15.46:0/1115243309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c573f9b4a0 msgr2=0x55c574122120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef477fe700 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c573f9b4a0 0x55c574122120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef477fe700 1 -- 172.21.15.46:0/1115243309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c573eb6490 con 0x55c5741186a0 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef477fe700 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 0x55c574122ba0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fef40009a20 tx=0x7fef4000bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df87e8212921c17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef2aff5700 1 -- 172.21.15.46:0/1115243309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40022070 con 0x55c5741186a0 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef2aff5700 1 -- 172.21.15.46:0/1115243309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fef400079d0 con 0x55c5741186a0 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef5bdceb80 1 -- 172.21.15.46:0/1115243309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c5740489d0 con 0x55c5741186a0 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef2aff5700 1 -- 172.21.15.46:0/1115243309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef4002e050 con 0x55c5741186a0 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef5bdceb80 10 client.15672 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.718+0000 7fef5bdceb80 1 -- 172.21.15.46:0/1115243309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c57412fb30 con 0x55c5741186a0 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.719+0000 7fef2aff5700 1 -- 172.21.15.46:0/1115243309 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c5740489d0 con 0x55c5741186a0 2024-05-09T00:14:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.719+0000 7fef2aff5700 1 -- 172.21.15.46:0/1115243309 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fef40011070 con 0x55c5741186a0 2024-05-09T00:14:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.719+0000 7fef5bdceb80 10 client.15672 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.719+0000 7fef5bdceb80 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.719+0000 7fef5bdceb80 20 client.15672 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.719+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c574132110 0x55c574134530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.719+0000 7fef5bdceb80 4 client.15672 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:14:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.719+0000 7fef5bdceb80 1 -- 172.21.15.46:0/1115243309 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c574042a70 con 0x55c574132110 2024-05-09T00:14:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.719+0000 7fef46ffd700 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c574132110 0x55c574134530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.720+0000 7fef46ffd700 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c574132110 0x55c574134530 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc4dc5bf7ac7f6f server_cookie=766a66b9ad54a17a in_seq=0 out_seq=0 2024-05-09T00:14:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.720+0000 7fef2aff5700 10 client.15672 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef2aff5700 1 -- 172.21.15.46:0/1115243309 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c574042a70 con 0x55c574132110 2024-05-09T00:14:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef2aff5700 10 client.15672 handle_command_reply: tid=0 2024-05-09T00:14:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef5bdceb80 1 client.15672 shutdown 2024-05-09T00:14:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef5bdceb80 1 -- 172.21.15.46:0/1115243309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 msgr2=0x55c574122ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 0x55c574122ba0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fef40009a20 tx=0x7fef4000bcd0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef5bdceb80 1 -- 172.21.15.46:0/1115243309 shutdown_connections 2024-05-09T00:14:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c574132110 0x55c574134530 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 0x55c574122ba0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c573f9b4a0 0x55c574122120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/1115243309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c574085160 0x55c574122660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef5bdceb80 1 -- 172.21.15.46:0/1115243309 >> 172.21.15.46:0/1115243309 conn(0x55c573ffb6a0 msgr2=0x55c5741131c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef2aff5700 0 client.15672 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef5bdceb80 1 -- 172.21.15.46:0/1115243309 shutdown_connections 2024-05-09T00:14:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef5bdceb80 1 -- 172.21.15.46:0/1115243309 wait complete. 2024-05-09T00:14:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.721+0000 7fef5bdceb80 20 client.15672 trim_cache size 0 max 16384 2024-05-09T00:14:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.736+0000 7fef5bdceb80 1 Processor -- start 2024-05-09T00:14:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.736+0000 7fef5bdceb80 1 -- start start 2024-05-09T00:14:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.736+0000 7fef5bdceb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c573f9b4a0 0x55c57411c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.736+0000 7fef5bdceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c574085160 0x55c5741180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.736+0000 7fef5bdceb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 0x55c57411ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.736+0000 7fef5bdceb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef481b1430 con 0x55c5741186a0 2024-05-09T00:14:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.736+0000 7fef5bdceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef481aecc0 con 0x55c574085160 2024-05-09T00:14:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.736+0000 7fef5bdceb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef481acaf0 con 0x55c573f9b4a0 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 0x55c57411ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 0x55c57411ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33168/0 (socket says 172.21.15.46:33168) 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef477fe700 1 -- 172.21.15.46:0/2833808429 learned_addr learned my addr 172.21.15.46:0/2833808429 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef467fc700 1 --2- 172.21.15.46:0/2833808429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c574085160 0x55c5741180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef46ffd700 1 --2- 172.21.15.46:0/2833808429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c573f9b4a0 0x55c57411c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef477fe700 1 -- 172.21.15.46:0/2833808429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c573f9b4a0 msgr2=0x55c57411c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef477fe700 1 --2- 172.21.15.46:0/2833808429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c573f9b4a0 0x55c57411c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef477fe700 1 -- 172.21.15.46:0/2833808429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c574085160 msgr2=0x55c5741180c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef477fe700 1 --2- 172.21.15.46:0/2833808429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c574085160 0x55c5741180c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef477fe700 1 -- 172.21.15.46:0/2833808429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c574114110 con 0x55c5741186a0 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef477fe700 1 --2- 172.21.15.46:0/2833808429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 0x55c57411ae00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fef4001cb70 tx=0x7fef4000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac857ef5ae91119e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef2cff9700 1 -- 172.21.15.46:0/2833808429 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40022040 con 0x55c5741186a0 2024-05-09T00:14:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef2cff9700 1 -- 172.21.15.46:0/2833808429 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef40005d20 con 0x55c5741186a0 2024-05-09T00:14:15.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.737+0000 7fef2cff9700 1 -- 172.21.15.46:0/2833808429 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40007a30 con 0x55c5741186a0 2024-05-09T00:14:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.738+0000 7fef5bdceb80 1 -- 172.21.15.46:0/2833808429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 msgr2=0x55c57411ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.738+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/2833808429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 0x55c57411ae00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fef4001cb70 tx=0x7fef4000b830 comp rx=0 tx=0).stop 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.738+0000 7fef5bdceb80 1 -- 172.21.15.46:0/2833808429 shutdown_connections 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.738+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/2833808429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5741186a0 0x55c57411ae00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.738+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/2833808429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c573f9b4a0 0x55c57411c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.738+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/2833808429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c574085160 0x55c5741180c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.738+0000 7fef5bdceb80 1 -- 172.21.15.46:0/2833808429 >> 172.21.15.46:0/2833808429 conn(0x55c573ffb6a0 msgr2=0x55c57411d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.738+0000 7fef5bdceb80 1 -- 172.21.15.46:0/2833808429 shutdown_connections 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.738+0000 7fef5bdceb80 1 -- 172.21.15.46:0/2833808429 wait complete. 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef5bdceb80 1 -- start start 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef5bdceb80 1 Processor -- start 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef5bdceb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c573f9b4a0 0x55c57411e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef5bdceb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c574085160 0x55c57411ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef5bdceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5741186a0 0x55c57411ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef5bdceb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef481ad180 con 0x55c573f9b4a0 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef5bdceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef481ae5f0 con 0x55c5741186a0 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef5bdceb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef481ad850 con 0x55c574085160 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef5bdceb80 10 client.0 _ll_register_callbacks cb 0x55c573eac6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c573f9b4a0 0x55c57411e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c573f9b4a0 0x55c57411e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33180/0 (socket says 172.21.15.46:33180) 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef46ffd700 1 -- 172.21.15.46:0/3649575679 learned_addr learned my addr 172.21.15.46:0/3649575679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef467fc700 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c574085160 0x55c57411ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef477fe700 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5741186a0 0x55c57411ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef46ffd700 1 -- 172.21.15.46:0/3649575679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c574085160 msgr2=0x55c57411ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.739+0000 7fef46ffd700 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c574085160 0x55c57411ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef46ffd700 1 -- 172.21.15.46:0/3649575679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5741186a0 msgr2=0x55c57411ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef46ffd700 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5741186a0 0x55c57411ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef46ffd700 1 -- 172.21.15.46:0/3649575679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57410cf60 con 0x55c573f9b4a0 2024-05-09T00:14:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef477fe700 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5741186a0 0x55c57411ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef467fc700 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c574085160 0x55c57411ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef46ffd700 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c573f9b4a0 0x55c57411e4f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fef380a9a90 tx=0x7fef3800c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1f2711300210cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef2aff5700 1 -- 172.21.15.46:0/3649575679 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef380a9610 con 0x55c573f9b4a0 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef2aff5700 1 -- 172.21.15.46:0/3649575679 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fef380a60f0 con 0x55c573f9b4a0 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef5bdceb80 1 -- 172.21.15.46:0/3649575679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef40005d20 con 0x55c573f9b4a0 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef2aff5700 1 -- 172.21.15.46:0/3649575679 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef380aa200 con 0x55c573f9b4a0 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef5bdceb80 10 client.15678 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef5bdceb80 1 -- 172.21.15.46:0/3649575679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c57412a5e0 con 0x55c573f9b4a0 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef2aff5700 1 -- 172.21.15.46:0/3649575679 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef40005d20 con 0x55c573f9b4a0 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef2aff5700 1 -- 172.21.15.46:0/3649575679 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fef380b24d0 con 0x55c573f9b4a0 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef5bdceb80 10 client.15678 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.740+0000 7fef5bdceb80 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.741+0000 7fef5bdceb80 20 client.15678 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.741+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c57412a810 0x55c57419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.741+0000 7fef5bdceb80 4 client.15678 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.741+0000 7fef5bdceb80 1 -- 172.21.15.46:0/3649575679 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c573eb6970 con 0x55c57412a810 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.741+0000 7fef467fc700 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c57412a810 0x55c57419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.741+0000 7fef2aff5700 10 client.15678 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.741+0000 7fef467fc700 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c57412a810 0x55c57419e8b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c86d36c7d2a643f server_cookie=5770f4da93f3f6f3 in_seq=0 out_seq=0 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.741+0000 7fef2aff5700 1 -- 172.21.15.46:0/3649575679 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c573eb6970 con 0x55c57412a810 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.741+0000 7fef2aff5700 10 client.15678 handle_command_reply: tid=0 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.741+0000 7fef5bdceb80 1 client.15678 shutdown 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef5bdceb80 1 -- 172.21.15.46:0/3649575679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c573f9b4a0 msgr2=0x55c57411e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c573f9b4a0 0x55c57411e4f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fef380a9a90 tx=0x7fef3800c6e0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef5bdceb80 1 -- 172.21.15.46:0/3649575679 shutdown_connections 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c57412a810 0x55c57419e8b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5741186a0 0x55c57411ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef2aff5700 0 client.15678 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:15.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c573f9b4a0 0x55c57411e4f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef5bdceb80 1 --2- 172.21.15.46:0/3649575679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c574085160 0x55c57411ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef5bdceb80 1 -- 172.21.15.46:0/3649575679 >> 172.21.15.46:0/3649575679 conn(0x55c573ffb6a0 msgr2=0x55c57411be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:15.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef5bdceb80 1 -- 172.21.15.46:0/3649575679 shutdown_connections 2024-05-09T00:14:15.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef5bdceb80 1 -- 172.21.15.46:0/3649575679 wait complete. 2024-05-09T00:14:15.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef5bdceb80 20 client.15678 trim_cache size 0 max 16384 2024-05-09T00:14:15.745 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:14:15.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef4d3b8700 1 -- 172.21.15.46:0/4020730264 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef300b2880 msgr2=0x7fef300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:15.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef4d3b8700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef300b2880 0x7fef300b4d90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef48101fd0 tx=0x7fef3c00a380 comp rx=0 tx=0).stop 2024-05-09T00:14:15.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef4d3b8700 1 -- 172.21.15.46:0/4020730264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef48102b20 msgr2=0x7fef4819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:15.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.742+0000 7fef4d3b8700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef48102b20 0x7fef4819ada0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fef3800e970 tx=0x7fef3800ec80 comp rx=0 tx=0).stop 2024-05-09T00:14:15.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.743+0000 7fef4d3b8700 1 -- 172.21.15.46:0/4020730264 shutdown_connections 2024-05-09T00:14:15.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.743+0000 7fef4d3b8700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef300b2880 0x7fef300b4d90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.743+0000 7fef4d3b8700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef48109890 0x7fef4819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.743+0000 7fef4d3b8700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef48102b20 0x7fef4819ada0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.743+0000 7fef4d3b8700 1 --2- 172.21.15.46:0/4020730264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef48108f40 0x7fef4819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:15.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.743+0000 7fef4d3b8700 1 -- 172.21.15.46:0/4020730264 >> 172.21.15.46:0/4020730264 conn(0x7fef480fe520 msgr2=0x7fef481086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:15.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.743+0000 7fef4d3b8700 1 -- 172.21.15.46:0/4020730264 shutdown_connections 2024-05-09T00:14:15.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:15.743+0000 7fef4d3b8700 1 -- 172.21.15.46:0/4020730264 wait complete. 2024-05-09T00:14:16.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:16 smithi046 ceph-mon[30151]: pgmap v463: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:14:17.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:16 smithi169 ceph-mon[34777]: pgmap v463: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:14:17.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:16 smithi142 ceph-mon[34533]: pgmap v463: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:14:17.550 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:14:17.550 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:14:17.818 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:19.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:18 smithi046 ceph-mon[30151]: pgmap v464: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:19.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:18 smithi169 ceph-mon[34777]: pgmap v464: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:19.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:18 smithi142 ceph-mon[34533]: pgmap v464: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:19.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.646+0000 7fe0f041d700 1 -- 172.21.15.46:0/47269626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8108740 msgr2=0x7fe0e81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:19.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.646+0000 7fe0f041d700 1 --2- 172.21.15.46:0/47269626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8108740 0x7fe0e81100a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000b0d0 tx=0x7fe0e000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.646+0000 7fe0f041d700 1 -- 172.21.15.46:0/47269626 shutdown_connections 2024-05-09T00:14:19.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.646+0000 7fe0f041d700 1 --2- 172.21.15.46:0/47269626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8108740 0x7fe0e81100a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.646+0000 7fe0f041d700 1 --2- 172.21.15.46:0/47269626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0e81034c0 0x7fe0e81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.646+0000 7fe0f041d700 1 --2- 172.21.15.46:0/47269626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e8107df0 0x7fe0e8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.646+0000 7fe0f041d700 1 -- 172.21.15.46:0/47269626 >> 172.21.15.46:0/47269626 conn(0x7fe0e80fedc0 msgr2=0x7fe0e81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:19.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.646+0000 7fe0f041d700 1 -- 172.21.15.46:0/47269626 shutdown_connections 2024-05-09T00:14:19.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.646+0000 7fe0f041d700 1 -- 172.21.15.46:0/47269626 wait complete. 2024-05-09T00:14:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.647+0000 7fe0f041d700 1 Processor -- start 2024-05-09T00:14:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.648+0000 7fe0f041d700 1 -- start start 2024-05-09T00:14:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.648+0000 7fe0f041d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0e81034c0 0x7fe0e81a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.648+0000 7fe0f041d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e8107df0 0x7fe0e81a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.648+0000 7fe0f041d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8108740 0x7fe0e81ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.648+0000 7fe0f041d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81144c0 con 0x7fe0e8108740 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.648+0000 7fe0f041d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e8114340 con 0x7fe0e81034c0 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.648+0000 7fe0f041d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e8114640 con 0x7fe0e8107df0 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.648+0000 7fe0ee1b9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0e81034c0 0x7fe0e81a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.648+0000 7fe0ee1b9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0e81034c0 0x7fe0e81a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:60402/0 (socket says 172.21.15.46:60402) 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.648+0000 7fe0ee1b9700 1 -- 172.21.15.46:0/2493012769 learned_addr learned my addr 172.21.15.46:0/2493012769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.648+0000 7fe0ee9ba700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8108740 0x7fe0e81ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.648+0000 7fe0ed9b8700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e8107df0 0x7fe0e81a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.649+0000 7fe0ee1b9700 1 -- 172.21.15.46:0/2493012769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e8107df0 msgr2=0x7fe0e81a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.649+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e8107df0 0x7fe0e81a2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.649+0000 7fe0ee1b9700 1 -- 172.21.15.46:0/2493012769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8108740 msgr2=0x7fe0e81ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.649+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8108740 0x7fe0e81ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.649+0000 7fe0ee1b9700 1 -- 172.21.15.46:0/2493012769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e81a2f80 con 0x7fe0e81034c0 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.649+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0e81034c0 0x7fe0e81a2140 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400e9a0 tx=0x7fe0e400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.649+0000 7fe0df7fe700 1 -- 172.21.15.46:0/2493012769 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e400ca10 con 0x7fe0e81034c0 2024-05-09T00:14:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.649+0000 7fe0df7fe700 1 -- 172.21.15.46:0/2493012769 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0e400cbb0 con 0x7fe0e81034c0 2024-05-09T00:14:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.649+0000 7fe0df7fe700 1 -- 172.21.15.46:0/2493012769 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4018680 con 0x7fe0e81034c0 2024-05-09T00:14:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.649+0000 7fe0f041d700 1 -- 172.21.15.46:0/2493012769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e81ad820 con 0x7fe0e81034c0 2024-05-09T00:14:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.649+0000 7fe0f041d700 1 -- 172.21.15.46:0/2493012769 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e81add10 con 0x7fe0e81034c0 2024-05-09T00:14:19.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.652+0000 7fe0df7fe700 1 -- 172.21.15.46:0/2493012769 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe0e4018820 con 0x7fe0e81034c0 2024-05-09T00:14:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.653+0000 7fe0df7fe700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe0d40b2880 0x7fe0d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.653+0000 7fe0fee33b80 1 Processor -- start 2024-05-09T00:14:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.653+0000 7fe0ed9b8700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe0d40b2880 0x7fe0d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.653+0000 7fe0df7fe700 1 -- 172.21.15.46:0/2493012769 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe0e40e2470 con 0x7fe0e81034c0 2024-05-09T00:14:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.653+0000 7fe0fee33b80 1 -- start start 2024-05-09T00:14:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0fee33b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da996aa1e0 0x55da99743570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0fee33b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da995c0520 0x55da99743ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0fee33b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da9973db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0fee33b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55da99660070 con 0x55da9973d720 2024-05-09T00:14:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0fee33b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55da995fe870 con 0x55da996aa1e0 2024-05-09T00:14:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0fee33b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55da9965c030 con 0x55da995c0520 2024-05-09T00:14:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0ed9b8700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe0d40b2880 0x7fe0d40b4d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8102240 tx=0x7fe0d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0ee9ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da9973db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0ee9ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da9973db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48384/0 (socket says 172.21.15.46:48384) 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0ee9ba700 1 -- 172.21.15.46:0/3737735930 learned_addr learned my addr 172.21.15.46:0/3737735930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/3737735930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da996aa1e0 0x55da99743570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0ed9b8700 1 --2- 172.21.15.46:0/3737735930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da995c0520 0x55da99743ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0ee9ba700 1 -- 172.21.15.46:0/3737735930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da995c0520 msgr2=0x55da99743ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0ee9ba700 1 --2- 172.21.15.46:0/3737735930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da995c0520 0x55da99743ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0ee9ba700 1 -- 172.21.15.46:0/3737735930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da996aa1e0 msgr2=0x55da99743570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0ee9ba700 1 --2- 172.21.15.46:0/3737735930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da996aa1e0 0x55da99743570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.654+0000 7fe0ee9ba700 1 -- 172.21.15.46:0/3737735930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da995f63f0 con 0x55da9973d720 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.655+0000 7fe0ee9ba700 1 --2- 172.21.15.46:0/3737735930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da9973db50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe0e00098f0 tx=0x7fe0e0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9791178f0f016359 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.655+0000 7fe0dcd47700 1 -- 172.21.15.46:0/3737735930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e00092e0 con 0x55da9973d720 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.655+0000 7fe0dcd47700 1 -- 172.21.15.46:0/3737735930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0e0020070 con 0x55da9973d720 2024-05-09T00:14:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.655+0000 7fe0dcd47700 1 -- 172.21.15.46:0/3737735930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e00053a0 con 0x55da9973d720 2024-05-09T00:14:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.656+0000 7fe0fee33b80 1 -- 172.21.15.46:0/3737735930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 msgr2=0x55da9973db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.656+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/3737735930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da9973db50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe0e00098f0 tx=0x7fe0e0009b80 comp rx=0 tx=0).stop 2024-05-09T00:14:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.656+0000 7fe0fee33b80 1 -- 172.21.15.46:0/3737735930 shutdown_connections 2024-05-09T00:14:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.656+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/3737735930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da9973db50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.656+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/3737735930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da996aa1e0 0x55da99743570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.656+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/3737735930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da995c0520 0x55da99743ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.656+0000 7fe0fee33b80 1 -- 172.21.15.46:0/3737735930 >> 172.21.15.46:0/3737735930 conn(0x55da99620720 msgr2=0x55da99738480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.656+0000 7fe0fee33b80 1 -- 172.21.15.46:0/3737735930 shutdown_connections 2024-05-09T00:14:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.656+0000 7fe0fee33b80 1 -- 172.21.15.46:0/3737735930 wait complete. 2024-05-09T00:14:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.656+0000 7fe0fee33b80 1 -- start start 2024-05-09T00:14:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.656+0000 7fe0fee33b80 1 Processor -- start 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0fee33b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da995c0520 0x55da99747190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0fee33b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da996aa1e0 0x55da997476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0fee33b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da99747c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0fee33b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55da994dbc20 con 0x55da9973d720 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0fee33b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55da9961a1d0 con 0x55da995c0520 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0fee33b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55da9973e560 con 0x55da996aa1e0 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0fee33b80 10 client.0 _ll_register_callbacks cb 0x55da994d1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0ee9ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da99747c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0ee1b9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da995c0520 0x55da99747190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0ee1b9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da995c0520 0x55da99747190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:60418/0 (socket says 172.21.15.46:60418) 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0ee1b9700 1 -- 172.21.15.46:0/1311675626 learned_addr learned my addr 172.21.15.46:0/1311675626 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0ed9b8700 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da996aa1e0 0x55da997476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0ee1b9700 1 -- 172.21.15.46:0/1311675626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da996aa1e0 msgr2=0x55da997476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da996aa1e0 0x55da997476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0ee1b9700 1 -- 172.21.15.46:0/1311675626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 msgr2=0x55da99747c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da99747c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0ee1b9700 1 -- 172.21.15.46:0/1311675626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da994db510 con 0x55da995c0520 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.657+0000 7fe0ee9ba700 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da99747c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.658+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da995c0520 0x55da99747190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40a9750 tx=0x7fe0e40a52d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5918f418062038ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.658+0000 7fe0d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.658+0000 7fe0d1ffb700 1 -- 172.21.15.46:0/1311675626 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40a5040 con 0x55da995c0520 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.658+0000 7fe0d1ffb700 1 -- 172.21.15.46:0/1311675626 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe0e40b2420 con 0x55da995c0520 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.658+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1311675626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e0020070 con 0x55da995c0520 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.658+0000 7fe0d1ffb700 1 -- 172.21.15.46:0/1311675626 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe0e40aa150 con 0x55da995c0520 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.658+0000 7fe0fee33b80 10 client.25361 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.658+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1311675626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da99754af0 con 0x55da995c0520 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.658+0000 7fe0d1ffb700 1 -- 172.21.15.46:0/1311675626 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0e0020070 con 0x55da995c0520 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.659+0000 7fe0d1ffb700 1 -- 172.21.15.46:0/1311675626 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe0e406f2b0 con 0x55da995c0520 2024-05-09T00:14:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.659+0000 7fe0fee33b80 10 client.25361 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.659+0000 7fe0fee33b80 10 client.25361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.659+0000 7fe0fee33b80 20 client.25361 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.659+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55da99756fc0 0x55da99759450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.659+0000 7fe0fee33b80 4 client.25361 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.659+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1311675626 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da9966da50 con 0x55da99756fc0 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.659+0000 7fe0ed9b8700 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55da99756fc0 0x55da99759450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.659+0000 7fe0ed9b8700 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55da99756fc0 0x55da99759450 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515fb9f801d703c1 server_cookie=fef56618c63225e3 in_seq=0 out_seq=0 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.659+0000 7fe0d1ffb700 10 client.25361 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.660+0000 7fe0d1ffb700 1 -- 172.21.15.46:0/1311675626 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da9966da50 con 0x55da99756fc0 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.660+0000 7fe0d1ffb700 10 client.25361 handle_command_reply: tid=0 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.660+0000 7fe0fee33b80 1 client.25361 shutdown 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.660+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1311675626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da995c0520 msgr2=0x55da99747190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.660+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da995c0520 0x55da99747190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40a9750 tx=0x7fe0e40a52d0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.660+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1311675626 shutdown_connections 2024-05-09T00:14:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.660+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55da99756fc0 0x55da99759450 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.660+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da99747c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.660+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da995c0520 0x55da99747190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.660+0000 7fe0d1ffb700 0 client.25361 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.660+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1311675626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da996aa1e0 0x55da997476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.660+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1311675626 >> 172.21.15.46:0/1311675626 conn(0x55da99620720 msgr2=0x55da99738240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.661+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1311675626 shutdown_connections 2024-05-09T00:14:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.661+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1311675626 wait complete. 2024-05-09T00:14:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.661+0000 7fe0fee33b80 20 client.25361 trim_cache size 0 max 16384 2024-05-09T00:14:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.681+0000 7fe0fee33b80 1 Processor -- start 2024-05-09T00:14:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.681+0000 7fe0fee33b80 1 -- start start 2024-05-09T00:14:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.681+0000 7fe0fee33b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da995c0520 0x55da997417e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.681+0000 7fe0fee33b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da996aa1e0 0x55da99749270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.681+0000 7fe0fee33b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da9973d720 0x55da99747a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.681+0000 7fe0fee33b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b7a90 con 0x55da995c0520 2024-05-09T00:14:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.681+0000 7fe0fee33b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b5320 con 0x55da9973d720 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.681+0000 7fe0fee33b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b3150 con 0x55da996aa1e0 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0ee1b9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da995c0520 0x55da997417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0ee1b9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da995c0520 0x55da997417e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48392/0 (socket says 172.21.15.46:48392) 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0ee1b9700 1 -- 172.21.15.46:0/1428124029 learned_addr learned my addr 172.21.15.46:0/1428124029 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0ed9b8700 1 --2- 172.21.15.46:0/1428124029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da996aa1e0 0x55da99749270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0ee1b9700 1 -- 172.21.15.46:0/1428124029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da996aa1e0 msgr2=0x55da99749270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0ee9ba700 1 --2- 172.21.15.46:0/1428124029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da9973d720 0x55da99747a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/1428124029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da996aa1e0 0x55da99749270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0ee1b9700 1 -- 172.21.15.46:0/1428124029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da9973d720 msgr2=0x55da99747a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/1428124029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da9973d720 0x55da99747a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0ee1b9700 1 -- 172.21.15.46:0/1428124029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da99746150 con 0x55da995c0520 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/1428124029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da995c0520 0x55da997417e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40a5ff0 tx=0x7fe0e40092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7b132554628af1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0d3fff700 1 -- 172.21.15.46:0/1428124029 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40a5070 con 0x55da995c0520 2024-05-09T00:14:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0d3fff700 1 -- 172.21.15.46:0/1428124029 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0e40b2040 con 0x55da995c0520 2024-05-09T00:14:19.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.682+0000 7fe0d3fff700 1 -- 172.21.15.46:0/1428124029 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40ade00 con 0x55da995c0520 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.683+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1428124029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da995c0520 msgr2=0x55da997417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.683+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1428124029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da995c0520 0x55da997417e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40a5ff0 tx=0x7fe0e40092d0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.683+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1428124029 shutdown_connections 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.683+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1428124029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da9973d720 0x55da99747a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.683+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1428124029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da995c0520 0x55da997417e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.683+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1428124029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da996aa1e0 0x55da99749270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.683+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1428124029 >> 172.21.15.46:0/1428124029 conn(0x55da99620720 msgr2=0x55da9973cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.683+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1428124029 shutdown_connections 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.683+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1428124029 wait complete. 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.683+0000 7fe0fee33b80 1 -- start start 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0fee33b80 1 Processor -- start 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0fee33b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da995c0520 0x55da9974f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0fee33b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da996aa1e0 0x55da9974a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0fee33b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da9974a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0fee33b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b37e0 con 0x55da9973d720 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0fee33b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b4c50 con 0x55da996aa1e0 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0fee33b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b3eb0 con 0x55da995c0520 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0fee33b80 10 client.0 _ll_register_callbacks cb 0x55da994d1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0ee9ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da9974a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0ee9ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da9974a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48408/0 (socket says 172.21.15.46:48408) 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0ee9ba700 1 -- 172.21.15.46:0/1097279870 learned_addr learned my addr 172.21.15.46:0/1097279870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da995c0520 0x55da9974f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0ed9b8700 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da996aa1e0 0x55da9974a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0ee9ba700 1 -- 172.21.15.46:0/1097279870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da995c0520 msgr2=0x55da9974f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0ee9ba700 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da995c0520 0x55da9974f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0ee9ba700 1 -- 172.21.15.46:0/1097279870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da996aa1e0 msgr2=0x55da9974a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:19.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0ee9ba700 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da996aa1e0 0x55da9974a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0ee9ba700 1 -- 172.21.15.46:0/1097279870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da9974add0 con 0x55da9973d720 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.684+0000 7fe0ed9b8700 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da996aa1e0 0x55da9974a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0ee9ba700 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da9974a860 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009bf0 tx=0x7fe0e0003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5c479d69550d18 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0d1ffb700 1 -- 172.21.15.46:0/1097279870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0032060 con 0x55da9973d720 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0d1ffb700 1 -- 172.21.15.46:0/1097279870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe0e00050b0 con 0x55da9973d720 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1097279870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e40b2040 con 0x55da9973d720 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0d1ffb700 1 -- 172.21.15.46:0/1097279870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe0e00052b0 con 0x55da9973d720 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0fee33b80 10 client.15696 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1097279870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da997c2560 con 0x55da9973d720 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0d1ffb700 1 -- 172.21.15.46:0/1097279870 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0e40b2040 con 0x55da9973d720 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0d1ffb700 1 -- 172.21.15.46:0/1097279870 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe0e0037070 con 0x55da9973d720 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0fee33b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0fee33b80 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:19.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0fee33b80 20 client.15696 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:19.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.685+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55da997c27e0 0x55da997c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:19.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.686+0000 7fe0fee33b80 4 client.15696 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:14:19.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.686+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1097279870 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55da99667af0 con 0x55da997c27e0 2024-05-09T00:14:19.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.686+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55da997c27e0 0x55da997c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:19.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.686+0000 7fe0ee1b9700 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55da997c27e0 0x55da997c4c00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d5e7ca8b71b982a server_cookie=2f13759390cb2ab9 in_seq=0 out_seq=0 2024-05-09T00:14:19.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.686+0000 7fe0d1ffb700 10 client.15696 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:19.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.687+0000 7fe0d1ffb700 1 -- 172.21.15.46:0/1097279870 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55da99667af0 con 0x55da997c27e0 2024-05-09T00:14:19.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.687+0000 7fe0d1ffb700 10 client.15696 handle_command_reply: tid=0 2024-05-09T00:14:19.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.687+0000 7fe0fee33b80 1 client.15696 shutdown 2024-05-09T00:14:19.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1097279870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 msgr2=0x55da9974a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:19.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da9974a860 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009bf0 tx=0x7fe0e0003e70 comp rx=0 tx=0).stop 2024-05-09T00:14:19.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1097279870 shutdown_connections 2024-05-09T00:14:19.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55da997c27e0 0x55da997c4c00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da9973d720 0x55da9974a860 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55da995c0520 0x55da9974f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0fee33b80 1 --2- 172.21.15.46:0/1097279870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da996aa1e0 0x55da9974a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0d1ffb700 0 client.15696 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:19.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1097279870 >> 172.21.15.46:0/1097279870 conn(0x55da99620720 msgr2=0x55da99738390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:19.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1097279870 shutdown_connections 2024-05-09T00:14:19.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0fee33b80 1 -- 172.21.15.46:0/1097279870 wait complete. 2024-05-09T00:14:19.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0fee33b80 20 client.15696 trim_cache size 0 max 16384 2024-05-09T00:14:19.690 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:14:19.690 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:14:19.691 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "22537362-4013-4f19-a990-a1e6d2df7d1e", 2024-05-09T00:14:19.691 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:14:19.691 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:14:19.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0f041d700 1 -- 172.21.15.46:0/2493012769 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe0d40b2880 msgr2=0x7fe0d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:19.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0f041d700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe0d40b2880 0x7fe0d40b4d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8102240 tx=0x7fe0d8009260 comp rx=0 tx=0).stop 2024-05-09T00:14:19.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0f041d700 1 -- 172.21.15.46:0/2493012769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0e81034c0 msgr2=0x7fe0e81a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:19.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.688+0000 7fe0f041d700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0e81034c0 0x7fe0e81a2140 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400e9a0 tx=0x7fe0e400ecb0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.689+0000 7fe0f041d700 1 -- 172.21.15.46:0/2493012769 shutdown_connections 2024-05-09T00:14:19.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.689+0000 7fe0f041d700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe0d40b2880 0x7fe0d40b4d90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.689+0000 7fe0f041d700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8108740 0x7fe0e81ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.689+0000 7fe0f041d700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0e81034c0 0x7fe0e81a2140 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.689+0000 7fe0f041d700 1 --2- 172.21.15.46:0/2493012769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e8107df0 0x7fe0e81a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:19.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.689+0000 7fe0f041d700 1 -- 172.21.15.46:0/2493012769 >> 172.21.15.46:0/2493012769 conn(0x7fe0e80fedc0 msgr2=0x7fe0e81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:19.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.689+0000 7fe0f041d700 1 -- 172.21.15.46:0/2493012769 shutdown_connections 2024-05-09T00:14:19.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:19.689+0000 7fe0f041d700 1 -- 172.21.15.46:0/2493012769 wait complete. 2024-05-09T00:14:20.466 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:14:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:20.979 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:20 smithi046 ceph-mon[30151]: pgmap v465: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:14:20.979 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:20 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:14:20.979 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:20 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:20.979 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:20 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:14:20.979 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:20 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:21.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:20 smithi169 ceph-mon[34777]: pgmap v465: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:14:21.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:20 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:14:21.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:20 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:21.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:20 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:14:21.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:20 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:21.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:20 smithi142 ceph-mon[34533]: pgmap v465: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:14:21.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:20 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:14:21.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:20 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:21.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:20 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:14:21.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:20 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:22.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.630+0000 7f42efdb8700 1 -- 172.21.15.46:0/2765977787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8107df0 msgr2=0x7f42e8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:22.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.630+0000 7f42efdb8700 1 --2- 172.21.15.46:0/2765977787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8107df0 0x7f42e8108200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0099c0 tx=0x7f42dc009cd0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.630+0000 7f42efdb8700 1 -- 172.21.15.46:0/2765977787 shutdown_connections 2024-05-09T00:14:22.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.630+0000 7f42efdb8700 1 --2- 172.21.15.46:0/2765977787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e8108740 0x7f42e81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.630+0000 7f42efdb8700 1 --2- 172.21.15.46:0/2765977787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e81034c0 0x7f42e81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.630+0000 7f42efdb8700 1 --2- 172.21.15.46:0/2765977787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8107df0 0x7f42e8108200 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.630+0000 7f42efdb8700 1 -- 172.21.15.46:0/2765977787 >> 172.21.15.46:0/2765977787 conn(0x7f42e80fedf0 msgr2=0x7f42e8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.630+0000 7f42efdb8700 1 -- 172.21.15.46:0/2765977787 shutdown_connections 2024-05-09T00:14:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.630+0000 7f42efdb8700 1 -- 172.21.15.46:0/2765977787 wait complete. 2024-05-09T00:14:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.631+0000 7f42efdb8700 1 Processor -- start 2024-05-09T00:14:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.631+0000 7f42efdb8700 1 -- start start 2024-05-09T00:14:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42efdb8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e81034c0 0x7f42e819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42efdb8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8107df0 0x7f42e819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42efdb8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e8108740 0x7f42e819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42efdb8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8113350 con 0x7f42e8107df0 2024-05-09T00:14:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42efdb8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81131d0 con 0x7f42e8108740 2024-05-09T00:14:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42efdb8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81134d0 con 0x7f42e81034c0 2024-05-09T00:14:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42ed353700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8107df0 0x7f42e819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42ed353700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8107df0 0x7f42e819c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48424/0 (socket says 172.21.15.46:48424) 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42ed353700 1 -- 172.21.15.46:0/3428157339 learned_addr learned my addr 172.21.15.46:0/3428157339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42edb54700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e81034c0 0x7f42e819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42ee355700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e8108740 0x7f42e819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42ed353700 1 -- 172.21.15.46:0/3428157339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e81034c0 msgr2=0x7f42e819bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42ed353700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e81034c0 0x7f42e819bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42ed353700 1 -- 172.21.15.46:0/3428157339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e8108740 msgr2=0x7f42e819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42ed353700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e8108740 0x7f42e819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42ed353700 1 -- 172.21.15.46:0/3428157339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e81a7060 con 0x7f42e8107df0 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42ee355700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e8108740 0x7f42e819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42ed353700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8107df0 0x7f42e819c250 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f42dc006010 tx=0x7f42dc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42daffd700 1 -- 172.21.15.46:0/3428157339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc027070 con 0x7f42e8107df0 2024-05-09T00:14:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.632+0000 7f42daffd700 1 -- 172.21.15.46:0/3428157339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42dc003fa0 con 0x7f42e8107df0 2024-05-09T00:14:22.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.633+0000 7f42daffd700 1 -- 172.21.15.46:0/3428157339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc022dd0 con 0x7f42e8107df0 2024-05-09T00:14:22.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.633+0000 7f42efdb8700 1 -- 172.21.15.46:0/3428157339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e81a72f0 con 0x7f42e8107df0 2024-05-09T00:14:22.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.633+0000 7f42efdb8700 1 -- 172.21.15.46:0/3428157339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e81a7740 con 0x7f42e8107df0 2024-05-09T00:14:22.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.635+0000 7f42daffd700 1 -- 172.21.15.46:0/3428157339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f42dc02a070 con 0x7f42e8107df0 2024-05-09T00:14:22.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.636+0000 7f42daffd700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f42d40b2880 0x7f42d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.636+0000 7f42edb54700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f42d40b2880 0x7f42d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.636+0000 7f42daffd700 1 -- 172.21.15.46:0/3428157339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f42dc0e2fc0 con 0x7f42e8107df0 2024-05-09T00:14:22.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.636+0000 7f42fe7ceb80 1 Processor -- start 2024-05-09T00:14:22.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42fe7ceb80 1 -- start start 2024-05-09T00:14:22.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42edb54700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f42d40b2880 0x7f42d40b4d90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f42e4005950 tx=0x7f42e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:22.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42fe7ceb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 0x564ed9cc24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42fe7ceb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cc2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42fe7ceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 0x564ed9cbcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42fe7ceb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ed9bdeff0 con 0x564ed9b3f4a0 2024-05-09T00:14:22.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42fe7ceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564ed9b7d7f0 con 0x564ed9cbc6a0 2024-05-09T00:14:22.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42fe7ceb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ed9bdafb0 con 0x564ed9c29160 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42ed353700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cc2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42ed353700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cc2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48438/0 (socket says 172.21.15.46:48438) 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42ed353700 1 -- 172.21.15.46:0/4292076565 learned_addr learned my addr 172.21.15.46:0/4292076565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42edb54700 1 --2- 172.21.15.46:0/4292076565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 0x564ed9cc24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42ee355700 1 --2- 172.21.15.46:0/4292076565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 0x564ed9cbcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42ed353700 1 -- 172.21.15.46:0/4292076565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 msgr2=0x564ed9cc24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42ed353700 1 --2- 172.21.15.46:0/4292076565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 0x564ed9cc24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42ed353700 1 -- 172.21.15.46:0/4292076565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 msgr2=0x564ed9cbcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42ed353700 1 --2- 172.21.15.46:0/4292076565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 0x564ed9cbcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.637+0000 7f42ed353700 1 -- 172.21.15.46:0/4292076565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed9b75370 con 0x564ed9b3f4a0 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.638+0000 7f42ed353700 1 --2- 172.21.15.46:0/4292076565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cc2a30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0aa4a0 tx=0x7f42dc022470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afca937d7a25896 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.638+0000 7f42d3fff700 1 -- 172.21.15.46:0/4292076565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0b10b0 con 0x564ed9b3f4a0 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.638+0000 7f42d3fff700 1 -- 172.21.15.46:0/4292076565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42dc0aa7a0 con 0x564ed9b3f4a0 2024-05-09T00:14:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.638+0000 7f42d3fff700 1 -- 172.21.15.46:0/4292076565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0aac00 con 0x564ed9b3f4a0 2024-05-09T00:14:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.639+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/4292076565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 msgr2=0x564ed9cc2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.639+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/4292076565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cc2a30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0aa4a0 tx=0x7f42dc022470 comp rx=0 tx=0).stop 2024-05-09T00:14:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.639+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/4292076565 shutdown_connections 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.639+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/4292076565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 0x564ed9cbcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.639+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/4292076565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 0x564ed9cc24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.639+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/4292076565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cc2a30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.639+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/4292076565 >> 172.21.15.46:0/4292076565 conn(0x564ed9b9f6a0 msgr2=0x564ed9cb7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.639+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/4292076565 shutdown_connections 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.639+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/4292076565 wait complete. 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.639+0000 7f42fe7ceb80 1 -- start start 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.639+0000 7f42fe7ceb80 1 Processor -- start 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42fe7ceb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9ccbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42fe7ceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9c29160 0x564ed9ccc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42fe7ceb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9cbc6a0 0x564ed9cc6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42fe7ceb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ed9a5aba0 con 0x564ed9b3f4a0 2024-05-09T00:14:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42fe7ceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564ed9b99150 con 0x564ed9c29160 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42fe7ceb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ed9cbd360 con 0x564ed9cbc6a0 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42fe7ceb80 10 client.0 _ll_register_callbacks cb 0x564ed9a506c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42edb54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9ccbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42edb54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9ccbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48452/0 (socket says 172.21.15.46:48452) 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42edb54700 1 -- 172.21.15.46:0/3255070905 learned_addr learned my addr 172.21.15.46:0/3255070905 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42ee355700 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9cbc6a0 0x564ed9cc6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42ed353700 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9c29160 0x564ed9ccc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42edb54700 1 -- 172.21.15.46:0/3255070905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9cbc6a0 msgr2=0x564ed9cc6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42edb54700 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9cbc6a0 0x564ed9cc6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42edb54700 1 -- 172.21.15.46:0/3255070905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9c29160 msgr2=0x564ed9ccc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42edb54700 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9c29160 0x564ed9ccc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.640+0000 7f42edb54700 1 -- 172.21.15.46:0/3255070905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed9a5a490 con 0x564ed9b3f4a0 2024-05-09T00:14:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42edb54700 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9ccbd50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f42e40114c0 tx=0x7f42e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345d4142db5ceca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42d17fa700 1 -- 172.21.15.46:0/3255070905 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4028670 con 0x564ed9b3f4a0 2024-05-09T00:14:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/3255070905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42dc0aa7a0 con 0x564ed9b3f4a0 2024-05-09T00:14:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42d17fa700 1 -- 172.21.15.46:0/3255070905 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f42e4028810 con 0x564ed9b3f4a0 2024-05-09T00:14:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42d17fa700 1 -- 172.21.15.46:0/3255070905 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42e4028a10 con 0x564ed9b3f4a0 2024-05-09T00:14:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42fe7ceb80 10 client.15708 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/3255070905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ed9cd39d0 con 0x564ed9b3f4a0 2024-05-09T00:14:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42d17fa700 1 -- 172.21.15.46:0/3255070905 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42dc0aa7a0 con 0x564ed9b3f4a0 2024-05-09T00:14:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42d17fa700 1 -- 172.21.15.46:0/3255070905 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f42e401fc00 con 0x564ed9b3f4a0 2024-05-09T00:14:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42fe7ceb80 10 client.15708 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42fe7ceb80 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42fe7ceb80 20 client.15708 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.641+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ed9cd6010 0x564ed9cd8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.642+0000 7f42fe7ceb80 4 client.15708 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:14:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.642+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/3255070905 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ed9bec9d0 con 0x564ed9cd6010 2024-05-09T00:14:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.642+0000 7f42ed353700 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ed9cd6010 0x564ed9cd8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.642+0000 7f42ed353700 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ed9cd6010 0x564ed9cd8490 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80ff4023679cc2fa server_cookie=5b38167191b3f579 in_seq=0 out_seq=0 2024-05-09T00:14:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.642+0000 7f42d17fa700 10 client.15708 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.643+0000 7f42d17fa700 1 -- 172.21.15.46:0/3255070905 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ed9bec9d0 con 0x564ed9cd6010 2024-05-09T00:14:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.643+0000 7f42d17fa700 10 client.15708 handle_command_reply: tid=0 2024-05-09T00:14:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.643+0000 7f42fe7ceb80 1 client.15708 shutdown 2024-05-09T00:14:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.643+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/3255070905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 msgr2=0x564ed9ccbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.643+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9ccbd50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f42e40114c0 tx=0x7f42e4004750 comp rx=0 tx=0).stop 2024-05-09T00:14:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.643+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/3255070905 shutdown_connections 2024-05-09T00:14:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.643+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ed9cd6010 0x564ed9cd8490 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.643+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9cbc6a0 0x564ed9cc6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.643+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9ccbd50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.643+0000 7f42d17fa700 0 client.15708 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.643+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/3255070905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9c29160 0x564ed9ccc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.643+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/3255070905 >> 172.21.15.46:0/3255070905 conn(0x564ed9b9f6a0 msgr2=0x564ed9cb71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.644+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/3255070905 shutdown_connections 2024-05-09T00:14:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.644+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/3255070905 wait complete. 2024-05-09T00:14:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.644+0000 7f42fe7ceb80 20 client.15708 trim_cache size 0 max 16384 2024-05-09T00:14:22.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42fe7ceb80 1 Processor -- start 2024-05-09T00:14:22.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42fe7ceb80 1 -- start start 2024-05-09T00:14:22.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42fe7ceb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cc0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42fe7ceb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 0x564ed9cc1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42fe7ceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 0x564ed9cbee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42fe7ceb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81b1530 con 0x564ed9b3f4a0 2024-05-09T00:14:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42fe7ceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81aedc0 con 0x564ed9cbc6a0 2024-05-09T00:14:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42fe7ceb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81acbf0 con 0x564ed9c29160 2024-05-09T00:14:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42edb54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cc0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42edb54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cc0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48462/0 (socket says 172.21.15.46:48462) 2024-05-09T00:14:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42edb54700 1 -- 172.21.15.46:0/84348461 learned_addr learned my addr 172.21.15.46:0/84348461 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42ed353700 1 --2- 172.21.15.46:0/84348461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 0x564ed9cc1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.664+0000 7f42ee355700 1 --2- 172.21.15.46:0/84348461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 0x564ed9cbee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.665+0000 7f42edb54700 1 -- 172.21.15.46:0/84348461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 msgr2=0x564ed9cc1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:22.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.665+0000 7f42edb54700 1 --2- 172.21.15.46:0/84348461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 0x564ed9cc1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.665+0000 7f42edb54700 1 -- 172.21.15.46:0/84348461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 msgr2=0x564ed9cbee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:22.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.665+0000 7f42edb54700 1 --2- 172.21.15.46:0/84348461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 0x564ed9cbee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.665+0000 7f42edb54700 1 -- 172.21.15.46:0/84348461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed9cc2bd0 con 0x564ed9b3f4a0 2024-05-09T00:14:22.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.665+0000 7f42edb54700 1 --2- 172.21.15.46:0/84348461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cc0720 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f42e400b480 tx=0x7f42e4011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4c6a7f0698243e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:22.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.665+0000 7f42d37fe700 1 -- 172.21.15.46:0/84348461 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4028db0 con 0x564ed9b3f4a0 2024-05-09T00:14:22.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.665+0000 7f42d37fe700 1 -- 172.21.15.46:0/84348461 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42e4023070 con 0x564ed9b3f4a0 2024-05-09T00:14:22.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.665+0000 7f42d37fe700 1 -- 172.21.15.46:0/84348461 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4004a20 con 0x564ed9b3f4a0 2024-05-09T00:14:22.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.666+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/84348461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 msgr2=0x564ed9cc0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.666+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/84348461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cc0720 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f42e400b480 tx=0x7f42e4011f20 comp rx=0 tx=0).stop 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.666+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/84348461 shutdown_connections 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.666+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/84348461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 0x564ed9cbee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.666+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/84348461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cc0720 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.666+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/84348461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 0x564ed9cc1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.666+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/84348461 >> 172.21.15.46:0/84348461 conn(0x564ed9b9f6a0 msgr2=0x564ed9cc1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.666+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/84348461 shutdown_connections 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.666+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/84348461 wait complete. 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42fe7ceb80 1 -- start start 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42fe7ceb80 1 Processor -- start 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42fe7ceb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42fe7ceb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 0x564ed9cce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42fe7ceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 0x564ed9d3f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42fe7ceb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81ad280 con 0x564ed9b3f4a0 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42fe7ceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81ae6f0 con 0x564ed9cbc6a0 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42fe7ceb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81ad950 con 0x564ed9c29160 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42fe7ceb80 10 client.0 _ll_register_callbacks cb 0x564ed9a506c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42edb54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42edb54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48464/0 (socket says 172.21.15.46:48464) 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42edb54700 1 -- 172.21.15.46:0/1896446920 learned_addr learned my addr 172.21.15.46:0/1896446920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.667+0000 7f42ed353700 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 0x564ed9cce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42ee355700 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 0x564ed9d3f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42edb54700 1 -- 172.21.15.46:0/1896446920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 msgr2=0x564ed9cce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42edb54700 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 0x564ed9cce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42edb54700 1 -- 172.21.15.46:0/1896446920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 msgr2=0x564ed9d3f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42edb54700 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 0x564ed9d3f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42edb54700 1 -- 172.21.15.46:0/1896446920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed9d3fd40 con 0x564ed9b3f4a0 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42edb54700 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cce430 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f42e40286a0 tx=0x7f42e4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b0bcfa3fcd987f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42d17fa700 1 -- 172.21.15.46:0/1896446920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4005430 con 0x564ed9b3f4a0 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42d17fa700 1 -- 172.21.15.46:0/1896446920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f42e4033710 con 0x564ed9b3f4a0 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/1896446920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42e4023070 con 0x564ed9b3f4a0 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42d17fa700 1 -- 172.21.15.46:0/1896446920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42e4011b60 con 0x564ed9b3f4a0 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.668+0000 7f42fe7ceb80 10 client.15714 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.669+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/1896446920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ed9d40e70 con 0x564ed9b3f4a0 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.669+0000 7f42d17fa700 1 -- 172.21.15.46:0/1896446920 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42e4023070 con 0x564ed9b3f4a0 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.669+0000 7f42d17fa700 1 -- 172.21.15.46:0/1896446920 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f42e4004b20 con 0x564ed9b3f4a0 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.669+0000 7f42fe7ceb80 10 client.15714 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.669+0000 7f42fe7ceb80 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.669+0000 7f42fe7ceb80 20 client.15714 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.669+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ed9d41460 0x564ed9d43880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.669+0000 7f42fe7ceb80 4 client.15714 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.669+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/1896446920 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ed9be6a70 con 0x564ed9d41460 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.669+0000 7f42ed353700 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ed9d41460 0x564ed9d43880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42d17fa700 10 client.15714 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42ed353700 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ed9d41460 0x564ed9d43880 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ec6f10f12e8b9a server_cookie=a31e18178d09b1cd in_seq=0 out_seq=0 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42d17fa700 1 -- 172.21.15.46:0/1896446920 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564ed9be6a70 con 0x564ed9d41460 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42d17fa700 10 client.15714 handle_command_reply: tid=0 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42fe7ceb80 1 client.15714 shutdown 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/1896446920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 msgr2=0x564ed9cce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cce430 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f42e40286a0 tx=0x7f42e4012000 comp rx=0 tx=0).stop 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/1896446920 shutdown_connections 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ed9d41460 0x564ed9d43880 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ed9cbc6a0 0x564ed9d3f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ed9b3f4a0 0x564ed9cce430 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42d17fa700 0 client.15714 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42fe7ceb80 1 --2- 172.21.15.46:0/1896446920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ed9c29160 0x564ed9cce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.670+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/1896446920 >> 172.21.15.46:0/1896446920 conn(0x564ed9b9f6a0 msgr2=0x564ed9cbfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:22.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/1896446920 shutdown_connections 2024-05-09T00:14:22.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42fe7ceb80 1 -- 172.21.15.46:0/1896446920 wait complete. 2024-05-09T00:14:22.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42fe7ceb80 20 client.15714 trim_cache size 0 max 16384 2024-05-09T00:14:22.673 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:14:22.673 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:14:22.673 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:14:22.673 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:14:22.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42efdb8700 1 -- 172.21.15.46:0/3428157339 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f42d40b2880 msgr2=0x7f42d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:22.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42efdb8700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f42d40b2880 0x7f42d40b4d90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f42e4005950 tx=0x7f42e40058e0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42efdb8700 1 -- 172.21.15.46:0/3428157339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8107df0 msgr2=0x7f42e819c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:22.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42efdb8700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8107df0 0x7f42e819c250 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f42dc006010 tx=0x7f42dc0047b0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42efdb8700 1 -- 172.21.15.46:0/3428157339 shutdown_connections 2024-05-09T00:14:22.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42efdb8700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f42d40b2880 0x7f42d40b4d90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42efdb8700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e8108740 0x7f42e819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42efdb8700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e81034c0 0x7f42e819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42efdb8700 1 --2- 172.21.15.46:0/3428157339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e8107df0 0x7f42e819c250 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:22.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42efdb8700 1 -- 172.21.15.46:0/3428157339 >> 172.21.15.46:0/3428157339 conn(0x7f42e80fedf0 msgr2=0x7f42e81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:22.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42efdb8700 1 -- 172.21.15.46:0/3428157339 shutdown_connections 2024-05-09T00:14:22.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:22.671+0000 7f42efdb8700 1 -- 172.21.15.46:0/3428157339 wait complete. 2024-05-09T00:14:22.988 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:22 smithi046 ceph-mon[30151]: pgmap v466: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:14:22.988 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:22 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:14:23.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:22 smithi169 ceph-mon[34777]: pgmap v466: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:14:23.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:22 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:14:23.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:22 smithi142 ceph-mon[34533]: pgmap v466: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:14:23.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:22 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:14:23.531 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:14:23.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:14:23.788 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:24.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:24 smithi046 ceph-mon[30151]: pgmap v467: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:25.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:24 smithi169 ceph-mon[34777]: pgmap v467: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:25.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:24 smithi142 ceph-mon[34533]: pgmap v467: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.613+0000 7fc10ce23700 1 -- 172.21.15.46:0/1366142797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc108108740 msgr2=0x7fc1081100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.613+0000 7fc10ce23700 1 --2- 172.21.15.46:0/1366142797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc108108740 0x7fc1081100a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc10000b0d0 tx=0x7fc10000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:14:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.613+0000 7fc10ce23700 1 -- 172.21.15.46:0/1366142797 shutdown_connections 2024-05-09T00:14:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.613+0000 7fc10ce23700 1 --2- 172.21.15.46:0/1366142797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc108108740 0x7fc1081100a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.613+0000 7fc10ce23700 1 --2- 172.21.15.46:0/1366142797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1081034c0 0x7fc1081038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.613+0000 7fc10ce23700 1 --2- 172.21.15.46:0/1366142797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108107df0 0x7fc108108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:25.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.613+0000 7fc10ce23700 1 -- 172.21.15.46:0/1366142797 >> 172.21.15.46:0/1366142797 conn(0x7fc1080fedf0 msgr2=0x7fc108101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:25.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.613+0000 7fc10ce23700 1 -- 172.21.15.46:0/1366142797 shutdown_connections 2024-05-09T00:14:25.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.613+0000 7fc10ce23700 1 -- 172.21.15.46:0/1366142797 wait complete. 2024-05-09T00:14:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.614+0000 7fc10ce23700 1 Processor -- start 2024-05-09T00:14:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.614+0000 7fc10ce23700 1 -- start start 2024-05-09T00:14:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.614+0000 7fc10ce23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1081034c0 0x7fc10819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.614+0000 7fc10ce23700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc108107df0 0x7fc10819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.614+0000 7fc10ce23700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108108740 0x7fc10819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.614+0000 7fc10ce23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc108113370 con 0x7fc1081034c0 2024-05-09T00:14:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.614+0000 7fc10ce23700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1081131f0 con 0x7fc108108740 2024-05-09T00:14:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.614+0000 7fc10ce23700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1081134f0 con 0x7fc108107df0 2024-05-09T00:14:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc10659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1081034c0 0x7fc10819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc10659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1081034c0 0x7fc10819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48484/0 (socket says 172.21.15.46:48484) 2024-05-09T00:14:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc10659c700 1 -- 172.21.15.46:0/3154094465 learned_addr learned my addr 172.21.15.46:0/3154094465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc106d9d700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108108740 0x7fc10819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc0fdd9b700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc108107df0 0x7fc10819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc10659c700 1 -- 172.21.15.46:0/3154094465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc108107df0 msgr2=0x7fc10819c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc10659c700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc108107df0 0x7fc10819c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc10659c700 1 -- 172.21.15.46:0/3154094465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108108740 msgr2=0x7fc10819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc10659c700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108108740 0x7fc10819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:25.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc10659c700 1 -- 172.21.15.46:0/3154094465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1081a6f80 con 0x7fc1081034c0 2024-05-09T00:14:25.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc10659c700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1081034c0 0x7fc10819bc30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800c920 tx=0x7fc0f800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:25.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc0fffff700 1 -- 172.21.15.46:0/3154094465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8007b00 con 0x7fc1081034c0 2024-05-09T00:14:25.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc10ce23700 1 -- 172.21.15.46:0/3154094465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1081a7270 con 0x7fc1081034c0 2024-05-09T00:14:25.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc0fffff700 1 -- 172.21.15.46:0/3154094465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f8004ce0 con 0x7fc1081034c0 2024-05-09T00:14:25.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc0fffff700 1 -- 172.21.15.46:0/3154094465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8005550 con 0x7fc1081034c0 2024-05-09T00:14:25.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.615+0000 7fc10ce23700 1 -- 172.21.15.46:0/3154094465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1081a7800 con 0x7fc1081034c0 2024-05-09T00:14:25.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.616+0000 7fc10ce23700 1 -- 172.21.15.46:0/3154094465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1080512c0 con 0x7fc1081034c0 2024-05-09T00:14:25.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.618+0000 7fc0fffff700 1 -- 172.21.15.46:0/3154094465 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc0f8007ca0 con 0x7fc1081034c0 2024-05-09T00:14:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.619+0000 7fc0fffff700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0f40b2900 0x7fc0f40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.619+0000 7fc0fdd9b700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0f40b2900 0x7fc0f40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.619+0000 7fc0fffff700 1 -- 172.21.15.46:0/3154094465 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc0f80e2140 con 0x7fc1081034c0 2024-05-09T00:14:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.619+0000 7fc0fdd9b700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0f40b2900 0x7fc0f40b4e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0005950 tx=0x7fc0f00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.623+0000 7fc0fffff700 1 -- 172.21.15.46:0/3154094465 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0f8014030 con 0x7fc1081034c0 2024-05-09T00:14:25.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.886+0000 7fc10ce23700 1 -- 172.21.15.46:0/3154094465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc10802e5f0 con 0x7fc1081034c0 2024-05-09T00:14:25.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.887+0000 7fc0fffff700 1 -- 172.21.15.46:0/3154094465 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7fc0f80a9ea0 con 0x7fc1081034c0 2024-05-09T00:14:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:14:25.885 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:14:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.890+0000 7fc10ce23700 1 -- 172.21.15.46:0/3154094465 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0f40b2900 msgr2=0x7fc0f40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.890+0000 7fc10ce23700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0f40b2900 0x7fc0f40b4e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0005950 tx=0x7fc0f00058e0 comp rx=0 tx=0).stop 2024-05-09T00:14:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.890+0000 7fc10ce23700 1 -- 172.21.15.46:0/3154094465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1081034c0 msgr2=0x7fc10819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.890+0000 7fc10ce23700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1081034c0 0x7fc10819bc30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800c920 tx=0x7fc0f800cc30 comp rx=0 tx=0).stop 2024-05-09T00:14:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.890+0000 7fc10ce23700 1 -- 172.21.15.46:0/3154094465 shutdown_connections 2024-05-09T00:14:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.890+0000 7fc10ce23700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0f40b2900 0x7fc0f40b4e10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.890+0000 7fc10ce23700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc108108740 0x7fc10819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.890+0000 7fc10ce23700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1081034c0 0x7fc10819bc30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.890+0000 7fc10ce23700 1 --2- 172.21.15.46:0/3154094465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc108107df0 0x7fc10819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.890+0000 7fc10ce23700 1 -- 172.21.15.46:0/3154094465 >> 172.21.15.46:0/3154094465 conn(0x7fc1080fedf0 msgr2=0x7fc108107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.890+0000 7fc10ce23700 1 -- 172.21.15.46:0/3154094465 shutdown_connections 2024-05-09T00:14:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:25.890+0000 7fc10ce23700 1 -- 172.21.15.46:0/3154094465 wait complete. 2024-05-09T00:14:25.889 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:14:26.540 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:14:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:26.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:26 smithi046 ceph-mon[30151]: pgmap v468: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:26.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:26 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3154094465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:14:27.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:26 smithi169 ceph-mon[34777]: pgmap v468: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:27.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:26 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3154094465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:14:27.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:26 smithi142 ceph-mon[34533]: pgmap v468: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:27.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:26 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3154094465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:14:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.501+0000 7f4017e3a700 1 -- 172.21.15.46:0/2029768139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40101045d0 msgr2=0x7f40101049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.501+0000 7f4017e3a700 1 --2- 172.21.15.46:0/2029768139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40101045d0 0x7f40101049e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f40040099c0 tx=0x7f4004009cd0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.501+0000 7f4017e3a700 1 -- 172.21.15.46:0/2029768139 shutdown_connections 2024-05-09T00:14:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.501+0000 7f4017e3a700 1 --2- 172.21.15.46:0/2029768139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4010109850 0x7f40101111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.501+0000 7f4017e3a700 1 --2- 172.21.15.46:0/2029768139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40101045d0 0x7f40101049e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.501+0000 7f4017e3a700 1 --2- 172.21.15.46:0/2029768139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4010108f00 0x7f4010109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.501+0000 7f4017e3a700 1 -- 172.21.15.46:0/2029768139 >> 172.21.15.46:0/2029768139 conn(0x7f40100ffea0 msgr2=0x7f40101022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.501+0000 7f4017e3a700 1 -- 172.21.15.46:0/2029768139 shutdown_connections 2024-05-09T00:14:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.501+0000 7f4017e3a700 1 -- 172.21.15.46:0/2029768139 wait complete. 2024-05-09T00:14:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f4017e3a700 1 Processor -- start 2024-05-09T00:14:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f4017e3a700 1 -- start start 2024-05-09T00:14:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f4017e3a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40101045d0 0x7f401019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f4017e3a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4010108f00 0x7f401019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f4017e3a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4010109850 0x7f401019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f4017e3a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4010114480 con 0x7f40101045d0 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f4017e3a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4010114300 con 0x7f4010109850 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f4017e3a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4010114600 con 0x7f4010108f00 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f4015bd6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40101045d0 0x7f401019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f4015bd6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40101045d0 0x7f401019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60916/0 (socket says 172.21.15.46:60916) 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f4015bd6700 1 -- 172.21.15.46:0/2922877153 learned_addr learned my addr 172.21.15.46:0/2922877153 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f40153d5700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4010108f00 0x7f401019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.502+0000 7f40163d7700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4010109850 0x7f401019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.503+0000 7f4015bd6700 1 -- 172.21.15.46:0/2922877153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4010108f00 msgr2=0x7f401019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.503+0000 7f4015bd6700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4010108f00 0x7f401019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.503+0000 7f4015bd6700 1 -- 172.21.15.46:0/2922877153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4010109850 msgr2=0x7f401019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.503+0000 7f4015bd6700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4010109850 0x7f401019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.503+0000 7f4015bd6700 1 -- 172.21.15.46:0/2922877153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401019fc50 con 0x7f40101045d0 2024-05-09T00:14:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.503+0000 7f4015bd6700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40101045d0 0x7f401019ae30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f400400b580 tx=0x7f40040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.503+0000 7f4002ffd700 1 -- 172.21.15.46:0/2922877153 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004027070 con 0x7f40101045d0 2024-05-09T00:14:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.503+0000 7f4002ffd700 1 -- 172.21.15.46:0/2922877153 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4004003f20 con 0x7f40101045d0 2024-05-09T00:14:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.503+0000 7f4017e3a700 1 -- 172.21.15.46:0/2922877153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401019fee0 con 0x7f40101045d0 2024-05-09T00:14:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.503+0000 7f4002ffd700 1 -- 172.21.15.46:0/2922877153 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004022d10 con 0x7f40101045d0 2024-05-09T00:14:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.503+0000 7f4017e3a700 1 -- 172.21.15.46:0/2922877153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40101a03c0 con 0x7f40101045d0 2024-05-09T00:14:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.506+0000 7f4002ffd700 1 -- 172.21.15.46:0/2922877153 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4004022470 con 0x7f40101045d0 2024-05-09T00:14:28.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.506+0000 7f4002ffd700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3ffc0b2880 0x7f3ffc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.507+0000 7f40153d5700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3ffc0b2880 0x7f3ffc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.507+0000 7f4002ffd700 1 -- 172.21.15.46:0/2922877153 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f40040e2f10 con 0x7f40101045d0 2024-05-09T00:14:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.507+0000 7f4026850b80 1 Processor -- start 2024-05-09T00:14:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.507+0000 7f40153d5700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3ffc0b2880 0x7f3ffc0b4d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f40101074f0 tx=0x7f400c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.507+0000 7f4026850b80 1 -- start start 2024-05-09T00:14:28.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f4026850b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c45240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f4026850b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 0x559e07c45780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f4026850b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07c3f400 0x559e07c3f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f4026850b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559e07b620a0 con 0x559e07ac24a0 2024-05-09T00:14:28.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f4026850b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559e07b079a0 con 0x559e07b3cd00 2024-05-09T00:14:28.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f4026850b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559e07b5e060 con 0x559e07c3f400 2024-05-09T00:14:28.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f40153d5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 0x559e07c45780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f40153d5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 0x559e07c45780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60920/0 (socket says 172.21.15.46:60920) 2024-05-09T00:14:28.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f40153d5700 1 -- 172.21.15.46:0/165850237 learned_addr learned my addr 172.21.15.46:0/165850237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:28.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f40163d7700 1 --2- 172.21.15.46:0/165850237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07c3f400 0x559e07c3f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f4015bd6700 1 --2- 172.21.15.46:0/165850237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c45240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f40153d5700 1 -- 172.21.15.46:0/165850237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07c3f400 msgr2=0x559e07c3f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f40153d5700 1 --2- 172.21.15.46:0/165850237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07c3f400 0x559e07c3f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f40153d5700 1 -- 172.21.15.46:0/165850237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 msgr2=0x559e07c45240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f40153d5700 1 --2- 172.21.15.46:0/165850237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c45240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f40153d5700 1 -- 172.21.15.46:0/165850237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e07af8500 con 0x559e07ac24a0 2024-05-09T00:14:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f40153d5700 1 --2- 172.21.15.46:0/165850237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 0x559e07c45780 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f400c010f40 tx=0x7f400c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910f1d5f2a687acc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.508+0000 7f3ffbfff700 1 -- 172.21.15.46:0/165850237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c00be60 con 0x559e07ac24a0 2024-05-09T00:14:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.509+0000 7f3ffbfff700 1 -- 172.21.15.46:0/165850237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f400c011db0 con 0x559e07ac24a0 2024-05-09T00:14:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.509+0000 7f3ffbfff700 1 -- 172.21.15.46:0/165850237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c01c690 con 0x559e07ac24a0 2024-05-09T00:14:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.509+0000 7f4026850b80 1 -- 172.21.15.46:0/165850237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 msgr2=0x559e07c45780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:28.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.509+0000 7f4026850b80 1 --2- 172.21.15.46:0/165850237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 0x559e07c45780 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f400c010f40 tx=0x7f400c013000 comp rx=0 tx=0).stop 2024-05-09T00:14:28.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.509+0000 7f4026850b80 1 -- 172.21.15.46:0/165850237 shutdown_connections 2024-05-09T00:14:28.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.509+0000 7f4026850b80 1 --2- 172.21.15.46:0/165850237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07c3f400 0x559e07c3f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.509+0000 7f4026850b80 1 --2- 172.21.15.46:0/165850237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c45240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.509+0000 7f4026850b80 1 --2- 172.21.15.46:0/165850237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 0x559e07c45780 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.509+0000 7f4026850b80 1 -- 172.21.15.46:0/165850237 >> 172.21.15.46:0/165850237 conn(0x559e07ac5f70 msgr2=0x559e07c3a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:28.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.510+0000 7f4026850b80 1 -- 172.21.15.46:0/165850237 shutdown_connections 2024-05-09T00:14:28.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.510+0000 7f4026850b80 1 -- 172.21.15.46:0/165850237 wait complete. 2024-05-09T00:14:28.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.510+0000 7f4026850b80 1 -- start start 2024-05-09T00:14:28.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.510+0000 7f4026850b80 1 Processor -- start 2024-05-09T00:14:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f4026850b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07ac24a0 0x559e07c49020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f4026850b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c49560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f4026850b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 0x559e07c49aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f4026850b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559e07c40160 con 0x559e07c3f400 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f4026850b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559e07afcb20 con 0x559e07b3cd00 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f4026850b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559e07c4c020 con 0x559e07ac24a0 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f4026850b80 10 client.0 _ll_register_callbacks cb 0x559e079d36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f40163d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 0x559e07c49aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f40163d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 0x559e07c49aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60928/0 (socket says 172.21.15.46:60928) 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f40163d7700 1 -- 172.21.15.46:0/4217786144 learned_addr learned my addr 172.21.15.46:0/4217786144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f40153d5700 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c49560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f4015bd6700 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07ac24a0 0x559e07c49020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f40163d7700 1 -- 172.21.15.46:0/4217786144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07ac24a0 msgr2=0x559e07c49020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f40163d7700 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07ac24a0 0x559e07c49020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f40163d7700 1 -- 172.21.15.46:0/4217786144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 msgr2=0x559e07c49560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f40163d7700 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c49560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.511+0000 7f40163d7700 1 -- 172.21.15.46:0/4217786144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e079dd490 con 0x559e07c3f400 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.512+0000 7f40163d7700 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 0x559e07c49aa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4008012d70 tx=0x7f4008010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690a5c5b2a42f74b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.512+0000 7f3ff97fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.512+0000 7f3ff97fa700 1 -- 172.21.15.46:0/4217786144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400801a070 con 0x559e07c3f400 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.512+0000 7f3ff97fa700 1 -- 172.21.15.46:0/4217786144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4008010c90 con 0x559e07c3f400 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.512+0000 7f4026850b80 1 -- 172.21.15.46:0/4217786144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f400c011db0 con 0x559e07c3f400 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.512+0000 7f3ff97fa700 1 -- 172.21.15.46:0/4217786144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4008025050 con 0x559e07c3f400 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.512+0000 7f4026850b80 10 client.15732 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.512+0000 7f4026850b80 1 -- 172.21.15.46:0/4217786144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e07c56a50 con 0x559e07c3f400 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.512+0000 7f3ff97fa700 1 -- 172.21.15.46:0/4217786144 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f400c011db0 con 0x559e07c3f400 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.512+0000 7f3ff97fa700 1 -- 172.21.15.46:0/4217786144 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4008015a30 con 0x559e07c3f400 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.513+0000 7f4026850b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.513+0000 7f4026850b80 10 client.15732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.513+0000 7f4026850b80 20 client.15732 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.513+0000 7f4026850b80 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559e07c58fe0 0x559e07c5b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.513+0000 7f4026850b80 4 client.15732 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:14:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.513+0000 7f4026850b80 1 -- 172.21.15.46:0/4217786144 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e07b6fa80 con 0x559e07c58fe0 2024-05-09T00:14:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.513+0000 7f4015bd6700 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559e07c58fe0 0x559e07c5b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.514+0000 7f4015bd6700 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559e07c58fe0 0x559e07c5b470 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad47e421e08816c3 server_cookie=1af01d14a84a845b in_seq=0 out_seq=0 2024-05-09T00:14:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.514+0000 7f3ff97fa700 10 client.15732 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:14:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.514+0000 7f3ff97fa700 1 -- 172.21.15.46:0/4217786144 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e07b6fa80 con 0x559e07c58fe0 2024-05-09T00:14:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.514+0000 7f3ff97fa700 10 client.15732 handle_command_reply: tid=0 2024-05-09T00:14:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f4026850b80 1 client.15732 shutdown 2024-05-09T00:14:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f4026850b80 1 -- 172.21.15.46:0/4217786144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 msgr2=0x559e07c49aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f4026850b80 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 0x559e07c49aa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4008012d70 tx=0x7f4008010520 comp rx=0 tx=0).stop 2024-05-09T00:14:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f4026850b80 1 -- 172.21.15.46:0/4217786144 shutdown_connections 2024-05-09T00:14:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f4026850b80 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559e07c58fe0 0x559e07c5b470 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f4026850b80 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 0x559e07c49aa0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f4026850b80 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07ac24a0 0x559e07c49020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f4026850b80 1 --2- 172.21.15.46:0/4217786144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c49560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f3ff97fa700 0 client.15732 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:14:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f4026850b80 1 -- 172.21.15.46:0/4217786144 >> 172.21.15.46:0/4217786144 conn(0x559e07ac5f70 msgr2=0x559e07c33ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f4026850b80 1 -- 172.21.15.46:0/4217786144 shutdown_connections 2024-05-09T00:14:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f4026850b80 1 -- 172.21.15.46:0/4217786144 wait complete. 2024-05-09T00:14:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.515+0000 7f4026850b80 20 client.15732 trim_cache size 0 max 16384 2024-05-09T00:14:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.530+0000 7f4026850b80 1 Processor -- start 2024-05-09T00:14:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.530+0000 7f4026850b80 1 -- start start 2024-05-09T00:14:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4026850b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 0x559e07c34a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4026850b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07b3cd00 0x559e07c47ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4026850b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07c3f400 0x559e07c54e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4026850b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f40101af240 con 0x559e07ac24a0 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4026850b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40101acad0 con 0x559e07c3f400 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4026850b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f40101aa900 con 0x559e07b3cd00 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4015bd6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 0x559e07c34a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4015bd6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 0x559e07c34a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60932/0 (socket says 172.21.15.46:60932) 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4015bd6700 1 -- 172.21.15.46:0/2094243809 learned_addr learned my addr 172.21.15.46:0/2094243809 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f40153d5700 1 --2- 172.21.15.46:0/2094243809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07b3cd00 0x559e07c47ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4015bd6700 1 -- 172.21.15.46:0/2094243809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07b3cd00 msgr2=0x559e07c47ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f40163d7700 1 --2- 172.21.15.46:0/2094243809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07c3f400 0x559e07c54e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4015bd6700 1 --2- 172.21.15.46:0/2094243809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07b3cd00 0x559e07c47ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4015bd6700 1 -- 172.21.15.46:0/2094243809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07c3f400 msgr2=0x559e07c54e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4015bd6700 1 --2- 172.21.15.46:0/2094243809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07c3f400 0x559e07c54e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4015bd6700 1 -- 172.21.15.46:0/2094243809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e07c48df0 con 0x559e07ac24a0 2024-05-09T00:14:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f40153d5700 1 --2- 172.21.15.46:0/2094243809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07b3cd00 0x559e07c47ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:14:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f4015bd6700 1 --2- 172.21.15.46:0/2094243809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 0x559e07c34a20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f40040a6640 tx=0x7f4004022810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f479b22ef168553 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f3ffb7fe700 1 -- 172.21.15.46:0/2094243809 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400406d030 con 0x559e07ac24a0 2024-05-09T00:14:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.531+0000 7f3ffb7fe700 1 -- 172.21.15.46:0/2094243809 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40040ae970 con 0x559e07ac24a0 2024-05-09T00:14:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.532+0000 7f3ffb7fe700 1 -- 172.21.15.46:0/2094243809 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40040920b0 con 0x559e07ac24a0 2024-05-09T00:14:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.532+0000 7f4026850b80 1 -- 172.21.15.46:0/2094243809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 msgr2=0x559e07c34a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.532+0000 7f4026850b80 1 --2- 172.21.15.46:0/2094243809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 0x559e07c34a20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f40040a6640 tx=0x7f4004022810 comp rx=0 tx=0).stop 2024-05-09T00:14:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.532+0000 7f4026850b80 1 -- 172.21.15.46:0/2094243809 shutdown_connections 2024-05-09T00:14:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.532+0000 7f4026850b80 1 --2- 172.21.15.46:0/2094243809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07c3f400 0x559e07c54e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.532+0000 7f4026850b80 1 --2- 172.21.15.46:0/2094243809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07ac24a0 0x559e07c34a20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.532+0000 7f4026850b80 1 --2- 172.21.15.46:0/2094243809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07b3cd00 0x559e07c47ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.532+0000 7f4026850b80 1 -- 172.21.15.46:0/2094243809 >> 172.21.15.46:0/2094243809 conn(0x559e07ac5f70 msgr2=0x559e07c477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.532+0000 7f4026850b80 1 -- 172.21.15.46:0/2094243809 shutdown_connections 2024-05-09T00:14:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.532+0000 7f4026850b80 1 -- 172.21.15.46:0/2094243809 wait complete. 2024-05-09T00:14:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.533+0000 7f4026850b80 1 -- start start 2024-05-09T00:14:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.533+0000 7f4026850b80 1 Processor -- start 2024-05-09T00:14:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.533+0000 7f4026850b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07ac24a0 0x559e07c51720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.533+0000 7f4026850b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c60270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.533+0000 7f4026850b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 0x559e07c607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f4026850b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f40101aaf90 con 0x559e07c3f400 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f4026850b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ac400 con 0x559e07b3cd00 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f4026850b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ab660 con 0x559e07ac24a0 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f4026850b80 10 client.0 _ll_register_callbacks cb 0x559e079d36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f40163d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 0x559e07c607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f40163d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 0x559e07c607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60942/0 (socket says 172.21.15.46:60942) 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f40163d7700 1 -- 172.21.15.46:0/4273850793 learned_addr learned my addr 172.21.15.46:0/4273850793 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f4015bd6700 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07ac24a0 0x559e07c51720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f40153d5700 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c60270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f40163d7700 1 -- 172.21.15.46:0/4273850793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07ac24a0 msgr2=0x559e07c51720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f40163d7700 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07ac24a0 0x559e07c51720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f40163d7700 1 -- 172.21.15.46:0/4273850793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 msgr2=0x559e07c60270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f40163d7700 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c60270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f40163d7700 1 -- 172.21.15.46:0/4273850793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e07c60eb0 con 0x559e07c3f400 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f40153d5700 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c60270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f40163d7700 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 0x559e07c607b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4008002230 tx=0x7f4008010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a579590b6da756c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f3ff97fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f3ff97fa700 1 -- 172.21.15.46:0/4273850793 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400801a040 con 0x559e07c3f400 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f4026850b80 1 -- 172.21.15.46:0/4273850793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40040ae970 con 0x559e07c3f400 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.534+0000 7f3ff97fa700 1 -- 172.21.15.46:0/4273850793 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4008005e70 con 0x559e07c3f400 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.535+0000 7f3ff97fa700 1 -- 172.21.15.46:0/4273850793 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4008003a00 con 0x559e07c3f400 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.535+0000 7f4026850b80 10 client.15738 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.535+0000 7f4026850b80 1 -- 172.21.15.46:0/4273850793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e07cc3650 con 0x559e07c3f400 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.535+0000 7f3ff97fa700 1 -- 172.21.15.46:0/4273850793 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40040ae970 con 0x559e07c3f400 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.535+0000 7f3ff97fa700 1 -- 172.21.15.46:0/4273850793 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f40080293f0 con 0x559e07c3f400 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.535+0000 7f4026850b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.535+0000 7f4026850b80 10 client.15738 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.535+0000 7f4026850b80 20 client.15738 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.535+0000 7f4026850b80 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559e07cc3d30 0x559e07cc6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.535+0000 7f4026850b80 4 client.15738 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.535+0000 7f4026850b80 1 -- 172.21.15.46:0/4273850793 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e07c44ca0 con 0x559e07cc3d30 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.535+0000 7f4015bd6700 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559e07cc3d30 0x559e07cc6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f3ff97fa700 10 client.15738 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4015bd6700 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559e07cc3d30 0x559e07cc6150 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9714a1dd3b03634 server_cookie=d7cc56acf16f02da in_seq=0 out_seq=0 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f3ff97fa700 1 -- 172.21.15.46:0/4273850793 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e07c44ca0 con 0x559e07cc3d30 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f3ff97fa700 10 client.15738 handle_command_reply: tid=0 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4026850b80 1 client.15738 shutdown 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4026850b80 1 -- 172.21.15.46:0/4273850793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 msgr2=0x559e07c607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4026850b80 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 0x559e07c607b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4008002230 tx=0x7f4008010f40 comp rx=0 tx=0).stop 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4026850b80 1 -- 172.21.15.46:0/4273850793 shutdown_connections 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4026850b80 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x559e07cc3d30 0x559e07cc6150 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4026850b80 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e07c3f400 0x559e07c607b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4026850b80 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559e07ac24a0 0x559e07c51720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4026850b80 1 --2- 172.21.15.46:0/4273850793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e07b3cd00 0x559e07c60270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f3ff97fa700 0 client.15738 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:14:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4026850b80 1 -- 172.21.15.46:0/4273850793 >> 172.21.15.46:0/4273850793 conn(0x559e07ac5f70 msgr2=0x559e07c34780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4026850b80 1 -- 172.21.15.46:0/4273850793 shutdown_connections 2024-05-09T00:14:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4026850b80 1 -- 172.21.15.46:0/4273850793 wait complete. 2024-05-09T00:14:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.536+0000 7f4026850b80 20 client.15738 trim_cache size 0 max 16384 2024-05-09T00:14:28.539 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:14:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.537+0000 7f4017e3a700 1 -- 172.21.15.46:0/2922877153 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3ffc0b2880 msgr2=0x7f3ffc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.537+0000 7f4017e3a700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3ffc0b2880 0x7f3ffc0b4d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f40101074f0 tx=0x7f400c00b410 comp rx=0 tx=0).stop 2024-05-09T00:14:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.537+0000 7f4017e3a700 1 -- 172.21.15.46:0/2922877153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40101045d0 msgr2=0x7f401019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.537+0000 7f4017e3a700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40101045d0 0x7f401019ae30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f400400b580 tx=0x7f40040047b0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.537+0000 7f4017e3a700 1 -- 172.21.15.46:0/2922877153 shutdown_connections 2024-05-09T00:14:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.537+0000 7f4017e3a700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3ffc0b2880 0x7f3ffc0b4d90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.537+0000 7f4017e3a700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4010109850 0x7f401019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.537+0000 7f4017e3a700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40101045d0 0x7f401019ae30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.537+0000 7f4017e3a700 1 --2- 172.21.15.46:0/2922877153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4010108f00 0x7f401019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.537+0000 7f4017e3a700 1 -- 172.21.15.46:0/2922877153 >> 172.21.15.46:0/2922877153 conn(0x7f40100ffea0 msgr2=0x7f4010102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.537+0000 7f4017e3a700 1 -- 172.21.15.46:0/2922877153 shutdown_connections 2024-05-09T00:14:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:28.537+0000 7f4017e3a700 1 -- 172.21.15.46:0/2922877153 wait complete. 2024-05-09T00:14:28.810 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:28 smithi046 ceph-mon[30151]: pgmap v469: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:29.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:28 smithi169 ceph-mon[34777]: pgmap v469: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:29.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:28 smithi142 ceph-mon[34533]: pgmap v469: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:29.231 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:14:29.484 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:30.974 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:30 smithi046 ceph-mon[30151]: pgmap v470: 97 pgs: 97 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:31.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:30 smithi169 ceph-mon[34777]: pgmap v470: 97 pgs: 97 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:31.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:30 smithi142 ceph-mon[34533]: pgmap v470: 97 pgs: 97 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:31.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.360+0000 7f15c2e9b700 1 -- 172.21.15.46:0/1514328723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15bc1054e0 msgr2=0x7f15bc1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:31.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.360+0000 7f15c2e9b700 1 --2- 172.21.15.46:0/1514328723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15bc1054e0 0x7f15bc1058f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009990 tx=0x7f15ac009ca0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.360+0000 7f15c2e9b700 1 -- 172.21.15.46:0/1514328723 shutdown_connections 2024-05-09T00:14:31.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.360+0000 7f15c2e9b700 1 --2- 172.21.15.46:0/1514328723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15bc10e910 0x7f15bc110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.360+0000 7f15c2e9b700 1 --2- 172.21.15.46:0/1514328723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15bc1054e0 0x7f15bc1058f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.360+0000 7f15c2e9b700 1 --2- 172.21.15.46:0/1514328723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15bc105e30 0x7f15bc10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.360+0000 7f15c2e9b700 1 -- 172.21.15.46:0/1514328723 >> 172.21.15.46:0/1514328723 conn(0x7f15bc0fef30 msgr2=0x7f15bc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:31.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.360+0000 7f15c2e9b700 1 -- 172.21.15.46:0/1514328723 shutdown_connections 2024-05-09T00:14:31.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.360+0000 7f15c2e9b700 1 -- 172.21.15.46:0/1514328723 wait complete. 2024-05-09T00:14:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15c2e9b700 1 Processor -- start 2024-05-09T00:14:31.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15c2e9b700 1 -- start start 2024-05-09T00:14:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15c2e9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15bc1054e0 0x7f15bc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15c2e9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15bc105e30 0x7f15bc19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15c2e9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15bc10e910 0x7f15bc19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15c2e9b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc114480 con 0x7f15bc105e30 2024-05-09T00:14:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15c2e9b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc114300 con 0x7f15bc1054e0 2024-05-09T00:14:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15c2e9b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc114600 con 0x7f15bc10e910 2024-05-09T00:14:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15bc105e30 0x7f15bc19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15bc105e30 0x7f15bc19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60956/0 (socket says 172.21.15.46:60956) 2024-05-09T00:14:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15bbfff700 1 -- 172.21.15.46:0/2352686462 learned_addr learned my addr 172.21.15.46:0/2352686462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15c1438700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15bc10e910 0x7f15bc19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15c0c37700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15bc1054e0 0x7f15bc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15bbfff700 1 -- 172.21.15.46:0/2352686462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15bc10e910 msgr2=0x7f15bc19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15bbfff700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15bc10e910 0x7f15bc19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15bbfff700 1 -- 172.21.15.46:0/2352686462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15bc1054e0 msgr2=0x7f15bc19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15bbfff700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15bc1054e0 0x7f15bc19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.361+0000 7f15bbfff700 1 -- 172.21.15.46:0/2352686462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc19fc70 con 0x7f15bc105e30 2024-05-09T00:14:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.362+0000 7f15bbfff700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15bc105e30 0x7f15bc19b390 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f15a400d980 tx=0x7f15a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.362+0000 7f15b9ffb700 1 -- 172.21.15.46:0/2352686462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4016070 con 0x7f15bc105e30 2024-05-09T00:14:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.362+0000 7f15b9ffb700 1 -- 172.21.15.46:0/2352686462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a4004830 con 0x7f15bc105e30 2024-05-09T00:14:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.362+0000 7f15c2e9b700 1 -- 172.21.15.46:0/2352686462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc19ff60 con 0x7f15bc105e30 2024-05-09T00:14:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.362+0000 7f15b9ffb700 1 -- 172.21.15.46:0/2352686462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4005540 con 0x7f15bc105e30 2024-05-09T00:14:31.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.362+0000 7f15c2e9b700 1 -- 172.21.15.46:0/2352686462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc1a7880 con 0x7f15bc105e30 2024-05-09T00:14:31.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.364+0000 7f15b9ffb700 1 -- 172.21.15.46:0/2352686462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f15a4004220 con 0x7f15bc105e30 2024-05-09T00:14:31.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.365+0000 7f15b9ffb700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f15a80b2880 0x7f15a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.365+0000 7f15c0c37700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f15a80b2880 0x7f15a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.365+0000 7f15b9ffb700 1 -- 172.21.15.46:0/2352686462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f15a40e1fd0 con 0x7f15bc105e30 2024-05-09T00:14:31.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15d18b1b80 1 Processor -- start 2024-05-09T00:14:31.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15d18b1b80 1 -- start start 2024-05-09T00:14:31.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15c0c37700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f15a80b2880 0x7f15a80b4d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00b5c0 tx=0x7f15ac005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:31.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15d18b1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 0x55e6873fdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15d18b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68727c4a0 0x55e6873fe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15d18b1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6874021e0 0x55e6874025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15d18b1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e68731c0a0 con 0x55e6872f6d00 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15d18b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e6872c19a0 con 0x55e68727c4a0 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15d18b1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e687318060 con 0x55e6874021e0 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15c0c37700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 0x55e6873fdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15c0c37700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 0x55e6873fdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60970/0 (socket says 172.21.15.46:60970) 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15c0c37700 1 -- 172.21.15.46:0/3972753799 learned_addr learned my addr 172.21.15.46:0/3972753799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15bbfff700 1 --2- 172.21.15.46:0/3972753799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68727c4a0 0x55e6873fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.366+0000 7f15c1438700 1 --2- 172.21.15.46:0/3972753799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6874021e0 0x55e6874025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.367+0000 7f15c0c37700 1 -- 172.21.15.46:0/3972753799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6874021e0 msgr2=0x55e6874025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.367+0000 7f15c0c37700 1 --2- 172.21.15.46:0/3972753799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6874021e0 0x55e6874025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.367+0000 7f15c0c37700 1 -- 172.21.15.46:0/3972753799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68727c4a0 msgr2=0x55e6873fe030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.367+0000 7f15c0c37700 1 --2- 172.21.15.46:0/3972753799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68727c4a0 0x55e6873fe030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.367+0000 7f15c0c37700 1 -- 172.21.15.46:0/3972753799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6872b2500 con 0x55e6872f6d00 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.367+0000 7f15c0c37700 1 --2- 172.21.15.46:0/3972753799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 0x55e6873fdad0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f15ac015000 tx=0x7f15ac01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d0bd4e55363b32 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:31.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.367+0000 7f15b2ffd700 1 -- 172.21.15.46:0/3972753799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac005460 con 0x55e6872f6d00 2024-05-09T00:14:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.367+0000 7f15b2ffd700 1 -- 172.21.15.46:0/3972753799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15ac005600 con 0x55e6872f6d00 2024-05-09T00:14:31.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.367+0000 7f15b2ffd700 1 -- 172.21.15.46:0/3972753799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00fac0 con 0x55e6872f6d00 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.368+0000 7f15d18b1b80 1 -- 172.21.15.46:0/3972753799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 msgr2=0x55e6873fdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.368+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/3972753799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 0x55e6873fdad0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f15ac015000 tx=0x7f15ac01e000 comp rx=0 tx=0).stop 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.368+0000 7f15d18b1b80 1 -- 172.21.15.46:0/3972753799 shutdown_connections 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.368+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/3972753799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6874021e0 0x55e6874025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.368+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/3972753799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 0x55e6873fdad0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.368+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/3972753799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68727c4a0 0x55e6873fe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.368+0000 7f15d18b1b80 1 -- 172.21.15.46:0/3972753799 >> 172.21.15.46:0/3972753799 conn(0x55e68727ff70 msgr2=0x55e6873f4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.368+0000 7f15d18b1b80 1 -- 172.21.15.46:0/3972753799 shutdown_connections 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.368+0000 7f15d18b1b80 1 -- 172.21.15.46:0/3972753799 wait complete. 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.368+0000 7f15d18b1b80 1 -- start start 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15d18b1b80 1 Processor -- start 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15d18b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68727c4a0 0x55e6874073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15d18b1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6872f6d00 0x55e6874078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15d18b1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6874021e0 0x55e687407e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15d18b1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e687402e40 con 0x55e6874021e0 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15d18b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e6872b6b20 con 0x55e68727c4a0 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15d18b1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e687402fc0 con 0x55e6872f6d00 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15d18b1b80 10 client.0 _ll_register_callbacks cb 0x55e68718d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15c1438700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6874021e0 0x55e687407e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15c1438700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6874021e0 0x55e687407e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60982/0 (socket says 172.21.15.46:60982) 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15c1438700 1 -- 172.21.15.46:0/2111575004 learned_addr learned my addr 172.21.15.46:0/2111575004 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15bbfff700 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6872f6d00 0x55e6874078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.369+0000 7f15c0c37700 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68727c4a0 0x55e6874073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f15c1438700 1 -- 172.21.15.46:0/2111575004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6872f6d00 msgr2=0x55e6874078f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f15c1438700 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6872f6d00 0x55e6874078f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f15c1438700 1 -- 172.21.15.46:0/2111575004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68727c4a0 msgr2=0x55e6874073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f15c1438700 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68727c4a0 0x55e6874073b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f15c1438700 1 -- 172.21.15.46:0/2111575004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e687197490 con 0x55e6874021e0 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f15c0c37700 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68727c4a0 0x55e6874073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f15c1438700 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6874021e0 0x55e687407e30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f15b4004070 tx=0x7f15b4004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6461e7d11e5681 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f159bfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f159bfff700 1 -- 172.21.15.46:0/2111575004 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b401a020 con 0x55e6874021e0 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f159bfff700 1 -- 172.21.15.46:0/2111575004 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f15b40079d0 con 0x55e6874021e0 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f15d18b1b80 1 -- 172.21.15.46:0/2111575004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15ac005600 con 0x55e6874021e0 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f159bfff700 1 -- 172.21.15.46:0/2111575004 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f15b4007bd0 con 0x55e6874021e0 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f15d18b1b80 10 client.15750 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f15d18b1b80 1 -- 172.21.15.46:0/2111575004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e687414cf0 con 0x55e6874021e0 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.370+0000 7f159bfff700 1 -- 172.21.15.46:0/2111575004 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15ac005600 con 0x55e6874021e0 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.371+0000 7f159bfff700 1 -- 172.21.15.46:0/2111575004 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f15b4021430 con 0x55e6874021e0 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.371+0000 7f15d18b1b80 10 client.15750 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.371+0000 7f15d18b1b80 10 client.15750 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.371+0000 7f15d18b1b80 20 client.15750 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.371+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55e687417330 0x55e687419790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.371+0000 7f15d18b1b80 4 client.15750 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.371+0000 7f15d18b1b80 1 -- 172.21.15.46:0/2111575004 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e687329a80 con 0x55e687417330 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.371+0000 7f15c0c37700 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55e687417330 0x55e687419790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.371+0000 7f159bfff700 10 client.15750 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.371+0000 7f15c0c37700 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55e687417330 0x55e687419790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6754f879d9b7b1db server_cookie=40c5eed902715ec4 in_seq=0 out_seq=0 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.372+0000 7f159bfff700 1 -- 172.21.15.46:0/2111575004 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e687329a80 con 0x55e687417330 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.372+0000 7f159bfff700 10 client.15750 handle_command_reply: tid=0 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.372+0000 7f15d18b1b80 1 client.15750 shutdown 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.372+0000 7f15d18b1b80 1 -- 172.21.15.46:0/2111575004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6874021e0 msgr2=0x55e687407e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.372+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6874021e0 0x55e687407e30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f15b4004070 tx=0x7f15b4004380 comp rx=0 tx=0).stop 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.372+0000 7f15d18b1b80 1 -- 172.21.15.46:0/2111575004 shutdown_connections 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.372+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55e687417330 0x55e687419790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.372+0000 7f159bfff700 0 client.15750 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.372+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6874021e0 0x55e687407e30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.372+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68727c4a0 0x55e6874073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.372+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/2111575004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6872f6d00 0x55e6874078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.372+0000 7f15d18b1b80 1 -- 172.21.15.46:0/2111575004 >> 172.21.15.46:0/2111575004 conn(0x55e68727ff70 msgr2=0x55e6873edff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.373+0000 7f15d18b1b80 1 -- 172.21.15.46:0/2111575004 shutdown_connections 2024-05-09T00:14:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.373+0000 7f15d18b1b80 1 -- 172.21.15.46:0/2111575004 wait complete. 2024-05-09T00:14:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.373+0000 7f15d18b1b80 20 client.15750 trim_cache size 0 max 16384 2024-05-09T00:14:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.388+0000 7f15d18b1b80 1 Processor -- start 2024-05-09T00:14:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.388+0000 7f15d18b1b80 1 -- start start 2024-05-09T00:14:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.388+0000 7f15d18b1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68727c4a0 0x55e687407c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.388+0000 7f15d18b1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6872f6d00 0x55e6873f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.388+0000 7f15d18b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6874021e0 0x55e687405b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.388+0000 7f15d18b1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc1af320 con 0x55e68727c4a0 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.388+0000 7f15d18b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc1acbb0 con 0x55e6874021e0 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.388+0000 7f15d18b1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc1aa9e0 con 0x55e6872f6d00 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.388+0000 7f15c0c37700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68727c4a0 0x55e687407c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15c0c37700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68727c4a0 0x55e687407c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60994/0 (socket says 172.21.15.46:60994) 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15c0c37700 1 -- 172.21.15.46:0/4085198398 learned_addr learned my addr 172.21.15.46:0/4085198398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15bbfff700 1 --2- 172.21.15.46:0/4085198398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6872f6d00 0x55e6873f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15c0c37700 1 -- 172.21.15.46:0/4085198398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6872f6d00 msgr2=0x55e6873f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15c0c37700 1 --2- 172.21.15.46:0/4085198398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6872f6d00 0x55e6873f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15c0c37700 1 -- 172.21.15.46:0/4085198398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6874021e0 msgr2=0x55e687405b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15c1438700 1 --2- 172.21.15.46:0/4085198398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6874021e0 0x55e687405b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15c0c37700 1 --2- 172.21.15.46:0/4085198398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6874021e0 0x55e687405b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15c0c37700 1 -- 172.21.15.46:0/4085198398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6873f5ed0 con 0x55e68727c4a0 2024-05-09T00:14:31.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15c0c37700 1 --2- 172.21.15.46:0/4085198398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68727c4a0 0x55e687407c70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f15ac006010 tx=0x7f15ac00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf83d7ce738e728a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:31.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15b27fc700 1 -- 172.21.15.46:0/4085198398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0056a0 con 0x55e68727c4a0 2024-05-09T00:14:31.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15b27fc700 1 -- 172.21.15.46:0/4085198398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15ac00bc80 con 0x55e68727c4a0 2024-05-09T00:14:31.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.389+0000 7f15b27fc700 1 -- 172.21.15.46:0/4085198398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac03c1e0 con 0x55e68727c4a0 2024-05-09T00:14:31.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.390+0000 7f15d18b1b80 1 -- 172.21.15.46:0/4085198398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68727c4a0 msgr2=0x55e687407c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:31.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.390+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/4085198398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68727c4a0 0x55e687407c70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f15ac006010 tx=0x7f15ac00f6a0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.390+0000 7f15d18b1b80 1 -- 172.21.15.46:0/4085198398 shutdown_connections 2024-05-09T00:14:31.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.390+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/4085198398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6874021e0 0x55e687405b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.390+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/4085198398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e68727c4a0 0x55e687407c70 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.390+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/4085198398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6872f6d00 0x55e6873f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.390+0000 7f15d18b1b80 1 -- 172.21.15.46:0/4085198398 >> 172.21.15.46:0/4085198398 conn(0x55e68727ff70 msgr2=0x55e6873f83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:31.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.390+0000 7f15d18b1b80 1 -- 172.21.15.46:0/4085198398 shutdown_connections 2024-05-09T00:14:31.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.390+0000 7f15d18b1b80 1 -- 172.21.15.46:0/4085198398 wait complete. 2024-05-09T00:14:31.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15d18b1b80 1 -- start start 2024-05-09T00:14:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15d18b1b80 1 Processor -- start 2024-05-09T00:14:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15d18b1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e68727c4a0 0x55e6874065b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15d18b1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 0x55e687406af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15d18b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6874021e0 0x55e687407030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15d18b1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc1ab070 con 0x55e6872f6d00 2024-05-09T00:14:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15d18b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc1ac4e0 con 0x55e6874021e0 2024-05-09T00:14:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15d18b1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc1ab740 con 0x55e68727c4a0 2024-05-09T00:14:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15d18b1b80 10 client.0 _ll_register_callbacks cb 0x55e68718d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 0x55e687406af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 0x55e687406af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32774/0 (socket says 172.21.15.46:32774) 2024-05-09T00:14:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15bbfff700 1 -- 172.21.15.46:0/992537591 learned_addr learned my addr 172.21.15.46:0/992537591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15c1438700 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6874021e0 0x55e687407030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.391+0000 7f15c0c37700 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e68727c4a0 0x55e6874065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f15bbfff700 1 -- 172.21.15.46:0/992537591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e68727c4a0 msgr2=0x55e6874065b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f15bbfff700 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e68727c4a0 0x55e6874065b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f15bbfff700 1 -- 172.21.15.46:0/992537591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6874021e0 msgr2=0x55e687407030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f15bbfff700 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6874021e0 0x55e687407030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f15bbfff700 1 -- 172.21.15.46:0/992537591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e68740e640 con 0x55e6872f6d00 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f15c0c37700 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e68727c4a0 0x55e6874065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f15c1438700 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6874021e0 0x55e687407030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f15bbfff700 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 0x55e687406af0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f15a40dfad0 tx=0x7f15a40a55e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5135a200eb68b130 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f159bfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f159bfff700 1 -- 172.21.15.46:0/992537591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a40ad9a0 con 0x55e6872f6d00 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f159bfff700 1 -- 172.21.15.46:0/992537591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f15a40910b0 con 0x55e6872f6d00 2024-05-09T00:14:31.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f15d18b1b80 1 -- 172.21.15.46:0/992537591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15ac00bc80 con 0x55e6872f6d00 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f159bfff700 1 -- 172.21.15.46:0/992537591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f15a40b2450 con 0x55e6872f6d00 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f15d18b1b80 10 client.15756 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f15d18b1b80 1 -- 172.21.15.46:0/992537591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e68740f8d0 con 0x55e6872f6d00 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.392+0000 7f159bfff700 1 -- 172.21.15.46:0/992537591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15ac00bc80 con 0x55e6872f6d00 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.393+0000 7f159bfff700 1 -- 172.21.15.46:0/992537591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f15a40b2050 con 0x55e6872f6d00 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.393+0000 7f15d18b1b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.393+0000 7f15d18b1b80 10 client.15756 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.393+0000 7f15d18b1b80 20 client.15756 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.393+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55e687417330 0x55e687483b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.393+0000 7f15d18b1b80 4 client.15756 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.393+0000 7f15d18b1b80 1 -- 172.21.15.46:0/992537591 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e687323b20 con 0x55e687417330 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.393+0000 7f15c0c37700 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55e687417330 0x55e687483b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.393+0000 7f159bfff700 10 client.15756 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.393+0000 7f15c0c37700 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55e687417330 0x55e687483b10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8c45ee8555479e85 server_cookie=28af27bb63f71bbb in_seq=0 out_seq=0 2024-05-09T00:14:31.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f159bfff700 1 -- 172.21.15.46:0/992537591 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e687323b20 con 0x55e687417330 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f159bfff700 10 client.15756 handle_command_reply: tid=0 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f15d18b1b80 1 client.15756 shutdown 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f15d18b1b80 1 -- 172.21.15.46:0/992537591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 msgr2=0x55e687406af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 0x55e687406af0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f15a40dfad0 tx=0x7f15a40a55e0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f15d18b1b80 1 -- 172.21.15.46:0/992537591 shutdown_connections 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55e687417330 0x55e687483b10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6874021e0 0x55e687407030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e68727c4a0 0x55e6874065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f15d18b1b80 1 --2- 172.21.15.46:0/992537591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6872f6d00 0x55e687406af0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f159bfff700 0 client.15756 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f15d18b1b80 1 -- 172.21.15.46:0/992537591 >> 172.21.15.46:0/992537591 conn(0x55e68727ff70 msgr2=0x55e6873fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f15d18b1b80 1 -- 172.21.15.46:0/992537591 shutdown_connections 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f15d18b1b80 1 -- 172.21.15.46:0/992537591 wait complete. 2024-05-09T00:14:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.394+0000 7f15d18b1b80 20 client.15756 trim_cache size 0 max 16384 2024-05-09T00:14:31.398 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:14:31.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.395+0000 7f15c2e9b700 1 -- 172.21.15.46:0/2352686462 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f15a80b2880 msgr2=0x7f15a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:31.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.395+0000 7f15c2e9b700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f15a80b2880 0x7f15a80b4d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00b5c0 tx=0x7f15ac005960 comp rx=0 tx=0).stop 2024-05-09T00:14:31.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.395+0000 7f15c2e9b700 1 -- 172.21.15.46:0/2352686462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15bc105e30 msgr2=0x7f15bc19b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:31.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.395+0000 7f15c2e9b700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15bc105e30 0x7f15bc19b390 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f15a400d980 tx=0x7f15a400dc90 comp rx=0 tx=0).stop 2024-05-09T00:14:31.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.395+0000 7f15c2e9b700 1 -- 172.21.15.46:0/2352686462 shutdown_connections 2024-05-09T00:14:31.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.395+0000 7f15c2e9b700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f15a80b2880 0x7f15a80b4d90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.395+0000 7f15c2e9b700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15bc10e910 0x7f15bc19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.395+0000 7f15c2e9b700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15bc1054e0 0x7f15bc19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.395+0000 7f15c2e9b700 1 --2- 172.21.15.46:0/2352686462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15bc105e30 0x7f15bc19b390 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:31.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.395+0000 7f15c2e9b700 1 -- 172.21.15.46:0/2352686462 >> 172.21.15.46:0/2352686462 conn(0x7f15bc0fef30 msgr2=0x7f15bc0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:31.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.395+0000 7f15c2e9b700 1 -- 172.21.15.46:0/2352686462 shutdown_connections 2024-05-09T00:14:31.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:31.395+0000 7f15c2e9b700 1 -- 172.21.15.46:0/2352686462 wait complete. 2024-05-09T00:14:32.297 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:14:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:32.939 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:32 smithi046 ceph-mon[30151]: pgmap v471: 97 pgs: 97 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:14:32.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:32 smithi142 ceph-mon[34533]: pgmap v471: 97 pgs: 97 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:14:33.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:32 smithi169 ceph-mon[34777]: pgmap v471: 97 pgs: 97 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:14:34.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.475+0000 7fed4b64a700 1 -- 172.21.15.46:0/901019044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed441034c0 msgr2=0x7fed441038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:34.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.475+0000 7fed4b64a700 1 --2- 172.21.15.46:0/901019044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed441034c0 0x7fed441038d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fed400099c0 tx=0x7fed40009cd0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.475+0000 7fed4b64a700 1 -- 172.21.15.46:0/901019044 shutdown_connections 2024-05-09T00:14:34.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.475+0000 7fed4b64a700 1 --2- 172.21.15.46:0/901019044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed44108740 0x7fed441100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.475+0000 7fed4b64a700 1 --2- 172.21.15.46:0/901019044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed441034c0 0x7fed441038d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.475+0000 7fed4b64a700 1 --2- 172.21.15.46:0/901019044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed44107df0 0x7fed44108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.475+0000 7fed4b64a700 1 -- 172.21.15.46:0/901019044 >> 172.21.15.46:0/901019044 conn(0x7fed440fedc0 msgr2=0x7fed441011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.475+0000 7fed4b64a700 1 -- 172.21.15.46:0/901019044 shutdown_connections 2024-05-09T00:14:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.475+0000 7fed4b64a700 1 -- 172.21.15.46:0/901019044 wait complete. 2024-05-09T00:14:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.476+0000 7fed4b64a700 1 Processor -- start 2024-05-09T00:14:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.476+0000 7fed4b64a700 1 -- start start 2024-05-09T00:14:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.476+0000 7fed4b64a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed441034c0 0x7fed4419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.476+0000 7fed4b64a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed44107df0 0x7fed4419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.476+0000 7fed4b64a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed44108740 0x7fed4419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.476+0000 7fed4b64a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed44113390 con 0x7fed44107df0 2024-05-09T00:14:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.476+0000 7fed4b64a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed44113210 con 0x7fed441034c0 2024-05-09T00:14:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.476+0000 7fed4b64a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed44113510 con 0x7fed44108740 2024-05-09T00:14:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.476+0000 7fed48be5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed44107df0 0x7fed4419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.476+0000 7fed48be5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed44107df0 0x7fed4419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32786/0 (socket says 172.21.15.46:32786) 2024-05-09T00:14:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.476+0000 7fed48be5700 1 -- 172.21.15.46:0/3246670777 learned_addr learned my addr 172.21.15.46:0/3246670777 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.476+0000 7fed493e6700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed441034c0 0x7fed4419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed49be7700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed44108740 0x7fed4419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed48be5700 1 -- 172.21.15.46:0/3246670777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed44108740 msgr2=0x7fed4419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed48be5700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed44108740 0x7fed4419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed48be5700 1 -- 172.21.15.46:0/3246670777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed441034c0 msgr2=0x7fed4419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed48be5700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed441034c0 0x7fed4419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed48be5700 1 -- 172.21.15.46:0/3246670777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4419fcc0 con 0x7fed44107df0 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed49be7700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed44108740 0x7fed4419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed493e6700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed441034c0 0x7fed4419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed48be5700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed44107df0 0x7fed4419b390 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fed3400d980 tx=0x7fed3400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed3a7fc700 1 -- 172.21.15.46:0/3246670777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34017070 con 0x7fed44107df0 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed4b64a700 1 -- 172.21.15.46:0/3246670777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4419ffb0 con 0x7fed44107df0 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed3a7fc700 1 -- 172.21.15.46:0/3246670777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed34010460 con 0x7fed44107df0 2024-05-09T00:14:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed3a7fc700 1 -- 172.21.15.46:0/3246670777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3400de60 con 0x7fed44107df0 2024-05-09T00:14:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.477+0000 7fed4b64a700 1 -- 172.21.15.46:0/3246670777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed441a78d0 con 0x7fed44107df0 2024-05-09T00:14:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.480+0000 7fed3a7fc700 1 -- 172.21.15.46:0/3246670777 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fed34010610 con 0x7fed44107df0 2024-05-09T00:14:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.481+0000 7fed3a7fc700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fed300b2880 0x7fed300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.481+0000 7fed5a060b80 1 Processor -- start 2024-05-09T00:14:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.481+0000 7fed493e6700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fed300b2880 0x7fed300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.481+0000 7fed3a7fc700 1 -- 172.21.15.46:0/3246670777 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fed340e1e10 con 0x7fed44107df0 2024-05-09T00:14:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.481+0000 7fed5a060b80 1 -- start start 2024-05-09T00:14:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.481+0000 7fed5a060b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206ae1d00 0x55b206be8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.481+0000 7fed5a060b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 0x55b206be9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed5a060b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206bed1e0 0x55b206bed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed5a060b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b206b070a0 con 0x55b206a674a0 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed5a060b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b206aac9a0 con 0x55b206bed1e0 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed5a060b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b206b03060 con 0x55b206ae1d00 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed48be5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 0x55b206be9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed493e6700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fed300b2880 0x7fed300b4d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fed40000c00 tx=0x7fed4001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed48be5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 0x55b206be9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32790/0 (socket says 172.21.15.46:32790) 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed48be5700 1 -- 172.21.15.46:0/3884811050 learned_addr learned my addr 172.21.15.46:0/3884811050 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed49be7700 1 --2- 172.21.15.46:0/3884811050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206bed1e0 0x55b206bed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed493e6700 1 --2- 172.21.15.46:0/3884811050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206ae1d00 0x55b206be8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed48be5700 1 -- 172.21.15.46:0/3884811050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206ae1d00 msgr2=0x55b206be8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed48be5700 1 --2- 172.21.15.46:0/3884811050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206ae1d00 0x55b206be8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed48be5700 1 -- 172.21.15.46:0/3884811050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206bed1e0 msgr2=0x55b206bed5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed48be5700 1 --2- 172.21.15.46:0/3884811050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206bed1e0 0x55b206bed5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed48be5700 1 -- 172.21.15.46:0/3884811050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b206a9d500 con 0x55b206a674a0 2024-05-09T00:14:34.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed48be5700 1 --2- 172.21.15.46:0/3884811050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 0x55b206be9030 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fed340a5470 tx=0x7fed340a8e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce81b16e8f4c39c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:34.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed2f7fe700 1 -- 172.21.15.46:0/3884811050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34020070 con 0x55b206a674a0 2024-05-09T00:14:34.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed2f7fe700 1 -- 172.21.15.46:0/3884811050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed340ad7a0 con 0x55b206a674a0 2024-05-09T00:14:34.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.482+0000 7fed2f7fe700 1 -- 172.21.15.46:0/3884811050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340a8850 con 0x55b206a674a0 2024-05-09T00:14:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.483+0000 7fed5a060b80 1 -- 172.21.15.46:0/3884811050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 msgr2=0x55b206be9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.483+0000 7fed5a060b80 1 --2- 172.21.15.46:0/3884811050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 0x55b206be9030 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fed340a5470 tx=0x7fed340a8e70 comp rx=0 tx=0).stop 2024-05-09T00:14:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.483+0000 7fed5a060b80 1 -- 172.21.15.46:0/3884811050 shutdown_connections 2024-05-09T00:14:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.483+0000 7fed5a060b80 1 --2- 172.21.15.46:0/3884811050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206bed1e0 0x55b206bed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.483+0000 7fed5a060b80 1 --2- 172.21.15.46:0/3884811050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206ae1d00 0x55b206be8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.483+0000 7fed5a060b80 1 --2- 172.21.15.46:0/3884811050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 0x55b206be9030 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.483+0000 7fed5a060b80 1 -- 172.21.15.46:0/3884811050 >> 172.21.15.46:0/3884811050 conn(0x55b206a6af70 msgr2=0x55b206bdf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.483+0000 7fed5a060b80 1 -- 172.21.15.46:0/3884811050 shutdown_connections 2024-05-09T00:14:34.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.483+0000 7fed5a060b80 1 -- 172.21.15.46:0/3884811050 wait complete. 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.484+0000 7fed5a060b80 1 -- start start 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.484+0000 7fed5a060b80 1 Processor -- start 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.484+0000 7fed5a060b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 0x55b206bf23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.484+0000 7fed5a060b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206ae1d00 0x55b206bf2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.484+0000 7fed5a060b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206bed1e0 0x55b206bf2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed5a060b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b206bedf00 con 0x55b206a674a0 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed5a060b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b206aa1b20 con 0x55b206ae1d00 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed5a060b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b206bf5470 con 0x55b206bed1e0 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed493e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 0x55b206bf23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed493e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 0x55b206bf23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32792/0 (socket says 172.21.15.46:32792) 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed493e6700 1 -- 172.21.15.46:0/1745556500 learned_addr learned my addr 172.21.15.46:0/1745556500 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed5a060b80 10 client.0 _ll_register_callbacks cb 0x55b2069786c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:34.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed48be5700 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206ae1d00 0x55b206bf2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed49be7700 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206bed1e0 0x55b206bf2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed493e6700 1 -- 172.21.15.46:0/1745556500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206bed1e0 msgr2=0x55b206bf2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed493e6700 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206bed1e0 0x55b206bf2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed493e6700 1 -- 172.21.15.46:0/1745556500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206ae1d00 msgr2=0x55b206bf2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed493e6700 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206ae1d00 0x55b206bf2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed493e6700 1 -- 172.21.15.46:0/1745556500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b206982490 con 0x55b206a674a0 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed493e6700 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 0x55b206bf23f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fed400099c0 tx=0x7fed40009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6b74367188d351 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.485+0000 7fed2cff9700 1 -- 172.21.15.46:0/1745556500 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40033060 con 0x55b206a674a0 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed2cff9700 1 -- 172.21.15.46:0/1745556500 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fed4002e430 con 0x55b206a674a0 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed5a060b80 1 -- 172.21.15.46:0/1745556500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed340ad7a0 con 0x55b206a674a0 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed2cff9700 1 -- 172.21.15.46:0/1745556500 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed4002e630 con 0x55b206a674a0 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed5a060b80 10 client.15768 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed5a060b80 1 -- 172.21.15.46:0/1745556500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b206bffd80 con 0x55b206a674a0 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed2cff9700 1 -- 172.21.15.46:0/1745556500 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed340ad7a0 con 0x55b206a674a0 2024-05-09T00:14:34.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed2cff9700 1 -- 172.21.15.46:0/1745556500 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fed4002d070 con 0x55b206a674a0 2024-05-09T00:14:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed5a060b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed5a060b80 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed5a060b80 20 client.15768 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed5a060b80 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b206c023c0 0x55b206c047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed5a060b80 4 client.15768 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:14:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.486+0000 7fed5a060b80 1 -- 172.21.15.46:0/1745556500 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b206b14a80 con 0x55b206c023c0 2024-05-09T00:14:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.487+0000 7fed48be5700 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b206c023c0 0x55b206c047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.487+0000 7fed48be5700 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b206c023c0 0x55b206c047e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cdc4bd2927b4340 server_cookie=439e0cae6c2e0040 in_seq=0 out_seq=0 2024-05-09T00:14:34.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.487+0000 7fed2cff9700 10 client.15768 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed2cff9700 1 -- 172.21.15.46:0/1745556500 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b206b14a80 con 0x55b206c023c0 2024-05-09T00:14:34.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed2cff9700 10 client.15768 handle_command_reply: tid=0 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed5a060b80 1 client.15768 shutdown 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed5a060b80 1 -- 172.21.15.46:0/1745556500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 msgr2=0x55b206bf23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed5a060b80 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 0x55b206bf23f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fed400099c0 tx=0x7fed40009310 comp rx=0 tx=0).stop 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed5a060b80 1 -- 172.21.15.46:0/1745556500 shutdown_connections 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed5a060b80 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b206c023c0 0x55b206c047e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed2cff9700 0 client.15768 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed5a060b80 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206bed1e0 0x55b206bf2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed5a060b80 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206a674a0 0x55b206bf23f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed5a060b80 1 --2- 172.21.15.46:0/1745556500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206ae1d00 0x55b206bf2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed5a060b80 1 -- 172.21.15.46:0/1745556500 >> 172.21.15.46:0/1745556500 conn(0x55b206a6af70 msgr2=0x55b206bd8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed5a060b80 1 -- 172.21.15.46:0/1745556500 shutdown_connections 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed5a060b80 1 -- 172.21.15.46:0/1745556500 wait complete. 2024-05-09T00:14:34.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.488+0000 7fed5a060b80 20 client.15768 trim_cache size 0 max 16384 2024-05-09T00:14:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.503+0000 7fed5a060b80 1 Processor -- start 2024-05-09T00:14:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.503+0000 7fed5a060b80 1 -- start start 2024-05-09T00:14:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed5a060b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206a674a0 0x55b206be94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed5a060b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206ae1d00 0x55b206bfe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed5a060b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206bed1e0 0x55b206c09f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed5a060b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed441af440 con 0x55b206bed1e0 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed5a060b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed441accd0 con 0x55b206a674a0 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed5a060b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed441aab00 con 0x55b206ae1d00 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed49be7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206bed1e0 0x55b206c09f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed49be7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206bed1e0 0x55b206c09f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32798/0 (socket says 172.21.15.46:32798) 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed49be7700 1 -- 172.21.15.46:0/2176818291 learned_addr learned my addr 172.21.15.46:0/2176818291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed48be5700 1 --2- 172.21.15.46:0/2176818291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206ae1d00 0x55b206bfe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed493e6700 1 --2- 172.21.15.46:0/2176818291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206a674a0 0x55b206be94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed49be7700 1 -- 172.21.15.46:0/2176818291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206ae1d00 msgr2=0x55b206bfe220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed49be7700 1 --2- 172.21.15.46:0/2176818291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206ae1d00 0x55b206bfe220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed49be7700 1 -- 172.21.15.46:0/2176818291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206a674a0 msgr2=0x55b206be94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed49be7700 1 --2- 172.21.15.46:0/2176818291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206a674a0 0x55b206be94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed49be7700 1 -- 172.21.15.46:0/2176818291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b206bdb9d0 con 0x55b206bed1e0 2024-05-09T00:14:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed493e6700 1 --2- 172.21.15.46:0/2176818291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206a674a0 0x55b206be94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.504+0000 7fed49be7700 1 --2- 172.21.15.46:0/2176818291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206bed1e0 0x55b206c09f90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fed3c008420 tx=0x7fed3c008730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f76e3faaf20fbe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.505+0000 7fed2effd700 1 -- 172.21.15.46:0/2176818291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c01a020 con 0x55b206bed1e0 2024-05-09T00:14:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.505+0000 7fed2effd700 1 -- 172.21.15.46:0/2176818291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed3c016920 con 0x55b206bed1e0 2024-05-09T00:14:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.505+0000 7fed2effd700 1 -- 172.21.15.46:0/2176818291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c015a40 con 0x55b206bed1e0 2024-05-09T00:14:34.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.505+0000 7fed5a060b80 1 -- 172.21.15.46:0/2176818291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206bed1e0 msgr2=0x55b206c09f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.505+0000 7fed5a060b80 1 --2- 172.21.15.46:0/2176818291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206bed1e0 0x55b206c09f90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fed3c008420 tx=0x7fed3c008730 comp rx=0 tx=0).stop 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.505+0000 7fed5a060b80 1 -- 172.21.15.46:0/2176818291 shutdown_connections 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.505+0000 7fed5a060b80 1 --2- 172.21.15.46:0/2176818291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206bed1e0 0x55b206c09f90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.505+0000 7fed5a060b80 1 --2- 172.21.15.46:0/2176818291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206a674a0 0x55b206be94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.505+0000 7fed5a060b80 1 --2- 172.21.15.46:0/2176818291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206ae1d00 0x55b206bfe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.506+0000 7fed5a060b80 1 -- 172.21.15.46:0/2176818291 >> 172.21.15.46:0/2176818291 conn(0x55b206a6af70 msgr2=0x55b206bd9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.506+0000 7fed5a060b80 1 -- 172.21.15.46:0/2176818291 shutdown_connections 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.506+0000 7fed5a060b80 1 -- 172.21.15.46:0/2176818291 wait complete. 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.506+0000 7fed5a060b80 1 -- start start 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.506+0000 7fed5a060b80 1 Processor -- start 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed5a060b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206a674a0 0x55b206bfa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed5a060b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206ae1d00 0x55b206beece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed5a060b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206bed1e0 0x55b206bef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed5a060b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ab190 con 0x55b206ae1d00 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed5a060b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ac600 con 0x55b206a674a0 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed5a060b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ab860 con 0x55b206bed1e0 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed5a060b80 10 client.0 _ll_register_callbacks cb 0x55b206c04d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed48be5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206ae1d00 0x55b206beece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed48be5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206ae1d00 0x55b206beece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32802/0 (socket says 172.21.15.46:32802) 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed48be5700 1 -- 172.21.15.46:0/1856028537 learned_addr learned my addr 172.21.15.46:0/1856028537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed493e6700 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206a674a0 0x55b206bfa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed49be7700 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206bed1e0 0x55b206bef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed48be5700 1 -- 172.21.15.46:0/1856028537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206bed1e0 msgr2=0x55b206bef220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed48be5700 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206bed1e0 0x55b206bef220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed48be5700 1 -- 172.21.15.46:0/1856028537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206a674a0 msgr2=0x55b206bfa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed48be5700 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206a674a0 0x55b206bfa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed48be5700 1 -- 172.21.15.46:0/1856028537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b206bef760 con 0x55b206ae1d00 2024-05-09T00:14:34.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed49be7700 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206bed1e0 0x55b206bef220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed48be5700 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206ae1d00 0x55b206beece0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fed34000c00 tx=0x7fed34009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f104a9d50fb9ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed2cff9700 1 -- 172.21.15.46:0/1856028537 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34068e60 con 0x55b206ae1d00 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed2cff9700 1 -- 172.21.15.46:0/1856028537 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fed340b1fc0 con 0x55b206ae1d00 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.507+0000 7fed5a060b80 1 -- 172.21.15.46:0/1856028537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed3c016920 con 0x55b206ae1d00 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.508+0000 7fed2cff9700 1 -- 172.21.15.46:0/1856028537 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed34020040 con 0x55b206ae1d00 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.508+0000 7fed5a060b80 10 client.15774 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.508+0000 7fed5a060b80 1 -- 172.21.15.46:0/1856028537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b206c6c680 con 0x55b206ae1d00 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.508+0000 7fed2cff9700 1 -- 172.21.15.46:0/1856028537 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed3c016920 con 0x55b206ae1d00 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.508+0000 7fed2cff9700 1 -- 172.21.15.46:0/1856028537 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fed340c3ce0 con 0x55b206ae1d00 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.508+0000 7fed5a060b80 10 client.15774 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.508+0000 7fed5a060b80 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.508+0000 7fed5a060b80 20 client.15774 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.508+0000 7fed5a060b80 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b206c023c0 0x55b206c6e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.508+0000 7fed5a060b80 4 client.15774 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.508+0000 7fed5a060b80 1 -- 172.21.15.46:0/1856028537 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b206b0eb20 con 0x55b206c023c0 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.508+0000 7fed493e6700 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b206c023c0 0x55b206c6e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed493e6700 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b206c023c0 0x55b206c6e860 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=449187ec4aa05b26 server_cookie=866691f55658e539 in_seq=0 out_seq=0 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed2cff9700 10 client.15774 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed2cff9700 1 -- 172.21.15.46:0/1856028537 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b206b0eb20 con 0x55b206c023c0 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed2cff9700 10 client.15774 handle_command_reply: tid=0 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed5a060b80 1 client.15774 shutdown 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed5a060b80 1 -- 172.21.15.46:0/1856028537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206ae1d00 msgr2=0x55b206beece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed5a060b80 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206ae1d00 0x55b206beece0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fed34000c00 tx=0x7fed34009650 comp rx=0 tx=0).stop 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed5a060b80 1 -- 172.21.15.46:0/1856028537 shutdown_connections 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed5a060b80 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b206c023c0 0x55b206c6e860 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed5a060b80 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b206bed1e0 0x55b206bef220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed5a060b80 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b206a674a0 0x55b206bfa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed2cff9700 0 client.15774 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:34.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed5a060b80 1 --2- 172.21.15.46:0/1856028537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b206ae1d00 0x55b206beece0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.509+0000 7fed5a060b80 1 -- 172.21.15.46:0/1856028537 >> 172.21.15.46:0/1856028537 conn(0x55b206a6af70 msgr2=0x55b206bec010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:34.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed5a060b80 1 -- 172.21.15.46:0/1856028537 shutdown_connections 2024-05-09T00:14:34.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed5a060b80 1 -- 172.21.15.46:0/1856028537 wait complete. 2024-05-09T00:14:34.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed5a060b80 20 client.15774 trim_cache size 0 max 16384 2024-05-09T00:14:34.512 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:14:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed4b64a700 1 -- 172.21.15.46:0/3246670777 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fed300b2880 msgr2=0x7fed300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed4b64a700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fed300b2880 0x7fed300b4d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fed40000c00 tx=0x7fed4001a040 comp rx=0 tx=0).stop 2024-05-09T00:14:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed4b64a700 1 -- 172.21.15.46:0/3246670777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed44107df0 msgr2=0x7fed4419b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed4b64a700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed44107df0 0x7fed4419b390 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fed3400d980 tx=0x7fed3400dd40 comp rx=0 tx=0).stop 2024-05-09T00:14:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed4b64a700 1 -- 172.21.15.46:0/3246670777 shutdown_connections 2024-05-09T00:14:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed4b64a700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fed300b2880 0x7fed300b4d90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed4b64a700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed44108740 0x7fed4419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed4b64a700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed441034c0 0x7fed4419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed4b64a700 1 --2- 172.21.15.46:0/3246670777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed44107df0 0x7fed4419b390 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:34.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.510+0000 7fed4b64a700 1 -- 172.21.15.46:0/3246670777 >> 172.21.15.46:0/3246670777 conn(0x7fed440fedc0 msgr2=0x7fed4406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:34.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.511+0000 7fed4b64a700 1 -- 172.21.15.46:0/3246670777 shutdown_connections 2024-05-09T00:14:34.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:34.511+0000 7fed4b64a700 1 -- 172.21.15.46:0/3246670777 wait complete. 2024-05-09T00:14:35.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:34 smithi046 ceph-mon[30151]: pgmap v472: 97 pgs: 97 active+clean; 1.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:14:35.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:34 smithi169 ceph-mon[34777]: pgmap v472: 97 pgs: 97 active+clean; 1.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:14:35.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:34 smithi142 ceph-mon[34533]: pgmap v472: 97 pgs: 97 active+clean; 1.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:14:36.195 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:14:36.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:14:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:36.858 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:36 smithi046 ceph-mon[30151]: pgmap v473: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:14:37.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:36 smithi169 ceph-mon[34777]: pgmap v473: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:14:37.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:36 smithi142 ceph-mon[34533]: pgmap v473: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:14:38.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.345+0000 7f95f35c9700 1 -- 172.21.15.46:0/1951191064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec103080 msgr2=0x7f95ec103490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.345+0000 7f95f35c9700 1 --2- 172.21.15.46:0/1951191064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec103080 0x7f95ec103490 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009990 tx=0x7f95e8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.345+0000 7f95f35c9700 1 -- 172.21.15.46:0/1951191064 shutdown_connections 2024-05-09T00:14:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.345+0000 7f95f35c9700 1 --2- 172.21.15.46:0/1951191064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec110c30 0x7f95ec113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.345+0000 7f95f35c9700 1 --2- 172.21.15.46:0/1951191064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec103080 0x7f95ec103490 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.345+0000 7f95f35c9700 1 --2- 172.21.15.46:0/1951191064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ec103a60 0x7f95ec1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.345+0000 7f95f35c9700 1 -- 172.21.15.46:0/1951191064 >> 172.21.15.46:0/1951191064 conn(0x7f95ec0fedc0 msgr2=0x7f95ec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.345+0000 7f95f35c9700 1 -- 172.21.15.46:0/1951191064 shutdown_connections 2024-05-09T00:14:38.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.345+0000 7f95f35c9700 1 -- 172.21.15.46:0/1951191064 wait complete. 2024-05-09T00:14:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.346+0000 7f95f35c9700 1 Processor -- start 2024-05-09T00:14:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.346+0000 7f95f35c9700 1 -- start start 2024-05-09T00:14:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.346+0000 7f95f35c9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec103080 0x7f95ec19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.346+0000 7f95f35c9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ec103a60 0x7f95ec19d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.346+0000 7f95f35c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec110c30 0x7f95ec1a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.346+0000 7f95f35c9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec116680 con 0x7f95ec110c30 2024-05-09T00:14:38.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.346+0000 7f95f35c9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec116500 con 0x7f95ec103a60 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.346+0000 7f95f35c9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec116800 con 0x7f95ec103080 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f1b66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec110c30 0x7f95ec1a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f1b66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec110c30 0x7f95ec1a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57018/0 (socket says 172.21.15.46:57018) 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f1b66700 1 -- 172.21.15.46:0/1080009686 learned_addr learned my addr 172.21.15.46:0/1080009686 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f0b64700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ec103a60 0x7f95ec19d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f1365700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec103080 0x7f95ec19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f1b66700 1 -- 172.21.15.46:0/1080009686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec103080 msgr2=0x7f95ec19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f1b66700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec103080 0x7f95ec19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f1b66700 1 -- 172.21.15.46:0/1080009686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ec103a60 msgr2=0x7f95ec19d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f1b66700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ec103a60 0x7f95ec19d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f1b66700 1 -- 172.21.15.46:0/1080009686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ec1a1d90 con 0x7f95ec110c30 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f1365700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec103080 0x7f95ec19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f1b66700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec110c30 0x7f95ec1a17a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f95e4007d70 tx=0x7f95e400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95e27fc700 1 -- 172.21.15.46:0/1080009686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4016020 con 0x7f95ec110c30 2024-05-09T00:14:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95e27fc700 1 -- 172.21.15.46:0/1080009686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e400ad40 con 0x7f95ec110c30 2024-05-09T00:14:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f35c9700 1 -- 172.21.15.46:0/1080009686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95ec1a2080 con 0x7f95ec110c30 2024-05-09T00:14:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95e27fc700 1 -- 172.21.15.46:0/1080009686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40115e0 con 0x7f95ec110c30 2024-05-09T00:14:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.347+0000 7f95f35c9700 1 -- 172.21.15.46:0/1080009686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95ec1a25c0 con 0x7f95ec110c30 2024-05-09T00:14:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.350+0000 7f95e27fc700 1 -- 172.21.15.46:0/1080009686 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f95e4019020 con 0x7f95ec110c30 2024-05-09T00:14:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.351+0000 7f95e27fc700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95d80b2900 0x7f95d80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.351+0000 7f95f1365700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95d80b2900 0x7f95d80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.351+0000 7f95e27fc700 1 -- 172.21.15.46:0/1080009686 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f95e40e21b0 con 0x7f95ec110c30 2024-05-09T00:14:38.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.351+0000 7f9601fdfb80 1 Processor -- start 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.351+0000 7f95f1365700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95d80b2900 0x7f95d80b4e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f95e8003800 tx=0x7f95e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.351+0000 7f9601fdfb80 1 -- start start 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f9601fdfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 0x563d3e1a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f9601fdfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e01e520 0x563d3e1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f9601fdfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e19b480 0x563d3e19b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f9601fdfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563d3e0be120 con 0x563d3e098d80 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f9601fdfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563d3e063a20 con 0x563d3e19b480 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f9601fdfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563d3e0ba0e0 con 0x563d3e01e520 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f95f1365700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 0x563d3e1a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f95f1365700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 0x563d3e1a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57028/0 (socket says 172.21.15.46:57028) 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f95f1365700 1 -- 172.21.15.46:0/2272578366 learned_addr learned my addr 172.21.15.46:0/2272578366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f95f0b64700 1 --2- 172.21.15.46:0/2272578366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e01e520 0x563d3e1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f95f1b66700 1 --2- 172.21.15.46:0/2272578366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e19b480 0x563d3e19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f95f1365700 1 -- 172.21.15.46:0/2272578366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e01e520 msgr2=0x563d3e1a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:38.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f95f1365700 1 --2- 172.21.15.46:0/2272578366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e01e520 0x563d3e1a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f95f1365700 1 -- 172.21.15.46:0/2272578366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e19b480 msgr2=0x563d3e19b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:38.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f95f1365700 1 --2- 172.21.15.46:0/2272578366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e19b480 0x563d3e19b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f95f1365700 1 -- 172.21.15.46:0/2272578366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d3e054580 con 0x563d3e098d80 2024-05-09T00:14:38.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f95f1b66700 1 --2- 172.21.15.46:0/2272578366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e19b480 0x563d3e19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:38.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.352+0000 7f95f1365700 1 --2- 172.21.15.46:0/2272578366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 0x563d3e1a12c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f95e80059d0 tx=0x7f95e801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ba2fc2f6445c79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:38.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.353+0000 7f95d77fe700 1 -- 172.21.15.46:0/2272578366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e800be40 con 0x563d3e098d80 2024-05-09T00:14:38.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.353+0000 7f95d77fe700 1 -- 172.21.15.46:0/2272578366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e800fda0 con 0x563d3e098d80 2024-05-09T00:14:38.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.353+0000 7f95d77fe700 1 -- 172.21.15.46:0/2272578366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8005390 con 0x563d3e098d80 2024-05-09T00:14:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.354+0000 7f9601fdfb80 1 -- 172.21.15.46:0/2272578366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 msgr2=0x563d3e1a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.354+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/2272578366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 0x563d3e1a12c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f95e80059d0 tx=0x7f95e801d000 comp rx=0 tx=0).stop 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.354+0000 7f9601fdfb80 1 -- 172.21.15.46:0/2272578366 shutdown_connections 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.354+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/2272578366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e19b480 0x563d3e19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.354+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/2272578366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 0x563d3e1a12c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.354+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/2272578366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e01e520 0x563d3e1a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.354+0000 7f9601fdfb80 1 -- 172.21.15.46:0/2272578366 >> 172.21.15.46:0/2272578366 conn(0x563d3e021ff0 msgr2=0x563d3e1962a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.354+0000 7f9601fdfb80 1 -- 172.21.15.46:0/2272578366 shutdown_connections 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.354+0000 7f9601fdfb80 1 -- 172.21.15.46:0/2272578366 wait complete. 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.354+0000 7f9601fdfb80 1 -- start start 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.354+0000 7f9601fdfb80 1 Processor -- start 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f9601fdfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e01e520 0x563d3e1a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f9601fdfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 0x563d3e1a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f9601fdfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e19b480 0x563d3e1a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f9601fdfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563d3e19c180 con 0x563d3e098d80 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f9601fdfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563d3e058ba0 con 0x563d3e01e520 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f9601fdfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563d3e1b12b0 con 0x563d3e19b480 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f9601fdfb80 10 client.0 _ll_register_callbacks cb 0x563d3df2f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f95f0b64700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 0x563d3e1a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f95f0b64700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 0x563d3e1a55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57042/0 (socket says 172.21.15.46:57042) 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f95f0b64700 1 -- 172.21.15.46:0/307591829 learned_addr learned my addr 172.21.15.46:0/307591829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f95f1b66700 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e19b480 0x563d3e1a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f95f1365700 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e01e520 0x563d3e1a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f95f0b64700 1 -- 172.21.15.46:0/307591829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e19b480 msgr2=0x563d3e1a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f95f0b64700 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e19b480 0x563d3e1a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f95f0b64700 1 -- 172.21.15.46:0/307591829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e01e520 msgr2=0x563d3e1a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f95f0b64700 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e01e520 0x563d3e1a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f95f0b64700 1 -- 172.21.15.46:0/307591829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d3df39510 con 0x563d3e098d80 2024-05-09T00:14:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.355+0000 7f95f1b66700 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e19b480 0x563d3e1a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f95f0b64700 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 0x563d3e1a55b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f95ec06c6f0 tx=0x7f95dc011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f74a9aaca94e81 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f95d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f95d4ff9700 1 -- 172.21.15.46:0/307591829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc00fad0 con 0x563d3e098d80 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f95d4ff9700 1 -- 172.21.15.46:0/307591829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f95dc00fc70 con 0x563d3e098d80 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f9601fdfb80 1 -- 172.21.15.46:0/307591829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e800fda0 con 0x563d3e098d80 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f95d4ff9700 1 -- 172.21.15.46:0/307591829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f95dc00fe70 con 0x563d3e098d80 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f9601fdfb80 10 client.15786 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f9601fdfb80 1 -- 172.21.15.46:0/307591829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563d3e1b2c30 con 0x563d3e098d80 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f95d4ff9700 1 -- 172.21.15.46:0/307591829 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e800fda0 con 0x563d3e098d80 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f95d4ff9700 1 -- 172.21.15.46:0/307591829 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f95dc018030 con 0x563d3e098d80 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f9601fdfb80 10 client.15786 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f9601fdfb80 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f9601fdfb80 20 client.15786 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.356+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563d3e1b5270 0x563d3e1b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.357+0000 7f9601fdfb80 4 client.15786 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:14:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.357+0000 7f9601fdfb80 1 -- 172.21.15.46:0/307591829 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d3e0cbb00 con 0x563d3e1b5270 2024-05-09T00:14:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.357+0000 7f95f1365700 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563d3e1b5270 0x563d3e1b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.357+0000 7f95d4ff9700 10 client.15786 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.357+0000 7f95f1365700 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563d3e1b5270 0x563d3e1b76b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991400ca7c324299 server_cookie=8535fb3da12d6c76 in_seq=0 out_seq=0 2024-05-09T00:14:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.358+0000 7f95d4ff9700 1 -- 172.21.15.46:0/307591829 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d3e0cbb00 con 0x563d3e1b5270 2024-05-09T00:14:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.358+0000 7f95d4ff9700 10 client.15786 handle_command_reply: tid=0 2024-05-09T00:14:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.358+0000 7f9601fdfb80 1 client.15786 shutdown 2024-05-09T00:14:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.358+0000 7f9601fdfb80 1 -- 172.21.15.46:0/307591829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 msgr2=0x563d3e1a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.358+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 0x563d3e1a55b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f95ec06c6f0 tx=0x7f95dc011bb0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.358+0000 7f9601fdfb80 1 -- 172.21.15.46:0/307591829 shutdown_connections 2024-05-09T00:14:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.358+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563d3e1b5270 0x563d3e1b76b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.359+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e19b480 0x563d3e1a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.359+0000 7f95d4ff9700 0 client.15786 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.359+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e01e520 0x563d3e1a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.359+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/307591829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e098d80 0x563d3e1a55b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.359+0000 7f9601fdfb80 1 -- 172.21.15.46:0/307591829 >> 172.21.15.46:0/307591829 conn(0x563d3e021ff0 msgr2=0x563d3e190070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.359+0000 7f9601fdfb80 1 -- 172.21.15.46:0/307591829 shutdown_connections 2024-05-09T00:14:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.359+0000 7f9601fdfb80 1 -- 172.21.15.46:0/307591829 wait complete. 2024-05-09T00:14:38.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.359+0000 7f9601fdfb80 20 client.15786 trim_cache size 0 max 16384 2024-05-09T00:14:38.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.379+0000 7f9601fdfb80 1 Processor -- start 2024-05-09T00:14:38.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.379+0000 7f9601fdfb80 1 -- start start 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f9601fdfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e021ff0 0x563d3e190aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f9601fdfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e01e520 0x563d3e1a5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f9601fdfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e098d80 0x563d3e19a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f9601fdfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1b35a0 con 0x563d3e021ff0 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f9601fdfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1b0e30 con 0x563d3e098d80 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f9601fdfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1aec60 con 0x563d3e01e520 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f95f1365700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e021ff0 0x563d3e190aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f95f1365700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e021ff0 0x563d3e190aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57056/0 (socket says 172.21.15.46:57056) 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f95f1365700 1 -- 172.21.15.46:0/3173186393 learned_addr learned my addr 172.21.15.46:0/3173186393 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f95f1b66700 1 --2- 172.21.15.46:0/3173186393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e098d80 0x563d3e19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f95f0b64700 1 --2- 172.21.15.46:0/3173186393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e01e520 0x563d3e1a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f95f1365700 1 -- 172.21.15.46:0/3173186393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e01e520 msgr2=0x563d3e1a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f95f1365700 1 --2- 172.21.15.46:0/3173186393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e01e520 0x563d3e1a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f95f1365700 1 -- 172.21.15.46:0/3173186393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e098d80 msgr2=0x563d3e19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f95f1365700 1 --2- 172.21.15.46:0/3173186393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e098d80 0x563d3e19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f95f1365700 1 -- 172.21.15.46:0/3173186393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d3e192a50 con 0x563d3e021ff0 2024-05-09T00:14:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.380+0000 7f95f1365700 1 --2- 172.21.15.46:0/3173186393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e021ff0 0x563d3e190aa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f95e800b5c0 tx=0x7f95e8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1164a68f8b0744 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.381+0000 7f95d6ffd700 1 -- 172.21.15.46:0/3173186393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e803c070 con 0x563d3e021ff0 2024-05-09T00:14:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.381+0000 7f95d6ffd700 1 -- 172.21.15.46:0/3173186393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e800fe80 con 0x563d3e021ff0 2024-05-09T00:14:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.381+0000 7f95d6ffd700 1 -- 172.21.15.46:0/3173186393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8005680 con 0x563d3e021ff0 2024-05-09T00:14:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.381+0000 7f9601fdfb80 1 -- 172.21.15.46:0/3173186393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e021ff0 msgr2=0x563d3e190aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.381+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/3173186393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e021ff0 0x563d3e190aa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f95e800b5c0 tx=0x7f95e8005040 comp rx=0 tx=0).stop 2024-05-09T00:14:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.381+0000 7f9601fdfb80 1 -- 172.21.15.46:0/3173186393 shutdown_connections 2024-05-09T00:14:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.381+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/3173186393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e098d80 0x563d3e19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.381+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/3173186393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e021ff0 0x563d3e190aa0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.381+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/3173186393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e01e520 0x563d3e1a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.381+0000 7f9601fdfb80 1 -- 172.21.15.46:0/3173186393 >> 172.21.15.46:0/3173186393 conn(0x563d3e1965a0 msgr2=0x563d3e1a3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.381+0000 7f9601fdfb80 1 -- 172.21.15.46:0/3173186393 shutdown_connections 2024-05-09T00:14:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.381+0000 7f9601fdfb80 1 -- 172.21.15.46:0/3173186393 wait complete. 2024-05-09T00:14:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.382+0000 7f9601fdfb80 1 -- start start 2024-05-09T00:14:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.382+0000 7f9601fdfb80 1 Processor -- start 2024-05-09T00:14:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.382+0000 7f9601fdfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e01e520 0x563d3e1ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.382+0000 7f9601fdfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e021ff0 0x563d3e1aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.382+0000 7f9601fdfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e098d80 0x563d3e1aa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.382+0000 7f9601fdfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1af2f0 con 0x563d3e01e520 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.382+0000 7f9601fdfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1b0760 con 0x563d3e021ff0 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.382+0000 7f9601fdfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1af9c0 con 0x563d3e098d80 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95f1365700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e01e520 0x563d3e1ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f9601fdfb80 10 client.0 _ll_register_callbacks cb 0x563d3e1a0d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95f1365700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e01e520 0x563d3e1ad410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57064/0 (socket says 172.21.15.46:57064) 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95f1365700 1 -- 172.21.15.46:0/1145855402 learned_addr learned my addr 172.21.15.46:0/1145855402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95f0b64700 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e021ff0 0x563d3e1aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95f1b66700 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e098d80 0x563d3e1aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95f1365700 1 -- 172.21.15.46:0/1145855402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e098d80 msgr2=0x563d3e1aa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95f1365700 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e098d80 0x563d3e1aa720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95f1365700 1 -- 172.21.15.46:0/1145855402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e021ff0 msgr2=0x563d3e1aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95f1365700 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e021ff0 0x563d3e1aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95f1365700 1 -- 172.21.15.46:0/1145855402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d3e1adca0 con 0x563d3e01e520 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95f1365700 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e01e520 0x563d3e1ad410 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f95e800ba30 tx=0x7f95e8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c45fae75bc6464 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95d4ff9700 1 -- 172.21.15.46:0/1145855402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8045060 con 0x563d3e01e520 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f95d4ff9700 1 -- 172.21.15.46:0/1145855402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f95e8003a70 con 0x563d3e01e520 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.383+0000 7f9601fdfb80 1 -- 172.21.15.46:0/1145855402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d3e0c5ba0 con 0x563d3e01e520 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.384+0000 7f95d4ff9700 1 -- 172.21.15.46:0/1145855402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f95e803c070 con 0x563d3e01e520 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.384+0000 7f9601fdfb80 10 client.15792 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.384+0000 7f9601fdfb80 1 -- 172.21.15.46:0/1145855402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563d3e2205b0 con 0x563d3e01e520 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.384+0000 7f95d4ff9700 1 -- 172.21.15.46:0/1145855402 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563d3e0c5ba0 con 0x563d3e01e520 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.384+0000 7f95d4ff9700 1 -- 172.21.15.46:0/1145855402 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f95e8036840 con 0x563d3e01e520 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.384+0000 7f9601fdfb80 10 client.15792 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.384+0000 7f9601fdfb80 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.384+0000 7f9601fdfb80 20 client.15792 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.384+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563d3e19b480 0x563d3e222840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.384+0000 7f9601fdfb80 4 client.15792 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:14:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.384+0000 7f9601fdfb80 1 -- 172.21.15.46:0/1145855402 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563d3df399f0 con 0x563d3e19b480 2024-05-09T00:14:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.384+0000 7f95f0b64700 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563d3e19b480 0x563d3e222840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.385+0000 7f95d4ff9700 10 client.15792 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.385+0000 7f95f0b64700 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563d3e19b480 0x563d3e222840 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29900c1f39d80be4 server_cookie=fb04999be9f249d in_seq=0 out_seq=0 2024-05-09T00:14:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f95d4ff9700 1 -- 172.21.15.46:0/1145855402 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563d3df399f0 con 0x563d3e19b480 2024-05-09T00:14:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f95d4ff9700 10 client.15792 handle_command_reply: tid=0 2024-05-09T00:14:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f9601fdfb80 1 client.15792 shutdown 2024-05-09T00:14:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f9601fdfb80 1 -- 172.21.15.46:0/1145855402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e01e520 msgr2=0x563d3e1ad410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e01e520 0x563d3e1ad410 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f95e800ba30 tx=0x7f95e8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f9601fdfb80 1 -- 172.21.15.46:0/1145855402 shutdown_connections 2024-05-09T00:14:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563d3e19b480 0x563d3e222840 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d3e098d80 0x563d3e1aa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f95d4ff9700 0 client.15792 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d3e01e520 0x563d3e1ad410 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f9601fdfb80 1 --2- 172.21.15.46:0/1145855402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d3e021ff0 0x563d3e1aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f9601fdfb80 1 -- 172.21.15.46:0/1145855402 >> 172.21.15.46:0/1145855402 conn(0x563d3e1965a0 msgr2=0x563d3e19a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.386+0000 7f9601fdfb80 1 -- 172.21.15.46:0/1145855402 shutdown_connections 2024-05-09T00:14:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f9601fdfb80 1 -- 172.21.15.46:0/1145855402 wait complete. 2024-05-09T00:14:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f9601fdfb80 20 client.15792 trim_cache size 0 max 16384 2024-05-09T00:14:38.388 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:14:38.388 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:14:38.389 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "fafbdb38-8f86-46b3-a4c5-b91dbb3a9c97", 2024-05-09T00:14:38.389 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:14:38.389 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:14:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f95f35c9700 1 -- 172.21.15.46:0/1080009686 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95d80b2900 msgr2=0x7f95d80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f95f35c9700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95d80b2900 0x7f95d80b4e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f95e8003800 tx=0x7f95e8005fb0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f95f35c9700 1 -- 172.21.15.46:0/1080009686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec110c30 msgr2=0x7f95ec1a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f95f35c9700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec110c30 0x7f95ec1a17a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f95e4007d70 tx=0x7f95e400a340 comp rx=0 tx=0).stop 2024-05-09T00:14:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f95f35c9700 1 -- 172.21.15.46:0/1080009686 shutdown_connections 2024-05-09T00:14:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f95f35c9700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f95d80b2900 0x7f95d80b4e10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f95f35c9700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec110c30 0x7f95ec1a17a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f95f35c9700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec103080 0x7f95ec19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f95f35c9700 1 --2- 172.21.15.46:0/1080009686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95ec103a60 0x7f95ec19d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f95f35c9700 1 -- 172.21.15.46:0/1080009686 >> 172.21.15.46:0/1080009686 conn(0x7f95ec0fedc0 msgr2=0x7f95ec1127a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f95f35c9700 1 -- 172.21.15.46:0/1080009686 shutdown_connections 2024-05-09T00:14:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:38.387+0000 7f95f35c9700 1 -- 172.21.15.46:0/1080009686 wait complete. 2024-05-09T00:14:39.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:38 smithi046 ceph-mon[30151]: pgmap v474: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:14:39.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:38 smithi169 ceph-mon[34777]: pgmap v474: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:14:39.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:14:39.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:38 smithi142 ceph-mon[34533]: pgmap v474: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:14:39.373 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:14:39.792 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:39 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:14:39.792 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:39 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:39.792 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:39 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:14:39.792 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:39 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:40.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:39 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:14:40.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:39 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:40.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:39 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:14:40.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:39 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:40.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:39 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:14:40.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:39 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:40.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:39 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:14:40.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:39 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:14:40.813 INFO:tasks.workunit.client.0.smithi046.stdout:1/600: open fd = 3 2024-05-09T00:14:40.813 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.813 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.813 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.813 INFO:tasks.workunit.client.0.smithi046.stdout:2/600: open fd = 3 2024-05-09T00:14:40.813 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.813 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.813 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.813 INFO:tasks.workunit.client.0.smithi046.stdout:3/600: open fd = 3 2024-05-09T00:14:40.813 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.813 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.814 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.814 INFO:tasks.workunit.client.0.smithi046.stdout:4/600: open fd = 3 2024-05-09T00:14:40.814 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.814 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.814 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.814 INFO:tasks.workunit.client.0.smithi046.stdout:5/600: open fd = 3 2024-05-09T00:14:40.814 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.814 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.814 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.814 INFO:tasks.workunit.client.0.smithi046.stdout:6/600: open fd = 3 2024-05-09T00:14:40.814 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.814 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:7/600: open fd = 3 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:8/600: open fd = 3 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:9/600: open fd = 3 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:10/600: open fd = 3 2024-05-09T00:14:40.815 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:11/600: open fd = 3 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:12/600: open fd = 3 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:13/600: open fd = 3 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.816 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:14/600: open fd = 3 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:15/600: open fd = 3 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:16/600: open fd = 3 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:17/600: open fd = 3 2024-05-09T00:14:40.817 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:18/600: open fd = 3 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:19/600: open fd = 3 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:20/600: open fd = 3 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.818 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:21/600: open fd = 3 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:22/600: open fd = 3 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:23/600: open fd = 3 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:24/600: open fd = 3 2024-05-09T00:14:40.819 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.820 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.820 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.820 INFO:tasks.workunit.client.0.smithi046.stdout:25/600: open fd = 3 2024-05-09T00:14:40.820 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.820 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.820 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.820 INFO:tasks.workunit.client.0.smithi046.stdout:26/600: open fd = 3 2024-05-09T00:14:40.820 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.820 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.820 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.820 INFO:tasks.workunit.client.0.smithi046.stdout:27/600: open fd = 3 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:28/600: open fd = 3 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:29/600: open fd = 3 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:30/600: open fd = 3 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.821 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:31/600: open fd = 3 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:32/600: open fd = 3 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:33/600: open fd = 3 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.822 INFO:tasks.workunit.client.0.smithi046.stdout:34/600: open fd = 3 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:35/600: open fd = 3 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:36/600: open fd = 3 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:37/600: open fd = 3 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.823 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:38/600: open fd = 3 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:39/600: open fd = 3 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:40/600: open fd = 3 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:41/600: open fd = 3 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.824 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:42/600: open fd = 3 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:43/600: open fd = 3 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:44/600: open fd = 3 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.825 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:45/600: open fd = 3 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:46/600: open fd = 3 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:47/600: open fd = 3 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:48/600: open fd = 3 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.826 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:49/600: open fd = 3 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:50/600: open fd = 3 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:51/600: open fd = 3 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.827 INFO:tasks.workunit.client.0.smithi046.stdout:52/600: open fd = 3 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:53/600: open fd = 3 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:54/600: open fd = 3 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:55/600: open fd = 3 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.828 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:56/600: open fd = 3 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:57/600: open fd = 3 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:58/600: open fd = 3 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:59/600: open fd = 3 2024-05-09T00:14:40.829 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:60/600: open fd = 3 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:61/600: open fd = 3 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:62/600: open fd = 3 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.830 INFO:tasks.workunit.client.0.smithi046.stdout:63/600: open fd = 3 2024-05-09T00:14:40.831 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.831 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:14:40.831 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:14:40.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:40 smithi046 ceph-mon[30151]: pgmap v475: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:41.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:40 smithi169 ceph-mon[34777]: pgmap v475: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:41.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:40 smithi142 ceph-mon[34533]: pgmap v475: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:41.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.199+0000 7fd207869700 1 -- 172.21.15.46:0/1475739640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2001034c0 msgr2=0x7fd2001038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:41.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.199+0000 7fd207869700 1 --2- 172.21.15.46:0/1475739640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2001034c0 0x7fd2001038d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0099c0 tx=0x7fd1fc009cd0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.199+0000 7fd207869700 1 -- 172.21.15.46:0/1475739640 shutdown_connections 2024-05-09T00:14:41.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.199+0000 7fd207869700 1 --2- 172.21.15.46:0/1475739640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd200108740 0x7fd2001100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.199+0000 7fd207869700 1 --2- 172.21.15.46:0/1475739640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2001034c0 0x7fd2001038d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.199+0000 7fd207869700 1 --2- 172.21.15.46:0/1475739640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd200107df0 0x7fd200108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.199+0000 7fd207869700 1 -- 172.21.15.46:0/1475739640 >> 172.21.15.46:0/1475739640 conn(0x7fd2000fedf0 msgr2=0x7fd200101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:41.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.199+0000 7fd207869700 1 -- 172.21.15.46:0/1475739640 shutdown_connections 2024-05-09T00:14:41.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.199+0000 7fd207869700 1 -- 172.21.15.46:0/1475739640 wait complete. 2024-05-09T00:14:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.200+0000 7fd207869700 1 Processor -- start 2024-05-09T00:14:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.200+0000 7fd207869700 1 -- start start 2024-05-09T00:14:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.200+0000 7fd207869700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2001034c0 0x7fd20019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.200+0000 7fd207869700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd200107df0 0x7fd20019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.200+0000 7fd207869700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd200108740 0x7fd20019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.200+0000 7fd207869700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd200113370 con 0x7fd200108740 2024-05-09T00:14:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.200+0000 7fd207869700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2001131f0 con 0x7fd200107df0 2024-05-09T00:14:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.200+0000 7fd207869700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd2001134f0 con 0x7fd2001034c0 2024-05-09T00:14:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.201+0000 7fd205e06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd200108740 0x7fd20019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.201+0000 7fd205e06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd200108740 0x7fd20019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57082/0 (socket says 172.21.15.46:57082) 2024-05-09T00:14:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.201+0000 7fd205e06700 1 -- 172.21.15.46:0/1341211802 learned_addr learned my addr 172.21.15.46:0/1341211802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.201+0000 7fd205605700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2001034c0 0x7fd20019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.201+0000 7fd204e04700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd200107df0 0x7fd20019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.201+0000 7fd205e06700 1 -- 172.21.15.46:0/1341211802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2001034c0 msgr2=0x7fd20019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.201+0000 7fd205e06700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2001034c0 0x7fd20019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.201+0000 7fd205e06700 1 -- 172.21.15.46:0/1341211802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd200107df0 msgr2=0x7fd20019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.201+0000 7fd205e06700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd200107df0 0x7fd20019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.201+0000 7fd205e06700 1 -- 172.21.15.46:0/1341211802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20019fd40 con 0x7fd200108740 2024-05-09T00:14:41.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.201+0000 7fd205e06700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd200108740 0x7fd20019f6a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8007ee0 tx=0x7fd1f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:41.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.202+0000 7fd1f67fc700 1 -- 172.21.15.46:0/1341211802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f800dd20 con 0x7fd200108740 2024-05-09T00:14:41.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.202+0000 7fd207869700 1 -- 172.21.15.46:0/1341211802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2001a0030 con 0x7fd200108740 2024-05-09T00:14:41.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.202+0000 7fd207869700 1 -- 172.21.15.46:0/1341211802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2001a7950 con 0x7fd200108740 2024-05-09T00:14:41.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.202+0000 7fd1f67fc700 1 -- 172.21.15.46:0/1341211802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f8004020 con 0x7fd200108740 2024-05-09T00:14:41.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.202+0000 7fd1f67fc700 1 -- 172.21.15.46:0/1341211802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8015660 con 0x7fd200108740 2024-05-09T00:14:41.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.205+0000 7fd1f67fc700 1 -- 172.21.15.46:0/1341211802 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd1f8015800 con 0x7fd200108740 2024-05-09T00:14:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.206+0000 7fd1f67fc700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd1ec0b2900 0x7fd1ec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.206+0000 7fd21627fb80 1 Processor -- start 2024-05-09T00:14:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.206+0000 7fd205605700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd1ec0b2900 0x7fd1ec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.206+0000 7fd1f67fc700 1 -- 172.21.15.46:0/1341211802 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd1f80e1a30 con 0x7fd200108740 2024-05-09T00:14:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.206+0000 7fd21627fb80 1 -- start start 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.206+0000 7fd21627fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b710006240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.206+0000 7fd21627fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 0x55b710006780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.206+0000 7fd21627fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 0x55b710000810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd21627fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b70ff230a0 con 0x55b70fe834a0 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd21627fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b70fec89a0 con 0x55b70fefdd00 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd21627fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b70ff1f060 con 0x55b710000400 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd205605700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd1ec0b2900 0x7fd1ec0b4e10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009990 tx=0x7fd1fc0036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd204e04700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 0x55b710006780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd204e04700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 0x55b710006780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57090/0 (socket says 172.21.15.46:57090) 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd204e04700 1 -- 172.21.15.46:0/4266117966 learned_addr learned my addr 172.21.15.46:0/4266117966 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd205e06700 1 --2- 172.21.15.46:0/4266117966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 0x55b710000810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd205605700 1 --2- 172.21.15.46:0/4266117966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b710006240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd204e04700 1 -- 172.21.15.46:0/4266117966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 msgr2=0x55b710000810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:41.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd204e04700 1 --2- 172.21.15.46:0/4266117966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 0x55b710000810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd204e04700 1 -- 172.21.15.46:0/4266117966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 msgr2=0x55b710006240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:41.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd204e04700 1 --2- 172.21.15.46:0/4266117966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b710006240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd204e04700 1 -- 172.21.15.46:0/4266117966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70feb9500 con 0x55b70fe834a0 2024-05-09T00:14:41.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd205605700 1 --2- 172.21.15.46:0/4266117966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b710006240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:41.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd204e04700 1 --2- 172.21.15.46:0/4266117966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 0x55b710006780 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000bcf0 tx=0x7fd1f00103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e941ae960956fac2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:41.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.207+0000 7fd1eb7fe700 1 -- 172.21.15.46:0/4266117966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0010d10 con 0x55b70fe834a0 2024-05-09T00:14:41.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.208+0000 7fd1eb7fe700 1 -- 172.21.15.46:0/4266117966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f00044e0 con 0x55b70fe834a0 2024-05-09T00:14:41.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.208+0000 7fd1eb7fe700 1 -- 172.21.15.46:0/4266117966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0005280 con 0x55b70fe834a0 2024-05-09T00:14:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.208+0000 7fd21627fb80 1 -- 172.21.15.46:0/4266117966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 msgr2=0x55b710006780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.208+0000 7fd21627fb80 1 --2- 172.21.15.46:0/4266117966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 0x55b710006780 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000bcf0 tx=0x7fd1f00103d0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.209+0000 7fd21627fb80 1 -- 172.21.15.46:0/4266117966 shutdown_connections 2024-05-09T00:14:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.209+0000 7fd21627fb80 1 --2- 172.21.15.46:0/4266117966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 0x55b710000810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.209+0000 7fd21627fb80 1 --2- 172.21.15.46:0/4266117966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b710006240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.209+0000 7fd21627fb80 1 --2- 172.21.15.46:0/4266117966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 0x55b710006780 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.209+0000 7fd21627fb80 1 -- 172.21.15.46:0/4266117966 >> 172.21.15.46:0/4266117966 conn(0x55b70fe86f70 msgr2=0x55b70fffb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.209+0000 7fd21627fb80 1 -- 172.21.15.46:0/4266117966 shutdown_connections 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.209+0000 7fd21627fb80 1 -- 172.21.15.46:0/4266117966 wait complete. 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.209+0000 7fd21627fb80 1 -- start start 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.209+0000 7fd21627fb80 1 Processor -- start 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd21627fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b70fe834a0 0x55b710009fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd21627fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fefdd00 0x55b71000a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd21627fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b710000400 0x55b71000aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd21627fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b710001160 con 0x55b70fefdd00 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd21627fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b70febdb20 con 0x55b710000400 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd21627fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b71000d020 con 0x55b70fe834a0 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd21627fb80 10 client.0 _ll_register_callbacks cb 0x55b70fd946c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd204e04700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fefdd00 0x55b71000a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd204e04700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fefdd00 0x55b71000a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57096/0 (socket says 172.21.15.46:57096) 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd204e04700 1 -- 172.21.15.46:0/2478595448 learned_addr learned my addr 172.21.15.46:0/2478595448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd205605700 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b70fe834a0 0x55b710009fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd205e06700 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b710000400 0x55b71000aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd204e04700 1 -- 172.21.15.46:0/2478595448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b70fe834a0 msgr2=0x55b710009fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd204e04700 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b70fe834a0 0x55b710009fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd204e04700 1 -- 172.21.15.46:0/2478595448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b710000400 msgr2=0x55b71000aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd204e04700 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b710000400 0x55b71000aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd204e04700 1 -- 172.21.15.46:0/2478595448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b70fd9e490 con 0x55b70fefdd00 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd205605700 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b70fe834a0 0x55b710009fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd204e04700 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fefdd00 0x55b71000a500 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0004d70 tx=0x7fd1f0004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b8b06d53add0c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.210+0000 7fd1e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd1e8ff9700 1 -- 172.21.15.46:0/2478595448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f002a060 con 0x55b70fefdd00 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd1e8ff9700 1 -- 172.21.15.46:0/2478595448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd1f0004e70 con 0x55b70fefdd00 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd21627fb80 1 -- 172.21.15.46:0/2478595448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1f00044e0 con 0x55b70fefdd00 2024-05-09T00:14:41.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd1e8ff9700 1 -- 172.21.15.46:0/2478595448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1f0005070 con 0x55b70fefdd00 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd21627fb80 10 client.15804 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd21627fb80 1 -- 172.21.15.46:0/2478595448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b710017a20 con 0x55b70fefdd00 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd1e8ff9700 1 -- 172.21.15.46:0/2478595448 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f00044e0 con 0x55b70fefdd00 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd1e8ff9700 1 -- 172.21.15.46:0/2478595448 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd1f002f070 con 0x55b70fefdd00 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd21627fb80 10 client.15804 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd21627fb80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd21627fb80 20 client.15804 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd21627fb80 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b710019ef0 0x55b71001c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd21627fb80 4 client.15804 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.211+0000 7fd21627fb80 1 -- 172.21.15.46:0/2478595448 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b70ff30a80 con 0x55b710019ef0 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.212+0000 7fd205605700 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b710019ef0 0x55b71001c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.212+0000 7fd205605700 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b710019ef0 0x55b71001c310 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f51e218045de3e36 server_cookie=a6d295f0fb9c62f9 in_seq=0 out_seq=0 2024-05-09T00:14:41.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.212+0000 7fd1e8ff9700 10 client.15804 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd1e8ff9700 1 -- 172.21.15.46:0/2478595448 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b70ff30a80 con 0x55b710019ef0 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd1e8ff9700 10 client.15804 handle_command_reply: tid=0 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd21627fb80 1 client.15804 shutdown 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd21627fb80 1 -- 172.21.15.46:0/2478595448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fefdd00 msgr2=0x55b71000a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd21627fb80 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fefdd00 0x55b71000a500 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0004d70 tx=0x7fd1f0004a30 comp rx=0 tx=0).stop 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd21627fb80 1 -- 172.21.15.46:0/2478595448 shutdown_connections 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd21627fb80 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b710019ef0 0x55b71001c310 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd21627fb80 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b710000400 0x55b71000aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd21627fb80 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b70fe834a0 0x55b710009fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd21627fb80 1 --2- 172.21.15.46:0/2478595448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fefdd00 0x55b71000a500 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd21627fb80 1 -- 172.21.15.46:0/2478595448 >> 172.21.15.46:0/2478595448 conn(0x55b70fe86f70 msgr2=0x55b70fff4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd1e8ff9700 0 client.15804 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd21627fb80 1 -- 172.21.15.46:0/2478595448 shutdown_connections 2024-05-09T00:14:41.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd21627fb80 1 -- 172.21.15.46:0/2478595448 wait complete. 2024-05-09T00:14:41.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.213+0000 7fd21627fb80 20 client.15804 trim_cache size 0 max 16384 2024-05-09T00:14:41.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.234+0000 7fd21627fb80 1 Processor -- start 2024-05-09T00:14:41.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.234+0000 7fd21627fb80 1 -- start start 2024-05-09T00:14:41.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.234+0000 7fd21627fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 0x55b70fffb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.234+0000 7fd21627fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b710016280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.234+0000 7fd21627fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 0x55b7100043e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.234+0000 7fd21627fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e400fc80 con 0x55b70fe834a0 2024-05-09T00:14:41.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.234+0000 7fd21627fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e400d510 con 0x55b70fefdd00 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.234+0000 7fd21627fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e400b340 con 0x55b710000400 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd205605700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 0x55b70fffb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd205605700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 0x55b70fffb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57100/0 (socket says 172.21.15.46:57100) 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd205605700 1 -- 172.21.15.46:0/2120634894 learned_addr learned my addr 172.21.15.46:0/2120634894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd205e06700 1 --2- 172.21.15.46:0/2120634894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 0x55b7100043e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd204e04700 1 --2- 172.21.15.46:0/2120634894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b710016280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd205605700 1 -- 172.21.15.46:0/2120634894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 msgr2=0x55b7100043e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd205605700 1 --2- 172.21.15.46:0/2120634894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 0x55b7100043e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd205605700 1 -- 172.21.15.46:0/2120634894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 msgr2=0x55b710016280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd205605700 1 --2- 172.21.15.46:0/2120634894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b710016280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd205605700 1 -- 172.21.15.46:0/2120634894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70fff79d0 con 0x55b70fe834a0 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd205605700 1 --2- 172.21.15.46:0/2120634894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 0x55b70fffb390 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0099c0 tx=0x7fd1fc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6046502bd4521 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:41.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd1eaffd700 1 -- 172.21.15.46:0/2120634894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc035070 con 0x55b70fe834a0 2024-05-09T00:14:41.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd1eaffd700 1 -- 172.21.15.46:0/2120634894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1fc030e50 con 0x55b70fe834a0 2024-05-09T00:14:41.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.235+0000 7fd1eaffd700 1 -- 172.21.15.46:0/2120634894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc00bdc0 con 0x55b70fe834a0 2024-05-09T00:14:41.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.236+0000 7fd21627fb80 1 -- 172.21.15.46:0/2120634894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 msgr2=0x55b70fffb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:41.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.236+0000 7fd21627fb80 1 --2- 172.21.15.46:0/2120634894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 0x55b70fffb390 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0099c0 tx=0x7fd1fc01e000 comp rx=0 tx=0).stop 2024-05-09T00:14:41.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.236+0000 7fd21627fb80 1 -- 172.21.15.46:0/2120634894 shutdown_connections 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.236+0000 7fd21627fb80 1 --2- 172.21.15.46:0/2120634894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 0x55b7100043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.236+0000 7fd21627fb80 1 --2- 172.21.15.46:0/2120634894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b70fe834a0 0x55b70fffb390 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.236+0000 7fd21627fb80 1 --2- 172.21.15.46:0/2120634894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b710016280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.236+0000 7fd21627fb80 1 -- 172.21.15.46:0/2120634894 >> 172.21.15.46:0/2120634894 conn(0x55b70fe86f70 msgr2=0x55b710003cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.236+0000 7fd21627fb80 1 -- 172.21.15.46:0/2120634894 shutdown_connections 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.236+0000 7fd21627fb80 1 -- 172.21.15.46:0/2120634894 wait complete. 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd21627fb80 1 -- start start 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd21627fb80 1 Processor -- start 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd21627fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b71001f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd21627fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 0x55b71007f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd21627fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b71007f9b0 0x55b71007fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd21627fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e400b9d0 con 0x55b71007f9b0 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd21627fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e400ce40 con 0x55b70fefdd00 2024-05-09T00:14:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd21627fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e400c0a0 con 0x55b710000400 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd21627fb80 10 client.0 _ll_register_callbacks cb 0x55b70fd946c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd205e06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b71007f9b0 0x55b71007fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd204e04700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 0x55b71007f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd205e06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b71007f9b0 0x55b71007fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57114/0 (socket says 172.21.15.46:57114) 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd205e06700 1 -- 172.21.15.46:0/58500885 learned_addr learned my addr 172.21.15.46:0/58500885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.237+0000 7fd205605700 1 --2- 172.21.15.46:0/58500885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b71001f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd205e06700 1 -- 172.21.15.46:0/58500885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 msgr2=0x55b71007f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd205e06700 1 --2- 172.21.15.46:0/58500885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 0x55b71007f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd205e06700 1 -- 172.21.15.46:0/58500885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 msgr2=0x55b71001f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd205e06700 1 --2- 172.21.15.46:0/58500885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b71001f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd205e06700 1 -- 172.21.15.46:0/58500885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b71001cb60 con 0x55b71007f9b0 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd205e06700 1 --2- 172.21.15.46:0/58500885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b71007f9b0 0x55b71007fe80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80df4f0 tx=0x7fd1f80a94f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5adc26e6ef09c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd1e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd1e8ff9700 1 -- 172.21.15.46:0/58500885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80a9010 con 0x55b71007f9b0 2024-05-09T00:14:41.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd1e8ff9700 1 -- 172.21.15.46:0/58500885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd1f80b1bf0 con 0x55b71007f9b0 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd21627fb80 1 -- 172.21.15.46:0/58500885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1fc030e50 con 0x55b71007f9b0 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd1e8ff9700 1 -- 172.21.15.46:0/58500885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd1f80ad3c0 con 0x55b71007f9b0 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd21627fb80 10 client.15810 fetch_fsmap learned FSMap version 26 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd21627fb80 1 -- 172.21.15.46:0/58500885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b710088d10 con 0x55b71007f9b0 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.238+0000 7fd1e8ff9700 1 -- 172.21.15.46:0/58500885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1fc030e50 con 0x55b71007f9b0 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.239+0000 7fd1e8ff9700 1 -- 172.21.15.46:0/58500885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd1f80a4b10 con 0x55b71007f9b0 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.239+0000 7fd21627fb80 10 client.15810 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.239+0000 7fd21627fb80 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.239+0000 7fd21627fb80 20 client.15810 populate_metadata read hostname 'smithi046' 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.239+0000 7fd21627fb80 1 --2- 172.21.15.46:0/58500885 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b7100893c0 0x55b71008b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.239+0000 7fd21627fb80 4 client.15810 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.239+0000 7fd21627fb80 1 -- 172.21.15.46:0/58500885 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b70ff2ab20 con 0x55b7100893c0 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.239+0000 7fd205605700 1 --2- 172.21.15.46:0/58500885 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b7100893c0 0x55b71008b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.239+0000 7fd1e8ff9700 10 client.15810 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.239+0000 7fd205605700 1 --2- 172.21.15.46:0/58500885 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b7100893c0 0x55b71008b7e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b15fb7dd40df2ec server_cookie=35720b7c4ed8c19c in_seq=0 out_seq=0 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd1e8ff9700 1 -- 172.21.15.46:0/58500885 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b70ff2ab20 con 0x55b7100893c0 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd1e8ff9700 10 client.15810 handle_command_reply: tid=0 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd21627fb80 1 client.15810 shutdown 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd21627fb80 1 -- 172.21.15.46:0/58500885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b71007f9b0 msgr2=0x55b71007fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd21627fb80 1 --2- 172.21.15.46:0/58500885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b71007f9b0 0x55b71007fe80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80df4f0 tx=0x7fd1f80a94f0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd21627fb80 1 -- 172.21.15.46:0/58500885 shutdown_connections 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd21627fb80 1 --2- 172.21.15.46:0/58500885 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b7100893c0 0x55b71008b7e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd21627fb80 1 --2- 172.21.15.46:0/58500885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b71007f9b0 0x55b71007fe80 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd21627fb80 1 --2- 172.21.15.46:0/58500885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70fefdd00 0x55b71001f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd21627fb80 1 --2- 172.21.15.46:0/58500885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b710000400 0x55b71007f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd21627fb80 1 -- 172.21.15.46:0/58500885 >> 172.21.15.46:0/58500885 conn(0x55b70fe86f70 msgr2=0x55b70fff4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd1e8ff9700 0 client.15810 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:14:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd21627fb80 1 -- 172.21.15.46:0/58500885 shutdown_connections 2024-05-09T00:14:41.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd21627fb80 1 -- 172.21.15.46:0/58500885 wait complete. 2024-05-09T00:14:41.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.240+0000 7fd21627fb80 20 client.15810 trim_cache size 0 max 16384 2024-05-09T00:14:41.243 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:14:41.243 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:14:41.243 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T00:14:41.243 INFO:teuthology.orchestra.run.smithi046.stdout: "fafbdb38-8f86-46b3-a4c5-b91dbb3a9c97": { 2024-05-09T00:14:41.243 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T00:14:41.243 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "fafbdb38-8f86-46b3-a4c5-b91dbb3a9c97", 2024-05-09T00:14:41.243 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T00:14:41.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:14:41.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:14:41.244 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:14:41.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.241+0000 7fd207869700 1 -- 172.21.15.46:0/1341211802 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd1ec0b2900 msgr2=0x7fd1ec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.241+0000 7fd207869700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd1ec0b2900 0x7fd1ec0b4e10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009990 tx=0x7fd1fc0036e0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.241+0000 7fd207869700 1 -- 172.21.15.46:0/1341211802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd200108740 msgr2=0x7fd20019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:14:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.241+0000 7fd207869700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd200108740 0x7fd20019f6a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8007ee0 tx=0x7fd1f800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.241+0000 7fd207869700 1 -- 172.21.15.46:0/1341211802 shutdown_connections 2024-05-09T00:14:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.241+0000 7fd207869700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd1ec0b2900 0x7fd1ec0b4e10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.241+0000 7fd207869700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd200108740 0x7fd20019f6a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.241+0000 7fd207869700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2001034c0 0x7fd20019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.241+0000 7fd207869700 1 --2- 172.21.15.46:0/1341211802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd200107df0 0x7fd20019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:14:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.241+0000 7fd207869700 1 -- 172.21.15.46:0/1341211802 >> 172.21.15.46:0/1341211802 conn(0x7fd2000fedf0 msgr2=0x7fd2001063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:14:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.241+0000 7fd207869700 1 -- 172.21.15.46:0/1341211802 shutdown_connections 2024-05-09T00:14:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:14:41.241+0000 7fd207869700 1 -- 172.21.15.46:0/1341211802 wait complete. 2024-05-09T00:14:42.071 INFO:tasks.cephfs.filesystem:scrub status for tag:fafbdb38-8f86-46b3-a4c5-b91dbb3a9c97 - {'path': '/', 'tag': 'fafbdb38-8f86-46b3-a4c5-b91dbb3a9c97', 'options': 'recursive,force'} 2024-05-09T00:14:42.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:42 smithi046 ceph-mon[30151]: pgmap v476: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:14:42.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:42 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:14:43.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:42 smithi169 ceph-mon[34777]: pgmap v476: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:14:43.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:42 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:14:43.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:42 smithi142 ceph-mon[34533]: pgmap v476: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:14:43.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:42 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:14:45.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:44 smithi046 ceph-mon[30151]: pgmap v477: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:45.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:44 smithi169 ceph-mon[34777]: pgmap v477: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:45.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:44 smithi142 ceph-mon[34533]: pgmap v477: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:47.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:46 smithi046 ceph-mon[30151]: pgmap v478: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:14:47.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:46 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:14:47.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:46 smithi169 ceph-mon[34777]: pgmap v478: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:14:47.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:46 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:14:47.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:46 smithi142 ceph-mon[34533]: pgmap v478: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:14:47.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:46 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:14:48.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:14:48.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:14:48.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:14:48.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:14:48.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:14:48.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:14:48.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:14:48.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:14:48.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:14:49.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:48 smithi046 ceph-mon[30151]: pgmap v479: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:49.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:48 smithi169 ceph-mon[34777]: pgmap v479: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:49.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:48 smithi142 ceph-mon[34533]: pgmap v479: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:51.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:50 smithi046 ceph-mon[30151]: pgmap v480: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:14:51.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:50 smithi169 ceph-mon[34777]: pgmap v480: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:14:51.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:50 smithi142 ceph-mon[34533]: pgmap v480: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:14:53.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:52 smithi046 ceph-mon[30151]: pgmap v481: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:14:53.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:52 smithi169 ceph-mon[34777]: pgmap v481: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:14:53.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:52 smithi142 ceph-mon[34533]: pgmap v481: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:14:55.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:54 smithi046 ceph-mon[30151]: pgmap v482: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:55.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:54 smithi169 ceph-mon[34777]: pgmap v482: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:55.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:54 smithi142 ceph-mon[34533]: pgmap v482: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:14:57.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:56 smithi046 ceph-mon[30151]: pgmap v483: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:57.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:56 smithi169 ceph-mon[34777]: pgmap v483: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:57.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:56 smithi142 ceph-mon[34533]: pgmap v483: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:14:59.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:14:58 smithi046 ceph-mon[30151]: pgmap v484: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:59.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:14:58 smithi169 ceph-mon[34777]: pgmap v484: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:14:59.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:14:58 smithi142 ceph-mon[34533]: pgmap v484: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:01.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:00 smithi046 ceph-mon[30151]: pgmap v485: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:01.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:00 smithi169 ceph-mon[34777]: pgmap v485: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:01.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:00 smithi142 ceph-mon[34533]: pgmap v485: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:03.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:02 smithi046 ceph-mon[30151]: pgmap v486: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:03.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:02 smithi169 ceph-mon[34777]: pgmap v486: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:03.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:02 smithi142 ceph-mon[34533]: pgmap v486: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:05.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:04 smithi046 ceph-mon[30151]: pgmap v487: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:05.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:04 smithi169 ceph-mon[34777]: pgmap v487: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:05.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:04 smithi142 ceph-mon[34533]: pgmap v487: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:07.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:06 smithi046 ceph-mon[30151]: pgmap v488: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:07.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:06 smithi169 ceph-mon[34777]: pgmap v488: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:07.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:06 smithi142 ceph-mon[34533]: pgmap v488: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:09.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:08 smithi046 ceph-mon[30151]: pgmap v489: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-05-09T00:15:09.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:08 smithi169 ceph-mon[34777]: pgmap v489: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-05-09T00:15:09.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:08 smithi142 ceph-mon[34533]: pgmap v489: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-05-09T00:15:11.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:10 smithi046 ceph-mon[30151]: pgmap v490: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-05-09T00:15:11.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:10 smithi169 ceph-mon[34777]: pgmap v490: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-05-09T00:15:11.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:10 smithi142 ceph-mon[34533]: pgmap v490: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-05-09T00:15:12.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:15:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:13.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:12 smithi046 ceph-mon[30151]: pgmap v491: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:13.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:12 smithi169 ceph-mon[34777]: pgmap v491: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:13.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:12 smithi142 ceph-mon[34533]: pgmap v491: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:14.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.278+0000 7f28be71c700 1 -- 172.21.15.46:0/3172982942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b81034c0 msgr2=0x7f28b81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:14.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.278+0000 7f28be71c700 1 --2- 172.21.15.46:0/3172982942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b81034c0 0x7f28b81038d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f28a80099c0 tx=0x7f28a8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.278+0000 7f28be71c700 1 -- 172.21.15.46:0/3172982942 shutdown_connections 2024-05-09T00:15:14.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.278+0000 7f28be71c700 1 --2- 172.21.15.46:0/3172982942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28b8108740 0x7f28b81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.278+0000 7f28be71c700 1 --2- 172.21.15.46:0/3172982942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b81034c0 0x7f28b81038d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.278+0000 7f28be71c700 1 --2- 172.21.15.46:0/3172982942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b8107df0 0x7f28b8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.278+0000 7f28be71c700 1 -- 172.21.15.46:0/3172982942 >> 172.21.15.46:0/3172982942 conn(0x7f28b80fedf0 msgr2=0x7f28b8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.278+0000 7f28be71c700 1 -- 172.21.15.46:0/3172982942 shutdown_connections 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.278+0000 7f28be71c700 1 -- 172.21.15.46:0/3172982942 wait complete. 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28be71c700 1 Processor -- start 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28be71c700 1 -- start start 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28be71c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b81034c0 0x7f28b8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28be71c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28b8107df0 0x7f28b819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28be71c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b8108740 0x7f28b819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28be71c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81133e0 con 0x7f28b81034c0 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28be71c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8113260 con 0x7f28b8108740 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28be71c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8113560 con 0x7f28b8107df0 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28b7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b81034c0 0x7f28b8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28b7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b81034c0 0x7f28b8199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34396/0 (socket says 172.21.15.46:34396) 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28b7fff700 1 -- 172.21.15.46:0/4171354836 learned_addr learned my addr 172.21.15.46:0/4171354836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28bccb9700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b8108740 0x7f28b819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28affff700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28b8107df0 0x7f28b819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28b7fff700 1 -- 172.21.15.46:0/4171354836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28b8107df0 msgr2=0x7f28b819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28b7fff700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28b8107df0 0x7f28b819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28b7fff700 1 -- 172.21.15.46:0/4171354836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b8108740 msgr2=0x7f28b819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28b7fff700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b8108740 0x7f28b819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.279+0000 7f28b7fff700 1 -- 172.21.15.46:0/4171354836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b81a91b0 con 0x7f28b81034c0 2024-05-09T00:15:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.280+0000 7f28b7fff700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b81034c0 0x7f28b8199b30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f28a8006010 tx=0x7f28a8003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.280+0000 7f28b5ffb700 1 -- 172.21.15.46:0/4171354836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8027070 con 0x7f28b81034c0 2024-05-09T00:15:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.280+0000 7f28b5ffb700 1 -- 172.21.15.46:0/4171354836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28a8004200 con 0x7f28b81034c0 2024-05-09T00:15:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.280+0000 7f28be71c700 1 -- 172.21.15.46:0/4171354836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b81a9440 con 0x7f28b81034c0 2024-05-09T00:15:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.280+0000 7f28b5ffb700 1 -- 172.21.15.46:0/4171354836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a800f460 con 0x7f28b81034c0 2024-05-09T00:15:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.280+0000 7f28be71c700 1 -- 172.21.15.46:0/4171354836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b81a98f0 con 0x7f28b81034c0 2024-05-09T00:15:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.283+0000 7f28b5ffb700 1 -- 172.21.15.46:0/4171354836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f28a802a070 con 0x7f28b81034c0 2024-05-09T00:15:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.283+0000 7f28b5ffb700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28a40b2900 0x7f28a40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.283+0000 7f28affff700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28a40b2900 0x7f28a40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.284+0000 7f28b5ffb700 1 -- 172.21.15.46:0/4171354836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f28a80e3270 con 0x7f28b81034c0 2024-05-09T00:15:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.284+0000 7f28affff700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28a40b2900 0x7f28a40b4e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f28a0006fd0 tx=0x7f28a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.284+0000 7f28cd132b80 1 Processor -- start 2024-05-09T00:15:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.284+0000 7f28cd132b80 1 -- start start 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.284+0000 7f28cd132b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac1c3160 0x561fac25c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.284+0000 7f28cd132b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 0x561fac25ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.284+0000 7f28cd132b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 0x561fac256ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.284+0000 7f28cd132b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561fac178ff0 con 0x561fac0d94a0 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.284+0000 7f28cd132b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561fac1177f0 con 0x561fac1c3160 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.284+0000 7f28cd132b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561fac174fb0 con 0x561fac2566a0 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 0x561fac25ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 0x561fac25ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34408/0 (socket says 172.21.15.46:34408) 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28affff700 1 -- 172.21.15.46:0/3674173689 learned_addr learned my addr 172.21.15.46:0/3674173689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28b7fff700 1 --2- 172.21.15.46:0/3674173689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac1c3160 0x561fac25c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28affff700 1 -- 172.21.15.46:0/3674173689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 msgr2=0x561fac256ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28affff700 1 --2- 172.21.15.46:0/3674173689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 0x561fac256ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28affff700 1 -- 172.21.15.46:0/3674173689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac1c3160 msgr2=0x561fac25c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28affff700 1 --2- 172.21.15.46:0/3674173689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac1c3160 0x561fac25c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28affff700 1 -- 172.21.15.46:0/3674173689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fac10f370 con 0x561fac0d94a0 2024-05-09T00:15:14.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28b7fff700 1 --2- 172.21.15.46:0/3674173689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac1c3160 0x561fac25c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:14.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28affff700 1 --2- 172.21.15.46:0/3674173689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 0x561fac25ca30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f28a000d6a0 tx=0x7f28a0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a79b2c20da7a0f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:14.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28ae7fc700 1 -- 172.21.15.46:0/3674173689 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a0024070 con 0x561fac0d94a0 2024-05-09T00:15:14.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28ae7fc700 1 -- 172.21.15.46:0/3674173689 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28a0014af0 con 0x561fac0d94a0 2024-05-09T00:15:14.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.285+0000 7f28ae7fc700 1 -- 172.21.15.46:0/3674173689 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a001fae0 con 0x561fac0d94a0 2024-05-09T00:15:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.286+0000 7f28cd132b80 1 -- 172.21.15.46:0/3674173689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 msgr2=0x561fac25ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.286+0000 7f28cd132b80 1 --2- 172.21.15.46:0/3674173689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 0x561fac25ca30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f28a000d6a0 tx=0x7f28a0010000 comp rx=0 tx=0).stop 2024-05-09T00:15:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.286+0000 7f28cd132b80 1 -- 172.21.15.46:0/3674173689 shutdown_connections 2024-05-09T00:15:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.286+0000 7f28cd132b80 1 --2- 172.21.15.46:0/3674173689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 0x561fac256ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.286+0000 7f28cd132b80 1 --2- 172.21.15.46:0/3674173689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac1c3160 0x561fac25c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.286+0000 7f28cd132b80 1 --2- 172.21.15.46:0/3674173689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 0x561fac25ca30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.286+0000 7f28cd132b80 1 -- 172.21.15.46:0/3674173689 >> 172.21.15.46:0/3674173689 conn(0x561fac1396a0 msgr2=0x561fac251400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.286+0000 7f28cd132b80 1 -- 172.21.15.46:0/3674173689 shutdown_connections 2024-05-09T00:15:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.286+0000 7f28cd132b80 1 -- 172.21.15.46:0/3674173689 wait complete. 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.287+0000 7f28cd132b80 1 -- start start 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.287+0000 7f28cd132b80 1 Processor -- start 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.287+0000 7f28cd132b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 0x561fac2602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.287+0000 7f28cd132b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac1c3160 0x561fac2607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.287+0000 7f28cd132b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac2566a0 0x561fac260d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.287+0000 7f28cd132b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561fabff4ba0 con 0x561fac0d94a0 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.287+0000 7f28cd132b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561fac133150 con 0x561fac2566a0 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.287+0000 7f28cd132b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561fac2631d0 con 0x561fac1c3160 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.287+0000 7f28b7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 0x561fac2602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.287+0000 7f28cd132b80 10 client.0 _ll_register_callbacks cb 0x561fabfea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.287+0000 7f28b7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 0x561fac2602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34414/0 (socket says 172.21.15.46:34414) 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.287+0000 7f28b7fff700 1 -- 172.21.15.46:0/3888111163 learned_addr learned my addr 172.21.15.46:0/3888111163 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28affff700 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac1c3160 0x561fac2607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28bccb9700 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac2566a0 0x561fac260d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28b7fff700 1 -- 172.21.15.46:0/3888111163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac1c3160 msgr2=0x561fac2607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28b7fff700 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac1c3160 0x561fac2607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28b7fff700 1 -- 172.21.15.46:0/3888111163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac2566a0 msgr2=0x561fac260d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28b7fff700 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac2566a0 0x561fac260d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28b7fff700 1 -- 172.21.15.46:0/3888111163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fabff4490 con 0x561fac0d94a0 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28bccb9700 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac2566a0 0x561fac260d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28b7fff700 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 0x561fac2602a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f28a80a7b90 tx=0x7f28a80aa820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d22c2ad95ad6af7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28977fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28977fe700 1 -- 172.21.15.46:0/3888111163 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a80afe60 con 0x561fac0d94a0 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28977fe700 1 -- 172.21.15.46:0/3888111163 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f28a8022470 con 0x561fac0d94a0 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28cd132b80 1 -- 172.21.15.46:0/3888111163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28a0014af0 con 0x561fac0d94a0 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28977fe700 1 -- 172.21.15.46:0/3888111163 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28a80ab020 con 0x561fac0d94a0 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28cd132b80 10 client.15822 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.288+0000 7f28cd132b80 1 -- 172.21.15.46:0/3888111163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561fac26dc70 con 0x561fac0d94a0 2024-05-09T00:15:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.289+0000 7f28977fe700 1 -- 172.21.15.46:0/3888111163 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28a80c6070 con 0x561fac0d94a0 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.289+0000 7f28977fe700 1 -- 172.21.15.46:0/3888111163 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f28a80bfce0 con 0x561fac0d94a0 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.289+0000 7f28cd132b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.289+0000 7f28cd132b80 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.289+0000 7f28cd132b80 20 client.15822 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.289+0000 7f28cd132b80 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561fac270190 0x561fac2725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.289+0000 7f28cd132b80 4 client.15822 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.289+0000 7f28cd132b80 1 -- 172.21.15.46:0/3888111163 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fac1869d0 con 0x561fac270190 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.289+0000 7f28affff700 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561fac270190 0x561fac2725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.290+0000 7f28affff700 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561fac270190 0x561fac2725f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8a5276142feda3 server_cookie=9c7cc9331709ef73 in_seq=0 out_seq=0 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.290+0000 7f28977fe700 10 client.15822 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.290+0000 7f28977fe700 1 -- 172.21.15.46:0/3888111163 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561fac1869d0 con 0x561fac270190 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.290+0000 7f28977fe700 10 client.15822 handle_command_reply: tid=0 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.290+0000 7f28cd132b80 1 client.15822 shutdown 2024-05-09T00:15:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.291+0000 7f28cd132b80 1 -- 172.21.15.46:0/3888111163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 msgr2=0x561fac2602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.291+0000 7f28cd132b80 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 0x561fac2602a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f28a80a7b90 tx=0x7f28a80aa820 comp rx=0 tx=0).stop 2024-05-09T00:15:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.291+0000 7f28cd132b80 1 -- 172.21.15.46:0/3888111163 shutdown_connections 2024-05-09T00:15:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.291+0000 7f28cd132b80 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561fac270190 0x561fac2725f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.291+0000 7f28cd132b80 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac2566a0 0x561fac260d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.291+0000 7f28977fe700 0 client.15822 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.291+0000 7f28cd132b80 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac0d94a0 0x561fac2602a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.291+0000 7f28cd132b80 1 --2- 172.21.15.46:0/3888111163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac1c3160 0x561fac2607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.291+0000 7f28cd132b80 1 -- 172.21.15.46:0/3888111163 >> 172.21.15.46:0/3888111163 conn(0x561fac1396a0 msgr2=0x561fac2511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.291+0000 7f28cd132b80 1 -- 172.21.15.46:0/3888111163 shutdown_connections 2024-05-09T00:15:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.291+0000 7f28cd132b80 1 -- 172.21.15.46:0/3888111163 wait complete. 2024-05-09T00:15:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.291+0000 7f28cd132b80 20 client.15822 trim_cache size 0 max 16384 2024-05-09T00:15:14.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28cd132b80 1 Processor -- start 2024-05-09T00:15:14.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28cd132b80 1 -- start start 2024-05-09T00:15:14.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28cd132b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac0d94a0 0x561fac25ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28cd132b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 0x561fac25a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28cd132b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 0x561fac260b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28cd132b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81b3650 con 0x561fac1c3160 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28cd132b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81b0ee0 con 0x561fac0d94a0 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28cd132b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81aed10 con 0x561fac2566a0 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 0x561fac25a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 0x561fac25a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34420/0 (socket says 172.21.15.46:34420) 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28affff700 1 -- 172.21.15.46:0/1534678834 learned_addr learned my addr 172.21.15.46:0/1534678834 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28b7fff700 1 --2- 172.21.15.46:0/1534678834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac0d94a0 0x561fac25ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28bccb9700 1 --2- 172.21.15.46:0/1534678834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 0x561fac260b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28affff700 1 -- 172.21.15.46:0/1534678834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 msgr2=0x561fac260b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28affff700 1 --2- 172.21.15.46:0/1534678834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 0x561fac260b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28affff700 1 -- 172.21.15.46:0/1534678834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac0d94a0 msgr2=0x561fac25ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28affff700 1 --2- 172.21.15.46:0/1534678834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac0d94a0 0x561fac25ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28affff700 1 -- 172.21.15.46:0/1534678834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fac2630a0 con 0x561fac1c3160 2024-05-09T00:15:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28b7fff700 1 --2- 172.21.15.46:0/1534678834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac0d94a0 0x561fac25ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.312+0000 7f28affff700 1 --2- 172.21.15.46:0/1534678834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 0x561fac25a830 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f28a000d6a0 tx=0x7f28a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b42233e39288fc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.313+0000 7f28adffb700 1 -- 172.21.15.46:0/1534678834 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a0024040 con 0x561fac1c3160 2024-05-09T00:15:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.313+0000 7f28adffb700 1 -- 172.21.15.46:0/1534678834 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28a0033030 con 0x561fac1c3160 2024-05-09T00:15:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.313+0000 7f28adffb700 1 -- 172.21.15.46:0/1534678834 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a002f530 con 0x561fac1c3160 2024-05-09T00:15:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.313+0000 7f28cd132b80 1 -- 172.21.15.46:0/1534678834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 msgr2=0x561fac25a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.313+0000 7f28cd132b80 1 --2- 172.21.15.46:0/1534678834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 0x561fac25a830 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f28a000d6a0 tx=0x7f28a0004750 comp rx=0 tx=0).stop 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.314+0000 7f28cd132b80 1 -- 172.21.15.46:0/1534678834 shutdown_connections 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.314+0000 7f28cd132b80 1 --2- 172.21.15.46:0/1534678834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 0x561fac260b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.314+0000 7f28cd132b80 1 --2- 172.21.15.46:0/1534678834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac0d94a0 0x561fac25ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.314+0000 7f28cd132b80 1 --2- 172.21.15.46:0/1534678834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 0x561fac25a830 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.314+0000 7f28cd132b80 1 -- 172.21.15.46:0/1534678834 >> 172.21.15.46:0/1534678834 conn(0x561fac1396a0 msgr2=0x561fac255d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.314+0000 7f28cd132b80 1 -- 172.21.15.46:0/1534678834 shutdown_connections 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.314+0000 7f28cd132b80 1 -- 172.21.15.46:0/1534678834 wait complete. 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.314+0000 7f28cd132b80 1 -- start start 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.314+0000 7f28cd132b80 1 Processor -- start 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28cd132b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac0d94a0 0x561fac277600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28cd132b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 0x561fac277b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28cd132b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 0x561fac278080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28cd132b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81af3a0 con 0x561fac1c3160 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28cd132b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81b0810 con 0x561fac0d94a0 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28cd132b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81afa70 con 0x561fac2566a0 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28cd132b80 10 client.0 _ll_register_callbacks cb 0x561fabfea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 0x561fac277b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 0x561fac277b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34430/0 (socket says 172.21.15.46:34430) 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28affff700 1 -- 172.21.15.46:0/2833083419 learned_addr learned my addr 172.21.15.46:0/2833083419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28bccb9700 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 0x561fac278080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28b7fff700 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac0d94a0 0x561fac277600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28affff700 1 -- 172.21.15.46:0/2833083419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 msgr2=0x561fac278080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28affff700 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 0x561fac278080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28affff700 1 -- 172.21.15.46:0/2833083419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac0d94a0 msgr2=0x561fac277600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28affff700 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac0d94a0 0x561fac277600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28affff700 1 -- 172.21.15.46:0/2833083419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fac2d9960 con 0x561fac1c3160 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28b7fff700 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac0d94a0 0x561fac277600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28affff700 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 0x561fac277b40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f28a0022040 tx=0x7f28a001f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32ed2288116e251 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.315+0000 7f28977fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28977fe700 1 -- 172.21.15.46:0/2833083419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a003f070 con 0x561fac1c3160 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28977fe700 1 -- 172.21.15.46:0/2833083419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f28a001f890 con 0x561fac1c3160 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28cd132b80 1 -- 172.21.15.46:0/2833083419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28a0033030 con 0x561fac1c3160 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28977fe700 1 -- 172.21.15.46:0/2833083419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28a004a050 con 0x561fac1c3160 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28cd132b80 10 client.15828 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28cd132b80 1 -- 172.21.15.46:0/2833083419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561fac2daaa0 con 0x561fac1c3160 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28977fe700 1 -- 172.21.15.46:0/2833083419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28a0033030 con 0x561fac1c3160 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28977fe700 1 -- 172.21.15.46:0/2833083419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f28a0013070 con 0x561fac1c3160 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28cd132b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28cd132b80 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28cd132b80 20 client.15828 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28cd132b80 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561fac2dafb0 0x561fac2dd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28cd132b80 4 client.15828 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.316+0000 7f28cd132b80 1 -- 172.21.15.46:0/2833083419 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561fac180a70 con 0x561fac2dafb0 2024-05-09T00:15:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.317+0000 7f28b7fff700 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561fac2dafb0 0x561fac2dd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.317+0000 7f28b7fff700 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561fac2dafb0 0x561fac2dd3d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4718c5143a30a57 server_cookie=6986a83e6906520e in_seq=0 out_seq=0 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.317+0000 7f28977fe700 10 client.15828 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.317+0000 7f28977fe700 1 -- 172.21.15.46:0/2833083419 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561fac180a70 con 0x561fac2dafb0 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.317+0000 7f28977fe700 10 client.15828 handle_command_reply: tid=0 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.317+0000 7f28cd132b80 1 client.15828 shutdown 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28cd132b80 1 -- 172.21.15.46:0/2833083419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 msgr2=0x561fac277b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28cd132b80 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 0x561fac277b40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f28a0022040 tx=0x7f28a001f680 comp rx=0 tx=0).stop 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28cd132b80 1 -- 172.21.15.46:0/2833083419 shutdown_connections 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28cd132b80 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561fac2dafb0 0x561fac2dd3d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28cd132b80 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561fac2566a0 0x561fac278080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28cd132b80 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fac0d94a0 0x561fac277600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28977fe700 0 client.15828 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28cd132b80 1 --2- 172.21.15.46:0/2833083419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fac1c3160 0x561fac277b40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28cd132b80 1 -- 172.21.15.46:0/2833083419 >> 172.21.15.46:0/2833083419 conn(0x561fac1396a0 msgr2=0x561fac255d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28cd132b80 1 -- 172.21.15.46:0/2833083419 shutdown_connections 2024-05-09T00:15:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28cd132b80 1 -- 172.21.15.46:0/2833083419 wait complete. 2024-05-09T00:15:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28cd132b80 20 client.15828 trim_cache size 0 max 16384 2024-05-09T00:15:14.321 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:15:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:15:14.321 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:15:14.321 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:15:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28be71c700 1 -- 172.21.15.46:0/4171354836 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28a40b2900 msgr2=0x7f28a40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28be71c700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28a40b2900 0x7f28a40b4e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f28a0006fd0 tx=0x7f28a0008040 comp rx=0 tx=0).stop 2024-05-09T00:15:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28be71c700 1 -- 172.21.15.46:0/4171354836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b81034c0 msgr2=0x7f28b8199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28be71c700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b81034c0 0x7f28b8199b30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f28a8006010 tx=0x7f28a8003860 comp rx=0 tx=0).stop 2024-05-09T00:15:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28be71c700 1 -- 172.21.15.46:0/4171354836 shutdown_connections 2024-05-09T00:15:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28be71c700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28a40b2900 0x7f28a40b4e10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28be71c700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b8108740 0x7f28b819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28be71c700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b81034c0 0x7f28b8199b30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28be71c700 1 --2- 172.21.15.46:0/4171354836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28b8107df0 0x7f28b819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.318+0000 7f28be71c700 1 -- 172.21.15.46:0/4171354836 >> 172.21.15.46:0/4171354836 conn(0x7f28b80fedf0 msgr2=0x7f28b81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.319+0000 7f28be71c700 1 -- 172.21.15.46:0/4171354836 shutdown_connections 2024-05-09T00:15:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:14.319+0000 7f28be71c700 1 -- 172.21.15.46:0/4171354836 wait complete. 2024-05-09T00:15:14.959 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:14 smithi046 ceph-mon[30151]: pgmap v492: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:14.993 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:15:14.994 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:15:15.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:14 smithi169 ceph-mon[34777]: pgmap v492: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:15.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:14 smithi142 ceph-mon[34533]: pgmap v492: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:15.252 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:17.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:16 smithi046 ceph-mon[30151]: pgmap v493: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.093+0000 7f33eb9af700 1 -- 172.21.15.46:0/3859808965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e41034c0 msgr2=0x7f33e41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.093+0000 7f33eb9af700 1 --2- 172.21.15.46:0/3859808965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e41034c0 0x7f33e41038d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f33e00099c0 tx=0x7f33e0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:15:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.093+0000 7f33eb9af700 1 -- 172.21.15.46:0/3859808965 shutdown_connections 2024-05-09T00:15:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.093+0000 7f33eb9af700 1 --2- 172.21.15.46:0/3859808965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e4108740 0x7f33e41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.093+0000 7f33eb9af700 1 --2- 172.21.15.46:0/3859808965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e41034c0 0x7f33e41038d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.093+0000 7f33eb9af700 1 --2- 172.21.15.46:0/3859808965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33e4107df0 0x7f33e4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.093+0000 7f33eb9af700 1 -- 172.21.15.46:0/3859808965 >> 172.21.15.46:0/3859808965 conn(0x7f33e40fedc0 msgr2=0x7f33e41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.093+0000 7f33eb9af700 1 -- 172.21.15.46:0/3859808965 shutdown_connections 2024-05-09T00:15:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.093+0000 7f33eb9af700 1 -- 172.21.15.46:0/3859808965 wait complete. 2024-05-09T00:15:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33eb9af700 1 Processor -- start 2024-05-09T00:15:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33eb9af700 1 -- start start 2024-05-09T00:15:17.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33eb9af700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e41034c0 0x7f33e419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:17.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33eb9af700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4107df0 0x7f33e419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:17.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33eb9af700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33e4108740 0x7f33e41a6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:17.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33eb9af700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4113370 con 0x7f33e4107df0 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33eb9af700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41131f0 con 0x7f33e4108740 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33eb9af700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41134f0 con 0x7f33e41034c0 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33e8f4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4107df0 0x7f33e419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33e8f4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4107df0 0x7f33e419c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60996/0 (socket says 172.21.15.46:60996) 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33e8f4a700 1 -- 172.21.15.46:0/2641975262 learned_addr learned my addr 172.21.15.46:0/2641975262 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33e974b700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e41034c0 0x7f33e419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33e9f4c700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33e4108740 0x7f33e41a6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33e8f4a700 1 -- 172.21.15.46:0/2641975262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e41034c0 msgr2=0x7f33e419bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33e8f4a700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e41034c0 0x7f33e419bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33e8f4a700 1 -- 172.21.15.46:0/2641975262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33e4108740 msgr2=0x7f33e41a6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33e8f4a700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33e4108740 0x7f33e41a6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33e8f4a700 1 -- 172.21.15.46:0/2641975262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e419ca00 con 0x7f33e4107df0 2024-05-09T00:15:17.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.094+0000 7f33e9f4c700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33e4108740 0x7f33e41a6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:15:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.095+0000 7f33e8f4a700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4107df0 0x7f33e419c150 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f33d400d950 tx=0x7f33d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.095+0000 7f33da7fc700 1 -- 172.21.15.46:0/2641975262 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d4016070 con 0x7f33e4107df0 2024-05-09T00:15:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.095+0000 7f33da7fc700 1 -- 172.21.15.46:0/2641975262 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d4004830 con 0x7f33e4107df0 2024-05-09T00:15:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.095+0000 7f33eb9af700 1 -- 172.21.15.46:0/2641975262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e41a72f0 con 0x7f33e4107df0 2024-05-09T00:15:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.095+0000 7f33da7fc700 1 -- 172.21.15.46:0/2641975262 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d4005440 con 0x7f33e4107df0 2024-05-09T00:15:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.095+0000 7f33eb9af700 1 -- 172.21.15.46:0/2641975262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e41a7830 con 0x7f33e4107df0 2024-05-09T00:15:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.096+0000 7f33eb9af700 1 -- 172.21.15.46:0/2641975262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e40512c0 con 0x7f33e4107df0 2024-05-09T00:15:17.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.098+0000 7f33da7fc700 1 -- 172.21.15.46:0/2641975262 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f33d4004220 con 0x7f33e4107df0 2024-05-09T00:15:17.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.098+0000 7f33da7fc700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33d00b2900 0x7f33d00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:17.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.098+0000 7f33e974b700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33d00b2900 0x7f33d00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:17.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.098+0000 7f33da7fc700 1 -- 172.21.15.46:0/2641975262 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f33d40e2430 con 0x7f33e4107df0 2024-05-09T00:15:17.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.099+0000 7f33e974b700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33d00b2900 0x7f33d00b4e10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000c00 tx=0x7f33e001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.102+0000 7f33da7fc700 1 -- 172.21.15.46:0/2641975262 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33d401f020 con 0x7f33e4107df0 2024-05-09T00:15:17.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:16 smithi169 ceph-mon[34777]: pgmap v493: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:17.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:16 smithi142 ceph-mon[34533]: pgmap v493: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:17.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.364+0000 7f33eb9af700 1 -- 172.21.15.46:0/2641975262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33e402e5f0 con 0x7f33e4107df0 2024-05-09T00:15:17.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.365+0000 7f33da7fc700 1 -- 172.21.15.46:0/2641975262 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f33d40aa150 con 0x7f33e4107df0 2024-05-09T00:15:17.365 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:15:17.365 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:15:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.368+0000 7f33eb9af700 1 -- 172.21.15.46:0/2641975262 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33d00b2900 msgr2=0x7f33d00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.368+0000 7f33eb9af700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33d00b2900 0x7f33d00b4e10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000c00 tx=0x7f33e001a040 comp rx=0 tx=0).stop 2024-05-09T00:15:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.368+0000 7f33eb9af700 1 -- 172.21.15.46:0/2641975262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4107df0 msgr2=0x7f33e419c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.368+0000 7f33eb9af700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4107df0 0x7f33e419c150 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f33d400d950 tx=0x7f33d400dc60 comp rx=0 tx=0).stop 2024-05-09T00:15:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.368+0000 7f33eb9af700 1 -- 172.21.15.46:0/2641975262 shutdown_connections 2024-05-09T00:15:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.368+0000 7f33eb9af700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33d00b2900 0x7f33d00b4e10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.368+0000 7f33eb9af700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33e4108740 0x7f33e41a6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.368+0000 7f33eb9af700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e41034c0 0x7f33e419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.368+0000 7f33eb9af700 1 --2- 172.21.15.46:0/2641975262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4107df0 0x7f33e419c150 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.368+0000 7f33eb9af700 1 -- 172.21.15.46:0/2641975262 >> 172.21.15.46:0/2641975262 conn(0x7f33e40fedc0 msgr2=0x7f33e41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.368+0000 7f33eb9af700 1 -- 172.21.15.46:0/2641975262 shutdown_connections 2024-05-09T00:15:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:17.368+0000 7f33eb9af700 1 -- 172.21.15.46:0/2641975262 wait complete. 2024-05-09T00:15:17.369 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:15:17.992 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:17 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2641975262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:15:17.994 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:15:18.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:17 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2641975262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:15:18.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:17 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2641975262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:15:18.254 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:19.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:18 smithi046 ceph-mon[30151]: pgmap v494: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:19.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:18 smithi169 ceph-mon[34777]: pgmap v494: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:19.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:18 smithi142 ceph-mon[34533]: pgmap v494: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.112+0000 7fe281e46700 1 -- 172.21.15.46:0/3415807534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27c1054a0 msgr2=0x7fe27c1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.112+0000 7fe281e46700 1 --2- 172.21.15.46:0/3415807534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27c1054a0 0x7fe27c1058b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009990 tx=0x7fe26c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.112+0000 7fe281e46700 1 -- 172.21.15.46:0/3415807534 shutdown_connections 2024-05-09T00:15:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.112+0000 7fe281e46700 1 --2- 172.21.15.46:0/3415807534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe27c10e910 0x7fe27c110e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.112+0000 7fe281e46700 1 --2- 172.21.15.46:0/3415807534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27c1054a0 0x7fe27c1058b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.112+0000 7fe281e46700 1 --2- 172.21.15.46:0/3415807534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe27c105df0 0x7fe27c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.112+0000 7fe281e46700 1 -- 172.21.15.46:0/3415807534 >> 172.21.15.46:0/3415807534 conn(0x7fe27c078e50 msgr2=0x7fe27c079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.112+0000 7fe281e46700 1 -- 172.21.15.46:0/3415807534 shutdown_connections 2024-05-09T00:15:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.112+0000 7fe281e46700 1 -- 172.21.15.46:0/3415807534 wait complete. 2024-05-09T00:15:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.113+0000 7fe281e46700 1 Processor -- start 2024-05-09T00:15:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.113+0000 7fe281e46700 1 -- start start 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.113+0000 7fe281e46700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27c1054a0 0x7fe27c19d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.113+0000 7fe281e46700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe27c105df0 0x7fe27c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.113+0000 7fe281e46700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe27c10e910 0x7fe27c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.113+0000 7fe281e46700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c104690 con 0x7fe27c1054a0 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.113+0000 7fe281e46700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c104510 con 0x7fe27c10e910 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.113+0000 7fe281e46700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c104810 con 0x7fe27c105df0 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.113+0000 7fe27b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27c1054a0 0x7fe27c19d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.113+0000 7fe27b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27c1054a0 0x7fe27c19d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32776/0 (socket says 172.21.15.46:32776) 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.113+0000 7fe27b7fe700 1 -- 172.21.15.46:0/2661697190 learned_addr learned my addr 172.21.15.46:0/2661697190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe27affd700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe27c105df0 0x7fe27c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe27bfff700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe27c10e910 0x7fe27c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe27b7fe700 1 -- 172.21.15.46:0/2661697190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe27c105df0 msgr2=0x7fe27c19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe27b7fe700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe27c105df0 0x7fe27c19d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe27b7fe700 1 -- 172.21.15.46:0/2661697190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe27c10e910 msgr2=0x7fe27c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:20.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe27b7fe700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe27c10e910 0x7fe27c1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe27b7fe700 1 -- 172.21.15.46:0/2661697190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c1a1e90 con 0x7fe27c1054a0 2024-05-09T00:15:20.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe27b7fe700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27c1054a0 0x7fe27c19d070 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe26c000c00 tx=0x7fe26c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe278ff9700 1 -- 172.21.15.46:0/2661697190 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c027070 con 0x7fe27c1054a0 2024-05-09T00:15:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe278ff9700 1 -- 172.21.15.46:0/2661697190 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe26c0040e0 con 0x7fe27c1054a0 2024-05-09T00:15:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe281e46700 1 -- 172.21.15.46:0/2661697190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27c1a2120 con 0x7fe27c1054a0 2024-05-09T00:15:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe278ff9700 1 -- 172.21.15.46:0/2661697190 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00f460 con 0x7fe27c1054a0 2024-05-09T00:15:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.114+0000 7fe281e46700 1 -- 172.21.15.46:0/2661697190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe27c1a2650 con 0x7fe27c1054a0 2024-05-09T00:15:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.117+0000 7fe278ff9700 1 -- 172.21.15.46:0/2661697190 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe26c02a070 con 0x7fe27c1054a0 2024-05-09T00:15:20.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.118+0000 7fe278ff9700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe2640b2900 0x7fe2640b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.118+0000 7fe27affd700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe2640b2900 0x7fe2640b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.118+0000 7fe278ff9700 1 -- 172.21.15.46:0/2661697190 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe26c0e3210 con 0x7fe27c1054a0 2024-05-09T00:15:20.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.118+0000 7fe29085cb80 1 Processor -- start 2024-05-09T00:15:20.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.118+0000 7fe29085cb80 1 -- start start 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe27affd700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe2640b2900 0x7fe2640b4e10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe27c10ff40 tx=0x7fe27000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe29085cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7b34d00 0x55bcf7c3d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe29085cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c3d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe29085cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7c37400 0x55bcf7c37810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe29085cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bcf7b5a0a0 con 0x55bcf7aba4a0 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe29085cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bcf7aff9a0 con 0x55bcf7c37400 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe29085cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bcf7b56060 con 0x55bcf7b34d00 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe27affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c3d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe27affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c3d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32784/0 (socket says 172.21.15.46:32784) 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe27affd700 1 -- 172.21.15.46:0/302393866 learned_addr learned my addr 172.21.15.46:0/302393866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe27b7fe700 1 --2- 172.21.15.46:0/302393866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7b34d00 0x55bcf7c3d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe27bfff700 1 --2- 172.21.15.46:0/302393866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7c37400 0x55bcf7c37810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe27affd700 1 -- 172.21.15.46:0/302393866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7b34d00 msgr2=0x55bcf7c3d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe27affd700 1 --2- 172.21.15.46:0/302393866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7b34d00 0x55bcf7c3d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe27affd700 1 -- 172.21.15.46:0/302393866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7c37400 msgr2=0x55bcf7c37810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:20.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe27affd700 1 --2- 172.21.15.46:0/302393866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7c37400 0x55bcf7c37810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe27affd700 1 -- 172.21.15.46:0/302393866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcf7af0500 con 0x55bcf7aba4a0 2024-05-09T00:15:20.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.119+0000 7fe27affd700 1 --2- 172.21.15.46:0/302393866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c3d780 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe27000fd70 tx=0x7fe270012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ab67ab0f9acc0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:20.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.120+0000 7fe261ffb700 1 -- 172.21.15.46:0/302393866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270016070 con 0x55bcf7aba4a0 2024-05-09T00:15:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.120+0000 7fe261ffb700 1 -- 172.21.15.46:0/302393866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe27000fe50 con 0x55bcf7aba4a0 2024-05-09T00:15:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.120+0000 7fe261ffb700 1 -- 172.21.15.46:0/302393866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27001b420 con 0x55bcf7aba4a0 2024-05-09T00:15:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.121+0000 7fe29085cb80 1 -- 172.21.15.46:0/302393866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 msgr2=0x55bcf7c3d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.121+0000 7fe29085cb80 1 --2- 172.21.15.46:0/302393866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c3d780 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe27000fd70 tx=0x7fe270012000 comp rx=0 tx=0).stop 2024-05-09T00:15:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.121+0000 7fe29085cb80 1 -- 172.21.15.46:0/302393866 shutdown_connections 2024-05-09T00:15:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.121+0000 7fe29085cb80 1 --2- 172.21.15.46:0/302393866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7c37400 0x55bcf7c37810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.121+0000 7fe29085cb80 1 --2- 172.21.15.46:0/302393866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7b34d00 0x55bcf7c3d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.121+0000 7fe29085cb80 1 --2- 172.21.15.46:0/302393866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c3d780 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.121+0000 7fe29085cb80 1 -- 172.21.15.46:0/302393866 >> 172.21.15.46:0/302393866 conn(0x55bcf7abdf70 msgr2=0x55bcf7c32220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.121+0000 7fe29085cb80 1 -- 172.21.15.46:0/302393866 shutdown_connections 2024-05-09T00:15:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.121+0000 7fe29085cb80 1 -- 172.21.15.46:0/302393866 wait complete. 2024-05-09T00:15:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.121+0000 7fe29085cb80 1 -- start start 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.121+0000 7fe29085cb80 1 Processor -- start 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe29085cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c40f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe29085cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7b34d00 0x55bcf7c414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe29085cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7c37400 0x55bcf7c419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe29085cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bcf7c38160 con 0x55bcf7aba4a0 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe29085cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bcf7af4b20 con 0x55bcf7b34d00 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe29085cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bcf7c44020 con 0x55bcf7c37400 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe29085cb80 10 client.0 _ll_register_callbacks cb 0x55bcf79cb6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe27b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c40f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe27b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c40f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32794/0 (socket says 172.21.15.46:32794) 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe27b7fe700 1 -- 172.21.15.46:0/2083611470 learned_addr learned my addr 172.21.15.46:0/2083611470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe27affd700 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7b34d00 0x55bcf7c414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe27bfff700 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7c37400 0x55bcf7c419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe27b7fe700 1 -- 172.21.15.46:0/2083611470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7c37400 msgr2=0x55bcf7c419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe27b7fe700 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7c37400 0x55bcf7c419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe27b7fe700 1 -- 172.21.15.46:0/2083611470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7b34d00 msgr2=0x55bcf7c414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe27b7fe700 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7b34d00 0x55bcf7c414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe27b7fe700 1 -- 172.21.15.46:0/2083611470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcf79d5490 con 0x55bcf7aba4a0 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe27bfff700 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7c37400 0x55bcf7c419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe27b7fe700 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c40f60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0098e0 tx=0x7fe26c0aadc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e53ec0eb60e4a19 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.122+0000 7fe25affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe25affd700 1 -- 172.21.15.46:0/2083611470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0aa1d0 con 0x55bcf7aba4a0 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe25affd700 1 -- 172.21.15.46:0/2083611470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe26c0cd0f0 con 0x55bcf7aba4a0 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe29085cb80 1 -- 172.21.15.46:0/2083611470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe27000fe50 con 0x55bcf7aba4a0 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe25affd700 1 -- 172.21.15.46:0/2083611470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe26c0a9ff0 con 0x55bcf7aba4a0 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe29085cb80 10 client.15846 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe29085cb80 1 -- 172.21.15.46:0/2083611470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcf7c4e970 con 0x55bcf7aba4a0 2024-05-09T00:15:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe25affd700 1 -- 172.21.15.46:0/2083611470 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe27000fe50 con 0x55bcf7aba4a0 2024-05-09T00:15:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe25affd700 1 -- 172.21.15.46:0/2083611470 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe26c0abbb0 con 0x55bcf7aba4a0 2024-05-09T00:15:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe29085cb80 10 client.15846 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe29085cb80 10 client.15846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:15:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe29085cb80 20 client.15846 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe29085cb80 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bcf7c50e40 0x55bcf7c53280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe29085cb80 4 client.15846 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:15:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.123+0000 7fe29085cb80 1 -- 172.21.15.46:0/2083611470 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcf7b67a80 con 0x55bcf7c50e40 2024-05-09T00:15:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.124+0000 7fe27affd700 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bcf7c50e40 0x55bcf7c53280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.124+0000 7fe25affd700 10 client.15846 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:15:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.124+0000 7fe27affd700 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bcf7c50e40 0x55bcf7c53280 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd5732c8da345882 server_cookie=d2c1ff6eb1ccca1d in_seq=0 out_seq=0 2024-05-09T00:15:20.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.125+0000 7fe25affd700 1 -- 172.21.15.46:0/2083611470 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bcf7b67a80 con 0x55bcf7c50e40 2024-05-09T00:15:20.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.125+0000 7fe25affd700 10 client.15846 handle_command_reply: tid=0 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.125+0000 7fe29085cb80 1 client.15846 shutdown 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.126+0000 7fe29085cb80 1 -- 172.21.15.46:0/2083611470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 msgr2=0x55bcf7c40f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.126+0000 7fe29085cb80 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c40f60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0098e0 tx=0x7fe26c0aadc0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.126+0000 7fe29085cb80 1 -- 172.21.15.46:0/2083611470 shutdown_connections 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.126+0000 7fe29085cb80 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bcf7c50e40 0x55bcf7c53280 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.126+0000 7fe29085cb80 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7c37400 0x55bcf7c419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.126+0000 7fe29085cb80 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c40f60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.126+0000 7fe25affd700 0 client.15846 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.126+0000 7fe29085cb80 1 --2- 172.21.15.46:0/2083611470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7b34d00 0x55bcf7c414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.126+0000 7fe29085cb80 1 -- 172.21.15.46:0/2083611470 >> 172.21.15.46:0/2083611470 conn(0x55bcf7abdf70 msgr2=0x55bcf7c2bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.126+0000 7fe29085cb80 1 -- 172.21.15.46:0/2083611470 shutdown_connections 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.126+0000 7fe29085cb80 1 -- 172.21.15.46:0/2083611470 wait complete. 2024-05-09T00:15:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.126+0000 7fe29085cb80 20 client.15846 trim_cache size 0 max 16384 2024-05-09T00:15:20.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.141+0000 7fe29085cb80 1 Processor -- start 2024-05-09T00:15:20.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.141+0000 7fe29085cb80 1 -- start start 2024-05-09T00:15:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.141+0000 7fe29085cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c2c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.141+0000 7fe29085cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7b34d00 0x55bcf7c36e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.141+0000 7fe29085cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7c37400 0x55bcf7c41820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.141+0000 7fe29085cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1b3630 con 0x55bcf7c37400 2024-05-09T00:15:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.141+0000 7fe29085cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1b0ec0 con 0x55bcf7aba4a0 2024-05-09T00:15:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.141+0000 7fe29085cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1aecf0 con 0x55bcf7b34d00 2024-05-09T00:15:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7c37400 0x55bcf7c41820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7c37400 0x55bcf7c41820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32804/0 (socket says 172.21.15.46:32804) 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27bfff700 1 -- 172.21.15.46:0/3832263977 learned_addr learned my addr 172.21.15.46:0/3832263977 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27b7fe700 1 --2- 172.21.15.46:0/3832263977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c2c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27affd700 1 --2- 172.21.15.46:0/3832263977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7b34d00 0x55bcf7c36e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27affd700 1 -- 172.21.15.46:0/3832263977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7aba4a0 msgr2=0x55bcf7c2c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27affd700 1 --2- 172.21.15.46:0/3832263977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c2c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27affd700 1 -- 172.21.15.46:0/3832263977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7c37400 msgr2=0x55bcf7c41820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27affd700 1 --2- 172.21.15.46:0/3832263977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7c37400 0x55bcf7c41820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27affd700 1 -- 172.21.15.46:0/3832263977 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcf7c32f40 con 0x55bcf7b34d00 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27b7fe700 1 --2- 172.21.15.46:0/3832263977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c2c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27bfff700 1 --2- 172.21.15.46:0/3832263977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7c37400 0x55bcf7c41820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe27affd700 1 --2- 172.21.15.46:0/3832263977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7b34d00 0x55bcf7c36e20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe27000a9d0 tx=0x7fe27000a940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73a578f89abde033 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.142+0000 7fe2617fa700 1 -- 172.21.15.46:0/3832263977 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270016070 con 0x55bcf7b34d00 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.143+0000 7fe2617fa700 1 -- 172.21.15.46:0/3832263977 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe270025c70 con 0x55bcf7b34d00 2024-05-09T00:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.143+0000 7fe2617fa700 1 -- 172.21.15.46:0/3832263977 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270005040 con 0x55bcf7b34d00 2024-05-09T00:15:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.143+0000 7fe29085cb80 1 -- 172.21.15.46:0/3832263977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7b34d00 msgr2=0x55bcf7c36e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.143+0000 7fe29085cb80 1 --2- 172.21.15.46:0/3832263977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7b34d00 0x55bcf7c36e20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe27000a9d0 tx=0x7fe27000a940 comp rx=0 tx=0).stop 2024-05-09T00:15:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.143+0000 7fe29085cb80 1 -- 172.21.15.46:0/3832263977 shutdown_connections 2024-05-09T00:15:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.143+0000 7fe29085cb80 1 --2- 172.21.15.46:0/3832263977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7c37400 0x55bcf7c41820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.143+0000 7fe29085cb80 1 --2- 172.21.15.46:0/3832263977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c2c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.143+0000 7fe29085cb80 1 --2- 172.21.15.46:0/3832263977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7b34d00 0x55bcf7c36e20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.143+0000 7fe29085cb80 1 -- 172.21.15.46:0/3832263977 >> 172.21.15.46:0/3832263977 conn(0x55bcf7abdf70 msgr2=0x55bcf7c42610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.143+0000 7fe29085cb80 1 -- 172.21.15.46:0/3832263977 shutdown_connections 2024-05-09T00:15:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.143+0000 7fe29085cb80 1 -- 172.21.15.46:0/3832263977 wait complete. 2024-05-09T00:15:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.144+0000 7fe29085cb80 1 -- start start 2024-05-09T00:15:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.144+0000 7fe29085cb80 1 Processor -- start 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe29085cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c49750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe29085cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7b34d00 0x55bcf7c3dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe29085cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7c37400 0x55bcf7c3e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe29085cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1af380 con 0x55bcf7b34d00 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe29085cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1b07f0 con 0x55bcf7aba4a0 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe29085cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c1afa50 con 0x55bcf7c37400 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe29085cb80 10 client.0 _ll_register_callbacks cb 0x55bcf79cb6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe27affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7b34d00 0x55bcf7c3dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe27b7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c49750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe27affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7b34d00 0x55bcf7c3dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32818/0 (socket says 172.21.15.46:32818) 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe27affd700 1 -- 172.21.15.46:0/3888423825 learned_addr learned my addr 172.21.15.46:0/3888423825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe27bfff700 1 --2- 172.21.15.46:0/3888423825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7c37400 0x55bcf7c3e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe27affd700 1 -- 172.21.15.46:0/3888423825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7c37400 msgr2=0x55bcf7c3e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe27affd700 1 --2- 172.21.15.46:0/3888423825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7c37400 0x55bcf7c3e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe27affd700 1 -- 172.21.15.46:0/3888423825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7aba4a0 msgr2=0x55bcf7c49750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe27affd700 1 --2- 172.21.15.46:0/3888423825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c49750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe27affd700 1 -- 172.21.15.46:0/3888423825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcf7c3e9b0 con 0x55bcf7b34d00 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe27affd700 1 --2- 172.21.15.46:0/3888423825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7b34d00 0x55bcf7c3dc20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe27000f820 tx=0x7fe27000fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a877f85f96c9641a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.145+0000 7fe25affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe25affd700 1 -- 172.21.15.46:0/3888423825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27003e060 con 0x55bcf7b34d00 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe29085cb80 1 -- 172.21.15.46:0/3888423825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe270025c70 con 0x55bcf7b34d00 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe25affd700 1 -- 172.21.15.46:0/3888423825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe270025470 con 0x55bcf7b34d00 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe25affd700 1 -- 172.21.15.46:0/3888423825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe270016050 con 0x55bcf7b34d00 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe29085cb80 10 client.15852 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe29085cb80 1 -- 172.21.15.46:0/3888423825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcf7cbb5d0 con 0x55bcf7b34d00 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe25affd700 1 -- 172.21.15.46:0/3888423825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe270025c70 con 0x55bcf7b34d00 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe25affd700 1 -- 172.21.15.46:0/3888423825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe27001bb70 con 0x55bcf7b34d00 2024-05-09T00:15:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe29085cb80 10 client.15852 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe29085cb80 10 client.15852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:15:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe29085cb80 20 client.15852 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe29085cb80 1 --2- 172.21.15.46:0/3888423825 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bcf7cbba30 0x55bcf7cbde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe29085cb80 4 client.15852 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:15:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.146+0000 7fe29085cb80 1 -- 172.21.15.46:0/3888423825 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bcf7b61b20 con 0x55bcf7cbba30 2024-05-09T00:15:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.147+0000 7fe27b7fe700 1 --2- 172.21.15.46:0/3888423825 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bcf7cbba30 0x55bcf7cbde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.147+0000 7fe25affd700 10 client.15852 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:15:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.147+0000 7fe27b7fe700 1 --2- 172.21.15.46:0/3888423825 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bcf7cbba30 0x55bcf7cbde50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82533c5143ef33a3 server_cookie=7783b544f1d677c8 in_seq=0 out_seq=0 2024-05-09T00:15:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.147+0000 7fe25affd700 1 -- 172.21.15.46:0/3888423825 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bcf7b61b20 con 0x55bcf7cbba30 2024-05-09T00:15:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.147+0000 7fe25affd700 10 client.15852 handle_command_reply: tid=0 2024-05-09T00:15:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe29085cb80 1 client.15852 shutdown 2024-05-09T00:15:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe29085cb80 1 -- 172.21.15.46:0/3888423825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7b34d00 msgr2=0x55bcf7c3dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe29085cb80 1 --2- 172.21.15.46:0/3888423825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7b34d00 0x55bcf7c3dc20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe27000f820 tx=0x7fe27000fe50 comp rx=0 tx=0).stop 2024-05-09T00:15:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe29085cb80 1 -- 172.21.15.46:0/3888423825 shutdown_connections 2024-05-09T00:15:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe29085cb80 1 --2- 172.21.15.46:0/3888423825 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bcf7cbba30 0x55bcf7cbde50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe29085cb80 1 --2- 172.21.15.46:0/3888423825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bcf7c37400 0x55bcf7c3e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe29085cb80 1 --2- 172.21.15.46:0/3888423825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcf7aba4a0 0x55bcf7c49750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe25affd700 0 client.15852 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:15:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe29085cb80 1 --2- 172.21.15.46:0/3888423825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf7b34d00 0x55bcf7c3dc20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe29085cb80 1 -- 172.21.15.46:0/3888423825 >> 172.21.15.46:0/3888423825 conn(0x55bcf7abdf70 msgr2=0x55bcf7c36b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe29085cb80 1 -- 172.21.15.46:0/3888423825 shutdown_connections 2024-05-09T00:15:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe29085cb80 1 -- 172.21.15.46:0/3888423825 wait complete. 2024-05-09T00:15:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.148+0000 7fe29085cb80 20 client.15852 trim_cache size 0 max 16384 2024-05-09T00:15:20.152 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:15:20.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.149+0000 7fe281e46700 1 -- 172.21.15.46:0/2661697190 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe2640b2900 msgr2=0x7fe2640b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:20.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.149+0000 7fe281e46700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe2640b2900 0x7fe2640b4e10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe27c10ff40 tx=0x7fe27000a300 comp rx=0 tx=0).stop 2024-05-09T00:15:20.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.149+0000 7fe281e46700 1 -- 172.21.15.46:0/2661697190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27c1054a0 msgr2=0x7fe27c19d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:20.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.149+0000 7fe281e46700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27c1054a0 0x7fe27c19d070 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe26c000c00 tx=0x7fe26c0047b0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.149+0000 7fe281e46700 1 -- 172.21.15.46:0/2661697190 shutdown_connections 2024-05-09T00:15:20.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.149+0000 7fe281e46700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe2640b2900 0x7fe2640b4e10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.149+0000 7fe281e46700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe27c10e910 0x7fe27c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.149+0000 7fe281e46700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27c1054a0 0x7fe27c19d070 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.149+0000 7fe281e46700 1 --2- 172.21.15.46:0/2661697190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe27c105df0 0x7fe27c19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:20.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.149+0000 7fe281e46700 1 -- 172.21.15.46:0/2661697190 >> 172.21.15.46:0/2661697190 conn(0x7fe27c078e50 msgr2=0x7fe27c1062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:20.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.149+0000 7fe281e46700 1 -- 172.21.15.46:0/2661697190 shutdown_connections 2024-05-09T00:15:20.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:20.149+0000 7fe281e46700 1 -- 172.21.15.46:0/2661697190 wait complete. 2024-05-09T00:15:20.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:15:21.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:20 smithi046 ceph-mon[30151]: pgmap v495: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:21.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:20 smithi169 ceph-mon[34777]: pgmap v495: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:21.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:20 smithi142 ceph-mon[34533]: pgmap v495: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:22.968 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:22 smithi046 ceph-mon[30151]: pgmap v496: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.089+0000 7f17e2b6d700 1 -- 172.21.15.46:0/3762049946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17dc108740 msgr2=0x7f17dc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.089+0000 7f17e2b6d700 1 --2- 172.21.15.46:0/3762049946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17dc108740 0x7f17dc1100a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f17d400b0d0 tx=0x7f17d400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.089+0000 7f17e2b6d700 1 -- 172.21.15.46:0/3762049946 shutdown_connections 2024-05-09T00:15:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.089+0000 7f17e2b6d700 1 --2- 172.21.15.46:0/3762049946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17dc108740 0x7f17dc1100a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.089+0000 7f17e2b6d700 1 --2- 172.21.15.46:0/3762049946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17dc1034c0 0x7f17dc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.089+0000 7f17e2b6d700 1 --2- 172.21.15.46:0/3762049946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17dc107df0 0x7f17dc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.089+0000 7f17e2b6d700 1 -- 172.21.15.46:0/3762049946 >> 172.21.15.46:0/3762049946 conn(0x7f17dc0fedf0 msgr2=0x7f17dc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.089+0000 7f17e2b6d700 1 -- 172.21.15.46:0/3762049946 shutdown_connections 2024-05-09T00:15:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.089+0000 7f17e2b6d700 1 -- 172.21.15.46:0/3762049946 wait complete. 2024-05-09T00:15:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.090+0000 7f17e2b6d700 1 Processor -- start 2024-05-09T00:15:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.090+0000 7f17e2b6d700 1 -- start start 2024-05-09T00:15:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.090+0000 7f17e2b6d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17dc1034c0 0x7f17dc19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.090+0000 7f17e2b6d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17dc107df0 0x7f17dc19c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.090+0000 7f17e2b6d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17dc108740 0x7f17dc19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.090+0000 7f17e2b6d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc113370 con 0x7f17dc108740 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.090+0000 7f17e2b6d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1131f0 con 0x7f17dc107df0 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.090+0000 7f17e2b6d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1134f0 con 0x7f17dc1034c0 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17e110a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17dc108740 0x7f17dc19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17e110a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17dc108740 0x7f17dc19c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32836/0 (socket says 172.21.15.46:32836) 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17e110a700 1 -- 172.21.15.46:0/1108734599 learned_addr learned my addr 172.21.15.46:0/1108734599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17e0909700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17dc1034c0 0x7f17dc19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17dbfff700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17dc107df0 0x7f17dc19c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17e110a700 1 -- 172.21.15.46:0/1108734599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17dc1034c0 msgr2=0x7f17dc19bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17e110a700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17dc1034c0 0x7f17dc19bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17e110a700 1 -- 172.21.15.46:0/1108734599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17dc107df0 msgr2=0x7f17dc19c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17e110a700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17dc107df0 0x7f17dc19c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17e110a700 1 -- 172.21.15.46:0/1108734599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc1a6f60 con 0x7f17dc108740 2024-05-09T00:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17dbfff700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17dc107df0 0x7f17dc19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17e110a700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17dc108740 0x7f17dc19c6b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f17d40095c0 tx=0x7f17d4007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17d9ffb700 1 -- 172.21.15.46:0/1108734599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d401f070 con 0x7f17dc108740 2024-05-09T00:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17e2b6d700 1 -- 172.21.15.46:0/1108734599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc1a71f0 con 0x7f17dc108740 2024-05-09T00:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17d9ffb700 1 -- 172.21.15.46:0/1108734599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d4003c00 con 0x7f17dc108740 2024-05-09T00:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.091+0000 7f17d9ffb700 1 -- 172.21.15.46:0/1108734599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d401b6a0 con 0x7f17dc108740 2024-05-09T00:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.092+0000 7f17e2b6d700 1 -- 172.21.15.46:0/1108734599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc1a7640 con 0x7f17dc108740 2024-05-09T00:15:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.094+0000 7f17d9ffb700 1 -- 172.21.15.46:0/1108734599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f17d4022070 con 0x7f17dc108740 2024-05-09T00:15:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.095+0000 7f17d9ffb700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17cc0b2880 0x7f17cc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.095+0000 7f17e0909700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17cc0b2880 0x7f17cc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.095+0000 7f17f1583b80 1 Processor -- start 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.095+0000 7f17d9ffb700 1 -- 172.21.15.46:0/1108734599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f17d400f020 con 0x7f17dc108740 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.095+0000 7f17f1583b80 1 -- start start 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17f1583b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb850d00 0x556fdb957ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17f1583b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 0x556fdb958030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17f1583b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 0x556fdb95c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17f1583b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556fdb8760a0 con 0x556fdb7d64a0 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17f1583b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556fdb81b9a0 con 0x556fdb850d00 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17f1583b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556fdb872060 con 0x556fdb95c1e0 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17e0909700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17cc0b2880 0x7f17cc0b4d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f17d0005950 tx=0x7f17d00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 0x556fdb958030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 0x556fdb958030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32844/0 (socket says 172.21.15.46:32844) 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17dbfff700 1 -- 172.21.15.46:0/3770179677 learned_addr learned my addr 172.21.15.46:0/3770179677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17e110a700 1 --2- 172.21.15.46:0/3770179677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 0x556fdb95c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17e0909700 1 --2- 172.21.15.46:0/3770179677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb850d00 0x556fdb957ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17dbfff700 1 -- 172.21.15.46:0/3770179677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 msgr2=0x556fdb95c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17dbfff700 1 --2- 172.21.15.46:0/3770179677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 0x556fdb95c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17dbfff700 1 -- 172.21.15.46:0/3770179677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb850d00 msgr2=0x556fdb957ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17dbfff700 1 --2- 172.21.15.46:0/3770179677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb850d00 0x556fdb957ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17dbfff700 1 -- 172.21.15.46:0/3770179677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fdb80c500 con 0x556fdb7d64a0 2024-05-09T00:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.096+0000 7f17dbfff700 1 --2- 172.21.15.46:0/3770179677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 0x556fdb958030 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009bf0 tx=0x7f17c8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18eb1072fd5df574 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.097+0000 7f17c6ffd700 1 -- 172.21.15.46:0/3770179677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8016070 con 0x556fdb7d64a0 2024-05-09T00:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.097+0000 7f17c6ffd700 1 -- 172.21.15.46:0/3770179677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c8004830 con 0x556fdb7d64a0 2024-05-09T00:15:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.097+0000 7f17c6ffd700 1 -- 172.21.15.46:0/3770179677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8005560 con 0x556fdb7d64a0 2024-05-09T00:15:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.098+0000 7f17f1583b80 1 -- 172.21.15.46:0/3770179677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 msgr2=0x556fdb958030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.098+0000 7f17f1583b80 1 --2- 172.21.15.46:0/3770179677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 0x556fdb958030 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009bf0 tx=0x7f17c8009f00 comp rx=0 tx=0).stop 2024-05-09T00:15:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.098+0000 7f17f1583b80 1 -- 172.21.15.46:0/3770179677 shutdown_connections 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.098+0000 7f17f1583b80 1 --2- 172.21.15.46:0/3770179677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 0x556fdb95c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.098+0000 7f17f1583b80 1 --2- 172.21.15.46:0/3770179677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb850d00 0x556fdb957ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.098+0000 7f17f1583b80 1 --2- 172.21.15.46:0/3770179677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 0x556fdb958030 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.098+0000 7f17f1583b80 1 -- 172.21.15.46:0/3770179677 >> 172.21.15.46:0/3770179677 conn(0x556fdb7d9f70 msgr2=0x556fdb94e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.098+0000 7f17f1583b80 1 -- 172.21.15.46:0/3770179677 shutdown_connections 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.098+0000 7f17f1583b80 1 -- 172.21.15.46:0/3770179677 wait complete. 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.098+0000 7f17f1583b80 1 -- start start 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.098+0000 7f17f1583b80 1 Processor -- start 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17f1583b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb7d64a0 0x556fdb9613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17f1583b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb850d00 0x556fdb961900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17f1583b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 0x556fdb961e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17f1583b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556fdb95cf00 con 0x556fdb850d00 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17f1583b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556fdb810b20 con 0x556fdb7d64a0 2024-05-09T00:15:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17f1583b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556fdb964470 con 0x556fdb95c1e0 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17f1583b80 10 client.0 _ll_register_callbacks cb 0x556fdb6e76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb850d00 0x556fdb961900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb850d00 0x556fdb961900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32860/0 (socket says 172.21.15.46:32860) 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17dbfff700 1 -- 172.21.15.46:0/3184860315 learned_addr learned my addr 172.21.15.46:0/3184860315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17e0909700 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb7d64a0 0x556fdb9613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17e110a700 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 0x556fdb961e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17dbfff700 1 -- 172.21.15.46:0/3184860315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 msgr2=0x556fdb961e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17dbfff700 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 0x556fdb961e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17dbfff700 1 -- 172.21.15.46:0/3184860315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb7d64a0 msgr2=0x556fdb9613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17dbfff700 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb7d64a0 0x556fdb9613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17dbfff700 1 -- 172.21.15.46:0/3184860315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fdb6f1490 con 0x556fdb850d00 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17dbfff700 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb850d00 0x556fdb961900 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f17c8005100 tx=0x7f17c8004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd0f66e2c833eaf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.099+0000 7f17bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17bbfff700 1 -- 172.21.15.46:0/3184860315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8028060 con 0x556fdb850d00 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17bbfff700 1 -- 172.21.15.46:0/3184860315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f17c800e8a0 con 0x556fdb850d00 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17f1583b80 1 -- 172.21.15.46:0/3184860315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c8004830 con 0x556fdb850d00 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17bbfff700 1 -- 172.21.15.46:0/3184860315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17c8016040 con 0x556fdb850d00 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17f1583b80 10 client.15864 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17f1583b80 1 -- 172.21.15.46:0/3184860315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556fdb96ed90 con 0x556fdb850d00 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17bbfff700 1 -- 172.21.15.46:0/3184860315 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c8004830 con 0x556fdb850d00 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17bbfff700 1 -- 172.21.15.46:0/3184860315 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f17c802d070 con 0x556fdb850d00 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17f1583b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17f1583b80 10 client.15864 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17f1583b80 20 client.15864 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17f1583b80 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556fdb9712c0 0x556fdb9736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17f1583b80 4 client.15864 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.100+0000 7f17f1583b80 1 -- 172.21.15.46:0/3184860315 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fdb883a80 con 0x556fdb9712c0 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.101+0000 7f17e0909700 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556fdb9712c0 0x556fdb9736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.101+0000 7f17e0909700 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556fdb9712c0 0x556fdb9736e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef599cdb0b2f2ff4 server_cookie=21a893effe6f3933 in_seq=0 out_seq=0 2024-05-09T00:15:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.101+0000 7f17bbfff700 10 client.15864 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.102+0000 7f17bbfff700 1 -- 172.21.15.46:0/3184860315 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556fdb883a80 con 0x556fdb9712c0 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.102+0000 7f17bbfff700 10 client.15864 handle_command_reply: tid=0 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.102+0000 7f17f1583b80 1 client.15864 shutdown 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.102+0000 7f17f1583b80 1 -- 172.21.15.46:0/3184860315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb850d00 msgr2=0x556fdb961900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.102+0000 7f17f1583b80 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb850d00 0x556fdb961900 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f17c8005100 tx=0x7f17c8004c10 comp rx=0 tx=0).stop 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.102+0000 7f17f1583b80 1 -- 172.21.15.46:0/3184860315 shutdown_connections 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.102+0000 7f17f1583b80 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556fdb9712c0 0x556fdb9736e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.102+0000 7f17f1583b80 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 0x556fdb961e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.102+0000 7f17f1583b80 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb7d64a0 0x556fdb9613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.102+0000 7f17f1583b80 1 --2- 172.21.15.46:0/3184860315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb850d00 0x556fdb961900 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.102+0000 7f17f1583b80 1 -- 172.21.15.46:0/3184860315 >> 172.21.15.46:0/3184860315 conn(0x556fdb7d9f70 msgr2=0x556fdb947ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.102+0000 7f17bbfff700 0 client.15864 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:15:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.103+0000 7f17f1583b80 1 -- 172.21.15.46:0/3184860315 shutdown_connections 2024-05-09T00:15:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.103+0000 7f17f1583b80 1 -- 172.21.15.46:0/3184860315 wait complete. 2024-05-09T00:15:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.103+0000 7f17f1583b80 20 client.15864 trim_cache size 0 max 16384 2024-05-09T00:15:23.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:22 smithi169 ceph-mon[34777]: pgmap v496: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.118+0000 7f17f1583b80 1 Processor -- start 2024-05-09T00:15:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.118+0000 7f17f1583b80 1 -- start start 2024-05-09T00:15:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.118+0000 7f17f1583b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 0x556fdb96d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.118+0000 7f17f1583b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb850d00 0x556fdb961950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.118+0000 7f17f1583b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 0x556fdb95b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.118+0000 7f17f1583b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1af2a0 con 0x556fdb7d64a0 2024-05-09T00:15:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.118+0000 7f17f1583b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1acb30 con 0x556fdb850d00 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.118+0000 7f17f1583b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1aa960 con 0x556fdb95c1e0 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.118+0000 7f17e0909700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 0x556fdb96d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.118+0000 7f17e0909700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 0x556fdb96d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32864/0 (socket says 172.21.15.46:32864) 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.118+0000 7f17e0909700 1 -- 172.21.15.46:0/1896819707 learned_addr learned my addr 172.21.15.46:0/1896819707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.118+0000 7f17dbfff700 1 --2- 172.21.15.46:0/1896819707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb850d00 0x556fdb961950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.119+0000 7f17e110a700 1 --2- 172.21.15.46:0/1896819707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 0x556fdb95b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.119+0000 7f17e0909700 1 -- 172.21.15.46:0/1896819707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 msgr2=0x556fdb95b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.119+0000 7f17e0909700 1 --2- 172.21.15.46:0/1896819707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 0x556fdb95b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.119+0000 7f17e0909700 1 -- 172.21.15.46:0/1896819707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb850d00 msgr2=0x556fdb961950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.119+0000 7f17e0909700 1 --2- 172.21.15.46:0/1896819707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb850d00 0x556fdb961950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.119+0000 7f17e0909700 1 -- 172.21.15.46:0/1896819707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fdb94a9d0 con 0x556fdb7d64a0 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.119+0000 7f17dbfff700 1 --2- 172.21.15.46:0/1896819707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb850d00 0x556fdb961950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.119+0000 7f17e0909700 1 --2- 172.21.15.46:0/1896819707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 0x556fdb96d5c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f17d000b570 tx=0x7f17d0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d452801997758ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.119+0000 7f17c67fc700 1 -- 172.21.15.46:0/1896819707 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0021ca0 con 0x556fdb7d64a0 2024-05-09T00:15:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.119+0000 7f17c67fc700 1 -- 172.21.15.46:0/1896819707 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d0031070 con 0x556fdb7d64a0 2024-05-09T00:15:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.119+0000 7f17c67fc700 1 -- 172.21.15.46:0/1896819707 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d00116a0 con 0x556fdb7d64a0 2024-05-09T00:15:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.120+0000 7f17f1583b80 1 -- 172.21.15.46:0/1896819707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 msgr2=0x556fdb96d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.120+0000 7f17f1583b80 1 --2- 172.21.15.46:0/1896819707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 0x556fdb96d5c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f17d000b570 tx=0x7f17d0012000 comp rx=0 tx=0).stop 2024-05-09T00:15:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.120+0000 7f17f1583b80 1 -- 172.21.15.46:0/1896819707 shutdown_connections 2024-05-09T00:15:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.120+0000 7f17f1583b80 1 --2- 172.21.15.46:0/1896819707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb95c1e0 0x556fdb95b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.120+0000 7f17f1583b80 1 --2- 172.21.15.46:0/1896819707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb7d64a0 0x556fdb96d5c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.120+0000 7f17f1583b80 1 --2- 172.21.15.46:0/1896819707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb850d00 0x556fdb961950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.120+0000 7f17f1583b80 1 -- 172.21.15.46:0/1896819707 >> 172.21.15.46:0/1896819707 conn(0x556fdb7d9f70 msgr2=0x556fdb94dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.120+0000 7f17f1583b80 1 -- 172.21.15.46:0/1896819707 shutdown_connections 2024-05-09T00:15:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.120+0000 7f17f1583b80 1 -- 172.21.15.46:0/1896819707 wait complete. 2024-05-09T00:15:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.120+0000 7f17f1583b80 1 -- start start 2024-05-09T00:15:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.120+0000 7f17f1583b80 1 Processor -- start 2024-05-09T00:15:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17f1583b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb7d64a0 0x556fdb9765e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17f1583b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb850d00 0x556fdb959000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17f1583b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb95c1e0 0x556fdb959540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17f1583b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1aaff0 con 0x556fdb95c1e0 2024-05-09T00:15:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17f1583b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1ac460 con 0x556fdb7d64a0 2024-05-09T00:15:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17f1583b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc1ab6c0 con 0x556fdb850d00 2024-05-09T00:15:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17f1583b80 10 client.0 _ll_register_callbacks cb 0x556fdb6e76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17e110a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb95c1e0 0x556fdb959540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17e110a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb95c1e0 0x556fdb959540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32878/0 (socket says 172.21.15.46:32878) 2024-05-09T00:15:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17e110a700 1 -- 172.21.15.46:0/162343134 learned_addr learned my addr 172.21.15.46:0/162343134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17dbfff700 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb850d00 0x556fdb959000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17e0909700 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb7d64a0 0x556fdb9765e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17e110a700 1 -- 172.21.15.46:0/162343134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb850d00 msgr2=0x556fdb959000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17e110a700 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb850d00 0x556fdb959000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17e110a700 1 -- 172.21.15.46:0/162343134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb7d64a0 msgr2=0x556fdb9765e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17e110a700 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb7d64a0 0x556fdb9765e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17e110a700 1 -- 172.21.15.46:0/162343134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fdb959a80 con 0x556fdb95c1e0 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17e110a700 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb95c1e0 0x556fdb959540 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f17d40a7fd0 tx=0x7f17d400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18df225d5f4e8d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.121+0000 7f17bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17bbfff700 1 -- 172.21.15.46:0/162343134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d40c2030 con 0x556fdb95c1e0 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17f1583b80 1 -- 172.21.15.46:0/162343134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d0031070 con 0x556fdb95c1e0 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17bbfff700 1 -- 172.21.15.46:0/162343134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f17d40a7350 con 0x556fdb95c1e0 2024-05-09T00:15:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17bbfff700 1 -- 172.21.15.46:0/162343134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17d40ab460 con 0x556fdb95c1e0 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17f1583b80 10 client.15870 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17f1583b80 1 -- 172.21.15.46:0/162343134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556fdb95ad40 con 0x556fdb95c1e0 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17bbfff700 1 -- 172.21.15.46:0/162343134 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d0031070 con 0x556fdb95c1e0 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17bbfff700 1 -- 172.21.15.46:0/162343134 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f17d4011070 con 0x556fdb95c1e0 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17f1583b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17f1583b80 10 client.15870 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17f1583b80 20 client.15870 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17f1583b80 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556fdb9dad10 0x556fdb9dd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17f1583b80 4 client.15870 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.122+0000 7f17f1583b80 1 -- 172.21.15.46:0/162343134 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556fdb87db20 con 0x556fdb9dad10 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.123+0000 7f17e0909700 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556fdb9dad10 0x556fdb9dd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.123+0000 7f17bbfff700 10 client.15870 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.123+0000 7f17e0909700 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556fdb9dad10 0x556fdb9dd130 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef1b829031c87d5 server_cookie=86010eb0c6d4e584 in_seq=0 out_seq=0 2024-05-09T00:15:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.123+0000 7f17bbfff700 1 -- 172.21.15.46:0/162343134 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556fdb87db20 con 0x556fdb9dad10 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.123+0000 7f17bbfff700 10 client.15870 handle_command_reply: tid=0 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.123+0000 7f17f1583b80 1 client.15870 shutdown 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.123+0000 7f17f1583b80 1 -- 172.21.15.46:0/162343134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb95c1e0 msgr2=0x556fdb959540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.123+0000 7f17f1583b80 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb95c1e0 0x556fdb959540 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f17d40a7fd0 tx=0x7f17d400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.123+0000 7f17f1583b80 1 -- 172.21.15.46:0/162343134 shutdown_connections 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.123+0000 7f17f1583b80 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556fdb9dad10 0x556fdb9dd130 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.123+0000 7f17f1583b80 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fdb95c1e0 0x556fdb959540 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17f1583b80 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fdb7d64a0 0x556fdb9765e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17bbfff700 0 client.15870 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17f1583b80 1 --2- 172.21.15.46:0/162343134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556fdb850d00 0x556fdb959000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17f1583b80 1 -- 172.21.15.46:0/162343134 >> 172.21.15.46:0/162343134 conn(0x556fdb7d9f70 msgr2=0x556fdb95b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17f1583b80 1 -- 172.21.15.46:0/162343134 shutdown_connections 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17f1583b80 1 -- 172.21.15.46:0/162343134 wait complete. 2024-05-09T00:15:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17f1583b80 20 client.15870 trim_cache size 0 max 16384 2024-05-09T00:15:23.129 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:15:23.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17e2b6d700 1 -- 172.21.15.46:0/1108734599 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17cc0b2880 msgr2=0x7f17cc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17e2b6d700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17cc0b2880 0x7f17cc0b4d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f17d0005950 tx=0x7f17d00058e0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17e2b6d700 1 -- 172.21.15.46:0/1108734599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17dc108740 msgr2=0x7f17dc19c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17e2b6d700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17dc108740 0x7f17dc19c6b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f17d40095c0 tx=0x7f17d4007ad0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17e2b6d700 1 -- 172.21.15.46:0/1108734599 shutdown_connections 2024-05-09T00:15:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17e2b6d700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f17cc0b2880 0x7f17cc0b4d90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17e2b6d700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17dc108740 0x7f17dc19c6b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.124+0000 7f17e2b6d700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17dc1034c0 0x7f17dc19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.125+0000 7f17e2b6d700 1 --2- 172.21.15.46:0/1108734599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17dc107df0 0x7f17dc19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.125+0000 7f17e2b6d700 1 -- 172.21.15.46:0/1108734599 >> 172.21.15.46:0/1108734599 conn(0x7f17dc0fedf0 msgr2=0x7f17dc107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:23.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.125+0000 7f17e2b6d700 1 -- 172.21.15.46:0/1108734599 shutdown_connections 2024-05-09T00:15:23.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:23.125+0000 7f17e2b6d700 1 -- 172.21.15.46:0/1108734599 wait complete. 2024-05-09T00:15:23.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:22 smithi142 ceph-mon[34533]: pgmap v496: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:23.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:15:24.113 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:25.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:24 smithi046 ceph-mon[30151]: pgmap v497: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:25.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:24 smithi169 ceph-mon[34777]: pgmap v497: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:25.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:24 smithi142 ceph-mon[34533]: pgmap v497: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.983+0000 7fd99a4bb700 1 -- 172.21.15.46:0/3983631289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd994108740 msgr2=0x7fd9941100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.983+0000 7fd99a4bb700 1 --2- 172.21.15.46:0/3983631289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd994108740 0x7fd9941100a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00b0d0 tx=0x7fd98c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.983+0000 7fd99a4bb700 1 -- 172.21.15.46:0/3983631289 shutdown_connections 2024-05-09T00:15:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.983+0000 7fd99a4bb700 1 --2- 172.21.15.46:0/3983631289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd994108740 0x7fd9941100a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.983+0000 7fd99a4bb700 1 --2- 172.21.15.46:0/3983631289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9941034c0 0x7fd9941038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.983+0000 7fd99a4bb700 1 --2- 172.21.15.46:0/3983631289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd994107df0 0x7fd994108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.983+0000 7fd99a4bb700 1 -- 172.21.15.46:0/3983631289 >> 172.21.15.46:0/3983631289 conn(0x7fd9940fedf0 msgr2=0x7fd994101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.983+0000 7fd99a4bb700 1 -- 172.21.15.46:0/3983631289 shutdown_connections 2024-05-09T00:15:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.983+0000 7fd99a4bb700 1 -- 172.21.15.46:0/3983631289 wait complete. 2024-05-09T00:15:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.984+0000 7fd99a4bb700 1 Processor -- start 2024-05-09T00:15:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.984+0000 7fd99a4bb700 1 -- start start 2024-05-09T00:15:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd99a4bb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9941034c0 0x7fd99419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd99a4bb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd994107df0 0x7fd99419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd99a4bb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd994108740 0x7fd99419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd99a4bb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd994113390 con 0x7fd994107df0 2024-05-09T00:15:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd99a4bb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd994113210 con 0x7fd994108740 2024-05-09T00:15:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd99a4bb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd994113510 con 0x7fd9941034c0 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd9937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd994107df0 0x7fd99419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd9937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd994107df0 0x7fd99419b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32890/0 (socket says 172.21.15.46:32890) 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd9937fe700 1 -- 172.21.15.46:0/161076961 learned_addr learned my addr 172.21.15.46:0/161076961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd993fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9941034c0 0x7fd99419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd998a58700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd994108740 0x7fd99419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd9937fe700 1 -- 172.21.15.46:0/161076961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9941034c0 msgr2=0x7fd99419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd9937fe700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9941034c0 0x7fd99419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd9937fe700 1 -- 172.21.15.46:0/161076961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd994108740 msgr2=0x7fd99419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd9937fe700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd994108740 0x7fd99419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd9937fe700 1 -- 172.21.15.46:0/161076961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99419fc70 con 0x7fd994107df0 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.985+0000 7fd9937fe700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd994107df0 0x7fd99419b3f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd98800c9e0 tx=0x7fd98800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.986+0000 7fd9917fa700 1 -- 172.21.15.46:0/161076961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988007aa0 con 0x7fd994107df0 2024-05-09T00:15:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.986+0000 7fd9917fa700 1 -- 172.21.15.46:0/161076961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd988007c40 con 0x7fd994107df0 2024-05-09T00:15:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.986+0000 7fd99a4bb700 1 -- 172.21.15.46:0/161076961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99419ff60 con 0x7fd994107df0 2024-05-09T00:15:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.986+0000 7fd9917fa700 1 -- 172.21.15.46:0/161076961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98800fe60 con 0x7fd994107df0 2024-05-09T00:15:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.986+0000 7fd99a4bb700 1 -- 172.21.15.46:0/161076961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9941a04a0 con 0x7fd994107df0 2024-05-09T00:15:25.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.988+0000 7fd9917fa700 1 -- 172.21.15.46:0/161076961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd988004870 con 0x7fd994107df0 2024-05-09T00:15:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.989+0000 7fd9917fa700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd97c0b2880 0x7fd97c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.989+0000 7fd9917fa700 1 -- 172.21.15.46:0/161076961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd9880e20b0 con 0x7fd994107df0 2024-05-09T00:15:25.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.989+0000 7fd993fff700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd97c0b2880 0x7fd97c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd993fff700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd97c0b2880 0x7fd97c0b4d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd984004630 tx=0x7fd98400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd9a8ed1b80 1 Processor -- start 2024-05-09T00:15:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd9a8ed1b80 1 -- start start 2024-05-09T00:15:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd9a8ed1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa92005ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd9a8ed1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa91e844a0 0x55aa92006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd9a8ed1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa9200a1b0 0x55aa9200a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:25.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd9a8ed1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55aa91f240a0 con 0x55aa91e844a0 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd9a8ed1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55aa91ec99a0 con 0x55aa91efed00 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd9a8ed1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55aa91f20060 con 0x55aa9200a1b0 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd9937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa91e844a0 0x55aa92006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd9937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa91e844a0 0x55aa92006050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32892/0 (socket says 172.21.15.46:32892) 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd9937fe700 1 -- 172.21.15.46:0/1559764541 learned_addr learned my addr 172.21.15.46:0/1559764541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.990+0000 7fd993fff700 1 --2- 172.21.15.46:0/1559764541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa92005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.991+0000 7fd998a58700 1 --2- 172.21.15.46:0/1559764541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa9200a1b0 0x55aa9200a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.991+0000 7fd9937fe700 1 -- 172.21.15.46:0/1559764541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa9200a1b0 msgr2=0x55aa9200a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.991+0000 7fd9937fe700 1 --2- 172.21.15.46:0/1559764541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa9200a1b0 0x55aa9200a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.991+0000 7fd9937fe700 1 -- 172.21.15.46:0/1559764541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 msgr2=0x55aa92005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.991+0000 7fd9937fe700 1 --2- 172.21.15.46:0/1559764541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa92005ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.991+0000 7fd9937fe700 1 -- 172.21.15.46:0/1559764541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa91eba500 con 0x55aa91e844a0 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.991+0000 7fd9937fe700 1 --2- 172.21.15.46:0/1559764541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa91e844a0 0x55aa92006050 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd9880a6940 tx=0x7fd9880a90f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c969f4dd736502 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:25.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.991+0000 7fd97a7fc700 1 -- 172.21.15.46:0/1559764541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988007df0 con 0x55aa91e844a0 2024-05-09T00:15:25.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.991+0000 7fd97a7fc700 1 -- 172.21.15.46:0/1559764541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9880a9dd0 con 0x55aa91e844a0 2024-05-09T00:15:25.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.991+0000 7fd97a7fc700 1 -- 172.21.15.46:0/1559764541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9880dfcc0 con 0x55aa91e844a0 2024-05-09T00:15:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.992+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1559764541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa91e844a0 msgr2=0x55aa92006050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.992+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/1559764541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa91e844a0 0x55aa92006050 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd9880a6940 tx=0x7fd9880a90f0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.992+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1559764541 shutdown_connections 2024-05-09T00:15:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.992+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/1559764541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa9200a1b0 0x55aa9200a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.992+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/1559764541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa92005ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.992+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/1559764541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa91e844a0 0x55aa92006050 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.992+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1559764541 >> 172.21.15.46:0/1559764541 conn(0x55aa91e87f70 msgr2=0x55aa91ffc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.992+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1559764541 shutdown_connections 2024-05-09T00:15:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.992+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1559764541 wait complete. 2024-05-09T00:15:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.992+0000 7fd9a8ed1b80 1 -- start start 2024-05-09T00:15:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd9a8ed1b80 1 Processor -- start 2024-05-09T00:15:25.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd9a8ed1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91e844a0 0x55aa9200f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd9a8ed1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa91efed00 0x55aa9200fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd9a8ed1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 0x55aa9200ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd9a8ed1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55aa9200aed0 con 0x55aa9200a1b0 2024-05-09T00:15:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd9a8ed1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55aa91ebeb20 con 0x55aa91e844a0 2024-05-09T00:15:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd9a8ed1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55aa92012460 con 0x55aa91efed00 2024-05-09T00:15:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd998a58700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 0x55aa9200ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd9a8ed1b80 10 client.0 _ll_register_callbacks cb 0x55aa91d956c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd998a58700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 0x55aa9200ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32894/0 (socket says 172.21.15.46:32894) 2024-05-09T00:15:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd998a58700 1 -- 172.21.15.46:0/2283395373 learned_addr learned my addr 172.21.15.46:0/2283395373 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd993fff700 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91e844a0 0x55aa9200f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.993+0000 7fd9937fe700 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa91efed00 0x55aa9200fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd998a58700 1 -- 172.21.15.46:0/2283395373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa91efed00 msgr2=0x55aa9200fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd998a58700 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa91efed00 0x55aa9200fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd998a58700 1 -- 172.21.15.46:0/2283395373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91e844a0 msgr2=0x55aa9200f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd998a58700 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91e844a0 0x55aa9200f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd998a58700 1 -- 172.21.15.46:0/2283395373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa91d9f490 con 0x55aa9200a1b0 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd998a58700 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 0x55aa9200ffd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd98c003c30 tx=0x7fd98c003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca97c438394d510 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd9737fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd9737fe700 1 -- 172.21.15.46:0/2283395373 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0079d0 con 0x55aa9200a1b0 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd9737fe700 1 -- 172.21.15.46:0/2283395373 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd98c007b70 con 0x55aa9200a1b0 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/2283395373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9880a9dd0 con 0x55aa9200a1b0 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd9737fe700 1 -- 172.21.15.46:0/2283395373 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd98c007d70 con 0x55aa9200a1b0 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd9a8ed1b80 10 client.15882 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/2283395373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa9201ce90 con 0x55aa9200a1b0 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.994+0000 7fd9737fe700 1 -- 172.21.15.46:0/2283395373 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9880a9dd0 con 0x55aa9200a1b0 2024-05-09T00:15:25.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.995+0000 7fd9737fe700 1 -- 172.21.15.46:0/2283395373 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd98c011070 con 0x55aa9200a1b0 2024-05-09T00:15:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.995+0000 7fd9a8ed1b80 10 client.15882 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.995+0000 7fd9a8ed1b80 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.995+0000 7fd9a8ed1b80 20 client.15882 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.995+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aa9201f4d0 0x55aa920218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.995+0000 7fd9a8ed1b80 4 client.15882 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:15:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.995+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/2283395373 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa91f31a80 con 0x55aa9201f4d0 2024-05-09T00:15:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.995+0000 7fd993fff700 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aa9201f4d0 0x55aa920218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.995+0000 7fd9737fe700 10 client.15882 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:25.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.995+0000 7fd993fff700 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aa9201f4d0 0x55aa920218f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf1dbd9af6a1f108 server_cookie=a263b649771d0182 in_seq=0 out_seq=0 2024-05-09T00:15:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.996+0000 7fd9737fe700 1 -- 172.21.15.46:0/2283395373 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa91f31a80 con 0x55aa9201f4d0 2024-05-09T00:15:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.996+0000 7fd9737fe700 10 client.15882 handle_command_reply: tid=0 2024-05-09T00:15:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.996+0000 7fd9a8ed1b80 1 client.15882 shutdown 2024-05-09T00:15:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.997+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/2283395373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 msgr2=0x55aa9200ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.997+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 0x55aa9200ffd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd98c003c30 tx=0x7fd98c003c60 comp rx=0 tx=0).stop 2024-05-09T00:15:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.997+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/2283395373 shutdown_connections 2024-05-09T00:15:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.997+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aa9201f4d0 0x55aa920218f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.997+0000 7fd9737fe700 0 client.15882 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.997+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 0x55aa9200ffd0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.997+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91e844a0 0x55aa9200f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.997+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/2283395373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa91efed00 0x55aa9200fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.997+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/2283395373 >> 172.21.15.46:0/2283395373 conn(0x55aa91e87f70 msgr2=0x55aa920003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.997+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/2283395373 shutdown_connections 2024-05-09T00:15:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.997+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/2283395373 wait complete. 2024-05-09T00:15:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:25.997+0000 7fd9a8ed1b80 20 client.15882 trim_cache size 0 max 16384 2024-05-09T00:15:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.012+0000 7fd9a8ed1b80 1 Processor -- start 2024-05-09T00:15:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.012+0000 7fd9a8ed1b80 1 -- start start 2024-05-09T00:15:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.012+0000 7fd9a8ed1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa91e844a0 0x55aa91ff6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.012+0000 7fd9a8ed1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa92009510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.012+0000 7fd9a8ed1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 0x55aa920270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd9a8ed1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941af250 con 0x55aa9200a1b0 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd9a8ed1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941acae0 con 0x55aa91efed00 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd9a8ed1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941aa930 con 0x55aa91e844a0 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd998a58700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 0x55aa920270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd998a58700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 0x55aa920270b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32900/0 (socket says 172.21.15.46:32900) 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd998a58700 1 -- 172.21.15.46:0/6662428 learned_addr learned my addr 172.21.15.46:0/6662428 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd9937fe700 1 --2- 172.21.15.46:0/6662428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa92009510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd993fff700 1 --2- 172.21.15.46:0/6662428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa91e844a0 0x55aa91ff6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd998a58700 1 -- 172.21.15.46:0/6662428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa91e844a0 msgr2=0x55aa91ff6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd998a58700 1 --2- 172.21.15.46:0/6662428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa91e844a0 0x55aa91ff6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd998a58700 1 -- 172.21.15.46:0/6662428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 msgr2=0x55aa92009510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd998a58700 1 --2- 172.21.15.46:0/6662428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa92009510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd998a58700 1 -- 172.21.15.46:0/6662428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa92011da0 con 0x55aa9200a1b0 2024-05-09T00:15:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd998a58700 1 --2- 172.21.15.46:0/6662428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 0x55aa920270b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009a20 tx=0x7fd98c009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9819993990d11eae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd979ffb700 1 -- 172.21.15.46:0/6662428 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0077d0 con 0x55aa9200a1b0 2024-05-09T00:15:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd979ffb700 1 -- 172.21.15.46:0/6662428 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd98c021070 con 0x55aa9200a1b0 2024-05-09T00:15:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.013+0000 7fd979ffb700 1 -- 172.21.15.46:0/6662428 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c026aa0 con 0x55aa9200a1b0 2024-05-09T00:15:26.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.014+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/6662428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 msgr2=0x55aa920270b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:26.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.014+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/6662428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 0x55aa920270b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009a20 tx=0x7fd98c009d00 comp rx=0 tx=0).stop 2024-05-09T00:15:26.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.014+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/6662428 shutdown_connections 2024-05-09T00:15:26.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.014+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/6662428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa9200a1b0 0x55aa920270b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.014+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/6662428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa91e844a0 0x55aa91ff6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.014+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/6662428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa92009510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.014+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/6662428 >> 172.21.15.46:0/6662428 conn(0x55aa91e87f70 msgr2=0x55aa9200fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.014+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/6662428 shutdown_connections 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.014+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/6662428 wait complete. 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.015+0000 7fd9a8ed1b80 1 -- start start 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.015+0000 7fd9a8ed1b80 1 Processor -- start 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.015+0000 7fd9a8ed1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa91e844a0 0x55aa920822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.015+0000 7fd9a8ed1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa920827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.015+0000 7fd9a8ed1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa9200a1b0 0x55aa92016ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.015+0000 7fd9a8ed1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941aafa0 con 0x55aa91e844a0 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.015+0000 7fd9a8ed1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941ac410 con 0x55aa91efed00 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.015+0000 7fd9a8ed1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941ab670 con 0x55aa9200a1b0 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.015+0000 7fd9a8ed1b80 10 client.0 _ll_register_callbacks cb 0x55aa91d956c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:26.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.015+0000 7fd9937fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa920827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.015+0000 7fd9937fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa920827e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:60102/0 (socket says 172.21.15.46:60102) 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.015+0000 7fd9937fe700 1 -- 172.21.15.46:0/1072642301 learned_addr learned my addr 172.21.15.46:0/1072642301 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd998a58700 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa9200a1b0 0x55aa92016ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd9937fe700 1 -- 172.21.15.46:0/1072642301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa9200a1b0 msgr2=0x55aa92016ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd9937fe700 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa9200a1b0 0x55aa92016ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd9937fe700 1 -- 172.21.15.46:0/1072642301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa91e844a0 msgr2=0x55aa920822a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd9937fe700 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa91e844a0 0x55aa920822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd9937fe700 1 -- 172.21.15.46:0/1072642301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa92017350 con 0x55aa91efed00 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd998a58700 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa9200a1b0 0x55aa92016ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd9937fe700 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa920827e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd9880a93d0 tx=0x7fd988005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e1a9d8d9289df7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd9737fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd9737fe700 1 -- 172.21.15.46:0/1072642301 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9880c9070 con 0x55aa91efed00 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1072642301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd98c021070 con 0x55aa91efed00 2024-05-09T00:15:26.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd9737fe700 1 -- 172.21.15.46:0/1072642301 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd9880a5cc0 con 0x55aa91efed00 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.016+0000 7fd9737fe700 1 -- 172.21.15.46:0/1072642301 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd98806d040 con 0x55aa91efed00 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.017+0000 7fd9a8ed1b80 10 client.25511 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.017+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1072642301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa92089970 con 0x55aa91efed00 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.017+0000 7fd9737fe700 1 -- 172.21.15.46:0/1072642301 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd98c021070 con 0x55aa91efed00 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.017+0000 7fd9737fe700 1 -- 172.21.15.46:0/1072642301 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd9880b20a0 con 0x55aa91efed00 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.017+0000 7fd9a8ed1b80 10 client.25511 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.017+0000 7fd9a8ed1b80 10 client.25511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.017+0000 7fd9a8ed1b80 20 client.25511 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.017+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aa92089ba0 0x55aa9208bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.017+0000 7fd9a8ed1b80 4 client.25511 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.017+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1072642301 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa91f2bb20 con 0x55aa92089ba0 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.017+0000 7fd993fff700 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aa92089ba0 0x55aa9208bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:26.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd993fff700 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aa92089ba0 0x55aa9208bfc0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88670fa0aacbfd5a server_cookie=11be84419ec3272 in_seq=0 out_seq=0 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9737fe700 10 client.25511 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9737fe700 1 -- 172.21.15.46:0/1072642301 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa91f2bb20 con 0x55aa92089ba0 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9737fe700 10 client.25511 handle_command_reply: tid=0 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9a8ed1b80 1 client.25511 shutdown 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1072642301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 msgr2=0x55aa920827e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa920827e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd9880a93d0 tx=0x7fd988005880 comp rx=0 tx=0).stop 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1072642301 shutdown_connections 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aa92089ba0 0x55aa9208bfc0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aa9200a1b0 0x55aa92016ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa91e844a0 0x55aa920822a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9a8ed1b80 1 --2- 172.21.15.46:0/1072642301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa91efed00 0x55aa920827e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9737fe700 0 client.25511 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:26.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1072642301 >> 172.21.15.46:0/1072642301 conn(0x55aa91e87f70 msgr2=0x55aa9200fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1072642301 shutdown_connections 2024-05-09T00:15:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.018+0000 7fd9a8ed1b80 1 -- 172.21.15.46:0/1072642301 wait complete. 2024-05-09T00:15:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd9a8ed1b80 20 client.25511 trim_cache size 0 max 16384 2024-05-09T00:15:26.023 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:15:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd99a4bb700 1 -- 172.21.15.46:0/161076961 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd97c0b2880 msgr2=0x7fd97c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd99a4bb700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd97c0b2880 0x7fd97c0b4d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd984004630 tx=0x7fd98400b3c0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd99a4bb700 1 -- 172.21.15.46:0/161076961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd994107df0 msgr2=0x7fd99419b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd99a4bb700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd994107df0 0x7fd99419b3f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd98800c9e0 tx=0x7fd98800ccf0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd99a4bb700 1 -- 172.21.15.46:0/161076961 shutdown_connections 2024-05-09T00:15:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd99a4bb700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd97c0b2880 0x7fd97c0b4d90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd99a4bb700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd994108740 0x7fd99419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd99a4bb700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9941034c0 0x7fd99419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd99a4bb700 1 --2- 172.21.15.46:0/161076961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd994107df0 0x7fd99419b3f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd99a4bb700 1 -- 172.21.15.46:0/161076961 >> 172.21.15.46:0/161076961 conn(0x7fd9940fedf0 msgr2=0x7fd994075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd99a4bb700 1 -- 172.21.15.46:0/161076961 shutdown_connections 2024-05-09T00:15:26.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:26.019+0000 7fd99a4bb700 1 -- 172.21.15.46:0/161076961 wait complete. 2024-05-09T00:15:27.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:26 smithi046 ceph-mon[30151]: pgmap v498: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:15:27.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:26 smithi169 ceph-mon[34777]: pgmap v498: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:15:27.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:26 smithi142 ceph-mon[34533]: pgmap v498: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:15:27.808 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:15:27.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:15:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:29.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:28 smithi046 ceph-mon[30151]: pgmap v499: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:29.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:28 smithi169 ceph-mon[34777]: pgmap v499: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:29.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:28 smithi142 ceph-mon[34533]: pgmap v499: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.916+0000 7f46d83db700 1 -- 172.21.15.46:0/1556375064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d01034c0 msgr2=0x7f46d01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.916+0000 7f46d83db700 1 --2- 172.21.15.46:0/1556375064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d01034c0 0x7f46d01038d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0099c0 tx=0x7f46cc009cd0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.916+0000 7f46d83db700 1 -- 172.21.15.46:0/1556375064 shutdown_connections 2024-05-09T00:15:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.916+0000 7f46d83db700 1 --2- 172.21.15.46:0/1556375064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d0108740 0x7f46d01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.916+0000 7f46d83db700 1 --2- 172.21.15.46:0/1556375064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d01034c0 0x7f46d01038d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.916+0000 7f46d83db700 1 --2- 172.21.15.46:0/1556375064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d0107df0 0x7f46d0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.916+0000 7f46d83db700 1 -- 172.21.15.46:0/1556375064 >> 172.21.15.46:0/1556375064 conn(0x7f46d00fedf0 msgr2=0x7f46d0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.916+0000 7f46d83db700 1 -- 172.21.15.46:0/1556375064 shutdown_connections 2024-05-09T00:15:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.916+0000 7f46d83db700 1 -- 172.21.15.46:0/1556375064 wait complete. 2024-05-09T00:15:29.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.917+0000 7f46d83db700 1 Processor -- start 2024-05-09T00:15:29.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.917+0000 7f46d83db700 1 -- start start 2024-05-09T00:15:29.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d83db700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d01034c0 0x7f46d019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d83db700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d0107df0 0x7f46d019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d83db700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d0108740 0x7f46d019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d83db700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01144c0 con 0x7f46d0108740 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d83db700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0114340 con 0x7f46d01034c0 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d83db700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0114640 con 0x7f46d0107df0 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d6978700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d0108740 0x7f46d019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d6978700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d0108740 0x7f46d019f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35826/0 (socket says 172.21.15.46:35826) 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d6978700 1 -- 172.21.15.46:0/2207322513 learned_addr learned my addr 172.21.15.46:0/2207322513 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d5976700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d0107df0 0x7f46d019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d6177700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d01034c0 0x7f46d019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d6978700 1 -- 172.21.15.46:0/2207322513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d0107df0 msgr2=0x7f46d019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d6978700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d0107df0 0x7f46d019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d6978700 1 -- 172.21.15.46:0/2207322513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d01034c0 msgr2=0x7f46d019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d6978700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d01034c0 0x7f46d019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d6978700 1 -- 172.21.15.46:0/2207322513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d019fd20 con 0x7f46d0108740 2024-05-09T00:15:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.918+0000 7f46d6978700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d0108740 0x7f46d019f680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f46c800ada0 tx=0x7f46c800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.919+0000 7f46c77fe700 1 -- 172.21.15.46:0/2207322513 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8017020 con 0x7f46d0108740 2024-05-09T00:15:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.919+0000 7f46d83db700 1 -- 172.21.15.46:0/2207322513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d01a0010 con 0x7f46d0108740 2024-05-09T00:15:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.919+0000 7f46c77fe700 1 -- 172.21.15.46:0/2207322513 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c800cd80 con 0x7f46d0108740 2024-05-09T00:15:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.919+0000 7f46c77fe700 1 -- 172.21.15.46:0/2207322513 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8005260 con 0x7f46d0108740 2024-05-09T00:15:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.919+0000 7f46d83db700 1 -- 172.21.15.46:0/2207322513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d01a7930 con 0x7f46d0108740 2024-05-09T00:15:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.922+0000 7f46c77fe700 1 -- 172.21.15.46:0/2207322513 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f46c80054c0 con 0x7f46d0108740 2024-05-09T00:15:29.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.922+0000 7f46c77fe700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f46bc0b2880 0x7f46bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.923+0000 7f46d6177700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f46bc0b2880 0x7f46bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.923+0000 7f46c77fe700 1 -- 172.21.15.46:0/2207322513 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f46c80e18f0 con 0x7f46d0108740 2024-05-09T00:15:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.923+0000 7f46e6df1b80 1 Processor -- start 2024-05-09T00:15:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.923+0000 7f46e6df1b80 1 -- start start 2024-05-09T00:15:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.923+0000 7f46d6177700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f46bc0b2880 0x7f46bc0b4d90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009990 tx=0x7f46cc0036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.923+0000 7f46e6df1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957dded80 0x55b957ee72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.923+0000 7f46e6df1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957ee7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.923+0000 7f46e6df1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957ee1480 0x55b957ee1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.923+0000 7f46e6df1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b957e04120 con 0x55b957dded80 2024-05-09T00:15:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.923+0000 7f46e6df1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b957da9a20 con 0x55b957ee1480 2024-05-09T00:15:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.923+0000 7f46e6df1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b957e000e0 con 0x55b957d64520 2024-05-09T00:15:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46d6177700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957dded80 0x55b957ee72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46d6177700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957dded80 0x55b957ee72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35832/0 (socket says 172.21.15.46:35832) 2024-05-09T00:15:29.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46d6177700 1 -- 172.21.15.46:0/2329470035 learned_addr learned my addr 172.21.15.46:0/2329470035 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:29.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46d5976700 1 --2- 172.21.15.46:0/2329470035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957ee7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46d6978700 1 --2- 172.21.15.46:0/2329470035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957ee1480 0x55b957ee1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46d6177700 1 -- 172.21.15.46:0/2329470035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 msgr2=0x55b957ee7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46d6177700 1 --2- 172.21.15.46:0/2329470035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957ee7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46d6177700 1 -- 172.21.15.46:0/2329470035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957ee1480 msgr2=0x55b957ee1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46d6177700 1 --2- 172.21.15.46:0/2329470035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957ee1480 0x55b957ee1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46d6177700 1 -- 172.21.15.46:0/2329470035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b957d9a580 con 0x55b957dded80 2024-05-09T00:15:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46d6978700 1 --2- 172.21.15.46:0/2329470035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957ee1480 0x55b957ee1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46d6177700 1 --2- 172.21.15.46:0/2329470035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957dded80 0x55b957ee72c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0099c0 tx=0x7f46cc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35f3cc7f5a5a52d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46c4d47700 1 -- 172.21.15.46:0/2329470035 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc005430 con 0x55b957dded80 2024-05-09T00:15:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46c4d47700 1 -- 172.21.15.46:0/2329470035 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc0055d0 con 0x55b957dded80 2024-05-09T00:15:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.924+0000 7f46c4d47700 1 -- 172.21.15.46:0/2329470035 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc00fa20 con 0x55b957dded80 2024-05-09T00:15:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.925+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2329470035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957dded80 msgr2=0x55b957ee72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.925+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/2329470035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957dded80 0x55b957ee72c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0099c0 tx=0x7f46cc01e000 comp rx=0 tx=0).stop 2024-05-09T00:15:29.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.925+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2329470035 shutdown_connections 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.925+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/2329470035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957ee1480 0x55b957ee1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.925+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/2329470035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957dded80 0x55b957ee72c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.925+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/2329470035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957ee7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.925+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2329470035 >> 172.21.15.46:0/2329470035 conn(0x55b957d67ff0 msgr2=0x55b957edc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.926+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2329470035 shutdown_connections 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.926+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2329470035 wait complete. 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.926+0000 7f46e6df1b80 1 -- start start 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.926+0000 7f46e6df1b80 1 Processor -- start 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.926+0000 7f46e6df1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957d64520 0x55b957eeb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.926+0000 7f46e6df1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957dded80 0x55b957eeb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.926+0000 7f46e6df1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957ee1480 0x55b957eebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.926+0000 7f46e6df1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b957ee2120 con 0x55b957d64520 2024-05-09T00:15:29.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.926+0000 7f46e6df1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b957d9eba0 con 0x55b957ee1480 2024-05-09T00:15:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.926+0000 7f46e6df1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b957ee22a0 con 0x55b957dded80 2024-05-09T00:15:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46e6df1b80 10 client.0 _ll_register_callbacks cb 0x55b957c75740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46d6177700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957d64520 0x55b957eeb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46d6177700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957d64520 0x55b957eeb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35848/0 (socket says 172.21.15.46:35848) 2024-05-09T00:15:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46d6177700 1 -- 172.21.15.46:0/2684709273 learned_addr learned my addr 172.21.15.46:0/2684709273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46d6978700 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957ee1480 0x55b957eebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46d5976700 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957dded80 0x55b957eeb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46d6177700 1 -- 172.21.15.46:0/2684709273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957dded80 msgr2=0x55b957eeb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46d6177700 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957dded80 0x55b957eeb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46d6177700 1 -- 172.21.15.46:0/2684709273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957ee1480 msgr2=0x55b957eebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46d6177700 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957ee1480 0x55b957eebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46d6177700 1 -- 172.21.15.46:0/2684709273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b957c7f510 con 0x55b957d64520 2024-05-09T00:15:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46d6978700 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957ee1480 0x55b957eebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:29.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46d6177700 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957d64520 0x55b957eeb070 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f46cc006010 tx=0x7f46cc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4463fea8f1dd71 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46b9ffb700 1 -- 172.21.15.46:0/2684709273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc03c060 con 0x55b957d64520 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46b9ffb700 1 -- 172.21.15.46:0/2684709273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f46cc00b9a0 con 0x55b957d64520 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.927+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2684709273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46cc0055d0 con 0x55b957d64520 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.928+0000 7f46b9ffb700 1 -- 172.21.15.46:0/2684709273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46cc00bba0 con 0x55b957d64520 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.928+0000 7f46e6df1b80 10 client.15897 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.928+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2684709273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b957ef8990 con 0x55b957d64520 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.928+0000 7f46b9ffb700 1 -- 172.21.15.46:0/2684709273 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc0055d0 con 0x55b957d64520 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.928+0000 7f46b9ffb700 1 -- 172.21.15.46:0/2684709273 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f46cc041070 con 0x55b957d64520 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.928+0000 7f46e6df1b80 10 client.15897 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.928+0000 7f46e6df1b80 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.928+0000 7f46e6df1b80 20 client.15897 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:29.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.928+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b957efae60 0x55b957efd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.928+0000 7f46e6df1b80 4 client.15897 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:15:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.928+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2684709273 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b957e11b00 con 0x55b957efae60 2024-05-09T00:15:29.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.928+0000 7f46d5976700 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b957efae60 0x55b957efd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.929+0000 7f46d5976700 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b957efae60 0x55b957efd280 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282641ac2eaaac33 server_cookie=6a83eda64cc0597c in_seq=0 out_seq=0 2024-05-09T00:15:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.929+0000 7f46b9ffb700 10 client.15897 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46b9ffb700 1 -- 172.21.15.46:0/2684709273 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b957e11b00 con 0x55b957efae60 2024-05-09T00:15:29.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46b9ffb700 10 client.15897 handle_command_reply: tid=0 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46e6df1b80 1 client.15897 shutdown 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2684709273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957d64520 msgr2=0x55b957eeb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957d64520 0x55b957eeb070 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f46cc006010 tx=0x7f46cc009310 comp rx=0 tx=0).stop 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2684709273 shutdown_connections 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b957efae60 0x55b957efd280 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957ee1480 0x55b957eebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957d64520 0x55b957eeb070 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/2684709273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957dded80 0x55b957eeb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46b9ffb700 0 client.15897 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2684709273 >> 172.21.15.46:0/2684709273 conn(0x55b957d67ff0 msgr2=0x55b957ed6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2684709273 shutdown_connections 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46e6df1b80 1 -- 172.21.15.46:0/2684709273 wait complete. 2024-05-09T00:15:29.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.930+0000 7f46e6df1b80 20 client.15897 trim_cache size 0 max 16384 2024-05-09T00:15:29.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46e6df1b80 1 Processor -- start 2024-05-09T00:15:29.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46e6df1b80 1 -- start start 2024-05-09T00:15:29.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46e6df1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957ee56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46e6df1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957dded80 0x55b957ef7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46e6df1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 0x55b957eeb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46e6df1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01b1470 con 0x55b957ee1480 2024-05-09T00:15:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46e6df1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01aed00 con 0x55b957dded80 2024-05-09T00:15:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46e6df1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01acb30 con 0x55b957d64520 2024-05-09T00:15:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46d6978700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 0x55b957eeb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46d6978700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 0x55b957eeb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35856/0 (socket says 172.21.15.46:35856) 2024-05-09T00:15:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46d6978700 1 -- 172.21.15.46:0/4157359046 learned_addr learned my addr 172.21.15.46:0/4157359046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46d5976700 1 --2- 172.21.15.46:0/4157359046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957dded80 0x55b957ef7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46d6177700 1 --2- 172.21.15.46:0/4157359046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957ee56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46d6978700 1 -- 172.21.15.46:0/4157359046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 msgr2=0x55b957ee56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46d6978700 1 --2- 172.21.15.46:0/4157359046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957ee56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46d6978700 1 -- 172.21.15.46:0/4157359046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957dded80 msgr2=0x55b957ef7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46d6978700 1 --2- 172.21.15.46:0/4157359046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957dded80 0x55b957ef7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46d6978700 1 -- 172.21.15.46:0/4157359046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b957ed8a50 con 0x55b957ee1480 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.951+0000 7f46d6177700 1 --2- 172.21.15.46:0/4157359046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957ee56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.952+0000 7f46d6978700 1 --2- 172.21.15.46:0/4157359046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 0x55b957eeb930 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c680 tx=0x7f46c80a8ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919fda6b64fc867e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.952+0000 7f46bbfff700 1 -- 172.21.15.46:0/4157359046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c80a8fa0 con 0x55b957ee1480 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.952+0000 7f46bbfff700 1 -- 172.21.15.46:0/4157359046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c80ad350 con 0x55b957ee1480 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.952+0000 7f46bbfff700 1 -- 172.21.15.46:0/4157359046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c80b18e0 con 0x55b957ee1480 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 -- 172.21.15.46:0/4157359046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 msgr2=0x55b957eeb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/4157359046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 0x55b957eeb930 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c680 tx=0x7f46c80a8ab0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 -- 172.21.15.46:0/4157359046 shutdown_connections 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/4157359046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 0x55b957eeb930 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/4157359046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957ee56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/4157359046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957dded80 0x55b957ef7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 -- 172.21.15.46:0/4157359046 >> 172.21.15.46:0/4157359046 conn(0x55b957d67ff0 msgr2=0x55b957ee0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 -- 172.21.15.46:0/4157359046 shutdown_connections 2024-05-09T00:15:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 -- 172.21.15.46:0/4157359046 wait complete. 2024-05-09T00:15:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 -- start start 2024-05-09T00:15:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 Processor -- start 2024-05-09T00:15:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957f00490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957dded80 0x55b957efe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 0x55b957efedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01ad1c0 con 0x55b957ee1480 2024-05-09T00:15:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01ae630 con 0x55b957dded80 2024-05-09T00:15:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.953+0000 7f46e6df1b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01ad890 con 0x55b957d64520 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46e6df1b80 10 client.0 _ll_register_callbacks cb 0x55b957ee6d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46d6978700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 0x55b957efedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46d6978700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 0x55b957efedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35860/0 (socket says 172.21.15.46:35860) 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46d6978700 1 -- 172.21.15.46:0/3029765881 learned_addr learned my addr 172.21.15.46:0/3029765881 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46d5976700 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957dded80 0x55b957efe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46d6177700 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957f00490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46d6978700 1 -- 172.21.15.46:0/3029765881 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 msgr2=0x55b957f00490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46d6978700 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957f00490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46d6978700 1 -- 172.21.15.46:0/3029765881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957dded80 msgr2=0x55b957efe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46d6978700 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957dded80 0x55b957efe870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46d6978700 1 -- 172.21.15.46:0/3029765881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b957eff2f0 con 0x55b957ee1480 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46d5976700 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957dded80 0x55b957efe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:15:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46d6978700 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 0x55b957efedb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f46c80a8f30 tx=0x7f46c80a9430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4761f0e440959db9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46b9ffb700 1 -- 172.21.15.46:0/3029765881 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c80b1af0 con 0x55b957ee1480 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46b9ffb700 1 -- 172.21.15.46:0/3029765881 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f46c80057b0 con 0x55b957ee1480 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.954+0000 7f46e6df1b80 1 -- 172.21.15.46:0/3029765881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46c80ad350 con 0x55b957ee1480 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.955+0000 7f46b9ffb700 1 -- 172.21.15.46:0/3029765881 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46c808a050 con 0x55b957ee1480 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.955+0000 7f46e6df1b80 10 client.15903 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.955+0000 7f46e6df1b80 1 -- 172.21.15.46:0/3029765881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b957f65f50 con 0x55b957ee1480 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.955+0000 7f46b9ffb700 1 -- 172.21.15.46:0/3029765881 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c80ad350 con 0x55b957ee1480 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.955+0000 7f46b9ffb700 1 -- 172.21.15.46:0/3029765881 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f46c8012070 con 0x55b957ee1480 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.955+0000 7f46e6df1b80 10 client.15903 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.955+0000 7f46e6df1b80 10 client.15903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.955+0000 7f46e6df1b80 20 client.15903 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.955+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b957f665d0 0x55b957f689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.955+0000 7f46e6df1b80 4 client.15903 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.955+0000 7f46e6df1b80 1 -- 172.21.15.46:0/3029765881 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b957e0bba0 con 0x55b957f665d0 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.955+0000 7f46d6177700 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b957f665d0 0x55b957f689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.956+0000 7f46d6177700 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b957f665d0 0x55b957f689f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b00c53cf339ac4 server_cookie=d170f48057c7d5f8 in_seq=0 out_seq=0 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.956+0000 7f46b9ffb700 10 client.15903 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.957+0000 7f46b9ffb700 1 -- 172.21.15.46:0/3029765881 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b957e0bba0 con 0x55b957f665d0 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.957+0000 7f46b9ffb700 10 client.15903 handle_command_reply: tid=0 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.957+0000 7f46e6df1b80 1 client.15903 shutdown 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.957+0000 7f46e6df1b80 1 -- 172.21.15.46:0/3029765881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 msgr2=0x55b957efedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.957+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 0x55b957efedb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f46c80a8f30 tx=0x7f46c80a9430 comp rx=0 tx=0).stop 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46e6df1b80 1 -- 172.21.15.46:0/3029765881 shutdown_connections 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55b957f665d0 0x55b957f689f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b957ee1480 0x55b957efedb0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b957d64520 0x55b957f00490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46e6df1b80 1 --2- 172.21.15.46:0/3029765881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957dded80 0x55b957efe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46b9ffb700 0 client.15903 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46e6df1b80 1 -- 172.21.15.46:0/3029765881 >> 172.21.15.46:0/3029765881 conn(0x55b957d67ff0 msgr2=0x55b957ee9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46e6df1b80 1 -- 172.21.15.46:0/3029765881 shutdown_connections 2024-05-09T00:15:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46e6df1b80 1 -- 172.21.15.46:0/3029765881 wait complete. 2024-05-09T00:15:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46e6df1b80 20 client.15903 trim_cache size 0 max 16384 2024-05-09T00:15:29.961 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:15:29.961 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:15:29.961 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "87fc943b-36d1-4cb5-b101-33bde082770f", 2024-05-09T00:15:29.961 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:15:29.962 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:15:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46d83db700 1 -- 172.21.15.46:0/2207322513 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f46bc0b2880 msgr2=0x7f46bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46d83db700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f46bc0b2880 0x7f46bc0b4d90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009990 tx=0x7f46cc0036e0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46d83db700 1 -- 172.21.15.46:0/2207322513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d0108740 msgr2=0x7f46d019f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46d83db700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d0108740 0x7f46d019f680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f46c800ada0 tx=0x7f46c800c330 comp rx=0 tx=0).stop 2024-05-09T00:15:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46d83db700 1 -- 172.21.15.46:0/2207322513 shutdown_connections 2024-05-09T00:15:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46d83db700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f46bc0b2880 0x7f46bc0b4d90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46d83db700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d0108740 0x7f46d019f680 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46d83db700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d01034c0 0x7f46d019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46d83db700 1 --2- 172.21.15.46:0/2207322513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d0107df0 0x7f46d019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.958+0000 7f46d83db700 1 -- 172.21.15.46:0/2207322513 >> 172.21.15.46:0/2207322513 conn(0x7f46d00fedf0 msgr2=0x7f46d0101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.959+0000 7f46d83db700 1 -- 172.21.15.46:0/2207322513 shutdown_connections 2024-05-09T00:15:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:29.959+0000 7f46d83db700 1 -- 172.21.15.46:0/2207322513 wait complete. 2024-05-09T00:15:30.810 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:15:31.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:30 smithi046 ceph-mon[30151]: pgmap v500: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:31.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:30 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:15:31.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:30 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:15:31.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:30 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:15:31.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:30 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:15:31.066 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:31.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:30 smithi169 ceph-mon[34777]: pgmap v500: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:31.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:30 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:15:31.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:30 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:15:31.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:30 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:15:31.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:30 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:15:31.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:30 smithi142 ceph-mon[34533]: pgmap v500: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:31.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:30 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:15:31.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:30 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:15:31.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:30 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:15:31.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:30 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:15:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.884+0000 7fc0143fc700 1 -- 172.21.15.46:0/3422437361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00c108740 msgr2=0x7fc00c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.884+0000 7fc0143fc700 1 --2- 172.21.15.46:0/3422437361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00c108740 0x7fc00c1100a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc00400b0d0 tx=0x7fc00400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.884+0000 7fc0143fc700 1 -- 172.21.15.46:0/3422437361 shutdown_connections 2024-05-09T00:15:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.884+0000 7fc0143fc700 1 --2- 172.21.15.46:0/3422437361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00c108740 0x7fc00c1100a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.884+0000 7fc0143fc700 1 --2- 172.21.15.46:0/3422437361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc00c1034c0 0x7fc00c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.884+0000 7fc0143fc700 1 --2- 172.21.15.46:0/3422437361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc00c107df0 0x7fc00c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.884+0000 7fc0143fc700 1 -- 172.21.15.46:0/3422437361 >> 172.21.15.46:0/3422437361 conn(0x7fc00c0fedf0 msgr2=0x7fc00c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.884+0000 7fc0143fc700 1 -- 172.21.15.46:0/3422437361 shutdown_connections 2024-05-09T00:15:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.884+0000 7fc0143fc700 1 -- 172.21.15.46:0/3422437361 wait complete. 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc0143fc700 1 Processor -- start 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc0143fc700 1 -- start start 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc0143fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc00c1034c0 0x7fc00c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc0143fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00c107df0 0x7fc00c19d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc0143fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc00c108740 0x7fc00c1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc0143fc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c113370 con 0x7fc00c107df0 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc0143fc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1131f0 con 0x7fc00c1034c0 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc0143fc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1134f0 con 0x7fc00c108740 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc011997700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00c107df0 0x7fc00c19d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc011997700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00c107df0 0x7fc00c19d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35880/0 (socket says 172.21.15.46:35880) 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc011997700 1 -- 172.21.15.46:0/3305118435 learned_addr learned my addr 172.21.15.46:0/3305118435 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc012198700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc00c1034c0 0x7fc00c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc012999700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc00c108740 0x7fc00c1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc011997700 1 -- 172.21.15.46:0/3305118435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc00c108740 msgr2=0x7fc00c1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc011997700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc00c108740 0x7fc00c1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc011997700 1 -- 172.21.15.46:0/3305118435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc00c1034c0 msgr2=0x7fc00c19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc011997700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc00c1034c0 0x7fc00c19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc011997700 1 -- 172.21.15.46:0/3305118435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c1a1e80 con 0x7fc00c107df0 2024-05-09T00:15:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.885+0000 7fc012999700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc00c108740 0x7fc00c1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.886+0000 7fc011997700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00c107df0 0x7fc00c19d550 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00d980 tx=0x7fbffc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.886+0000 7fc0037fe700 1 -- 172.21.15.46:0/3305118435 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc017070 con 0x7fc00c107df0 2024-05-09T00:15:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.886+0000 7fc0037fe700 1 -- 172.21.15.46:0/3305118435 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbffc009de0 con 0x7fc00c107df0 2024-05-09T00:15:32.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:32 smithi046 ceph-mon[30151]: pgmap v501: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:32.887 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:32 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:15:32.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.886+0000 7fc0143fc700 1 -- 172.21.15.46:0/3305118435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c1a2170 con 0x7fc00c107df0 2024-05-09T00:15:32.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.886+0000 7fc0143fc700 1 -- 172.21.15.46:0/3305118435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c1a9a90 con 0x7fc00c107df0 2024-05-09T00:15:32.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.886+0000 7fc0037fe700 1 -- 172.21.15.46:0/3305118435 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc00f3c0 con 0x7fc00c107df0 2024-05-09T00:15:32.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.889+0000 7fc0037fe700 1 -- 172.21.15.46:0/3305118435 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbffc009930 con 0x7fc00c107df0 2024-05-09T00:15:32.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.890+0000 7fc0037fe700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbff80b2880 0x7fbff80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.890+0000 7fc012198700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbff80b2880 0x7fbff80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.890+0000 7fc0037fe700 1 -- 172.21.15.46:0/3305118435 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbffc0e2020 con 0x7fc00c107df0 2024-05-09T00:15:32.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.890+0000 7fc012198700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbff80b2880 0x7fbff80b4d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc008005950 tx=0x7fc00800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:32.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.891+0000 7fc022e12b80 1 Processor -- start 2024-05-09T00:15:32.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.891+0000 7fc022e12b80 1 -- start start 2024-05-09T00:15:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.891+0000 7fc022e12b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5ee160 0x55c91f6874f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc022e12b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5044a0 0x55c91f687a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc022e12b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 0x55c91f681ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc022e12b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c91f5a3ff0 con 0x55c91f6816a0 2024-05-09T00:15:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc022e12b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c91f5427f0 con 0x55c91f5044a0 2024-05-09T00:15:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc022e12b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c91f59ffb0 con 0x55c91f5ee160 2024-05-09T00:15:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc012999700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 0x55c91f681ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc012999700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 0x55c91f681ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35882/0 (socket says 172.21.15.46:35882) 2024-05-09T00:15:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc012999700 1 -- 172.21.15.46:0/3694418379 learned_addr learned my addr 172.21.15.46:0/3694418379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc012198700 1 --2- 172.21.15.46:0/3694418379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5ee160 0x55c91f6874f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc011997700 1 --2- 172.21.15.46:0/3694418379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5044a0 0x55c91f687a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc012999700 1 -- 172.21.15.46:0/3694418379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5ee160 msgr2=0x55c91f6874f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc012999700 1 --2- 172.21.15.46:0/3694418379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5ee160 0x55c91f6874f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc012999700 1 -- 172.21.15.46:0/3694418379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5044a0 msgr2=0x55c91f687a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc012999700 1 --2- 172.21.15.46:0/3694418379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5044a0 0x55c91f687a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc012999700 1 -- 172.21.15.46:0/3694418379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c91f53a370 con 0x55c91f6816a0 2024-05-09T00:15:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.892+0000 7fc012999700 1 --2- 172.21.15.46:0/3694418379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 0x55c91f681ad0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc00400cfa0 tx=0x7fc00400cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc14ad78f2facc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.893+0000 7fc000d47700 1 -- 172.21.15.46:0/3694418379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004022070 con 0x55c91f6816a0 2024-05-09T00:15:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.893+0000 7fc000d47700 1 -- 172.21.15.46:0/3694418379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0040092d0 con 0x55c91f6816a0 2024-05-09T00:15:32.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.893+0000 7fc000d47700 1 -- 172.21.15.46:0/3694418379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004007d20 con 0x55c91f6816a0 2024-05-09T00:15:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.893+0000 7fc022e12b80 1 -- 172.21.15.46:0/3694418379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 msgr2=0x55c91f681ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.893+0000 7fc022e12b80 1 --2- 172.21.15.46:0/3694418379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 0x55c91f681ad0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc00400cfa0 tx=0x7fc00400cfd0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.893+0000 7fc022e12b80 1 -- 172.21.15.46:0/3694418379 shutdown_connections 2024-05-09T00:15:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 --2- 172.21.15.46:0/3694418379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 0x55c91f681ad0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 --2- 172.21.15.46:0/3694418379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5ee160 0x55c91f6874f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 --2- 172.21.15.46:0/3694418379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5044a0 0x55c91f687a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 -- 172.21.15.46:0/3694418379 >> 172.21.15.46:0/3694418379 conn(0x55c91f5646a0 msgr2=0x55c91f67c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 -- 172.21.15.46:0/3694418379 shutdown_connections 2024-05-09T00:15:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 -- 172.21.15.46:0/3694418379 wait complete. 2024-05-09T00:15:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 -- start start 2024-05-09T00:15:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 Processor -- start 2024-05-09T00:15:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 0x55c91f68b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5ee160 0x55c91f68b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f6816a0 0x55c91f68bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c91f41fba0 con 0x55c91f5044a0 2024-05-09T00:15:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c91f55e150 con 0x55c91f5ee160 2024-05-09T00:15:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.894+0000 7fc022e12b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c91f68e1d0 con 0x55c91f6816a0 2024-05-09T00:15:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc022e12b80 10 client.0 _ll_register_callbacks cb 0x55c91f4156c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc012198700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 0x55c91f68b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc012198700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 0x55c91f68b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35888/0 (socket says 172.21.15.46:35888) 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc012198700 1 -- 172.21.15.46:0/2836249112 learned_addr learned my addr 172.21.15.46:0/2836249112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc012999700 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f6816a0 0x55c91f68bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc011997700 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5ee160 0x55c91f68b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc012198700 1 -- 172.21.15.46:0/2836249112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f6816a0 msgr2=0x55c91f68bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc012198700 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f6816a0 0x55c91f68bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc012198700 1 -- 172.21.15.46:0/2836249112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5ee160 msgr2=0x55c91f68b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc012198700 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5ee160 0x55c91f68b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc012198700 1 -- 172.21.15.46:0/2836249112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91f41f490 con 0x55c91f5044a0 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc012198700 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 0x55c91f68b0f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc0080116f0 tx=0x7fc0080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6455d18d6b0b68eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fbff5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fbff5ffb700 1 -- 172.21.15.46:0/2836249112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008028470 con 0x55c91f5044a0 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fbff5ffb700 1 -- 172.21.15.46:0/2836249112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc008028610 con 0x55c91f5044a0 2024-05-09T00:15:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.895+0000 7fc022e12b80 1 -- 172.21.15.46:0/2836249112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0040092d0 con 0x55c91f5044a0 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.896+0000 7fbff5ffb700 1 -- 172.21.15.46:0/2836249112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc008028810 con 0x55c91f5044a0 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.896+0000 7fc022e12b80 10 client.15915 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.896+0000 7fc022e12b80 1 -- 172.21.15.46:0/2836249112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c91f698ae0 con 0x55c91f5044a0 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.896+0000 7fbff5ffb700 1 -- 172.21.15.46:0/2836249112 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0040092d0 con 0x55c91f5044a0 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.896+0000 7fbff5ffb700 1 -- 172.21.15.46:0/2836249112 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc00801faf0 con 0x55c91f5044a0 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.896+0000 7fc022e12b80 10 client.15915 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.896+0000 7fc022e12b80 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.896+0000 7fc022e12b80 20 client.15915 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.896+0000 7fc022e12b80 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c91f69b120 0x55c91f69d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.896+0000 7fc022e12b80 4 client.15915 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.896+0000 7fc022e12b80 1 -- 172.21.15.46:0/2836249112 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c91f5b19d0 con 0x55c91f69b120 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.896+0000 7fc011997700 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c91f69b120 0x55c91f69d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.897+0000 7fc011997700 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c91f69b120 0x55c91f69d580 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733a86fec2014127 server_cookie=a6b8b8da37ffb538 in_seq=0 out_seq=0 2024-05-09T00:15:32.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.897+0000 7fbff5ffb700 10 client.15915 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:32.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fbff5ffb700 1 -- 172.21.15.46:0/2836249112 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c91f5b19d0 con 0x55c91f69b120 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fbff5ffb700 10 client.15915 handle_command_reply: tid=0 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fc022e12b80 1 client.15915 shutdown 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fc022e12b80 1 -- 172.21.15.46:0/2836249112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 msgr2=0x55c91f68b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fc022e12b80 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 0x55c91f68b0f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc0080116f0 tx=0x7fc0080047c0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fc022e12b80 1 -- 172.21.15.46:0/2836249112 shutdown_connections 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fc022e12b80 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c91f69b120 0x55c91f69d580 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fc022e12b80 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f6816a0 0x55c91f68bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fc022e12b80 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 0x55c91f68b0f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fc022e12b80 1 --2- 172.21.15.46:0/2836249112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5ee160 0x55c91f68b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fc022e12b80 1 -- 172.21.15.46:0/2836249112 >> 172.21.15.46:0/2836249112 conn(0x55c91f5646a0 msgr2=0x55c91f67c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fbff5ffb700 0 client.15915 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fc022e12b80 1 -- 172.21.15.46:0/2836249112 shutdown_connections 2024-05-09T00:15:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fc022e12b80 1 -- 172.21.15.46:0/2836249112 wait complete. 2024-05-09T00:15:32.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.898+0000 7fc022e12b80 20 client.15915 trim_cache size 0 max 16384 2024-05-09T00:15:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.918+0000 7fc022e12b80 1 Processor -- start 2024-05-09T00:15:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc022e12b80 1 -- start start 2024-05-09T00:15:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc022e12b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 0x55c91f68b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc022e12b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5ee160 0x55c91f683e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc022e12b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f6816a0 0x55c91f6861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc022e12b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1b36f0 con 0x55c91f5044a0 2024-05-09T00:15:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc022e12b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1b0f80 con 0x55c91f6816a0 2024-05-09T00:15:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc022e12b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1aedb0 con 0x55c91f5ee160 2024-05-09T00:15:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc012198700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 0x55c91f68b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc012198700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 0x55c91f68b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35894/0 (socket says 172.21.15.46:35894) 2024-05-09T00:15:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc012198700 1 -- 172.21.15.46:0/3359748807 learned_addr learned my addr 172.21.15.46:0/3359748807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc012999700 1 --2- 172.21.15.46:0/3359748807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f6816a0 0x55c91f6861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc011997700 1 --2- 172.21.15.46:0/3359748807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5ee160 0x55c91f683e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc012198700 1 -- 172.21.15.46:0/3359748807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5ee160 msgr2=0x55c91f683e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc012198700 1 --2- 172.21.15.46:0/3359748807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5ee160 0x55c91f683e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc012198700 1 -- 172.21.15.46:0/3359748807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f6816a0 msgr2=0x55c91f6861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc012198700 1 --2- 172.21.15.46:0/3359748807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f6816a0 0x55c91f6861e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc012198700 1 -- 172.21.15.46:0/3359748807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c91f687bc0 con 0x55c91f5044a0 2024-05-09T00:15:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc012999700 1 --2- 172.21.15.46:0/3359748807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f6816a0 0x55c91f6861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:15:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.919+0000 7fc012198700 1 --2- 172.21.15.46:0/3359748807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 0x55c91f68b9b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc00800ba80 tx=0x7fc00800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74ca04a722fe25e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.920+0000 7fbff7fff700 1 -- 172.21.15.46:0/3359748807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00801e550 con 0x55c91f5044a0 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.920+0000 7fbff7fff700 1 -- 172.21.15.46:0/3359748807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc008023070 con 0x55c91f5044a0 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.920+0000 7fbff7fff700 1 -- 172.21.15.46:0/3359748807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00801ed20 con 0x55c91f5044a0 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.920+0000 7fc022e12b80 1 -- 172.21.15.46:0/3359748807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 msgr2=0x55c91f68b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.920+0000 7fc022e12b80 1 --2- 172.21.15.46:0/3359748807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 0x55c91f68b9b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc00800ba80 tx=0x7fc00800be30 comp rx=0 tx=0).stop 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.920+0000 7fc022e12b80 1 -- 172.21.15.46:0/3359748807 shutdown_connections 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.920+0000 7fc022e12b80 1 --2- 172.21.15.46:0/3359748807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f6816a0 0x55c91f6861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.920+0000 7fc022e12b80 1 --2- 172.21.15.46:0/3359748807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f5044a0 0x55c91f68b9b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.920+0000 7fc022e12b80 1 --2- 172.21.15.46:0/3359748807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5ee160 0x55c91f683e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.920+0000 7fc022e12b80 1 -- 172.21.15.46:0/3359748807 >> 172.21.15.46:0/3359748807 conn(0x55c91f5646a0 msgr2=0x55c91f69d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.921+0000 7fc022e12b80 1 -- 172.21.15.46:0/3359748807 shutdown_connections 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.921+0000 7fc022e12b80 1 -- 172.21.15.46:0/3359748807 wait complete. 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.921+0000 7fc022e12b80 1 -- start start 2024-05-09T00:15:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.921+0000 7fc022e12b80 1 Processor -- start 2024-05-09T00:15:32.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.921+0000 7fc022e12b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5044a0 0x55c91f6a2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.921+0000 7fc022e12b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5ee160 0x55c91f6a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.921+0000 7fc022e12b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 0x55c91f6a3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.921+0000 7fc022e12b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1af440 con 0x55c91f6816a0 2024-05-09T00:15:32.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.921+0000 7fc022e12b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1b08b0 con 0x55c91f5ee160 2024-05-09T00:15:32.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.921+0000 7fc022e12b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1afb10 con 0x55c91f5044a0 2024-05-09T00:15:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.921+0000 7fc022e12b80 10 client.0 _ll_register_callbacks cb 0x55c91f4156c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.921+0000 7fc012999700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 0x55c91f6a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc012999700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 0x55c91f6a3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35904/0 (socket says 172.21.15.46:35904) 2024-05-09T00:15:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc012999700 1 -- 172.21.15.46:0/1016905821 learned_addr learned my addr 172.21.15.46:0/1016905821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc012198700 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5044a0 0x55c91f6a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc011997700 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5ee160 0x55c91f6a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc012999700 1 -- 172.21.15.46:0/1016905821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5044a0 msgr2=0x55c91f6a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc012999700 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5044a0 0x55c91f6a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc012999700 1 -- 172.21.15.46:0/1016905821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5ee160 msgr2=0x55c91f6a2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc012999700 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5ee160 0x55c91f6a2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc012999700 1 -- 172.21.15.46:0/1016905821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91f691750 con 0x55c91f6816a0 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc012999700 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 0x55c91f6a3190 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc004009a20 tx=0x7fc004004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d28f974257f7b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fbff5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fbff5ffb700 1 -- 172.21.15.46:0/1016905821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004022040 con 0x55c91f6816a0 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fbff5ffb700 1 -- 172.21.15.46:0/1016905821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc004007b30 con 0x55c91f6816a0 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc022e12b80 1 -- 172.21.15.46:0/1016905821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc008023070 con 0x55c91f6816a0 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fbff5ffb700 1 -- 172.21.15.46:0/1016905821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc004007670 con 0x55c91f6816a0 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc022e12b80 10 client.15921 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.922+0000 7fc022e12b80 1 -- 172.21.15.46:0/1016905821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c91f705b30 con 0x55c91f6816a0 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.923+0000 7fbff5ffb700 1 -- 172.21.15.46:0/1016905821 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc008023070 con 0x55c91f6816a0 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.923+0000 7fbff5ffb700 1 -- 172.21.15.46:0/1016905821 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc004011070 con 0x55c91f6816a0 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.923+0000 7fc022e12b80 10 client.15921 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.923+0000 7fc022e12b80 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.923+0000 7fc022e12b80 20 client.15921 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.923+0000 7fc022e12b80 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c91f705d00 0x55c91f708120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.923+0000 7fc022e12b80 4 client.15921 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.923+0000 7fc022e12b80 1 -- 172.21.15.46:0/1016905821 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c91f5aba70 con 0x55c91f705d00 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.923+0000 7fc012198700 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c91f705d00 0x55c91f708120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fbff5ffb700 10 client.15921 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc012198700 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c91f705d00 0x55c91f708120 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97623bbf86faaaae server_cookie=5375a40dd2f5f241 in_seq=0 out_seq=0 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fbff5ffb700 1 -- 172.21.15.46:0/1016905821 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c91f5aba70 con 0x55c91f705d00 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fbff5ffb700 10 client.15921 handle_command_reply: tid=0 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc022e12b80 1 client.15921 shutdown 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc022e12b80 1 -- 172.21.15.46:0/1016905821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 msgr2=0x55c91f6a3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc022e12b80 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 0x55c91f6a3190 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc004009a20 tx=0x7fc004004c80 comp rx=0 tx=0).stop 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc022e12b80 1 -- 172.21.15.46:0/1016905821 shutdown_connections 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc022e12b80 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c91f705d00 0x55c91f708120 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc022e12b80 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c91f6816a0 0x55c91f6a3190 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc022e12b80 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c91f5044a0 0x55c91f6a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fbff5ffb700 0 client.15921 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:32.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc022e12b80 1 --2- 172.21.15.46:0/1016905821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c91f5ee160 0x55c91f6a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc022e12b80 1 -- 172.21.15.46:0/1016905821 >> 172.21.15.46:0/1016905821 conn(0x55c91f5646a0 msgr2=0x55c91f684e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:32.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc022e12b80 1 -- 172.21.15.46:0/1016905821 shutdown_connections 2024-05-09T00:15:32.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc022e12b80 1 -- 172.21.15.46:0/1016905821 wait complete. 2024-05-09T00:15:32.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.924+0000 7fc022e12b80 20 client.15921 trim_cache size 0 max 16384 2024-05-09T00:15:32.929 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:15:32.929 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:15:32.930 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:15:32.930 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:15:32.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.925+0000 7fc0143fc700 1 -- 172.21.15.46:0/3305118435 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbff80b2880 msgr2=0x7fbff80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:32.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.925+0000 7fc0143fc700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbff80b2880 0x7fbff80b4d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc008005950 tx=0x7fc00800b410 comp rx=0 tx=0).stop 2024-05-09T00:15:32.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.925+0000 7fc0143fc700 1 -- 172.21.15.46:0/3305118435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00c107df0 msgr2=0x7fc00c19d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:32.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.925+0000 7fc0143fc700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00c107df0 0x7fc00c19d550 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00d980 tx=0x7fbffc00dc90 comp rx=0 tx=0).stop 2024-05-09T00:15:32.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.925+0000 7fc0143fc700 1 -- 172.21.15.46:0/3305118435 shutdown_connections 2024-05-09T00:15:32.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.925+0000 7fc0143fc700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbff80b2880 0x7fbff80b4d90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.925+0000 7fc0143fc700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc00c108740 0x7fc00c1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.925+0000 7fc0143fc700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc00c1034c0 0x7fc00c19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.925+0000 7fc0143fc700 1 --2- 172.21.15.46:0/3305118435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00c107df0 0x7fc00c19d550 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:32.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.925+0000 7fc0143fc700 1 -- 172.21.15.46:0/3305118435 >> 172.21.15.46:0/3305118435 conn(0x7fc00c0fedf0 msgr2=0x7fc00c10ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:32.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.925+0000 7fc0143fc700 1 -- 172.21.15.46:0/3305118435 shutdown_connections 2024-05-09T00:15:32.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:32.925+0000 7fc0143fc700 1 -- 172.21.15.46:0/3305118435 wait complete. 2024-05-09T00:15:33.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:32 smithi169 ceph-mon[34777]: pgmap v501: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:33.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:32 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:15:33.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:32 smithi142 ceph-mon[34533]: pgmap v501: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:33.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:32 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:15:33.781 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:15:33.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:15:34.033 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:35.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:34 smithi169 ceph-mon[34777]: pgmap v502: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:35.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:34 smithi046 ceph-mon[30151]: pgmap v502: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:35.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:34 smithi142 ceph-mon[34533]: pgmap v502: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:35.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.716+0000 7f4c160a2700 1 -- 172.21.15.46:0/1530483464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c10107df0 msgr2=0x7f4c10108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:35.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.716+0000 7f4c160a2700 1 --2- 172.21.15.46:0/1530483464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c10107df0 0x7f4c10108200 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4c040099c0 tx=0x7f4c04009cd0 comp rx=0 tx=0).stop 2024-05-09T00:15:35.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.716+0000 7f4c160a2700 1 -- 172.21.15.46:0/1530483464 shutdown_connections 2024-05-09T00:15:35.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.716+0000 7f4c160a2700 1 --2- 172.21.15.46:0/1530483464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c10108740 0x7f4c101100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:35.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.716+0000 7f4c160a2700 1 --2- 172.21.15.46:0/1530483464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c101034c0 0x7f4c101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:35.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.716+0000 7f4c160a2700 1 --2- 172.21.15.46:0/1530483464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c10107df0 0x7f4c10108200 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:35.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.716+0000 7f4c160a2700 1 -- 172.21.15.46:0/1530483464 >> 172.21.15.46:0/1530483464 conn(0x7f4c100fedc0 msgr2=0x7f4c101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:35.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.716+0000 7f4c160a2700 1 -- 172.21.15.46:0/1530483464 shutdown_connections 2024-05-09T00:15:35.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.716+0000 7f4c160a2700 1 -- 172.21.15.46:0/1530483464 wait complete. 2024-05-09T00:15:35.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.717+0000 7f4c160a2700 1 Processor -- start 2024-05-09T00:15:35.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.717+0000 7f4c160a2700 1 -- start start 2024-05-09T00:15:35.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.717+0000 7f4c160a2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c101034c0 0x7f4c1019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:35.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.717+0000 7f4c160a2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c10107df0 0x7f4c1019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:35.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.717+0000 7f4c160a2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c10108740 0x7f4c1019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:35.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.717+0000 7f4c160a2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c10113350 con 0x7f4c101034c0 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.717+0000 7f4c160a2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4c101131d0 con 0x7f4c10108740 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.717+0000 7f4c160a2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c101134d0 con 0x7f4c10107df0 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.717+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c101034c0 0x7f4c1019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.717+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c101034c0 0x7f4c1019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35922/0 (socket says 172.21.15.46:35922) 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.717+0000 7f4c0f7fe700 1 -- 172.21.15.46:0/802809530 learned_addr learned my addr 172.21.15.46:0/802809530 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c0ffff700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c10108740 0x7f4c1019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c0effd700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c10107df0 0x7f4c1019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c0f7fe700 1 -- 172.21.15.46:0/802809530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c10107df0 msgr2=0x7f4c1019c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c0f7fe700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c10107df0 0x7f4c1019c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c0f7fe700 1 -- 172.21.15.46:0/802809530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c10108740 msgr2=0x7f4c1019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c0f7fe700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c10108740 0x7f4c1019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c0f7fe700 1 -- 172.21.15.46:0/802809530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c101a6fc0 con 0x7f4c101034c0 2024-05-09T00:15:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c0ffff700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c10108740 0x7f4c1019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:35.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c0f7fe700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c101034c0 0x7f4c1019bc50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000d950 tx=0x7f4c0000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:35.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c0cff9700 1 -- 172.21.15.46:0/802809530 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00016070 con 0x7f4c101034c0 2024-05-09T00:15:35.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c0cff9700 1 -- 172.21.15.46:0/802809530 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c00010460 con 0x7f4c101034c0 2024-05-09T00:15:35.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c0cff9700 1 -- 172.21.15.46:0/802809530 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000f620 con 0x7f4c101034c0 2024-05-09T00:15:35.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c160a2700 1 -- 172.21.15.46:0/802809530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c101a72b0 con 0x7f4c101034c0 2024-05-09T00:15:35.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.718+0000 7f4c160a2700 1 -- 172.21.15.46:0/802809530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c101a7770 con 0x7f4c101034c0 2024-05-09T00:15:35.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.719+0000 7f4c160a2700 1 -- 172.21.15.46:0/802809530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c100512c0 con 0x7f4c101034c0 2024-05-09T00:15:35.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.721+0000 7f4c0cff9700 1 -- 172.21.15.46:0/802809530 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4c00019070 con 0x7f4c101034c0 2024-05-09T00:15:35.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.722+0000 7f4c0cff9700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4bf80b2880 0x7f4bf80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:35.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.722+0000 7f4c0effd700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4bf80b2880 0x7f4bf80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:35.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.722+0000 7f4c0cff9700 1 -- 172.21.15.46:0/802809530 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4c000e1ec0 con 0x7f4c101034c0 2024-05-09T00:15:35.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.722+0000 7f4c0effd700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4bf80b2880 0x7f4bf80b4d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4c04000c00 tx=0x7f4c0401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:35.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.725+0000 7f4c0cff9700 1 -- 172.21.15.46:0/802809530 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4c000a9ba0 con 0x7f4c101034c0 2024-05-09T00:15:35.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.987+0000 7f4c160a2700 1 -- 172.21.15.46:0/802809530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c1002e7b0 con 0x7f4c101034c0 2024-05-09T00:15:35.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.988+0000 7f4c0cff9700 1 -- 172.21.15.46:0/802809530 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f4c000a94a0 con 0x7f4c101034c0 2024-05-09T00:15:35.987 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:15:35.987 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:15:35.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.990+0000 7f4c160a2700 1 -- 172.21.15.46:0/802809530 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4bf80b2880 msgr2=0x7f4bf80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:35.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.990+0000 7f4c160a2700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4bf80b2880 0x7f4bf80b4d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4c04000c00 tx=0x7f4c0401a040 comp rx=0 tx=0).stop 2024-05-09T00:15:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.990+0000 7f4c160a2700 1 -- 172.21.15.46:0/802809530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c101034c0 msgr2=0x7f4c1019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.990+0000 7f4c160a2700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c101034c0 0x7f4c1019bc50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000d950 tx=0x7f4c0000dd10 comp rx=0 tx=0).stop 2024-05-09T00:15:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.990+0000 7f4c160a2700 1 -- 172.21.15.46:0/802809530 shutdown_connections 2024-05-09T00:15:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.990+0000 7f4c160a2700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4bf80b2880 0x7f4bf80b4d90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.990+0000 7f4c160a2700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c10108740 0x7f4c1019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.990+0000 7f4c160a2700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c101034c0 0x7f4c1019bc50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.990+0000 7f4c160a2700 1 --2- 172.21.15.46:0/802809530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c10107df0 0x7f4c1019c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.990+0000 7f4c160a2700 1 -- 172.21.15.46:0/802809530 >> 172.21.15.46:0/802809530 conn(0x7f4c100fedc0 msgr2=0x7f4c101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.990+0000 7f4c160a2700 1 -- 172.21.15.46:0/802809530 shutdown_connections 2024-05-09T00:15:35.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:35.990+0000 7f4c160a2700 1 -- 172.21.15.46:0/802809530 wait complete. 2024-05-09T00:15:35.990 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:15:36.771 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:15:37.029 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:37.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:36 smithi046 ceph-mon[30151]: pgmap v503: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:37.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:36 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/802809530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:15:37.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:36 smithi169 ceph-mon[34777]: pgmap v503: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:37.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:36 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/802809530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:15:37.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:36 smithi142 ceph-mon[34533]: pgmap v503: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:37.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:36 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/802809530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:15:38.187 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:37 smithi046 ceph-mon[30151]: pgmap v504: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:38.191 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:37 smithi142 ceph-mon[34533]: pgmap v504: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:38.354 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:37 smithi169 ceph-mon[34777]: pgmap v504: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.791+0000 7f1e93ebf700 1 -- 172.21.15.46:0/1585278958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8c105e70 msgr2=0x7f1e8c10e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.791+0000 7f1e93ebf700 1 --2- 172.21.15.46:0/1585278958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8c105e70 0x7f1e8c10e3c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009990 tx=0x7f1e80009ca0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.791+0000 7f1e93ebf700 1 -- 172.21.15.46:0/1585278958 shutdown_connections 2024-05-09T00:15:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.791+0000 7f1e93ebf700 1 --2- 172.21.15.46:0/1585278958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e8c10e900 0x7f1e8c110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.791+0000 7f1e93ebf700 1 --2- 172.21.15.46:0/1585278958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e8c105520 0x7f1e8c105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.791+0000 7f1e93ebf700 1 --2- 172.21.15.46:0/1585278958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8c105e70 0x7f1e8c10e3c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.791+0000 7f1e93ebf700 1 -- 172.21.15.46:0/1585278958 >> 172.21.15.46:0/1585278958 conn(0x7f1e8c0fef30 msgr2=0x7f1e8c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.791+0000 7f1e93ebf700 1 -- 172.21.15.46:0/1585278958 shutdown_connections 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.791+0000 7f1e93ebf700 1 -- 172.21.15.46:0/1585278958 wait complete. 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e93ebf700 1 Processor -- start 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e93ebf700 1 -- start start 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e93ebf700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e8c105520 0x7f1e8c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e93ebf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8c105e70 0x7f1e8c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e93ebf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e8c10e900 0x7f1e8c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e93ebf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c114460 con 0x7f1e8c105e70 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e93ebf700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1142e0 con 0x7f1e8c105520 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e93ebf700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1145e0 con 0x7f1e8c10e900 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e9145a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8c105e70 0x7f1e8c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e9145a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8c105e70 0x7f1e8c19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33144/0 (socket says 172.21.15.46:33144) 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e9145a700 1 -- 172.21.15.46:0/2829064686 learned_addr learned my addr 172.21.15.46:0/2829064686 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e8c105520 0x7f1e8c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e9245c700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e8c10e900 0x7f1e8c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e9145a700 1 -- 172.21.15.46:0/2829064686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e8c10e900 msgr2=0x7f1e8c19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e9145a700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e8c10e900 0x7f1e8c19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e9145a700 1 -- 172.21.15.46:0/2829064686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e8c105520 msgr2=0x7f1e8c19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e9145a700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e8c105520 0x7f1e8c19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e9145a700 1 -- 172.21.15.46:0/2829064686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c19fd00 con 0x7f1e8c105e70 2024-05-09T00:15:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.792+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e8c105520 0x7f1e8c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.793+0000 7f1e9145a700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8c105e70 0x7f1e8c19b3d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1e800057c0 tx=0x7f1e800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.793+0000 7f1e7effd700 1 -- 172.21.15.46:0/2829064686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80027070 con 0x7f1e8c105e70 2024-05-09T00:15:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.793+0000 7f1e93ebf700 1 -- 172.21.15.46:0/2829064686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c19ff90 con 0x7f1e8c105e70 2024-05-09T00:15:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.793+0000 7f1e7effd700 1 -- 172.21.15.46:0/2829064686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e80004020 con 0x7f1e8c105e70 2024-05-09T00:15:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.793+0000 7f1e7effd700 1 -- 172.21.15.46:0/2829064686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80022c40 con 0x7f1e8c105e70 2024-05-09T00:15:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.793+0000 7f1e93ebf700 1 -- 172.21.15.46:0/2829064686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c1a7910 con 0x7f1e8c105e70 2024-05-09T00:15:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.796+0000 7f1e7effd700 1 -- 172.21.15.46:0/2829064686 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1e8002a070 con 0x7f1e8c105e70 2024-05-09T00:15:38.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.796+0000 7f1e7effd700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1e780b2900 0x7f1e780b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.797+0000 7f1e7effd700 1 -- 172.21.15.46:0/2829064686 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1e800e3140 con 0x7f1e8c105e70 2024-05-09T00:15:38.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.797+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1e780b2900 0x7f1e780b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.797+0000 7f1ea28d5b80 1 Processor -- start 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.797+0000 7f1ea28d5b80 1 -- start start 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.797+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1e780b2900 0x7f1e780b4e10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1e88005950 tx=0x7f1e880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.797+0000 7f1ea28d5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8e0d00 0x55c7fd9e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.797+0000 7f1ea28d5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8664a0 0x55c7fd9e8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.797+0000 7f1ea28d5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fd9ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.797+0000 7f1ea28d5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c7fd9060a0 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.797+0000 7f1ea28d5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c7fd8ab9a0 con 0x55c7fd8e0d00 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.797+0000 7f1ea28d5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c7fd902060 con 0x55c7fd8664a0 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e9245c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fd9ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e9145a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8664a0 0x55c7fd9e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e9245c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fd9ec5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33160/0 (socket says 172.21.15.46:33160) 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e9245c700 1 -- 172.21.15.46:0/444827335 learned_addr learned my addr 172.21.15.46:0/444827335 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/444827335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8e0d00 0x55c7fd9e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e9245c700 1 -- 172.21.15.46:0/444827335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8664a0 msgr2=0x55c7fd9e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e9245c700 1 --2- 172.21.15.46:0/444827335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8664a0 0x55c7fd9e8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e9245c700 1 -- 172.21.15.46:0/444827335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8e0d00 msgr2=0x55c7fd9e7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e9245c700 1 --2- 172.21.15.46:0/444827335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8e0d00 0x55c7fd9e7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e9245c700 1 -- 172.21.15.46:0/444827335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7fd89c500 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e9145a700 1 --2- 172.21.15.46:0/444827335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8664a0 0x55c7fd9e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:15:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e9245c700 1 --2- 172.21.15.46:0/444827335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fd9ec5f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009d50 tx=0x7f1e840077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e56058e49b5e776 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.798+0000 7f1e77fff700 1 -- 172.21.15.46:0/444827335 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84003a00 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.799+0000 7f1e77fff700 1 -- 172.21.15.46:0/444827335 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e84003ba0 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.799+0000 7f1e77fff700 1 -- 172.21.15.46:0/444827335 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e840048b0 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.799+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/444827335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 msgr2=0x55c7fd9ec5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.799+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/444827335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fd9ec5f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009d50 tx=0x7f1e840077e0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.799+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/444827335 shutdown_connections 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.799+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/444827335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fd9ec5f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.799+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/444827335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8e0d00 0x55c7fd9e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.800+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/444827335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8664a0 0x55c7fd9e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.800+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/444827335 >> 172.21.15.46:0/444827335 conn(0x55c7fd869f70 msgr2=0x55c7fd9de220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.800+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/444827335 shutdown_connections 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.800+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/444827335 wait complete. 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.800+0000 7f1ea28d5b80 1 -- start start 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.800+0000 7f1ea28d5b80 1 Processor -- start 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1ea28d5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8664a0 0x55c7fd9f1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1ea28d5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8e0d00 0x55c7fd9f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1ea28d5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fd9f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1ea28d5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c7fd9ecf00 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1ea28d5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c7fd8a0b20 con 0x55c7fd8664a0 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1ea28d5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c7fd9f4470 con 0x55c7fd8e0d00 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1ea28d5b80 10 client.0 _ll_register_callbacks cb 0x55c7fd7776c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e9245c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fd9f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e9245c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fd9f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33166/0 (socket says 172.21.15.46:33166) 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e9245c700 1 -- 172.21.15.46:0/2007094506 learned_addr learned my addr 172.21.15.46:0/2007094506 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8664a0 0x55c7fd9f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e9145a700 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8e0d00 0x55c7fd9f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e9245c700 1 -- 172.21.15.46:0/2007094506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8e0d00 msgr2=0x55c7fd9f18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e9245c700 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8e0d00 0x55c7fd9f18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e9245c700 1 -- 172.21.15.46:0/2007094506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8664a0 msgr2=0x55c7fd9f1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e9245c700 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8664a0 0x55c7fd9f1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e9245c700 1 -- 172.21.15.46:0/2007094506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7fd781490 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8664a0 0x55c7fd9f1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:15:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e9245c700 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fd9f1e00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1e84000f80 tx=0x7f1e84004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fee8ecf8e0a7d01 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.801+0000 7f1e757fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1e757fa700 1 -- 172.21.15.46:0/2007094506 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84022070 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1e757fa700 1 -- 172.21.15.46:0/2007094506 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1e8401e420 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/2007094506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e84003ba0 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1e757fa700 1 -- 172.21.15.46:0/2007094506 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e8402e050 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1ea28d5b80 10 client.15939 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/2007094506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c7fd9fed10 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1e757fa700 1 -- 172.21.15.46:0/2007094506 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e84003ba0 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1e757fa700 1 -- 172.21.15.46:0/2007094506 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1e84014480 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1ea28d5b80 10 client.15939 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1ea28d5b80 10 client.15939 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1ea28d5b80 20 client.15939 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c7fda011e0 0x55c7fda03600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1ea28d5b80 4 client.15939 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:15:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.802+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/2007094506 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7fd913a80 con 0x55c7fda011e0 2024-05-09T00:15:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.803+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c7fda011e0 0x55c7fda03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.803+0000 7f1e757fa700 10 client.15939 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:15:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.803+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c7fda011e0 0x55c7fda03600 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=282822d4256d39c5 server_cookie=ccdb7b3c0784b5f7 in_seq=0 out_seq=0 2024-05-09T00:15:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.804+0000 7f1e757fa700 1 -- 172.21.15.46:0/2007094506 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c7fd913a80 con 0x55c7fda011e0 2024-05-09T00:15:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.804+0000 7f1e757fa700 10 client.15939 handle_command_reply: tid=0 2024-05-09T00:15:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.804+0000 7f1ea28d5b80 1 client.15939 shutdown 2024-05-09T00:15:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.804+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/2007094506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 msgr2=0x55c7fd9f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.804+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fd9f1e00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1e84000f80 tx=0x7f1e84004450 comp rx=0 tx=0).stop 2024-05-09T00:15:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.805+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/2007094506 shutdown_connections 2024-05-09T00:15:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.805+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c7fda011e0 0x55c7fda03600 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.805+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fd9f1e00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.805+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8664a0 0x55c7fd9f1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.805+0000 7f1e757fa700 0 client.15939 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:15:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.805+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/2007094506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8e0d00 0x55c7fd9f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.805+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/2007094506 >> 172.21.15.46:0/2007094506 conn(0x55c7fd869f70 msgr2=0x55c7fd9e23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.805+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/2007094506 shutdown_connections 2024-05-09T00:15:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.805+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/2007094506 wait complete. 2024-05-09T00:15:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.805+0000 7f1ea28d5b80 20 client.15939 trim_cache size 0 max 16384 2024-05-09T00:15:38.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.820+0000 7f1ea28d5b80 1 Processor -- start 2024-05-09T00:15:38.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.820+0000 7f1ea28d5b80 1 -- start start 2024-05-09T00:15:38.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.820+0000 7f1ea28d5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd8664a0 0x55c7fd9fd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.820+0000 7f1ea28d5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8e0d00 0x55c7fd9f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.820+0000 7f1ea28d5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd9ec1e0 0x55c7fda06620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.820+0000 7f1ea28d5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1af310 con 0x55c7fd8664a0 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.820+0000 7f1ea28d5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1acba0 con 0x55c7fd8e0d00 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.820+0000 7f1ea28d5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1aa9f0 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.820+0000 7f1e91c5b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd8664a0 0x55c7fd9fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.820+0000 7f1e91c5b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd8664a0 0x55c7fd9fd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33170/0 (socket says 172.21.15.46:33170) 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.820+0000 7f1e91c5b700 1 -- 172.21.15.46:0/1844882007 learned_addr learned my addr 172.21.15.46:0/1844882007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.820+0000 7f1e9145a700 1 --2- 172.21.15.46:0/1844882007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8e0d00 0x55c7fd9f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.821+0000 7f1e9245c700 1 --2- 172.21.15.46:0/1844882007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd9ec1e0 0x55c7fda06620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.821+0000 7f1e91c5b700 1 -- 172.21.15.46:0/1844882007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd9ec1e0 msgr2=0x55c7fda06620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.821+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/1844882007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd9ec1e0 0x55c7fda06620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.821+0000 7f1e91c5b700 1 -- 172.21.15.46:0/1844882007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8e0d00 msgr2=0x55c7fd9f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.821+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/1844882007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8e0d00 0x55c7fd9f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.821+0000 7f1e91c5b700 1 -- 172.21.15.46:0/1844882007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7fd9da9d0 con 0x55c7fd8664a0 2024-05-09T00:15:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.821+0000 7f1e9245c700 1 --2- 172.21.15.46:0/1844882007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd9ec1e0 0x55c7fda06620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:15:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.821+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/1844882007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd8664a0 0x55c7fd9fd590 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55c7fd9e8b40 tx=0x7f1e88015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2cd8683db423959 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.821+0000 7f1e777fe700 1 -- 172.21.15.46:0/1844882007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8802e070 con 0x55c7fd8664a0 2024-05-09T00:15:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.821+0000 7f1e777fe700 1 -- 172.21.15.46:0/1844882007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e88003d30 con 0x55c7fd8664a0 2024-05-09T00:15:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.821+0000 7f1e777fe700 1 -- 172.21.15.46:0/1844882007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88011440 con 0x55c7fd8664a0 2024-05-09T00:15:38.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.822+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1844882007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd8664a0 msgr2=0x55c7fd9fd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:38.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.822+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/1844882007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd8664a0 0x55c7fd9fd590 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55c7fd9e8b40 tx=0x7f1e88015000 comp rx=0 tx=0).stop 2024-05-09T00:15:38.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.822+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1844882007 shutdown_connections 2024-05-09T00:15:38.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.822+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/1844882007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd9ec1e0 0x55c7fda06620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.822+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/1844882007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd8664a0 0x55c7fd9fd590 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.822+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/1844882007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8e0d00 0x55c7fd9f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.822+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1844882007 >> 172.21.15.46:0/1844882007 conn(0x55c7fd869f70 msgr2=0x55c7fd9eba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:38.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.822+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1844882007 shutdown_connections 2024-05-09T00:15:38.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.822+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1844882007 wait complete. 2024-05-09T00:15:38.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.822+0000 7f1ea28d5b80 1 -- start start 2024-05-09T00:15:38.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1ea28d5b80 1 Processor -- start 2024-05-09T00:15:38.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1ea28d5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8664a0 0x55c7fda06760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1ea28d5b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8e0d00 0x55c7fda06ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1ea28d5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fda05370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1ea28d5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1ab060 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1ea28d5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1ac4d0 con 0x55c7fd8664a0 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1ea28d5b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1ab730 con 0x55c7fd8e0d00 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1ea28d5b80 10 client.0 _ll_register_callbacks cb 0x55c7fd7776c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1e9245c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fda05370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1e91c5b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8664a0 0x55c7fda06760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1e9245c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fda05370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33176/0 (socket says 172.21.15.46:33176) 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1e9245c700 1 -- 172.21.15.46:0/1094958424 learned_addr learned my addr 172.21.15.46:0/1094958424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1e9145a700 1 --2- 172.21.15.46:0/1094958424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8e0d00 0x55c7fda06ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.823+0000 7f1e9245c700 1 -- 172.21.15.46:0/1094958424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8e0d00 msgr2=0x55c7fda06ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1e9245c700 1 --2- 172.21.15.46:0/1094958424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8e0d00 0x55c7fda06ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1e9245c700 1 -- 172.21.15.46:0/1094958424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8664a0 msgr2=0x55c7fda06760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1e9245c700 1 --2- 172.21.15.46:0/1094958424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8664a0 0x55c7fda06760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1e9245c700 1 -- 172.21.15.46:0/1094958424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7fda05b90 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1e9245c700 1 --2- 172.21.15.46:0/1094958424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fda05370 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1e84004420 tx=0x7f1e84009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b28b30f8d670b44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1e757fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1e757fa700 1 -- 172.21.15.46:0/1094958424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84022040 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1e757fa700 1 -- 172.21.15.46:0/1094958424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1e84002030 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1094958424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e88003d30 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1e757fa700 1 -- 172.21.15.46:0/1094958424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e840095e0 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1ea28d5b80 10 client.15945 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1094958424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c7fda6b1e0 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1e757fa700 1 -- 172.21.15.46:0/1094958424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e88003d30 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.824+0000 7f1e757fa700 1 -- 172.21.15.46:0/1094958424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1e840263b0 con 0x55c7fd9ec1e0 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.825+0000 7f1ea28d5b80 10 client.15945 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.825+0000 7f1ea28d5b80 10 client.15945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.825+0000 7f1ea28d5b80 20 client.15945 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.825+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/1094958424 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c7fda6b8c0 0x55c7fda6dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:38.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.825+0000 7f1ea28d5b80 4 client.15945 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.825+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1094958424 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7fd90db20 con 0x55c7fda6b8c0 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.825+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/1094958424 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c7fda6b8c0 0x55c7fda6dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1e91c5b700 1 --2- 172.21.15.46:0/1094958424 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c7fda6b8c0 0x55c7fda6dce0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=107a5ef8e1f93188 server_cookie=331cb8f5f8f410d6 in_seq=0 out_seq=0 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1e757fa700 10 client.15945 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1e757fa700 1 -- 172.21.15.46:0/1094958424 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7fd90db20 con 0x55c7fda6b8c0 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1e757fa700 10 client.15945 handle_command_reply: tid=0 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1ea28d5b80 1 client.15945 shutdown 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1094958424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 msgr2=0x55c7fda05370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/1094958424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fda05370 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1e84004420 tx=0x7f1e84009cd0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1094958424 shutdown_connections 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/1094958424 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55c7fda6b8c0 0x55c7fda6dce0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/1094958424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fd9ec1e0 0x55c7fda05370 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/1094958424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7fd8664a0 0x55c7fda06760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1ea28d5b80 1 --2- 172.21.15.46:0/1094958424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7fd8e0d00 0x55c7fda06ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1e757fa700 0 client.15945 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:15:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1094958424 >> 172.21.15.46:0/1094958424 conn(0x55c7fd869f70 msgr2=0x55c7fd9efb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1094958424 shutdown_connections 2024-05-09T00:15:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1ea28d5b80 1 -- 172.21.15.46:0/1094958424 wait complete. 2024-05-09T00:15:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.826+0000 7f1ea28d5b80 20 client.15945 trim_cache size 0 max 16384 2024-05-09T00:15:38.829 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:15:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.827+0000 7f1e93ebf700 1 -- 172.21.15.46:0/2829064686 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1e780b2900 msgr2=0x7f1e780b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.827+0000 7f1e93ebf700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1e780b2900 0x7f1e780b4e10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1e88005950 tx=0x7f1e880058e0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.827+0000 7f1e93ebf700 1 -- 172.21.15.46:0/2829064686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8c105e70 msgr2=0x7f1e8c19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.827+0000 7f1e93ebf700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8c105e70 0x7f1e8c19b3d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1e800057c0 tx=0x7f1e800057f0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.827+0000 7f1e93ebf700 1 -- 172.21.15.46:0/2829064686 shutdown_connections 2024-05-09T00:15:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.827+0000 7f1e93ebf700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1e780b2900 0x7f1e780b4e10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.827+0000 7f1e93ebf700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e8c10e900 0x7f1e8c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.827+0000 7f1e93ebf700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e8c105520 0x7f1e8c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.827+0000 7f1e93ebf700 1 --2- 172.21.15.46:0/2829064686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8c105e70 0x7f1e8c19b3d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.827+0000 7f1e93ebf700 1 -- 172.21.15.46:0/2829064686 >> 172.21.15.46:0/2829064686 conn(0x7f1e8c0fef30 msgr2=0x7f1e8c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.827+0000 7f1e93ebf700 1 -- 172.21.15.46:0/2829064686 shutdown_connections 2024-05-09T00:15:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:38.827+0000 7f1e93ebf700 1 -- 172.21.15.46:0/2829064686 wait complete. 2024-05-09T00:15:39.617 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:15:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:40.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:40 smithi046 ceph-mon[30151]: pgmap v505: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:15:40.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:40 smithi169 ceph-mon[34777]: pgmap v505: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:15:40.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:40 smithi142 ceph-mon[34533]: pgmap v505: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:15:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.678+0000 7f9cd4d75700 1 -- 172.21.15.46:0/2577934681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd0107df0 msgr2=0x7f9cd0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.678+0000 7f9cd4d75700 1 --2- 172.21.15.46:0/2577934681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd0107df0 0x7f9cd0108200 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80099c0 tx=0x7f9cb8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.678+0000 7f9cd4d75700 1 -- 172.21.15.46:0/2577934681 shutdown_connections 2024-05-09T00:15:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.678+0000 7f9cd4d75700 1 --2- 172.21.15.46:0/2577934681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd0108740 0x7f9cd01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.678+0000 7f9cd4d75700 1 --2- 172.21.15.46:0/2577934681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cd01034c0 0x7f9cd01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.678+0000 7f9cd4d75700 1 --2- 172.21.15.46:0/2577934681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd0107df0 0x7f9cd0108200 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.678+0000 7f9cd4d75700 1 -- 172.21.15.46:0/2577934681 >> 172.21.15.46:0/2577934681 conn(0x7f9cd00fedc0 msgr2=0x7f9cd01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.678+0000 7f9cd4d75700 1 -- 172.21.15.46:0/2577934681 shutdown_connections 2024-05-09T00:15:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.678+0000 7f9cd4d75700 1 -- 172.21.15.46:0/2577934681 wait complete. 2024-05-09T00:15:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.679+0000 7f9cd4d75700 1 Processor -- start 2024-05-09T00:15:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cd4d75700 1 -- start start 2024-05-09T00:15:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cd4d75700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd01034c0 0x7f9cd019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cd4d75700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd0107df0 0x7f9cd019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cd4d75700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cd0108740 0x7f9cd019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cd4d75700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd0113440 con 0x7f9cd01034c0 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cd4d75700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01132c0 con 0x7f9cd0108740 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cd4d75700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01135c0 con 0x7f9cd0107df0 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd01034c0 0x7f9cd019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd01034c0 0x7f9cd019bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33190/0 (socket says 172.21.15.46:33190) 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cce59c700 1 -- 172.21.15.46:0/3701283757 learned_addr learned my addr 172.21.15.46:0/3701283757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cc5d9b700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd0107df0 0x7f9cd019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cced9d700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cd0108740 0x7f9cd019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cce59c700 1 -- 172.21.15.46:0/3701283757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd0107df0 msgr2=0x7f9cd019c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cce59c700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd0107df0 0x7f9cd019c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cce59c700 1 -- 172.21.15.46:0/3701283757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cd0108740 msgr2=0x7f9cd019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cce59c700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cd0108740 0x7f9cd019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.680+0000 7f9cce59c700 1 -- 172.21.15.46:0/3701283757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd01a70b0 con 0x7f9cd01034c0 2024-05-09T00:15:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.681+0000 7f9cce59c700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd01034c0 0x7f9cd019bd60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000b530 tx=0x7f9cc000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.681+0000 7f9cc7fff700 1 -- 172.21.15.46:0/3701283757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0015020 con 0x7f9cd01034c0 2024-05-09T00:15:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.681+0000 7f9cd4d75700 1 -- 172.21.15.46:0/3701283757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd01a73a0 con 0x7f9cd01034c0 2024-05-09T00:15:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.681+0000 7f9cc7fff700 1 -- 172.21.15.46:0/3701283757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc0011ce0 con 0x7f9cd01034c0 2024-05-09T00:15:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.681+0000 7f9cc7fff700 1 -- 172.21.15.46:0/3701283757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0010d20 con 0x7f9cd01034c0 2024-05-09T00:15:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.681+0000 7f9cd4d75700 1 -- 172.21.15.46:0/3701283757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd01a7900 con 0x7f9cd01034c0 2024-05-09T00:15:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.684+0000 7f9cc7fff700 1 -- 172.21.15.46:0/3701283757 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9cc0018020 con 0x7f9cd01034c0 2024-05-09T00:15:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.684+0000 7f9cc7fff700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9cbc0b2880 0x7f9cbc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.685+0000 7f9cc5d9b700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9cbc0b2880 0x7f9cbc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.685+0000 7f9cc7fff700 1 -- 172.21.15.46:0/3701283757 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9cc00e1a30 con 0x7f9cd01034c0 2024-05-09T00:15:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.685+0000 7f9ce378bb80 1 Processor -- start 2024-05-09T00:15:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.685+0000 7f9ce378bb80 1 -- start start 2024-05-09T00:15:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.685+0000 7f9cc5d9b700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9cbc0b2880 0x7f9cbc0b4d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8000c00 tx=0x7f9cb801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.685+0000 7f9ce378bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a9fd00 0x5609a1ba8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.685+0000 7f9ce378bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a254a0 0x5609a1ba8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.685+0000 7f9ce378bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 0x5609a1ba2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.685+0000 7f9ce378bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609a1ac50a0 con 0x5609a1ba2400 2024-05-09T00:15:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.685+0000 7f9ce378bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5609a1a6a9a0 con 0x5609a1a254a0 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.685+0000 7f9ce378bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5609a1ac1060 con 0x5609a1a9fd00 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cced9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 0x5609a1ba2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cced9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 0x5609a1ba2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33192/0 (socket says 172.21.15.46:33192) 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cced9d700 1 -- 172.21.15.46:0/2833957215 learned_addr learned my addr 172.21.15.46:0/2833957215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cc5d9b700 1 --2- 172.21.15.46:0/2833957215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a254a0 0x5609a1ba8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cce59c700 1 --2- 172.21.15.46:0/2833957215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a9fd00 0x5609a1ba8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cced9d700 1 -- 172.21.15.46:0/2833957215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a9fd00 msgr2=0x5609a1ba8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cced9d700 1 --2- 172.21.15.46:0/2833957215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a9fd00 0x5609a1ba8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cced9d700 1 -- 172.21.15.46:0/2833957215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a254a0 msgr2=0x5609a1ba8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cced9d700 1 --2- 172.21.15.46:0/2833957215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a254a0 0x5609a1ba8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cced9d700 1 -- 172.21.15.46:0/2833957215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a1a5b500 con 0x5609a1ba2400 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cc5d9b700 1 --2- 172.21.15.46:0/2833957215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a254a0 0x5609a1ba8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cced9d700 1 --2- 172.21.15.46:0/2833957215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 0x5609a1ba2810 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8007de0 tx=0x7f9cc800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24113c333ef32fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:41.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cc4d99700 1 -- 172.21.15.46:0/2833957215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8015020 con 0x5609a1ba2400 2024-05-09T00:15:41.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cc4d99700 1 -- 172.21.15.46:0/2833957215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc8011900 con 0x5609a1ba2400 2024-05-09T00:15:41.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.686+0000 7f9cc4d99700 1 -- 172.21.15.46:0/2833957215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc800a070 con 0x5609a1ba2400 2024-05-09T00:15:41.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.687+0000 7f9ce378bb80 1 -- 172.21.15.46:0/2833957215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 msgr2=0x5609a1ba2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:41.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.687+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/2833957215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 0x5609a1ba2810 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8007de0 tx=0x7f9cc800c490 comp rx=0 tx=0).stop 2024-05-09T00:15:41.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.687+0000 7f9ce378bb80 1 -- 172.21.15.46:0/2833957215 shutdown_connections 2024-05-09T00:15:41.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.687+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/2833957215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 0x5609a1ba2810 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.687+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/2833957215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a9fd00 0x5609a1ba8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.687+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/2833957215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a254a0 0x5609a1ba8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.687+0000 7f9ce378bb80 1 -- 172.21.15.46:0/2833957215 >> 172.21.15.46:0/2833957215 conn(0x5609a1a28f70 msgr2=0x5609a1b9d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:41.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.687+0000 7f9ce378bb80 1 -- 172.21.15.46:0/2833957215 shutdown_connections 2024-05-09T00:15:41.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.687+0000 7f9ce378bb80 1 -- 172.21.15.46:0/2833957215 wait complete. 2024-05-09T00:15:41.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9ce378bb80 1 -- start start 2024-05-09T00:15:41.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9ce378bb80 1 Processor -- start 2024-05-09T00:15:41.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9ce378bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a254a0 0x5609a1bac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9ce378bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a9fd00 0x5609a1bac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9ce378bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 0x5609a1bacbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9ce378bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609a1ba3160 con 0x5609a1ba2400 2024-05-09T00:15:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9ce378bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5609a1a5fb20 con 0x5609a1a9fd00 2024-05-09T00:15:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9ce378bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5609a1baf020 con 0x5609a1a254a0 2024-05-09T00:15:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9ce378bb80 10 client.0 _ll_register_callbacks cb 0x5609a19366c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9cced9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 0x5609a1bacbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9cc5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a9fd00 0x5609a1bac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9cc5d9b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a9fd00 0x5609a1bac6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:34574/0 (socket says 172.21.15.46:34574) 2024-05-09T00:15:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9cce59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a254a0 0x5609a1bac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.688+0000 7f9cc5d9b700 1 -- 172.21.15.46:0/112018162 learned_addr learned my addr 172.21.15.46:0/112018162 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9cced9d700 1 -- 172.21.15.46:0/112018162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a254a0 msgr2=0x5609a1bac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:41.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9cced9d700 1 --2- 172.21.15.46:0/112018162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a254a0 0x5609a1bac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9cced9d700 1 -- 172.21.15.46:0/112018162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a9fd00 msgr2=0x5609a1bac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9cced9d700 1 --2- 172.21.15.46:0/112018162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a9fd00 0x5609a1bac6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9cced9d700 1 -- 172.21.15.46:0/112018162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a1940490 con 0x5609a1ba2400 2024-05-09T00:15:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9cced9d700 1 --2- 172.21.15.46:0/112018162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 0x5609a1bacbf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8008010 tx=0x7f9cc800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdfd628794cfc3a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9cb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9cb5ffb700 1 -- 172.21.15.46:0/112018162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc800a070 con 0x5609a1ba2400 2024-05-09T00:15:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9ce378bb80 1 -- 172.21.15.46:0/112018162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cc8011900 con 0x5609a1ba2400 2024-05-09T00:15:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9cb5ffb700 1 -- 172.21.15.46:0/112018162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9cc801a430 con 0x5609a1ba2400 2024-05-09T00:15:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9cb5ffb700 1 -- 172.21.15.46:0/112018162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9cc8015040 con 0x5609a1ba2400 2024-05-09T00:15:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9ce378bb80 10 client.15957 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.689+0000 7f9ce378bb80 1 -- 172.21.15.46:0/112018162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609a1bb9b50 con 0x5609a1ba2400 2024-05-09T00:15:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.690+0000 7f9cb5ffb700 1 -- 172.21.15.46:0/112018162 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc8011900 con 0x5609a1ba2400 2024-05-09T00:15:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.690+0000 7f9cb5ffb700 1 -- 172.21.15.46:0/112018162 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9cc8024660 con 0x5609a1ba2400 2024-05-09T00:15:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.690+0000 7f9ce378bb80 10 client.15957 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.690+0000 7f9ce378bb80 10 client.15957 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:15:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.690+0000 7f9ce378bb80 20 client.15957 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.690+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/112018162 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5609a1bbc080 0x5609a1bbe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.690+0000 7f9ce378bb80 4 client.15957 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:15:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.690+0000 7f9ce378bb80 1 -- 172.21.15.46:0/112018162 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609a1ad2a80 con 0x5609a1bbc080 2024-05-09T00:15:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.690+0000 7f9cce59c700 1 --2- 172.21.15.46:0/112018162 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5609a1bbc080 0x5609a1bbe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.691+0000 7f9cb5ffb700 10 client.15957 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:15:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.691+0000 7f9cce59c700 1 --2- 172.21.15.46:0/112018162 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5609a1bbc080 0x5609a1bbe4a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13052bc0a4e27c4e server_cookie=34b37c12145fcc8f in_seq=0 out_seq=0 2024-05-09T00:15:41.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.691+0000 7f9cb5ffb700 1 -- 172.21.15.46:0/112018162 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5609a1ad2a80 con 0x5609a1bbc080 2024-05-09T00:15:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.691+0000 7f9cb5ffb700 10 client.15957 handle_command_reply: tid=0 2024-05-09T00:15:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.691+0000 7f9ce378bb80 1 client.15957 shutdown 2024-05-09T00:15:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.692+0000 7f9ce378bb80 1 -- 172.21.15.46:0/112018162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 msgr2=0x5609a1bacbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.692+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/112018162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 0x5609a1bacbf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8008010 tx=0x7f9cc800cf20 comp rx=0 tx=0).stop 2024-05-09T00:15:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.692+0000 7f9ce378bb80 1 -- 172.21.15.46:0/112018162 shutdown_connections 2024-05-09T00:15:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.692+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/112018162 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5609a1bbc080 0x5609a1bbe4a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.692+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/112018162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1ba2400 0x5609a1bacbf0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.692+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/112018162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a254a0 0x5609a1bac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.692+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/112018162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a9fd00 0x5609a1bac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.692+0000 7f9ce378bb80 1 -- 172.21.15.46:0/112018162 >> 172.21.15.46:0/112018162 conn(0x5609a1a28f70 msgr2=0x5609a1b96ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.692+0000 7f9cb5ffb700 0 client.15957 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:15:41.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.692+0000 7f9ce378bb80 1 -- 172.21.15.46:0/112018162 shutdown_connections 2024-05-09T00:15:41.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.692+0000 7f9ce378bb80 1 -- 172.21.15.46:0/112018162 wait complete. 2024-05-09T00:15:41.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.692+0000 7f9ce378bb80 20 client.15957 trim_cache size 0 max 16384 2024-05-09T00:15:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.707+0000 7f9ce378bb80 1 Processor -- start 2024-05-09T00:15:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.707+0000 7f9ce378bb80 1 -- start start 2024-05-09T00:15:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.707+0000 7f9ce378bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a28f70 0x5609a1ba75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.707+0000 7f9ce378bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a254a0 0x5609a1b97a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.707+0000 7f9ce378bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a9fd00 0x5609a1baca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.707+0000 7f9ce378bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01b1650 con 0x5609a1a28f70 2024-05-09T00:15:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.707+0000 7f9ce378bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01aeee0 con 0x5609a1a254a0 2024-05-09T00:15:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.707+0000 7f9ce378bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01acd10 con 0x5609a1a9fd00 2024-05-09T00:15:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.707+0000 7f9cce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a28f70 0x5609a1ba75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a28f70 0x5609a1ba75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33198/0 (socket says 172.21.15.46:33198) 2024-05-09T00:15:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cce59c700 1 -- 172.21.15.46:0/4109905104 learned_addr learned my addr 172.21.15.46:0/4109905104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cc5d9b700 1 --2- 172.21.15.46:0/4109905104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a254a0 0x5609a1b97a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cced9d700 1 --2- 172.21.15.46:0/4109905104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a9fd00 0x5609a1baca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cce59c700 1 -- 172.21.15.46:0/4109905104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a9fd00 msgr2=0x5609a1baca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cce59c700 1 --2- 172.21.15.46:0/4109905104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a9fd00 0x5609a1baca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cce59c700 1 -- 172.21.15.46:0/4109905104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a254a0 msgr2=0x5609a1b97a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cce59c700 1 --2- 172.21.15.46:0/4109905104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a254a0 0x5609a1b97a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cce59c700 1 -- 172.21.15.46:0/4109905104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a1bacf70 con 0x5609a1a28f70 2024-05-09T00:15:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cce59c700 1 --2- 172.21.15.46:0/4109905104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a28f70 0x5609a1ba75d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000bcf0 tx=0x7f9cc001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1439b49f601c2b01 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cb7fff700 1 -- 172.21.15.46:0/4109905104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00c2070 con 0x5609a1a28f70 2024-05-09T00:15:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cb7fff700 1 -- 172.21.15.46:0/4109905104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc00b1eb0 con 0x5609a1a28f70 2024-05-09T00:15:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.708+0000 7f9cb7fff700 1 -- 172.21.15.46:0/4109905104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00b1c80 con 0x5609a1a28f70 2024-05-09T00:15:41.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.709+0000 7f9ce378bb80 1 -- 172.21.15.46:0/4109905104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a28f70 msgr2=0x5609a1ba75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:41.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.709+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/4109905104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a28f70 0x5609a1ba75d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000bcf0 tx=0x7f9cc001c040 comp rx=0 tx=0).stop 2024-05-09T00:15:41.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.709+0000 7f9ce378bb80 1 -- 172.21.15.46:0/4109905104 shutdown_connections 2024-05-09T00:15:41.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.709+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/4109905104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a9fd00 0x5609a1baca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.709+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/4109905104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a28f70 0x5609a1ba75d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.709+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/4109905104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a254a0 0x5609a1b97a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.709+0000 7f9ce378bb80 1 -- 172.21.15.46:0/4109905104 >> 172.21.15.46:0/4109905104 conn(0x5609a1b9d520 msgr2=0x5609a1ba13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.709+0000 7f9ce378bb80 1 -- 172.21.15.46:0/4109905104 shutdown_connections 2024-05-09T00:15:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.709+0000 7f9ce378bb80 1 -- 172.21.15.46:0/4109905104 wait complete. 2024-05-09T00:15:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.709+0000 7f9ce378bb80 1 -- start start 2024-05-09T00:15:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.709+0000 7f9ce378bb80 1 Processor -- start 2024-05-09T00:15:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9ce378bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a254a0 0x5609a1bb40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9ce378bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a28f70 0x5609a1bb45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9ce378bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a9fd00 0x5609a1ba8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9ce378bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01ad3a0 con 0x5609a1a254a0 2024-05-09T00:15:41.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9ce378bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01ae810 con 0x5609a1a9fd00 2024-05-09T00:15:41.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9ce378bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01ada70 con 0x5609a1a28f70 2024-05-09T00:15:41.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9ce378bb80 10 client.0 _ll_register_callbacks cb 0x5609a19366c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:41.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a254a0 0x5609a1bb40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a254a0 0x5609a1bb40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33200/0 (socket says 172.21.15.46:33200) 2024-05-09T00:15:41.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cce59c700 1 -- 172.21.15.46:0/3694653354 learned_addr learned my addr 172.21.15.46:0/3694653354 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:41.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cced9d700 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a9fd00 0x5609a1ba8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cc5d9b700 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a28f70 0x5609a1bb45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cce59c700 1 -- 172.21.15.46:0/3694653354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a28f70 msgr2=0x5609a1bb45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:41.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cce59c700 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a28f70 0x5609a1bb45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cce59c700 1 -- 172.21.15.46:0/3694653354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a9fd00 msgr2=0x5609a1ba8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cce59c700 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a9fd00 0x5609a1ba8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cce59c700 1 -- 172.21.15.46:0/3694653354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a1ba9380 con 0x5609a1a254a0 2024-05-09T00:15:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cc5d9b700 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a28f70 0x5609a1bb45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cce59c700 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a254a0 0x5609a1bb40a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00df500 tx=0x7f9cc00107c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11311c3f19d317b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.710+0000 7f9cb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9cb5ffb700 1 -- 172.21.15.46:0/3694653354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0088060 con 0x5609a1a254a0 2024-05-09T00:15:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9cb5ffb700 1 -- 172.21.15.46:0/3694653354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9cc00b1a20 con 0x5609a1a254a0 2024-05-09T00:15:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9ce378bb80 1 -- 172.21.15.46:0/3694653354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cc00b1eb0 con 0x5609a1a254a0 2024-05-09T00:15:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9cb5ffb700 1 -- 172.21.15.46:0/3694653354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9cc00c2070 con 0x5609a1a254a0 2024-05-09T00:15:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9ce378bb80 10 client.15963 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:41.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9ce378bb80 1 -- 172.21.15.46:0/3694653354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609a1c25c40 con 0x5609a1a254a0 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9cb5ffb700 1 -- 172.21.15.46:0/3694653354 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc00b1eb0 con 0x5609a1a254a0 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9cb5ffb700 1 -- 172.21.15.46:0/3694653354 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9cc008f020 con 0x5609a1a254a0 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9ce378bb80 10 client.15963 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9ce378bb80 10 client.15963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9ce378bb80 20 client.15963 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5609a1ba2400 0x5609a1c28360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9ce378bb80 4 client.15963 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.711+0000 7f9ce378bb80 1 -- 172.21.15.46:0/3694653354 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609a1accb20 con 0x5609a1ba2400 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.712+0000 7f9cc5d9b700 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5609a1ba2400 0x5609a1c28360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.712+0000 7f9cc5d9b700 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5609a1ba2400 0x5609a1c28360 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9b3315ca0124bae server_cookie=f88d68cc3584c1b4 in_seq=0 out_seq=0 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.712+0000 7f9cb5ffb700 10 client.15963 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.712+0000 7f9cb5ffb700 1 -- 172.21.15.46:0/3694653354 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609a1accb20 con 0x5609a1ba2400 2024-05-09T00:15:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.712+0000 7f9cb5ffb700 10 client.15963 handle_command_reply: tid=0 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.712+0000 7f9ce378bb80 1 client.15963 shutdown 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.712+0000 7f9ce378bb80 1 -- 172.21.15.46:0/3694653354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a254a0 msgr2=0x5609a1bb40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.712+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a254a0 0x5609a1bb40a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00df500 tx=0x7f9cc00107c0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9ce378bb80 1 -- 172.21.15.46:0/3694653354 shutdown_connections 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5609a1ba2400 0x5609a1c28360 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609a1a9fd00 0x5609a1ba8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609a1a254a0 0x5609a1bb40a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9ce378bb80 1 --2- 172.21.15.46:0/3694653354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5609a1a28f70 0x5609a1bb45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9cb5ffb700 0 client.15963 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9ce378bb80 1 -- 172.21.15.46:0/3694653354 >> 172.21.15.46:0/3694653354 conn(0x5609a1b9d520 msgr2=0x5609a1b97780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9ce378bb80 1 -- 172.21.15.46:0/3694653354 shutdown_connections 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9ce378bb80 1 -- 172.21.15.46:0/3694653354 wait complete. 2024-05-09T00:15:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9ce378bb80 20 client.15963 trim_cache size 0 max 16384 2024-05-09T00:15:41.722 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:15:41.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9cd4d75700 1 -- 172.21.15.46:0/3701283757 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9cbc0b2880 msgr2=0x7f9cbc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:41.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9cd4d75700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9cbc0b2880 0x7f9cbc0b4d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8000c00 tx=0x7f9cb801a040 comp rx=0 tx=0).stop 2024-05-09T00:15:41.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9cd4d75700 1 -- 172.21.15.46:0/3701283757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd01034c0 msgr2=0x7f9cd019bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:41.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9cd4d75700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd01034c0 0x7f9cd019bd60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000b530 tx=0x7f9cc000b8f0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9cd4d75700 1 -- 172.21.15.46:0/3701283757 shutdown_connections 2024-05-09T00:15:41.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9cd4d75700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9cbc0b2880 0x7f9cbc0b4d90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9cd4d75700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9cd0108740 0x7f9cd019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9cd4d75700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd01034c0 0x7f9cd019bd60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9cd4d75700 1 --2- 172.21.15.46:0/3701283757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd0107df0 0x7f9cd019c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:41.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.713+0000 7f9cd4d75700 1 -- 172.21.15.46:0/3701283757 >> 172.21.15.46:0/3701283757 conn(0x7f9cd00fedc0 msgr2=0x7f9cd01075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:41.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.714+0000 7f9cd4d75700 1 -- 172.21.15.46:0/3701283757 shutdown_connections 2024-05-09T00:15:41.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:41.714+0000 7f9cd4d75700 1 -- 172.21.15.46:0/3701283757 wait complete. 2024-05-09T00:15:42.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:15:42.793 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:42.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:42 smithi046 ceph-mon[30151]: pgmap v506: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:42.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:42 smithi169 ceph-mon[34777]: pgmap v506: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:42.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:42 smithi142 ceph-mon[34533]: pgmap v506: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:44.328 INFO:tasks.workunit.client.0.smithi046.stdout:64/600: open fd = 3 2024-05-09T00:15:44.328 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.328 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.328 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.328 INFO:tasks.workunit.client.0.smithi046.stdout:65/600: open fd = 3 2024-05-09T00:15:44.328 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:66/600: open fd = 3 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:67/600: open fd = 3 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:68/600: open fd = 3 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.329 INFO:tasks.workunit.client.0.smithi046.stdout:69/600: open fd = 3 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:70/600: open fd = 3 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:71/600: open fd = 3 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:72/600: open fd = 3 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.330 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:73/600: open fd = 3 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:74/600: open fd = 3 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:75/600: open fd = 3 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:76/600: open fd = 3 2024-05-09T00:15:44.331 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:77/600: open fd = 3 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:78/600: open fd = 3 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:79/600: open fd = 3 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.332 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:80/600: open fd = 3 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:81/600: open fd = 3 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:82/600: open fd = 3 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:83/600: open fd = 3 2024-05-09T00:15:44.333 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:84/600: open fd = 3 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:85/600: open fd = 3 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:86/600: open fd = 3 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.334 INFO:tasks.workunit.client.0.smithi046.stdout:87/600: open fd = 3 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:88/600: open fd = 3 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:89/600: open fd = 3 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:90/600: open fd = 3 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.335 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:91/600: open fd = 3 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:92/600: open fd = 3 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:93/600: open fd = 3 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:94/600: open fd = 3 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.336 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:95/600: open fd = 3 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:96/600: open fd = 3 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:97/600: open fd = 3 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:98/600: open fd = 3 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.337 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:99/600: open fd = 3 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:100/600: open fd = 3 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:101/600: open fd = 3 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:102/600: open fd = 3 2024-05-09T00:15:44.338 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:103/600: open fd = 3 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:104/600: open fd = 3 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:105/600: open fd = 3 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:106/600: open fd = 3 2024-05-09T00:15:44.339 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:107/600: open fd = 3 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:108/600: open fd = 3 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:109/600: open fd = 3 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:110/600: open fd = 3 2024-05-09T00:15:44.340 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:111/600: open fd = 3 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:112/600: open fd = 3 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:113/600: open fd = 3 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.341 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:114/600: open fd = 3 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:115/600: open fd = 3 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:116/600: open fd = 3 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:117/600: open fd = 3 2024-05-09T00:15:44.342 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:118/600: open fd = 3 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:119/600: open fd = 3 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:120/600: open fd = 3 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.343 INFO:tasks.workunit.client.0.smithi046.stdout:121/600: open fd = 3 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:122/600: open fd = 3 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:123/600: open fd = 3 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:124/600: open fd = 3 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.344 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.345 INFO:tasks.workunit.client.0.smithi046.stdout:125/600: open fd = 3 2024-05-09T00:15:44.345 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.345 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.345 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:15:44.345 INFO:tasks.workunit.client.0.smithi046.stdout:126/600: open fd = 3 2024-05-09T00:15:44.345 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:15:44.749 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:44 smithi046 ceph-mon[30151]: pgmap v507: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.761+0000 7f84e799e700 1 -- 172.21.15.46:0/2251935901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e0107e10 msgr2=0x7f84e0108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.761+0000 7f84e799e700 1 --2- 172.21.15.46:0/2251935901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e0107e10 0x7f84e0108220 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f84d00099c0 tx=0x7f84d0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.761+0000 7f84e799e700 1 -- 172.21.15.46:0/2251935901 shutdown_connections 2024-05-09T00:15:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.761+0000 7f84e799e700 1 --2- 172.21.15.46:0/2251935901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84e0108760 0x7f84e01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.761+0000 7f84e799e700 1 --2- 172.21.15.46:0/2251935901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e01034e0 0x7f84e01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.761+0000 7f84e799e700 1 --2- 172.21.15.46:0/2251935901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e0107e10 0x7f84e0108220 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.761+0000 7f84e799e700 1 -- 172.21.15.46:0/2251935901 >> 172.21.15.46:0/2251935901 conn(0x7f84e00fef30 msgr2=0x7f84e0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.761+0000 7f84e799e700 1 -- 172.21.15.46:0/2251935901 shutdown_connections 2024-05-09T00:15:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.761+0000 7f84e799e700 1 -- 172.21.15.46:0/2251935901 wait complete. 2024-05-09T00:15:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.762+0000 7f84e799e700 1 Processor -- start 2024-05-09T00:15:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.762+0000 7f84e799e700 1 -- start start 2024-05-09T00:15:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.762+0000 7f84e799e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e01034e0 0x7f84e019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.762+0000 7f84e799e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e0107e10 0x7f84e019b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e799e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84e0108760 0x7f84e019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e799e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84e0114530 con 0x7f84e0107e10 2024-05-09T00:15:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e799e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01143b0 con 0x7f84e0108760 2024-05-09T00:15:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e799e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01146b0 con 0x7f84e01034e0 2024-05-09T00:15:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e4f39700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e0107e10 0x7f84e019b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e4f39700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e0107e10 0x7f84e019b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33226/0 (socket says 172.21.15.46:33226) 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e4f39700 1 -- 172.21.15.46:0/1559092255 learned_addr learned my addr 172.21.15.46:0/1559092255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e573a700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e01034e0 0x7f84e019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e5f3b700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84e0108760 0x7f84e019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e4f39700 1 -- 172.21.15.46:0/1559092255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e01034e0 msgr2=0x7f84e019af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e4f39700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e01034e0 0x7f84e019af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e4f39700 1 -- 172.21.15.46:0/1559092255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84e0108760 msgr2=0x7f84e019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e4f39700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84e0108760 0x7f84e019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e4f39700 1 -- 172.21.15.46:0/1559092255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e019fd50 con 0x7f84e0107e10 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e4f39700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e0107e10 0x7f84e019b4c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f84d0006010 tx=0x7f84d00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84d67fc700 1 -- 172.21.15.46:0/1559092255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0026070 con 0x7f84e0107e10 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.763+0000 7f84e799e700 1 -- 172.21.15.46:0/1559092255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e019ffe0 con 0x7f84e0107e10 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.764+0000 7f84d67fc700 1 -- 172.21.15.46:0/1559092255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d0003f20 con 0x7f84e0107e10 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.764+0000 7f84e799e700 1 -- 172.21.15.46:0/1559092255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e01a7960 con 0x7f84e0107e10 2024-05-09T00:15:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.764+0000 7f84d67fc700 1 -- 172.21.15.46:0/1559092255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d000fcc0 con 0x7f84e0107e10 2024-05-09T00:15:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.766+0000 7f84d67fc700 1 -- 172.21.15.46:0/1559092255 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f84d0029070 con 0x7f84e0107e10 2024-05-09T00:15:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.767+0000 7f84d67fc700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f84cc0b2880 0x7f84cc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.767+0000 7f84e573a700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f84cc0b2880 0x7f84cc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.767+0000 7f84d67fc700 1 -- 172.21.15.46:0/1559092255 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f84d00e2cb0 con 0x7f84e0107e10 2024-05-09T00:15:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.767+0000 7f84f63b4b80 1 Processor -- start 2024-05-09T00:15:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.767+0000 7f84f63b4b80 1 -- start start 2024-05-09T00:15:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84e573a700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f84cc0b2880 0x7f84cc0b4d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f84dc004630 tx=0x7f84dc00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84f63b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 0x555d58d8cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84f63b4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 0x555d58d8d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84f63b4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58d911e0 0x555d58d915f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84f63b4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d58cab0a0 con 0x555d58c0b4a0 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84f63b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555d58c509a0 con 0x555d58c85d00 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84f63b4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555d58ca7060 con 0x555d58d911e0 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84e4f39700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 0x555d58d8d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84e4f39700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 0x555d58d8d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33242/0 (socket says 172.21.15.46:33242) 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84e4f39700 1 -- 172.21.15.46:0/1163583466 learned_addr learned my addr 172.21.15.46:0/1163583466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84e5f3b700 1 --2- 172.21.15.46:0/1163583466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58d911e0 0x555d58d915f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84e573a700 1 --2- 172.21.15.46:0/1163583466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 0x555d58d8cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84e4f39700 1 -- 172.21.15.46:0/1163583466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58d911e0 msgr2=0x555d58d915f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84e4f39700 1 --2- 172.21.15.46:0/1163583466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58d911e0 0x555d58d915f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84e4f39700 1 -- 172.21.15.46:0/1163583466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 msgr2=0x555d58d8cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84e4f39700 1 --2- 172.21.15.46:0/1163583466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 0x555d58d8cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84e4f39700 1 -- 172.21.15.46:0/1163583466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d58c41500 con 0x555d58c0b4a0 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.768+0000 7f84e4f39700 1 --2- 172.21.15.46:0/1163583466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 0x555d58d8d030 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f84d0000c00 tx=0x7f84d000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a647732ea12922 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.769+0000 7f84cb7fe700 1 -- 172.21.15.46:0/1163583466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d00aa360 con 0x555d58c0b4a0 2024-05-09T00:15:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.769+0000 7f84cb7fe700 1 -- 172.21.15.46:0/1163583466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d00ae710 con 0x555d58c0b4a0 2024-05-09T00:15:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.769+0000 7f84cb7fe700 1 -- 172.21.15.46:0/1163583466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d00b2f00 con 0x555d58c0b4a0 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.769+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1163583466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 msgr2=0x555d58d8d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.769+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/1163583466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 0x555d58d8d030 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f84d0000c00 tx=0x7f84d000f790 comp rx=0 tx=0).stop 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.770+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1163583466 shutdown_connections 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.770+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/1163583466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58d911e0 0x555d58d915f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.770+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/1163583466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 0x555d58d8cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.770+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/1163583466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 0x555d58d8d030 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.770+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1163583466 >> 172.21.15.46:0/1163583466 conn(0x555d58c0ef70 msgr2=0x555d58d83220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.770+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1163583466 shutdown_connections 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.770+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1163583466 wait complete. 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.770+0000 7f84f63b4b80 1 -- start start 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.770+0000 7f84f63b4b80 1 Processor -- start 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84f63b4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58c0b4a0 0x555d58d9bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84f63b4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c85d00 0x555d58d9c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84f63b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58d911e0 0x555d58d962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84f63b4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d58d91d50 con 0x555d58c85d00 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84f63b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555d58c45b20 con 0x555d58d911e0 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84f63b4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555d58d91ed0 con 0x555d58c0b4a0 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84f63b4b80 10 client.0 _ll_register_callbacks cb 0x555d58b1c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84e4f39700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c85d00 0x555d58d9c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84e4f39700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c85d00 0x555d58d9c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33250/0 (socket says 172.21.15.46:33250) 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84e4f39700 1 -- 172.21.15.46:0/3041019268 learned_addr learned my addr 172.21.15.46:0/3041019268 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84e5f3b700 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58d911e0 0x555d58d962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84e573a700 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58c0b4a0 0x555d58d9bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84e4f39700 1 -- 172.21.15.46:0/3041019268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58c0b4a0 msgr2=0x555d58d9bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84e4f39700 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58c0b4a0 0x555d58d9bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84e4f39700 1 -- 172.21.15.46:0/3041019268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58d911e0 msgr2=0x555d58d962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84e4f39700 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58d911e0 0x555d58d962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84e4f39700 1 -- 172.21.15.46:0/3041019268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d58b26490 con 0x555d58c85d00 2024-05-09T00:15:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84e573a700 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58c0b4a0 0x555d58d9bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84e4f39700 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c85d00 0x555d58d9c400 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f84d00aa010 tx=0x7f84d00b31b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7ef0b00fdc467b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.771+0000 7f84c8ff9700 1 -- 172.21.15.46:0/3041019268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d00c4060 con 0x555d58c85d00 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84c8ff9700 1 -- 172.21.15.46:0/3041019268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f84d00b2eb0 con 0x555d58c85d00 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3041019268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d00ae710 con 0x555d58c85d00 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84c8ff9700 1 -- 172.21.15.46:0/3041019268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84d0003a70 con 0x555d58c85d00 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84f63b4b80 10 client.15975 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3041019268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555d58d91d50 con 0x555d58c85d00 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84c8ff9700 1 -- 172.21.15.46:0/3041019268 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d00ae710 con 0x555d58c85d00 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84c8ff9700 1 -- 172.21.15.46:0/3041019268 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f84d00a5d60 con 0x555d58c85d00 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84f63b4b80 10 client.15975 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84f63b4b80 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84f63b4b80 20 client.15975 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555d58da5de0 0x555d58da8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84f63b4b80 4 client.15975 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.772+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3041019268 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d58cb8a80 con 0x555d58da5de0 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.773+0000 7f84e573a700 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555d58da5de0 0x555d58da8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.773+0000 7f84e573a700 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555d58da5de0 0x555d58da8200 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50fa5401818ef89 server_cookie=cf72cf7a375ecc8b in_seq=0 out_seq=0 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.773+0000 7f84c8ff9700 10 client.15975 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84c8ff9700 1 -- 172.21.15.46:0/3041019268 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d58cb8a80 con 0x555d58da5de0 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84c8ff9700 10 client.15975 handle_command_reply: tid=0 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84f63b4b80 1 client.15975 shutdown 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3041019268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c85d00 msgr2=0x555d58d9c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c85d00 0x555d58d9c400 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f84d00aa010 tx=0x7f84d00b31b0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3041019268 shutdown_connections 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555d58da5de0 0x555d58da8200 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58d911e0 0x555d58d962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58c0b4a0 0x555d58d9bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84c8ff9700 0 client.15975 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:44.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/3041019268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c85d00 0x555d58d9c400 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3041019268 >> 172.21.15.46:0/3041019268 conn(0x555d58c0ef70 msgr2=0x555d58d7cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:44.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3041019268 shutdown_connections 2024-05-09T00:15:44.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.774+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3041019268 wait complete. 2024-05-09T00:15:44.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.775+0000 7f84f63b4b80 20 client.15975 trim_cache size 0 max 16384 2024-05-09T00:15:44.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84f63b4b80 1 Processor -- start 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84f63b4b80 1 -- start start 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84f63b4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58c0b4a0 0x555d58d96280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84f63b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 0x555d58dad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84f63b4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58d911e0 0x555d58dade90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84f63b4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01af2c0 con 0x555d58d911e0 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84f63b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01acb50 con 0x555d58c85d00 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84f63b4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01aa9c0 con 0x555d58c0b4a0 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84e5f3b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58d911e0 0x555d58dade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84e5f3b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58d911e0 0x555d58dade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33254/0 (socket says 172.21.15.46:33254) 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84e5f3b700 1 -- 172.21.15.46:0/3289146372 learned_addr learned my addr 172.21.15.46:0/3289146372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84e573a700 1 --2- 172.21.15.46:0/3289146372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58c0b4a0 0x555d58d96280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84e4f39700 1 --2- 172.21.15.46:0/3289146372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 0x555d58dad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84e5f3b700 1 -- 172.21.15.46:0/3289146372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58c0b4a0 msgr2=0x555d58d96280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84e5f3b700 1 --2- 172.21.15.46:0/3289146372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58c0b4a0 0x555d58d96280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84e5f3b700 1 -- 172.21.15.46:0/3289146372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 msgr2=0x555d58dad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:44.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84e5f3b700 1 --2- 172.21.15.46:0/3289146372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 0x555d58dad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84e5f3b700 1 -- 172.21.15.46:0/3289146372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d58d7f9d0 con 0x555d58d911e0 2024-05-09T00:15:44.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.790+0000 7f84e4f39700 1 --2- 172.21.15.46:0/3289146372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 0x555d58dad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:15:44.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.791+0000 7f84e5f3b700 1 --2- 172.21.15.46:0/3289146372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58d911e0 0x555d58dade90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009e10 tx=0x7f84d80077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4761cf7fb0c745 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.791+0000 7f84caffd700 1 -- 172.21.15.46:0/3289146372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d801b070 con 0x555d58d911e0 2024-05-09T00:15:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.791+0000 7f84caffd700 1 -- 172.21.15.46:0/3289146372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d80178f0 con 0x555d58d911e0 2024-05-09T00:15:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.791+0000 7f84caffd700 1 -- 172.21.15.46:0/3289146372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8016840 con 0x555d58d911e0 2024-05-09T00:15:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.792+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3289146372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58d911e0 msgr2=0x555d58dade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.792+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/3289146372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58d911e0 0x555d58dade90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009e10 tx=0x7f84d80077e0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.792+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3289146372 shutdown_connections 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.792+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/3289146372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58d911e0 0x555d58dade90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.792+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/3289146372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58c0b4a0 0x555d58d96280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.792+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/3289146372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 0x555d58dad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.792+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3289146372 >> 172.21.15.46:0/3289146372 conn(0x555d58c0ef70 msgr2=0x555d58d95b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.792+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3289146372 shutdown_connections 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.792+0000 7f84f63b4b80 1 -- 172.21.15.46:0/3289146372 wait complete. 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.792+0000 7f84f63b4b80 1 -- start start 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.792+0000 7f84f63b4b80 1 Processor -- start 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84f63b4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 0x555d58da2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84f63b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 0x555d58d8eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84f63b4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58d911e0 0x555d58d8f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84f63b4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01ab010 con 0x555d58c0b4a0 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84f63b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01ac480 con 0x555d58c85d00 2024-05-09T00:15:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84f63b4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01ab6e0 con 0x555d58d911e0 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84f63b4b80 10 client.0 _ll_register_callbacks cb 0x555d58b1c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84e573a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 0x555d58da2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84e573a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 0x555d58da2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33264/0 (socket says 172.21.15.46:33264) 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84e573a700 1 -- 172.21.15.46:0/1873566450 learned_addr learned my addr 172.21.15.46:0/1873566450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84e5f3b700 1 --2- 172.21.15.46:0/1873566450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58d911e0 0x555d58d8f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84e573a700 1 -- 172.21.15.46:0/1873566450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58d911e0 msgr2=0x555d58d8f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84e573a700 1 --2- 172.21.15.46:0/1873566450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58d911e0 0x555d58d8f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84e573a700 1 -- 172.21.15.46:0/1873566450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 msgr2=0x555d58d8eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84e573a700 1 --2- 172.21.15.46:0/1873566450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 0x555d58d8eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84e573a700 1 -- 172.21.15.46:0/1873566450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d58d8f610 con 0x555d58c0b4a0 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84e573a700 1 --2- 172.21.15.46:0/1873566450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 0x555d58da2550 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x555d58d95b40 tx=0x7f84dc013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa3979169dcb56 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.793+0000 7f84c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84c8ff9700 1 -- 172.21.15.46:0/1873566450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004830 con 0x555d58c0b4a0 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84c8ff9700 1 -- 172.21.15.46:0/1873566450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f84dc0049d0 con 0x555d58c0b4a0 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1873566450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d80178f0 con 0x555d58c0b4a0 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84c8ff9700 1 -- 172.21.15.46:0/1873566450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84dc004bd0 con 0x555d58c0b4a0 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84f63b4b80 10 client.15981 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1873566450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555d58e10120 con 0x555d58c0b4a0 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84c8ff9700 1 -- 172.21.15.46:0/1873566450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d80178f0 con 0x555d58c0b4a0 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84c8ff9700 1 -- 172.21.15.46:0/1873566450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f84dc02e070 con 0x555d58c0b4a0 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84f63b4b80 10 client.15981 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84f63b4b80 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84f63b4b80 20 client.15981 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/1873566450 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555d58e10860 0x555d58e12c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84f63b4b80 4 client.15981 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.794+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1873566450 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d58cb2b20 con 0x555d58e10860 2024-05-09T00:15:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.795+0000 7f84e4f39700 1 --2- 172.21.15.46:0/1873566450 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555d58e10860 0x555d58e12c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.795+0000 7f84e4f39700 1 --2- 172.21.15.46:0/1873566450 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555d58e10860 0x555d58e12c80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a4588bdcd452f7 server_cookie=4c44008b57f0dfa7 in_seq=0 out_seq=0 2024-05-09T00:15:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.795+0000 7f84c8ff9700 10 client.15981 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.795+0000 7f84c8ff9700 1 -- 172.21.15.46:0/1873566450 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d58cb2b20 con 0x555d58e10860 2024-05-09T00:15:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.795+0000 7f84c8ff9700 10 client.15981 handle_command_reply: tid=0 2024-05-09T00:15:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.795+0000 7f84f63b4b80 1 client.15981 shutdown 2024-05-09T00:15:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.795+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1873566450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 msgr2=0x555d58da2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.795+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/1873566450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 0x555d58da2550 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x555d58d95b40 tx=0x7f84dc013070 comp rx=0 tx=0).stop 2024-05-09T00:15:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.795+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1873566450 shutdown_connections 2024-05-09T00:15:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.795+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/1873566450 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555d58e10860 0x555d58e12c80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/1873566450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555d58d911e0 0x555d58d8f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/1873566450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d58c0b4a0 0x555d58da2550 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84c8ff9700 0 client.15981 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84f63b4b80 1 --2- 172.21.15.46:0/1873566450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d58c85d00 0x555d58d8eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1873566450 >> 172.21.15.46:0/1873566450 conn(0x555d58c0ef70 msgr2=0x555d58d82b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1873566450 shutdown_connections 2024-05-09T00:15:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84f63b4b80 1 -- 172.21.15.46:0/1873566450 wait complete. 2024-05-09T00:15:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84f63b4b80 20 client.15981 trim_cache size 0 max 16384 2024-05-09T00:15:44.799 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:15:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84e799e700 1 -- 172.21.15.46:0/1559092255 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f84cc0b2880 msgr2=0x7f84cc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84e799e700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f84cc0b2880 0x7f84cc0b4d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f84dc004630 tx=0x7f84dc00b3c0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84e799e700 1 -- 172.21.15.46:0/1559092255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e0107e10 msgr2=0x7f84e019b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84e799e700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e0107e10 0x7f84e019b4c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f84d0006010 tx=0x7f84d00047b0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84e799e700 1 -- 172.21.15.46:0/1559092255 shutdown_connections 2024-05-09T00:15:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84e799e700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f84cc0b2880 0x7f84cc0b4d90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84e799e700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84e0108760 0x7f84e019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84e799e700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e01034e0 0x7f84e019af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84e799e700 1 --2- 172.21.15.46:0/1559092255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e0107e10 0x7f84e019b4c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.796+0000 7f84e799e700 1 -- 172.21.15.46:0/1559092255 >> 172.21.15.46:0/1559092255 conn(0x7f84e00fef30 msgr2=0x7f84e0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.797+0000 7f84e799e700 1 -- 172.21.15.46:0/1559092255 shutdown_connections 2024-05-09T00:15:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:44.797+0000 7f84e799e700 1 -- 172.21.15.46:0/1559092255 wait complete. 2024-05-09T00:15:44.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:44 smithi169 ceph-mon[34777]: pgmap v507: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:44.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:44 smithi142 ceph-mon[34533]: pgmap v507: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:46.691 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:15:46.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:15:46.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:46 smithi046 ceph-mon[30151]: pgmap v508: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:46.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:46 smithi169 ceph-mon[34777]: pgmap v508: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:46.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:46 smithi142 ceph-mon[34533]: pgmap v508: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:47.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:47 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:15:47.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:47 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:15:47.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:47 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:15:48.769 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:48 smithi046 ceph-mon[30151]: pgmap v509: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:48.770 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:48 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:15:48.770 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:48 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:15:48.770 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:48 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:15:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.793+0000 7f3574074700 1 -- 172.21.15.46:0/1292926479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c1034c0 msgr2=0x7f356c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.793+0000 7f3574074700 1 --2- 172.21.15.46:0/1292926479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c1034c0 0x7f356c1038d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f35680099c0 tx=0x7f3568009cd0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.793+0000 7f3574074700 1 -- 172.21.15.46:0/1292926479 shutdown_connections 2024-05-09T00:15:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.793+0000 7f3574074700 1 --2- 172.21.15.46:0/1292926479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f356c108740 0x7f356c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.793+0000 7f3574074700 1 --2- 172.21.15.46:0/1292926479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c1034c0 0x7f356c1038d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.793+0000 7f3574074700 1 --2- 172.21.15.46:0/1292926479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f356c107df0 0x7f356c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.793+0000 7f3574074700 1 -- 172.21.15.46:0/1292926479 >> 172.21.15.46:0/1292926479 conn(0x7f356c0fedc0 msgr2=0x7f356c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.793+0000 7f3574074700 1 -- 172.21.15.46:0/1292926479 shutdown_connections 2024-05-09T00:15:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.793+0000 7f3574074700 1 -- 172.21.15.46:0/1292926479 wait complete. 2024-05-09T00:15:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f3574074700 1 Processor -- start 2024-05-09T00:15:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f3574074700 1 -- start start 2024-05-09T00:15:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f3574074700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f356c1034c0 0x7f356c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f3574074700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c107df0 0x7f356c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f3574074700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f356c108740 0x7f356c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f3574074700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f356c113370 con 0x7f356c107df0 2024-05-09T00:15:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f3574074700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1131f0 con 0x7f356c108740 2024-05-09T00:15:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f3574074700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1134f0 con 0x7f356c1034c0 2024-05-09T00:15:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f357160f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c107df0 0x7f356c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f357160f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c107df0 0x7f356c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35798/0 (socket says 172.21.15.46:35798) 2024-05-09T00:15:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f357160f700 1 -- 172.21.15.46:0/2289648459 learned_addr learned my addr 172.21.15.46:0/2289648459 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f3571e10700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f356c1034c0 0x7f356c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f3572611700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f356c108740 0x7f356c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f357160f700 1 -- 172.21.15.46:0/2289648459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f356c1034c0 msgr2=0x7f356c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.794+0000 7f357160f700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f356c1034c0 0x7f356c19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.795+0000 7f357160f700 1 -- 172.21.15.46:0/2289648459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f356c108740 msgr2=0x7f356c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.795+0000 7f357160f700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f356c108740 0x7f356c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.795+0000 7f357160f700 1 -- 172.21.15.46:0/2289648459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356c19fce0 con 0x7f356c107df0 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.795+0000 7f3572611700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f356c108740 0x7f356c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.795+0000 7f357160f700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c107df0 0x7f356c19b3b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f355c00c8f0 tx=0x7f355c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.795+0000 7f3562ffd700 1 -- 172.21.15.46:0/2289648459 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c007940 con 0x7f356c107df0 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.795+0000 7f3562ffd700 1 -- 172.21.15.46:0/2289648459 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f355c00f450 con 0x7f356c107df0 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.795+0000 7f3574074700 1 -- 172.21.15.46:0/2289648459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356c19ffd0 con 0x7f356c107df0 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.795+0000 7f3562ffd700 1 -- 172.21.15.46:0/2289648459 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c018660 con 0x7f356c107df0 2024-05-09T00:15:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.795+0000 7f3574074700 1 -- 172.21.15.46:0/2289648459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356c1a78f0 con 0x7f356c107df0 2024-05-09T00:15:48.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.798+0000 7f3562ffd700 1 -- 172.21.15.46:0/2289648459 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f355c018800 con 0x7f356c107df0 2024-05-09T00:15:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3562ffd700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f35580b2900 0x7f35580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3582a8ab80 1 Processor -- start 2024-05-09T00:15:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3571e10700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f35580b2900 0x7f35580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3562ffd700 1 -- 172.21.15.46:0/2289648459 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f355c0e20c0 con 0x7f356c107df0 2024-05-09T00:15:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3582a8ab80 1 -- start start 2024-05-09T00:15:48.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3582a8ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb904d80 0x55aabba0d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3582a8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba0d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3582a8ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabba07480 0x55aabba07890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3582a8ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55aabb92a120 con 0x55aabba07480 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3582a8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55aabb8cfa20 con 0x55aabb88a520 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3582a8ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55aabb9260e0 con 0x55aabb904d80 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3571e10700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f35580b2900 0x7f35580b4e10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3568009990 tx=0x7f35680036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3572611700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabba07480 0x55aabba07890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3572611700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabba07480 0x55aabba07890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35804/0 (socket says 172.21.15.46:35804) 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.799+0000 7f3572611700 1 -- 172.21.15.46:0/3437173861 learned_addr learned my addr 172.21.15.46:0/3437173861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.800+0000 7f357160f700 1 --2- 172.21.15.46:0/3437173861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba0d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.800+0000 7f3571e10700 1 --2- 172.21.15.46:0/3437173861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb904d80 0x55aabba0d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.800+0000 7f3572611700 1 -- 172.21.15.46:0/3437173861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb904d80 msgr2=0x55aabba0d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.800+0000 7f3572611700 1 --2- 172.21.15.46:0/3437173861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb904d80 0x55aabba0d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.800+0000 7f3572611700 1 -- 172.21.15.46:0/3437173861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 msgr2=0x55aabba0d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.800+0000 7f3572611700 1 --2- 172.21.15.46:0/3437173861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba0d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.800+0000 7f3572611700 1 -- 172.21.15.46:0/3437173861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabb8c0580 con 0x55aabba07480 2024-05-09T00:15:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.800+0000 7f3572611700 1 --2- 172.21.15.46:0/3437173861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabba07480 0x55aabba07890 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3564009dd0 tx=0x7f35640077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8e9855032df4ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.800+0000 7f3557fff700 1 -- 172.21.15.46:0/3437173861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564004bb0 con 0x55aabba07480 2024-05-09T00:15:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.800+0000 7f3557fff700 1 -- 172.21.15.46:0/3437173861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3564004d50 con 0x55aabba07480 2024-05-09T00:15:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.800+0000 7f3557fff700 1 -- 172.21.15.46:0/3437173861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564014670 con 0x55aabba07480 2024-05-09T00:15:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.801+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3437173861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabba07480 msgr2=0x55aabba07890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.801+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/3437173861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabba07480 0x55aabba07890 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3564009dd0 tx=0x7f35640077e0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.801+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3437173861 shutdown_connections 2024-05-09T00:15:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.801+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/3437173861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabba07480 0x55aabba07890 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.801+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/3437173861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb904d80 0x55aabba0d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.801+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/3437173861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba0d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.801+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3437173861 >> 172.21.15.46:0/3437173861 conn(0x55aabb88dff0 msgr2=0x55aabba022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.801+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3437173861 shutdown_connections 2024-05-09T00:15:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.801+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3437173861 wait complete. 2024-05-09T00:15:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.802+0000 7f3582a8ab80 1 -- start start 2024-05-09T00:15:48.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.802+0000 7f3582a8ab80 1 Processor -- start 2024-05-09T00:15:48.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.802+0000 7f3582a8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba111f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.802+0000 7f3582a8ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 0x55aabba11730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.802+0000 7f3582a8ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabba07480 0x55aabba11c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.802+0000 7f3582a8ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55aabba081e0 con 0x55aabb904d80 2024-05-09T00:15:48.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.802+0000 7f3582a8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55aabb8c4ba0 con 0x55aabb88a520 2024-05-09T00:15:48.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.802+0000 7f3582a8ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55aabba140a0 con 0x55aabba07480 2024-05-09T00:15:48.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.802+0000 7f3582a8ab80 10 client.0 _ll_register_callbacks cb 0x55aabb79b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:48.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.802+0000 7f357160f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 0x55aabba11730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f357160f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 0x55aabba11730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35806/0 (socket says 172.21.15.46:35806) 2024-05-09T00:15:48.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f357160f700 1 -- 172.21.15.46:0/3108566847 learned_addr learned my addr 172.21.15.46:0/3108566847 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:48.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f3571e10700 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba111f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f3572611700 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabba07480 0x55aabba11c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f357160f700 1 -- 172.21.15.46:0/3108566847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabba07480 msgr2=0x55aabba11c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:48.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f357160f700 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabba07480 0x55aabba11c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f357160f700 1 -- 172.21.15.46:0/3108566847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 msgr2=0x55aabba111f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:48.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f357160f700 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba111f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f357160f700 1 -- 172.21.15.46:0/3108566847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aabb7a5510 con 0x55aabb904d80 2024-05-09T00:15:48.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f357160f700 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 0x55aabba11730 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f355c0a93a0 tx=0x7f355c0a9290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8042b1c334407a79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f35557fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f35557fa700 1 -- 172.21.15.46:0/3108566847 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c0c2070 con 0x55aabb904d80 2024-05-09T00:15:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f35557fa700 1 -- 172.21.15.46:0/3108566847 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f355c017360 con 0x55aabb904d80 2024-05-09T00:15:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3108566847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3564004d50 con 0x55aabb904d80 2024-05-09T00:15:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f35557fa700 1 -- 172.21.15.46:0/3108566847 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f355c017560 con 0x55aabb904d80 2024-05-09T00:15:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f3582a8ab80 10 client.15993 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.803+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3108566847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aabba1ec30 con 0x55aabb904d80 2024-05-09T00:15:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.804+0000 7f35557fa700 1 -- 172.21.15.46:0/3108566847 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3564004d50 con 0x55aabb904d80 2024-05-09T00:15:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.804+0000 7f35557fa700 1 -- 172.21.15.46:0/3108566847 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f355c0b2070 con 0x55aabb904d80 2024-05-09T00:15:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.804+0000 7f3582a8ab80 10 client.15993 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.804+0000 7f3582a8ab80 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.804+0000 7f3582a8ab80 20 client.15993 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.804+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aabba21270 0x55aabba23690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.804+0000 7f3582a8ab80 4 client.15993 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.804+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3108566847 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aabb937b00 con 0x55aabba21270 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.804+0000 7f3571e10700 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aabba21270 0x55aabba23690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.805+0000 7f35557fa700 10 client.15993 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.805+0000 7f3571e10700 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aabba21270 0x55aabba23690 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23675de720ef0163 server_cookie=4ca7a90ff3fe77b in_seq=0 out_seq=0 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.805+0000 7f35557fa700 1 -- 172.21.15.46:0/3108566847 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aabb937b00 con 0x55aabba21270 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.805+0000 7f35557fa700 10 client.15993 handle_command_reply: tid=0 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.805+0000 7f3582a8ab80 1 client.15993 shutdown 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.805+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3108566847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 msgr2=0x55aabba11730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.805+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 0x55aabba11730 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f355c0a93a0 tx=0x7f355c0a9290 comp rx=0 tx=0).stop 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.806+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3108566847 shutdown_connections 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.806+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aabba21270 0x55aabba23690 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.806+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabba07480 0x55aabba11c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.806+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.806+0000 7f35557fa700 0 client.15993 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.806+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/3108566847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 0x55aabba11730 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.806+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3108566847 >> 172.21.15.46:0/3108566847 conn(0x55aabb88dff0 msgr2=0x55aabb9fc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.806+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3108566847 shutdown_connections 2024-05-09T00:15:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.806+0000 7f3582a8ab80 1 -- 172.21.15.46:0/3108566847 wait complete. 2024-05-09T00:15:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.806+0000 7f3582a8ab80 20 client.15993 trim_cache size 0 max 16384 2024-05-09T00:15:48.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.826+0000 7f3582a8ab80 1 Processor -- start 2024-05-09T00:15:48.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.826+0000 7f3582a8ab80 1 -- start start 2024-05-09T00:15:48.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.826+0000 7f3582a8ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb88dff0 0x55aabb9fcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3582a8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba10150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3582a8ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 0x55aabba06b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3582a8ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1b1580 con 0x55aabb904d80 2024-05-09T00:15:48.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3582a8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1aee10 con 0x55aabb88a520 2024-05-09T00:15:48.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3582a8ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1acc40 con 0x55aabb88dff0 2024-05-09T00:15:48.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3572611700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 0x55aabba06b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3572611700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 0x55aabba06b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35820/0 (socket says 172.21.15.46:35820) 2024-05-09T00:15:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3572611700 1 -- 172.21.15.46:0/2697622505 learned_addr learned my addr 172.21.15.46:0/2697622505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f357160f700 1 --2- 172.21.15.46:0/2697622505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba10150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3571e10700 1 --2- 172.21.15.46:0/2697622505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb88dff0 0x55aabb9fcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3572611700 1 -- 172.21.15.46:0/2697622505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb88dff0 msgr2=0x55aabb9fcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3572611700 1 --2- 172.21.15.46:0/2697622505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb88dff0 0x55aabb9fcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3572611700 1 -- 172.21.15.46:0/2697622505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 msgr2=0x55aabba10150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3572611700 1 --2- 172.21.15.46:0/2697622505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba10150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3572611700 1 -- 172.21.15.46:0/2697622505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabba0f760 con 0x55aabb904d80 2024-05-09T00:15:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f3572611700 1 --2- 172.21.15.46:0/2697622505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 0x55aabba06b80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f35640058a0 tx=0x7f3564005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b5036c9b1742ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f35577fe700 1 -- 172.21.15.46:0/2697622505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564006000 con 0x55aabb904d80 2024-05-09T00:15:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.827+0000 7f35577fe700 1 -- 172.21.15.46:0/2697622505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3564023030 con 0x55aabb904d80 2024-05-09T00:15:48.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.828+0000 7f35577fe700 1 -- 172.21.15.46:0/2697622505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356401f670 con 0x55aabb904d80 2024-05-09T00:15:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.828+0000 7f3582a8ab80 1 -- 172.21.15.46:0/2697622505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 msgr2=0x55aabba06b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.828+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/2697622505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 0x55aabba06b80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f35640058a0 tx=0x7f3564005b00 comp rx=0 tx=0).stop 2024-05-09T00:15:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.828+0000 7f3582a8ab80 1 -- 172.21.15.46:0/2697622505 shutdown_connections 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.828+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/2697622505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb904d80 0x55aabba06b80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.828+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/2697622505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb88dff0 0x55aabb9fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.828+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/2697622505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba10150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.828+0000 7f3582a8ab80 1 -- 172.21.15.46:0/2697622505 >> 172.21.15.46:0/2697622505 conn(0x55aabba025a0 msgr2=0x55aabba06470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.828+0000 7f3582a8ab80 1 -- 172.21.15.46:0/2697622505 shutdown_connections 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.828+0000 7f3582a8ab80 1 -- 172.21.15.46:0/2697622505 wait complete. 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.829+0000 7f3582a8ab80 1 -- start start 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.829+0000 7f3582a8ab80 1 Processor -- start 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.829+0000 7f3582a8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba19850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.829+0000 7f3582a8ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb88dff0 0x55aabba16100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.829+0000 7f3582a8ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb904d80 0x55aabba16680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.829+0000 7f3582a8ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1ad2d0 con 0x55aabb88dff0 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.829+0000 7f3582a8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1ae740 con 0x55aabb88a520 2024-05-09T00:15:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.829+0000 7f3582a8ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1ad9a0 con 0x55aabb904d80 2024-05-09T00:15:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.829+0000 7f3582a8ab80 10 client.0 _ll_register_callbacks cb 0x55aabb79b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.829+0000 7f357160f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb88dff0 0x55aabba16100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.829+0000 7f357160f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb88dff0 0x55aabba16100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35830/0 (socket says 172.21.15.46:35830) 2024-05-09T00:15:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.829+0000 7f357160f700 1 -- 172.21.15.46:0/1587933353 learned_addr learned my addr 172.21.15.46:0/1587933353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f3571e10700 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba19850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f3572611700 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb904d80 0x55aabba16680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f357160f700 1 -- 172.21.15.46:0/1587933353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb904d80 msgr2=0x55aabba16680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f357160f700 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb904d80 0x55aabba16680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f357160f700 1 -- 172.21.15.46:0/1587933353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 msgr2=0x55aabba19850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f357160f700 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba19850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f357160f700 1 -- 172.21.15.46:0/1587933353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aabba16e20 con 0x55aabb88dff0 2024-05-09T00:15:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f3572611700 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb904d80 0x55aabba16680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f3571e10700 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba19850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f357160f700 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb88dff0 0x55aabba16100 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f355c017700 tx=0x7f355c0c3f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a3f5d828360411 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f35557fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f35557fa700 1 -- 172.21.15.46:0/1587933353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c0c2070 con 0x55aabb88dff0 2024-05-09T00:15:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f3582a8ab80 1 -- 172.21.15.46:0/1587933353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3564023030 con 0x55aabb88dff0 2024-05-09T00:15:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f35557fa700 1 -- 172.21.15.46:0/1587933353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f355c0a5c90 con 0x55aabb88dff0 2024-05-09T00:15:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f35557fa700 1 -- 172.21.15.46:0/1587933353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f355c0a9900 con 0x55aabb88dff0 2024-05-09T00:15:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f3582a8ab80 10 client.15999 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.830+0000 7f3582a8ab80 1 -- 172.21.15.46:0/1587933353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aabba8c9e0 con 0x55aabb88dff0 2024-05-09T00:15:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.831+0000 7f35557fa700 1 -- 172.21.15.46:0/1587933353 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3564023030 con 0x55aabb88dff0 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.831+0000 7f35557fa700 1 -- 172.21.15.46:0/1587933353 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f355c007ae0 con 0x55aabb88dff0 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.831+0000 7f3582a8ab80 10 client.15999 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.831+0000 7f3582a8ab80 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.831+0000 7f3582a8ab80 20 client.15999 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.831+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aabba07480 0x55aabba8eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.831+0000 7f3582a8ab80 4 client.15999 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.831+0000 7f3582a8ab80 1 -- 172.21.15.46:0/1587933353 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aabb931ba0 con 0x55aabba07480 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.831+0000 7f3571e10700 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aabba07480 0x55aabba8eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.831+0000 7f35557fa700 10 client.15999 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.831+0000 7f3571e10700 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aabba07480 0x55aabba8eeb0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=314ecdac14072d98 server_cookie=cb9ce32fa39e5b93 in_seq=0 out_seq=0 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.832+0000 7f35557fa700 1 -- 172.21.15.46:0/1587933353 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aabb931ba0 con 0x55aabba07480 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.832+0000 7f35557fa700 10 client.15999 handle_command_reply: tid=0 2024-05-09T00:15:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.832+0000 7f3582a8ab80 1 client.15999 shutdown 2024-05-09T00:15:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.833+0000 7f3582a8ab80 1 -- 172.21.15.46:0/1587933353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb88dff0 msgr2=0x55aabba16100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.833+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb88dff0 0x55aabba16100 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f355c017700 tx=0x7f355c0c3f90 comp rx=0 tx=0).stop 2024-05-09T00:15:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.833+0000 7f3582a8ab80 1 -- 172.21.15.46:0/1587933353 shutdown_connections 2024-05-09T00:15:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.833+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55aabba07480 0x55aabba8eeb0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.833+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55aabb904d80 0x55aabba16680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.833+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aabb88a520 0x55aabba19850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.833+0000 7f35557fa700 0 client.15999 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.833+0000 7f3582a8ab80 1 --2- 172.21.15.46:0/1587933353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aabb88dff0 0x55aabba16100 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.833+0000 7f3582a8ab80 1 -- 172.21.15.46:0/1587933353 >> 172.21.15.46:0/1587933353 conn(0x55aabba025a0 msgr2=0x55aabb9fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.833+0000 7f3582a8ab80 1 -- 172.21.15.46:0/1587933353 shutdown_connections 2024-05-09T00:15:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.833+0000 7f3582a8ab80 1 -- 172.21.15.46:0/1587933353 wait complete. 2024-05-09T00:15:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.833+0000 7f3582a8ab80 20 client.15999 trim_cache size 0 max 16384 2024-05-09T00:15:48.837 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:15:48.837 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:15:48.837 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "42d56421-f258-4a7d-b559-c93c5b3494af", 2024-05-09T00:15:48.837 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:15:48.837 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:15:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.834+0000 7f3574074700 1 -- 172.21.15.46:0/2289648459 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f35580b2900 msgr2=0x7f35580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.834+0000 7f3574074700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f35580b2900 0x7f35580b4e10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3568009990 tx=0x7f35680036e0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.834+0000 7f3574074700 1 -- 172.21.15.46:0/2289648459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c107df0 msgr2=0x7f356c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.834+0000 7f3574074700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c107df0 0x7f356c19b3b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f355c00c8f0 tx=0x7f355c00cc00 comp rx=0 tx=0).stop 2024-05-09T00:15:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.834+0000 7f3574074700 1 -- 172.21.15.46:0/2289648459 shutdown_connections 2024-05-09T00:15:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.834+0000 7f3574074700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f35580b2900 0x7f35580b4e10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.834+0000 7f3574074700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f356c108740 0x7f356c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.834+0000 7f3574074700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f356c1034c0 0x7f356c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.834+0000 7f3574074700 1 --2- 172.21.15.46:0/2289648459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c107df0 0x7f356c19b3b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.834+0000 7f3574074700 1 -- 172.21.15.46:0/2289648459 >> 172.21.15.46:0/2289648459 conn(0x7f356c0fedc0 msgr2=0x7f356c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.834+0000 7f3574074700 1 -- 172.21.15.46:0/2289648459 shutdown_connections 2024-05-09T00:15:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:48.834+0000 7f3574074700 1 -- 172.21.15.46:0/2289648459 wait complete. 2024-05-09T00:15:48.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:48 smithi169 ceph-mon[34777]: pgmap v509: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:48.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:48 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:15:48.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:48 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:15:48.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:48 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:15:48.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:48 smithi142 ceph-mon[34533]: pgmap v509: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:48.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:48 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:15:48.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:48 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:15:48.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:48 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:15:49.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:49 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:15:49.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:49 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:15:49.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:49 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:15:49.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:49 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:15:49.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:15:49.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:49 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:15:49.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:49 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:15:49.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:49 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:15:49.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:49 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:15:49.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:49 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:15:49.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:49 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:15:49.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:49 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:15:49.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:49 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:15:49.942 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:15:50.768 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:50 smithi046 ceph-mon[30151]: pgmap v510: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:50.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:50 smithi169 ceph-mon[34777]: pgmap v510: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:50.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:50 smithi142 ceph-mon[34533]: pgmap v510: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.804+0000 7f842b54a700 1 -- 172.21.15.46:0/1147334388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8424103270 msgr2=0x7f8424103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.804+0000 7f842b54a700 1 --2- 172.21.15.46:0/1147334388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8424103270 0x7f8424103680 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8420009990 tx=0x7f8420009ca0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.804+0000 7f842b54a700 1 -- 172.21.15.46:0/1147334388 shutdown_connections 2024-05-09T00:15:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.804+0000 7f842b54a700 1 --2- 172.21.15.46:0/1147334388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84241086c0 0x7f8424108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.804+0000 7f842b54a700 1 --2- 172.21.15.46:0/1147334388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8424103270 0x7f8424103680 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.804+0000 7f842b54a700 1 --2- 172.21.15.46:0/1147334388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8424103bc0 0x7f84241080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.804+0000 7f842b54a700 1 -- 172.21.15.46:0/1147334388 >> 172.21.15.46:0/1147334388 conn(0x7f84240fef30 msgr2=0x7f8424101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.804+0000 7f842b54a700 1 -- 172.21.15.46:0/1147334388 shutdown_connections 2024-05-09T00:15:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.804+0000 7f842b54a700 1 -- 172.21.15.46:0/1147334388 wait complete. 2024-05-09T00:15:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f842b54a700 1 Processor -- start 2024-05-09T00:15:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f842b54a700 1 -- start start 2024-05-09T00:15:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f842b54a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8424103270 0x7f84241a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f842b54a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8424103bc0 0x7f84241a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f842b54a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84241086c0 0x7f84241a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f842b54a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f842411ab40 con 0x7f8424103270 2024-05-09T00:15:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f842b54a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f842411a9c0 con 0x7f84241086c0 2024-05-09T00:15:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f842b54a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f842411acc0 con 0x7f8424103bc0 2024-05-09T00:15:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f84292e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8424103270 0x7f84241a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f84292e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8424103270 0x7f84241a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35852/0 (socket says 172.21.15.46:35852) 2024-05-09T00:15:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f84292e6700 1 -- 172.21.15.46:0/2859818478 learned_addr learned my addr 172.21.15.46:0/2859818478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f8428ae5700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8424103bc0 0x7f84241a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.805+0000 7f8429ae7700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84241086c0 0x7f84241a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.806+0000 7f84292e6700 1 -- 172.21.15.46:0/2859818478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8424103bc0 msgr2=0x7f84241a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.806+0000 7f84292e6700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8424103bc0 0x7f84241a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.806+0000 7f84292e6700 1 -- 172.21.15.46:0/2859818478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84241086c0 msgr2=0x7f84241a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.806+0000 7f84292e6700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84241086c0 0x7f84241a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.806+0000 7f84292e6700 1 -- 172.21.15.46:0/2859818478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84241a6440 con 0x7f8424103270 2024-05-09T00:15:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.806+0000 7f8428ae5700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8424103bc0 0x7f84241a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.806+0000 7f84292e6700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8424103270 0x7f84241a1570 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8420000c00 tx=0x7f84200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.806+0000 7f841a7fc700 1 -- 172.21.15.46:0/2859818478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420027070 con 0x7f8424103270 2024-05-09T00:15:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.806+0000 7f841a7fc700 1 -- 172.21.15.46:0/2859818478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84200040e0 con 0x7f8424103270 2024-05-09T00:15:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.806+0000 7f842b54a700 1 -- 172.21.15.46:0/2859818478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84241a66d0 con 0x7f8424103270 2024-05-09T00:15:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.806+0000 7f841a7fc700 1 -- 172.21.15.46:0/2859818478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842000f460 con 0x7f8424103270 2024-05-09T00:15:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.806+0000 7f842b54a700 1 -- 172.21.15.46:0/2859818478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84241ae050 con 0x7f8424103270 2024-05-09T00:15:51.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.809+0000 7f841a7fc700 1 -- 172.21.15.46:0/2859818478 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f842002a070 con 0x7f8424103270 2024-05-09T00:15:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.810+0000 7f841a7fc700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f84100b2900 0x7f84100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.810+0000 7f8439f60b80 1 Processor -- start 2024-05-09T00:15:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.810+0000 7f8439f60b80 1 -- start start 2024-05-09T00:15:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.810+0000 7f841a7fc700 1 -- 172.21.15.46:0/2859818478 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f84200e31e0 con 0x7f8424103270 2024-05-09T00:15:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.810+0000 7f8428ae5700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f84100b2900 0x7f84100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8439f60b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b37dd00 0x562c6b486240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8439f60b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b3034a0 0x562c6b486780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8439f60b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b480400 0x562c6b480810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8439f60b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562c6b3a30a0 con 0x562c6b480400 2024-05-09T00:15:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8439f60b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562c6b3489a0 con 0x562c6b37dd00 2024-05-09T00:15:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8439f60b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562c6b39f060 con 0x562c6b3034a0 2024-05-09T00:15:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8429ae7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b480400 0x562c6b480810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8429ae7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b480400 0x562c6b480810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35864/0 (socket says 172.21.15.46:35864) 2024-05-09T00:15:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8429ae7700 1 -- 172.21.15.46:0/3705888720 learned_addr learned my addr 172.21.15.46:0/3705888720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8428ae5700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f84100b2900 0x7f84100b4e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8424103a80 tx=0x7f8414008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8428ae5700 1 --2- 172.21.15.46:0/3705888720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b3034a0 0x562c6b486780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f84292e6700 1 --2- 172.21.15.46:0/3705888720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b37dd00 0x562c6b486240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8429ae7700 1 -- 172.21.15.46:0/3705888720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b3034a0 msgr2=0x562c6b486780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8429ae7700 1 --2- 172.21.15.46:0/3705888720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b3034a0 0x562c6b486780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8429ae7700 1 -- 172.21.15.46:0/3705888720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b37dd00 msgr2=0x562c6b486240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8429ae7700 1 --2- 172.21.15.46:0/3705888720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b37dd00 0x562c6b486240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8429ae7700 1 -- 172.21.15.46:0/3705888720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c6b339500 con 0x562c6b480400 2024-05-09T00:15:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8428ae5700 1 --2- 172.21.15.46:0/3705888720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b3034a0 0x562c6b486780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.811+0000 7f8429ae7700 1 --2- 172.21.15.46:0/3705888720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b480400 0x562c6b480810 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f841c00a430 tx=0x7f841c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4990ba3753214af4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.812+0000 7f840f7fe700 1 -- 172.21.15.46:0/3705888720 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c013590 con 0x562c6b480400 2024-05-09T00:15:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.812+0000 7f840f7fe700 1 -- 172.21.15.46:0/3705888720 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f841c019070 con 0x562c6b480400 2024-05-09T00:15:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.812+0000 7f840f7fe700 1 -- 172.21.15.46:0/3705888720 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c014ab0 con 0x562c6b480400 2024-05-09T00:15:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.812+0000 7f8439f60b80 1 -- 172.21.15.46:0/3705888720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b480400 msgr2=0x562c6b480810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.812+0000 7f8439f60b80 1 --2- 172.21.15.46:0/3705888720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b480400 0x562c6b480810 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f841c00a430 tx=0x7f841c009650 comp rx=0 tx=0).stop 2024-05-09T00:15:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.813+0000 7f8439f60b80 1 -- 172.21.15.46:0/3705888720 shutdown_connections 2024-05-09T00:15:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.813+0000 7f8439f60b80 1 --2- 172.21.15.46:0/3705888720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b480400 0x562c6b480810 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.813+0000 7f8439f60b80 1 --2- 172.21.15.46:0/3705888720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b37dd00 0x562c6b486240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.813+0000 7f8439f60b80 1 --2- 172.21.15.46:0/3705888720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b3034a0 0x562c6b486780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.813+0000 7f8439f60b80 1 -- 172.21.15.46:0/3705888720 >> 172.21.15.46:0/3705888720 conn(0x562c6b306f70 msgr2=0x562c6b47b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.813+0000 7f8439f60b80 1 -- 172.21.15.46:0/3705888720 shutdown_connections 2024-05-09T00:15:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.813+0000 7f8439f60b80 1 -- 172.21.15.46:0/3705888720 wait complete. 2024-05-09T00:15:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.813+0000 7f8439f60b80 1 -- start start 2024-05-09T00:15:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.813+0000 7f8439f60b80 1 Processor -- start 2024-05-09T00:15:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8439f60b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b3034a0 0x562c6b48a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8439f60b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b48a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8439f60b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b480400 0x562c6b48aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8439f60b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562c6b481160 con 0x562c6b37dd00 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8439f60b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562c6b33db20 con 0x562c6b480400 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8439f60b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562c6b48d020 con 0x562c6b3034a0 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8439f60b80 10 client.0 _ll_register_callbacks cb 0x562c6b2146c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8428ae5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b48a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8428ae5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b48a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35878/0 (socket says 172.21.15.46:35878) 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8428ae5700 1 -- 172.21.15.46:0/2635553880 learned_addr learned my addr 172.21.15.46:0/2635553880 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8429ae7700 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b480400 0x562c6b48aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f84292e6700 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b3034a0 0x562c6b48a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8428ae5700 1 -- 172.21.15.46:0/2635553880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b3034a0 msgr2=0x562c6b48a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8428ae5700 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b3034a0 0x562c6b48a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8428ae5700 1 -- 172.21.15.46:0/2635553880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b480400 msgr2=0x562c6b48aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8428ae5700 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b480400 0x562c6b48aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8428ae5700 1 -- 172.21.15.46:0/2635553880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6b21e490 con 0x562c6b37dd00 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8429ae7700 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b480400 0x562c6b48aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f84292e6700 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b3034a0 0x562c6b48a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f8428ae5700 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b48a580 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f841400de60 tx=0x7f8414004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ca8b79efec6c68 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.814+0000 7f840cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f840cff9700 1 -- 172.21.15.46:0/2635553880 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414025060 con 0x562c6b37dd00 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f840cff9700 1 -- 172.21.15.46:0/2635553880 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f841400f040 con 0x562c6b37dd00 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f8439f60b80 1 -- 172.21.15.46:0/2635553880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f841c019070 con 0x562c6b37dd00 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f840cff9700 1 -- 172.21.15.46:0/2635553880 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8414014e60 con 0x562c6b37dd00 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f8439f60b80 10 client.16011 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f8439f60b80 1 -- 172.21.15.46:0/2635553880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c6b497ab0 con 0x562c6b37dd00 2024-05-09T00:15:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f840cff9700 1 -- 172.21.15.46:0/2635553880 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f841c019070 con 0x562c6b37dd00 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f840cff9700 1 -- 172.21.15.46:0/2635553880 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f841402a070 con 0x562c6b37dd00 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f8439f60b80 10 client.16011 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f8439f60b80 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f8439f60b80 20 client.16011 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f8439f60b80 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562c6b49a140 0x562c6b49c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f8439f60b80 4 client.16011 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.815+0000 7f8439f60b80 1 -- 172.21.15.46:0/2635553880 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c6b3b0a80 con 0x562c6b49a140 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.816+0000 7f84292e6700 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562c6b49a140 0x562c6b49c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.816+0000 7f84292e6700 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562c6b49a140 0x562c6b49c5c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44de50762b70f159 server_cookie=350535287db5d0e4 in_seq=0 out_seq=0 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.816+0000 7f840cff9700 10 client.16011 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.817+0000 7f840cff9700 1 -- 172.21.15.46:0/2635553880 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562c6b3b0a80 con 0x562c6b49a140 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.817+0000 7f840cff9700 10 client.16011 handle_command_reply: tid=0 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.817+0000 7f8439f60b80 1 client.16011 shutdown 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.817+0000 7f8439f60b80 1 -- 172.21.15.46:0/2635553880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 msgr2=0x562c6b48a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.817+0000 7f8439f60b80 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b48a580 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f841400de60 tx=0x7f8414004750 comp rx=0 tx=0).stop 2024-05-09T00:15:51.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.817+0000 7f8439f60b80 1 -- 172.21.15.46:0/2635553880 shutdown_connections 2024-05-09T00:15:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.817+0000 7f8439f60b80 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562c6b49a140 0x562c6b49c5c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.817+0000 7f8439f60b80 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b480400 0x562c6b48aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.817+0000 7f8439f60b80 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b3034a0 0x562c6b48a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.817+0000 7f8439f60b80 1 --2- 172.21.15.46:0/2635553880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b48a580 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.817+0000 7f8439f60b80 1 -- 172.21.15.46:0/2635553880 >> 172.21.15.46:0/2635553880 conn(0x562c6b306f70 msgr2=0x562c6b474ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.817+0000 7f840cff9700 0 client.16011 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.818+0000 7f8439f60b80 1 -- 172.21.15.46:0/2635553880 shutdown_connections 2024-05-09T00:15:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.818+0000 7f8439f60b80 1 -- 172.21.15.46:0/2635553880 wait complete. 2024-05-09T00:15:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.818+0000 7f8439f60b80 20 client.16011 trim_cache size 0 max 16384 2024-05-09T00:15:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f8439f60b80 1 Processor -- start 2024-05-09T00:15:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f8439f60b80 1 -- start start 2024-05-09T00:15:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f8439f60b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b3034a0 0x562c6b488ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f8439f60b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b475700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f8439f60b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b480400 0x562c6b495e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f8439f60b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84241b7a80 con 0x562c6b37dd00 2024-05-09T00:15:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f8439f60b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84241b5310 con 0x562c6b3034a0 2024-05-09T00:15:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f8439f60b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84241b3160 con 0x562c6b480400 2024-05-09T00:15:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f8428ae5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b475700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f8428ae5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b475700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35886/0 (socket says 172.21.15.46:35886) 2024-05-09T00:15:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f8428ae5700 1 -- 172.21.15.46:0/2418362958 learned_addr learned my addr 172.21.15.46:0/2418362958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f8429ae7700 1 --2- 172.21.15.46:0/2418362958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b480400 0x562c6b495e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.839+0000 7f84292e6700 1 --2- 172.21.15.46:0/2418362958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b3034a0 0x562c6b488ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.840+0000 7f8428ae5700 1 -- 172.21.15.46:0/2418362958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b480400 msgr2=0x562c6b495e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.840+0000 7f8428ae5700 1 --2- 172.21.15.46:0/2418362958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b480400 0x562c6b495e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.840+0000 7f8428ae5700 1 -- 172.21.15.46:0/2418362958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b3034a0 msgr2=0x562c6b488ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:51.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.840+0000 7f8428ae5700 1 --2- 172.21.15.46:0/2418362958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b3034a0 0x562c6b488ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.840+0000 7f8428ae5700 1 -- 172.21.15.46:0/2418362958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c6b4779d0 con 0x562c6b37dd00 2024-05-09T00:15:51.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.840+0000 7f8429ae7700 1 --2- 172.21.15.46:0/2418362958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b480400 0x562c6b495e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:51.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.840+0000 7f84292e6700 1 --2- 172.21.15.46:0/2418362958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b3034a0 0x562c6b488ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:15:51.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.840+0000 7f8428ae5700 1 --2- 172.21.15.46:0/2418362958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b475700 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8414016010 tx=0x7f841400df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a95ca4935d22dcc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:51.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.840+0000 7f840effd700 1 -- 172.21.15.46:0/2418362958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414030030 con 0x562c6b37dd00 2024-05-09T00:15:51.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.840+0000 7f840effd700 1 -- 172.21.15.46:0/2418362958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8414014aa0 con 0x562c6b37dd00 2024-05-09T00:15:51.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.840+0000 7f840effd700 1 -- 172.21.15.46:0/2418362958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414006040 con 0x562c6b37dd00 2024-05-09T00:15:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.841+0000 7f8439f60b80 1 -- 172.21.15.46:0/2418362958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 msgr2=0x562c6b475700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.841+0000 7f8439f60b80 1 --2- 172.21.15.46:0/2418362958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b475700 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8414016010 tx=0x7f841400df40 comp rx=0 tx=0).stop 2024-05-09T00:15:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.841+0000 7f8439f60b80 1 -- 172.21.15.46:0/2418362958 shutdown_connections 2024-05-09T00:15:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.841+0000 7f8439f60b80 1 --2- 172.21.15.46:0/2418362958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b480400 0x562c6b495e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.841+0000 7f8439f60b80 1 --2- 172.21.15.46:0/2418362958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b3034a0 0x562c6b488ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.841+0000 7f8439f60b80 1 --2- 172.21.15.46:0/2418362958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b475700 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.841+0000 7f8439f60b80 1 -- 172.21.15.46:0/2418362958 >> 172.21.15.46:0/2418362958 conn(0x562c6b306f70 msgr2=0x562c6b474ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.841+0000 7f8439f60b80 1 -- 172.21.15.46:0/2418362958 shutdown_connections 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.841+0000 7f8439f60b80 1 -- 172.21.15.46:0/2418362958 wait complete. 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.841+0000 7f8439f60b80 1 -- start start 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.841+0000 7f8439f60b80 1 Processor -- start 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8439f60b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b3034a0 0x562c6b492760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8439f60b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b4a1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8439f60b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b480400 0x562c6b4a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8439f60b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84241b37d0 con 0x562c6b37dd00 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8439f60b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84241b4c40 con 0x562c6b3034a0 2024-05-09T00:15:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8439f60b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84241b3ea0 con 0x562c6b480400 2024-05-09T00:15:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8439f60b80 10 client.0 _ll_register_callbacks cb 0x562c6b480dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:15:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8428ae5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b4a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8428ae5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b4a1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35896/0 (socket says 172.21.15.46:35896) 2024-05-09T00:15:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8428ae5700 1 -- 172.21.15.46:0/3811539429 learned_addr learned my addr 172.21.15.46:0/3811539429 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:15:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f84292e6700 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b3034a0 0x562c6b492760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8429ae7700 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b480400 0x562c6b4a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8428ae5700 1 -- 172.21.15.46:0/3811539429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b480400 msgr2=0x562c6b4a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8428ae5700 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b480400 0x562c6b4a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8428ae5700 1 -- 172.21.15.46:0/3811539429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b3034a0 msgr2=0x562c6b492760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:15:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8428ae5700 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b3034a0 0x562c6b492760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.842+0000 7f8428ae5700 1 -- 172.21.15.46:0/3811539429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6b4a1fd0 con 0x562c6b37dd00 2024-05-09T00:15:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.843+0000 7f8428ae5700 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b4a1390 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8414005d30 tx=0x7f8414005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e60e95d7fcf6d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.843+0000 7f840cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.843+0000 7f840cff9700 1 -- 172.21.15.46:0/3811539429 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414030040 con 0x562c6b37dd00 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.843+0000 7f840cff9700 1 -- 172.21.15.46:0/3811539429 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8414012760 con 0x562c6b37dd00 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.843+0000 7f8439f60b80 1 -- 172.21.15.46:0/3811539429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8414014aa0 con 0x562c6b37dd00 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.843+0000 7f840cff9700 1 -- 172.21.15.46:0/3811539429 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8414044050 con 0x562c6b37dd00 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.843+0000 7f8439f60b80 10 client.16017 fetch_fsmap learned FSMap version 26 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.843+0000 7f8439f60b80 1 -- 172.21.15.46:0/3811539429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c6b504830 con 0x562c6b37dd00 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.843+0000 7f840cff9700 1 -- 172.21.15.46:0/3811539429 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8414014aa0 con 0x562c6b37dd00 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.844+0000 7f840cff9700 1 -- 172.21.15.46:0/3811539429 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f841402a040 con 0x562c6b37dd00 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.844+0000 7f8439f60b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.844+0000 7f8439f60b80 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.844+0000 7f8439f60b80 20 client.16017 populate_metadata read hostname 'smithi046' 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.844+0000 7f8439f60b80 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562c6b504e50 0x562c6b507270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:15:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.844+0000 7f8439f60b80 4 client.16017 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.844+0000 7f8439f60b80 1 -- 172.21.15.46:0/3811539429 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562c6b3aab20 con 0x562c6b504e50 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.844+0000 7f84292e6700 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562c6b504e50 0x562c6b507270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.844+0000 7f84292e6700 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562c6b504e50 0x562c6b507270 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70195f78e3323f35 server_cookie=e9dc6f1066914e4a in_seq=0 out_seq=0 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.844+0000 7f840cff9700 10 client.16017 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f840cff9700 1 -- 172.21.15.46:0/3811539429 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562c6b3aab20 con 0x562c6b504e50 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f840cff9700 10 client.16017 handle_command_reply: tid=0 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f8439f60b80 1 client.16017 shutdown 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f8439f60b80 1 -- 172.21.15.46:0/3811539429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 msgr2=0x562c6b4a1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f8439f60b80 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b4a1390 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8414005d30 tx=0x7f8414005d60 comp rx=0 tx=0).stop 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f8439f60b80 1 -- 172.21.15.46:0/3811539429 shutdown_connections 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f8439f60b80 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562c6b504e50 0x562c6b507270 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f840cff9700 0 client.16017 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f8439f60b80 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562c6b480400 0x562c6b4a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f8439f60b80 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c6b3034a0 0x562c6b492760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f8439f60b80 1 --2- 172.21.15.46:0/3811539429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562c6b37dd00 0x562c6b4a1390 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f8439f60b80 1 -- 172.21.15.46:0/3811539429 >> 172.21.15.46:0/3811539429 conn(0x562c6b306f70 msgr2=0x562c6b48b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f8439f60b80 1 -- 172.21.15.46:0/3811539429 shutdown_connections 2024-05-09T00:15:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f8439f60b80 1 -- 172.21.15.46:0/3811539429 wait complete. 2024-05-09T00:15:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.845+0000 7f8439f60b80 20 client.16017 trim_cache size 0 max 16384 2024-05-09T00:15:51.850 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:15:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:15:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T00:15:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "42d56421-f258-4a7d-b559-c93c5b3494af": { 2024-05-09T00:15:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T00:15:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "42d56421-f258-4a7d-b559-c93c5b3494af", 2024-05-09T00:15:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T00:15:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:15:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:15:51.851 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:15:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.846+0000 7f842b54a700 1 -- 172.21.15.46:0/2859818478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f84100b2900 msgr2=0x7f84100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.846+0000 7f842b54a700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f84100b2900 0x7f84100b4e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8424103a80 tx=0x7f8414008040 comp rx=0 tx=0).stop 2024-05-09T00:15:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.846+0000 7f842b54a700 1 -- 172.21.15.46:0/2859818478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8424103270 msgr2=0x7f84241a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:15:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.846+0000 7f842b54a700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8424103270 0x7f84241a1570 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8420000c00 tx=0x7f84200047b0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.846+0000 7f842b54a700 1 -- 172.21.15.46:0/2859818478 shutdown_connections 2024-05-09T00:15:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.846+0000 7f842b54a700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f84100b2900 0x7f84100b4e10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.846+0000 7f842b54a700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84241086c0 0x7f84241a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.846+0000 7f842b54a700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8424103270 0x7f84241a1570 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.846+0000 7f842b54a700 1 --2- 172.21.15.46:0/2859818478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8424103bc0 0x7f84241a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:15:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.846+0000 7f842b54a700 1 -- 172.21.15.46:0/2859818478 >> 172.21.15.46:0/2859818478 conn(0x7f84240fef30 msgr2=0x7f84241060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:15:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.846+0000 7f842b54a700 1 -- 172.21.15.46:0/2859818478 shutdown_connections 2024-05-09T00:15:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:15:51.846+0000 7f842b54a700 1 -- 172.21.15.46:0/2859818478 wait complete. 2024-05-09T00:15:52.662 INFO:tasks.cephfs.filesystem:scrub status for tag:42d56421-f258-4a7d-b559-c93c5b3494af - {'path': '/', 'tag': '42d56421-f258-4a7d-b559-c93c5b3494af', 'options': 'recursive,force'} 2024-05-09T00:15:52.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:52 smithi046 ceph-mon[30151]: pgmap v511: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:52.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:52 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:15:52.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:52 smithi169 ceph-mon[34777]: pgmap v511: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:52.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:52 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:15:52.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:52 smithi142 ceph-mon[34533]: pgmap v511: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:15:52.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:52 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:15:54.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:54 smithi046 ceph-mon[30151]: pgmap v512: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:54.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:54 smithi169 ceph-mon[34777]: pgmap v512: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:54.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:54 smithi142 ceph-mon[34533]: pgmap v512: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:15:56.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:56 smithi046 ceph-mon[30151]: pgmap v513: 97 pgs: 97 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:56.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:56 smithi169 ceph-mon[34777]: pgmap v513: 97 pgs: 97 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:56.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:56 smithi142 ceph-mon[34533]: pgmap v513: 97 pgs: 97 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:15:58.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:15:58 smithi046 ceph-mon[30151]: pgmap v514: 97 pgs: 97 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:58.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:15:58 smithi169 ceph-mon[34777]: pgmap v514: 97 pgs: 97 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:15:58.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:15:58 smithi142 ceph-mon[34533]: pgmap v514: 97 pgs: 97 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:00.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:00 smithi046 ceph-mon[30151]: pgmap v515: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:16:00.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:00 smithi169 ceph-mon[34777]: pgmap v515: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:16:00.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:00 smithi142 ceph-mon[34533]: pgmap v515: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:16:02.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:02 smithi046 ceph-mon[30151]: pgmap v516: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:02.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:02 smithi169 ceph-mon[34777]: pgmap v516: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:02.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:02 smithi142 ceph-mon[34533]: pgmap v516: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:04.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:04 smithi046 ceph-mon[30151]: pgmap v517: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:04.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:04 smithi169 ceph-mon[34777]: pgmap v517: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:04.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:04 smithi142 ceph-mon[34533]: pgmap v517: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:06.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:06 smithi046 ceph-mon[30151]: pgmap v518: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:16:06.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:06 smithi169 ceph-mon[34777]: pgmap v518: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:16:06.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:06 smithi142 ceph-mon[34533]: pgmap v518: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:16:08.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:08 smithi046 ceph-mon[30151]: pgmap v519: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:08.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:08 smithi169 ceph-mon[34777]: pgmap v519: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:08.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:08 smithi142 ceph-mon[34533]: pgmap v519: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:10.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:10 smithi046 ceph-mon[30151]: pgmap v520: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:10.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:10 smithi169 ceph-mon[34777]: pgmap v520: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:10.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:10 smithi142 ceph-mon[34533]: pgmap v520: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:12.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:12 smithi046 ceph-mon[30151]: pgmap v521: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:12.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:12 smithi169 ceph-mon[34777]: pgmap v521: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:12.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:12 smithi142 ceph-mon[34533]: pgmap v521: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:14.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:14 smithi046 ceph-mon[30151]: pgmap v522: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:14.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:14 smithi169 ceph-mon[34777]: pgmap v522: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:14.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:14 smithi142 ceph-mon[34533]: pgmap v522: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:16.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:16 smithi046 ceph-mon[30151]: pgmap v523: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:16.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:16 smithi169 ceph-mon[34777]: pgmap v523: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:16.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:16 smithi142 ceph-mon[34533]: pgmap v523: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:18.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:18 smithi046 ceph-mon[30151]: pgmap v524: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:18.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:18 smithi169 ceph-mon[34777]: pgmap v524: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:18.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:18 smithi142 ceph-mon[34533]: pgmap v524: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:20.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:20 smithi169 ceph-mon[34777]: pgmap v525: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:20.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:20 smithi142 ceph-mon[34533]: pgmap v525: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:21.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:20 smithi046 ceph-mon[30151]: pgmap v525: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:22.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:16:22.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:22 smithi046 ceph-mon[30151]: pgmap v526: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:22.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:22 smithi169 ceph-mon[34777]: pgmap v526: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:16:22.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:22 smithi142 ceph-mon[34533]: pgmap v526: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:24.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:24 smithi169 ceph-mon[34777]: pgmap v527: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:24.885 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:24 smithi046 ceph-mon[30151]: pgmap v527: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:24.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:24 smithi142 ceph-mon[34533]: pgmap v527: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.008+0000 7fcac2328700 1 -- 172.21.15.46:0/3608650319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcabc108f30 msgr2=0x7fcabc109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.008+0000 7fcac2328700 1 --2- 172.21.15.46:0/3608650319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcabc108f30 0x7fcabc109340 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40099c0 tx=0x7fcaa4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.008+0000 7fcac2328700 1 -- 172.21.15.46:0/3608650319 shutdown_connections 2024-05-09T00:16:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.008+0000 7fcac2328700 1 --2- 172.21.15.46:0/3608650319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcabc109880 0x7fcabc1111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.008+0000 7fcac2328700 1 --2- 172.21.15.46:0/3608650319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcabc104650 0x7fcabc104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.008+0000 7fcac2328700 1 --2- 172.21.15.46:0/3608650319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcabc108f30 0x7fcabc109340 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.008+0000 7fcac2328700 1 -- 172.21.15.46:0/3608650319 >> 172.21.15.46:0/3608650319 conn(0x7fcabc079f00 msgr2=0x7fcabc07a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.008+0000 7fcac2328700 1 -- 172.21.15.46:0/3608650319 shutdown_connections 2024-05-09T00:16:25.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.008+0000 7fcac2328700 1 -- 172.21.15.46:0/3608650319 wait complete. 2024-05-09T00:16:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcac2328700 1 Processor -- start 2024-05-09T00:16:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcac2328700 1 -- start start 2024-05-09T00:16:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcac2328700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcabc104650 0x7fcabc075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcac2328700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcabc108f30 0x7fcabc076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcac2328700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcabc109880 0x7fcabc076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcac2328700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc114480 con 0x7fcabc104650 2024-05-09T00:16:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcac2328700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc114300 con 0x7fcabc109880 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcac2328700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc114600 con 0x7fcabc108f30 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcabbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcabc104650 0x7fcabc075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcabbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcabc104650 0x7fcabc075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51694/0 (socket says 172.21.15.46:51694) 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcabbfff700 1 -- 172.21.15.46:0/3148197705 learned_addr learned my addr 172.21.15.46:0/3148197705 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcac08c5700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcabc109880 0x7fcabc076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcab3fff700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcabc108f30 0x7fcabc076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcabbfff700 1 -- 172.21.15.46:0/3148197705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcabc108f30 msgr2=0x7fcabc076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.009+0000 7fcabbfff700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcabc108f30 0x7fcabc076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.010+0000 7fcabbfff700 1 -- 172.21.15.46:0/3148197705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcabc109880 msgr2=0x7fcabc076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.010+0000 7fcabbfff700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcabc109880 0x7fcabc076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.010+0000 7fcabbfff700 1 -- 172.21.15.46:0/3148197705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc1a9140 con 0x7fcabc104650 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.010+0000 7fcac08c5700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcabc109880 0x7fcabc076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.010+0000 7fcabbfff700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcabc104650 0x7fcabc075ac0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c9b0 tx=0x7fcaac00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.010+0000 7fcab9ffb700 1 -- 172.21.15.46:0/3148197705 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac007b00 con 0x7fcabc104650 2024-05-09T00:16:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.010+0000 7fcab9ffb700 1 -- 172.21.15.46:0/3148197705 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaac007ca0 con 0x7fcabc104650 2024-05-09T00:16:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.010+0000 7fcac2328700 1 -- 172.21.15.46:0/3148197705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc1a9430 con 0x7fcabc104650 2024-05-09T00:16:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.010+0000 7fcab9ffb700 1 -- 172.21.15.46:0/3148197705 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac018730 con 0x7fcabc104650 2024-05-09T00:16:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.010+0000 7fcac2328700 1 -- 172.21.15.46:0/3148197705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1a9990 con 0x7fcabc104650 2024-05-09T00:16:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.013+0000 7fcab9ffb700 1 -- 172.21.15.46:0/3148197705 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcaac00f450 con 0x7fcabc104650 2024-05-09T00:16:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.013+0000 7fcab9ffb700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcaa80b2900 0x7fcaa80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcab3fff700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcaa80b2900 0x7fcaa80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcab9ffb700 1 -- 172.21.15.46:0/3148197705 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fcaac0e2270 con 0x7fcabc104650 2024-05-09T00:16:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcad0d3eb80 1 Processor -- start 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcad0d3eb80 1 -- start start 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcab3fff700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcaa80b2900 0x7fcaa80b4e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4000c00 tx=0x7fcaa401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcad0d3eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d9597b9160 0x55d9598524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcad0d3eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d959852a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcad0d3eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d95984c6a0 0x55d95984cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcad0d3eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d95976eff0 con 0x55d9596cf4a0 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcad0d3eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d95970d7f0 con 0x55d95984c6a0 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcad0d3eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d95976afb0 con 0x55d9597b9160 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d959852a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d959852a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51704/0 (socket says 172.21.15.46:51704) 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.014+0000 7fcab3fff700 1 -- 172.21.15.46:0/3457629709 learned_addr learned my addr 172.21.15.46:0/3457629709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.015+0000 7fcabbfff700 1 --2- 172.21.15.46:0/3457629709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d9597b9160 0x55d9598524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.015+0000 7fcac08c5700 1 --2- 172.21.15.46:0/3457629709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d95984c6a0 0x55d95984cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.015+0000 7fcab3fff700 1 -- 172.21.15.46:0/3457629709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d9597b9160 msgr2=0x55d9598524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.015+0000 7fcab3fff700 1 --2- 172.21.15.46:0/3457629709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d9597b9160 0x55d9598524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.015+0000 7fcab3fff700 1 -- 172.21.15.46:0/3457629709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d95984c6a0 msgr2=0x55d95984cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.015+0000 7fcab3fff700 1 --2- 172.21.15.46:0/3457629709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d95984c6a0 0x55d95984cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.015+0000 7fcab3fff700 1 -- 172.21.15.46:0/3457629709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d959705370 con 0x55d9596cf4a0 2024-05-09T00:16:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.015+0000 7fcab3fff700 1 --2- 172.21.15.46:0/3457629709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d959852a30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40099c0 tx=0x7fcaa401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7707799b3b66827 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.015+0000 7fcab27fc700 1 -- 172.21.15.46:0/3457629709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa400fac0 con 0x55d9596cf4a0 2024-05-09T00:16:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.015+0000 7fcab27fc700 1 -- 172.21.15.46:0/3457629709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa4031070 con 0x55d9596cf4a0 2024-05-09T00:16:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.015+0000 7fcab27fc700 1 -- 172.21.15.46:0/3457629709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4022d90 con 0x55d9596cf4a0 2024-05-09T00:16:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.016+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3457629709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 msgr2=0x55d959852a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.016+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3457629709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d959852a30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40099c0 tx=0x7fcaa401d000 comp rx=0 tx=0).stop 2024-05-09T00:16:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.016+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3457629709 shutdown_connections 2024-05-09T00:16:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.016+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3457629709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d95984c6a0 0x55d95984cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.016+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3457629709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d9597b9160 0x55d9598524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.016+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3457629709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d959852a30 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.016+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3457629709 >> 172.21.15.46:0/3457629709 conn(0x55d95972f6a0 msgr2=0x55d959847400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.016+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3457629709 shutdown_connections 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.016+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3457629709 wait complete. 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.017+0000 7fcad0d3eb80 1 -- start start 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.017+0000 7fcad0d3eb80 1 Processor -- start 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.017+0000 7fcad0d3eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d9596cf4a0 0x55d959856230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.017+0000 7fcad0d3eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9597b9160 0x55d959856770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.017+0000 7fcad0d3eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d95984c6a0 0x55d959856cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.017+0000 7fcad0d3eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d9595eaba0 con 0x55d9597b9160 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.017+0000 7fcad0d3eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d959729150 con 0x55d95984c6a0 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.017+0000 7fcad0d3eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d9598591d0 con 0x55d9596cf4a0 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.017+0000 7fcad0d3eb80 10 client.0 _ll_register_callbacks cb 0x55d9595e06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.017+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9597b9160 0x55d959856770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.017+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9597b9160 0x55d959856770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51720/0 (socket says 172.21.15.46:51720) 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.017+0000 7fcab3fff700 1 -- 172.21.15.46:0/2074435158 learned_addr learned my addr 172.21.15.46:0/2074435158 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fcabbfff700 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d9596cf4a0 0x55d959856230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fcac08c5700 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d95984c6a0 0x55d959856cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fcab3fff700 1 -- 172.21.15.46:0/2074435158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d9596cf4a0 msgr2=0x55d959856230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fcab3fff700 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d9596cf4a0 0x55d959856230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fcab3fff700 1 -- 172.21.15.46:0/2074435158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d95984c6a0 msgr2=0x55d959856cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fcab3fff700 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d95984c6a0 0x55d959856cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fcab3fff700 1 -- 172.21.15.46:0/2074435158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9595ea490 con 0x55d9597b9160 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fcac08c5700 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d95984c6a0 0x55d959856cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fcab3fff700 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9597b9160 0x55d959856770 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009350 tx=0x7fcaa4022d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9399f4ed72d8b819 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fca9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fca9b7fe700 1 -- 172.21.15.46:0/2074435158 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4039070 con 0x55d9597b9160 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fca9b7fe700 1 -- 172.21.15.46:0/2074435158 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fcaa400fcd0 con 0x55d9597b9160 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/2074435158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa4031070 con 0x55d9597b9160 2024-05-09T00:16:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fca9b7fe700 1 -- 172.21.15.46:0/2074435158 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcaa400be90 con 0x55d9597b9160 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fcad0d3eb80 10 client.16029 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.018+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/2074435158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d959863b80 con 0x55d9597b9160 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.019+0000 7fca9b7fe700 1 -- 172.21.15.46:0/2074435158 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa4031070 con 0x55d9597b9160 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.019+0000 7fca9b7fe700 1 -- 172.21.15.46:0/2074435158 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcaa403d400 con 0x55d9597b9160 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.019+0000 7fcad0d3eb80 10 client.16029 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.019+0000 7fcad0d3eb80 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.019+0000 7fcad0d3eb80 20 client.16029 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.019+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d959866050 0x55d959868470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.019+0000 7fcad0d3eb80 4 client.16029 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.019+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/2074435158 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d95977c9d0 con 0x55d959866050 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.019+0000 7fcabbfff700 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d959866050 0x55d959868470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.020+0000 7fca9b7fe700 10 client.16029 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.020+0000 7fcabbfff700 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d959866050 0x55d959868470 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1653d9a95d8f07 server_cookie=ad2713d48c7437ba in_seq=0 out_seq=0 2024-05-09T00:16:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fca9b7fe700 1 -- 172.21.15.46:0/2074435158 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d95977c9d0 con 0x55d959866050 2024-05-09T00:16:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fca9b7fe700 10 client.16029 handle_command_reply: tid=0 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fcad0d3eb80 1 client.16029 shutdown 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/2074435158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9597b9160 msgr2=0x55d959856770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9597b9160 0x55d959856770 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009350 tx=0x7fcaa4022d10 comp rx=0 tx=0).stop 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/2074435158 shutdown_connections 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d959866050 0x55d959868470 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d95984c6a0 0x55d959856cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d9596cf4a0 0x55d959856230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/2074435158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9597b9160 0x55d959856770 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fca9b7fe700 0 client.16029 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/2074435158 >> 172.21.15.46:0/2074435158 conn(0x55d95972f6a0 msgr2=0x55d9598471c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/2074435158 shutdown_connections 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/2074435158 wait complete. 2024-05-09T00:16:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.021+0000 7fcad0d3eb80 20 client.16029 trim_cache size 0 max 16384 2024-05-09T00:16:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcad0d3eb80 1 Processor -- start 2024-05-09T00:16:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcad0d3eb80 1 -- start start 2024-05-09T00:16:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcad0d3eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d959862220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcad0d3eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9597b9160 0x55d959850510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcad0d3eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d95984c6a0 0x55d9598567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcad0d3eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1b3540 con 0x55d9596cf4a0 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcad0d3eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1b0dd0 con 0x55d9597b9160 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcad0d3eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1aec20 con 0x55d95984c6a0 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcabbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d959862220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcabbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d959862220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51730/0 (socket says 172.21.15.46:51730) 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcabbfff700 1 -- 172.21.15.46:0/3053902182 learned_addr learned my addr 172.21.15.46:0/3053902182 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcac08c5700 1 --2- 172.21.15.46:0/3053902182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d95984c6a0 0x55d9598567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcabbfff700 1 -- 172.21.15.46:0/3053902182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d95984c6a0 msgr2=0x55d9598567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcab3fff700 1 --2- 172.21.15.46:0/3053902182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9597b9160 0x55d959850510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcabbfff700 1 --2- 172.21.15.46:0/3053902182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d95984c6a0 0x55d9598567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcabbfff700 1 -- 172.21.15.46:0/3053902182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9597b9160 msgr2=0x55d959850510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcabbfff700 1 --2- 172.21.15.46:0/3053902182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9597b9160 0x55d959850510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcabbfff700 1 -- 172.21.15.46:0/3053902182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d959855200 con 0x55d9596cf4a0 2024-05-09T00:16:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcac08c5700 1 --2- 172.21.15.46:0/3053902182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d95984c6a0 0x55d9598567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:16:25.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcab3fff700 1 --2- 172.21.15.46:0/3053902182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9597b9160 0x55d959850510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:16:25.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.042+0000 7fcabbfff700 1 --2- 172.21.15.46:0/3053902182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d959862220 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00afd0 tx=0x7fcaac0b2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cc444c3400f6c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:25.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.043+0000 7fcab1ffb700 1 -- 172.21.15.46:0/3053902182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac0c2030 con 0x55d9596cf4a0 2024-05-09T00:16:25.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.043+0000 7fcab1ffb700 1 -- 172.21.15.46:0/3053902182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaac0c5030 con 0x55d9596cf4a0 2024-05-09T00:16:25.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.043+0000 7fcab1ffb700 1 -- 172.21.15.46:0/3053902182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00f990 con 0x55d9596cf4a0 2024-05-09T00:16:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.043+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3053902182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 msgr2=0x55d959862220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.043+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3053902182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d959862220 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00afd0 tx=0x7fcaac0b2040 comp rx=0 tx=0).stop 2024-05-09T00:16:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.043+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3053902182 shutdown_connections 2024-05-09T00:16:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.043+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3053902182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d95984c6a0 0x55d9598567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.043+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3053902182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d959862220 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.043+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3053902182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9597b9160 0x55d959850510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.043+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3053902182 >> 172.21.15.46:0/3053902182 conn(0x55d95972f6a0 msgr2=0x55d95984fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.044+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3053902182 shutdown_connections 2024-05-09T00:16:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.044+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3053902182 wait complete. 2024-05-09T00:16:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.044+0000 7fcad0d3eb80 1 -- start start 2024-05-09T00:16:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.044+0000 7fcad0d3eb80 1 Processor -- start 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcad0d3eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d9598697d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcad0d3eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9597b9160 0x55d959869d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcad0d3eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d95984c6a0 0x55d9598cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcad0d3eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1af290 con 0x55d9596cf4a0 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcad0d3eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1b0700 con 0x55d9597b9160 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcad0d3eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1af960 con 0x55d95984c6a0 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcad0d3eb80 10 client.0 _ll_register_callbacks cb 0x55d9595e06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcabbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d9598697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcabbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d9598697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51744/0 (socket says 172.21.15.46:51744) 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcabbfff700 1 -- 172.21.15.46:0/3968861531 learned_addr learned my addr 172.21.15.46:0/3968861531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcab3fff700 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9597b9160 0x55d959869d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcac08c5700 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d95984c6a0 0x55d9598cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcabbfff700 1 -- 172.21.15.46:0/3968861531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d95984c6a0 msgr2=0x55d9598cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcabbfff700 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d95984c6a0 0x55d9598cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcabbfff700 1 -- 172.21.15.46:0/3968861531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9597b9160 msgr2=0x55d959869d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:25.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcabbfff700 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9597b9160 0x55d959869d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcabbfff700 1 -- 172.21.15.46:0/3968861531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9598cfae0 con 0x55d9596cf4a0 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcabbfff700 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d9598697d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0a58c0 tx=0x7fcaac0a9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e9ea4aca8593ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fca9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fca9b7fe700 1 -- 172.21.15.46:0/3968861531 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00f740 con 0x55d9596cf4a0 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fca9b7fe700 1 -- 172.21.15.46:0/3968861531 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fcaac00f990 con 0x55d9596cf4a0 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.045+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3968861531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaac0c5030 con 0x55d9596cf4a0 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.046+0000 7fca9b7fe700 1 -- 172.21.15.46:0/3968861531 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcaac0a5b20 con 0x55d9596cf4a0 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.046+0000 7fcad0d3eb80 10 client.16035 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.046+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3968861531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9598d0be0 con 0x55d9596cf4a0 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.046+0000 7fca9b7fe700 1 -- 172.21.15.46:0/3968861531 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaac0c5030 con 0x55d9596cf4a0 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.046+0000 7fca9b7fe700 1 -- 172.21.15.46:0/3968861531 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcaac0bece0 con 0x55d9596cf4a0 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.046+0000 7fcad0d3eb80 10 client.16035 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.046+0000 7fcad0d3eb80 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.046+0000 7fcad0d3eb80 20 client.16035 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.046+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d9598d1230 0x55d9598d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.046+0000 7fcad0d3eb80 4 client.16035 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.046+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3968861531 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d959776a70 con 0x55d9598d1230 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.046+0000 7fcab3fff700 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d9598d1230 0x55d9598d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fca9b7fe700 10 client.16035 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fcab3fff700 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d9598d1230 0x55d9598d3650 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87cb46d446038dd server_cookie=6a0e8f151b6ed01a in_seq=0 out_seq=0 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fca9b7fe700 1 -- 172.21.15.46:0/3968861531 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d959776a70 con 0x55d9598d1230 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fca9b7fe700 10 client.16035 handle_command_reply: tid=0 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fcad0d3eb80 1 client.16035 shutdown 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3968861531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 msgr2=0x55d9598697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d9598697d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0a58c0 tx=0x7fcaac0a9500 comp rx=0 tx=0).stop 2024-05-09T00:16:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3968861531 shutdown_connections 2024-05-09T00:16:25.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d9598d1230 0x55d9598d3650 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d95984c6a0 0x55d9598cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9596cf4a0 0x55d9598697d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fcad0d3eb80 1 --2- 172.21.15.46:0/3968861531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9597b9160 0x55d959869d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3968861531 >> 172.21.15.46:0/3968861531 conn(0x55d95972f6a0 msgr2=0x55d95984fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:25.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.047+0000 7fca9b7fe700 0 client.16035 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:25.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3968861531 shutdown_connections 2024-05-09T00:16:25.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcad0d3eb80 1 -- 172.21.15.46:0/3968861531 wait complete. 2024-05-09T00:16:25.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcad0d3eb80 20 client.16035 trim_cache size 0 max 16384 2024-05-09T00:16:25.052 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:16:25.053 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:16:25.053 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:16:25.053 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:16:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcac2328700 1 -- 172.21.15.46:0/3148197705 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcaa80b2900 msgr2=0x7fcaa80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcac2328700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcaa80b2900 0x7fcaa80b4e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4000c00 tx=0x7fcaa401a040 comp rx=0 tx=0).stop 2024-05-09T00:16:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcac2328700 1 -- 172.21.15.46:0/3148197705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcabc104650 msgr2=0x7fcabc075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcac2328700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcabc104650 0x7fcabc075ac0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c9b0 tx=0x7fcaac00cd70 comp rx=0 tx=0).stop 2024-05-09T00:16:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcac2328700 1 -- 172.21.15.46:0/3148197705 shutdown_connections 2024-05-09T00:16:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcac2328700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcaa80b2900 0x7fcaa80b4e10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcac2328700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcabc109880 0x7fcabc076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcac2328700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcabc104650 0x7fcabc075ac0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcac2328700 1 --2- 172.21.15.46:0/3148197705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcabc108f30 0x7fcabc076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcac2328700 1 -- 172.21.15.46:0/3148197705 >> 172.21.15.46:0/3148197705 conn(0x7fcabc079f00 msgr2=0x7fcabc102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcac2328700 1 -- 172.21.15.46:0/3148197705 shutdown_connections 2024-05-09T00:16:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:25.048+0000 7fcac2328700 1 -- 172.21.15.46:0/3148197705 wait complete. 2024-05-09T00:16:25.794 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:16:25.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:16:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:16:26.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:26 smithi046 ceph-mon[30151]: pgmap v528: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:16:26.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:26 smithi169 ceph-mon[34777]: pgmap v528: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:16:26.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:26 smithi142 ceph-mon[34533]: pgmap v528: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:16:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.955+0000 7f7f93cba700 1 -- 172.21.15.46:0/595892290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f8c107e10 msgr2=0x7f7f8c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.955+0000 7f7f93cba700 1 --2- 172.21.15.46:0/595892290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f8c107e10 0x7f7f8c108220 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0099c0 tx=0x7f7f7c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:16:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.955+0000 7f7f93cba700 1 -- 172.21.15.46:0/595892290 shutdown_connections 2024-05-09T00:16:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.955+0000 7f7f93cba700 1 --2- 172.21.15.46:0/595892290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c108760 0x7f7f8c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.955+0000 7f7f93cba700 1 --2- 172.21.15.46:0/595892290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c1034e0 0x7f7f8c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.955+0000 7f7f93cba700 1 --2- 172.21.15.46:0/595892290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f8c107e10 0x7f7f8c108220 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.955+0000 7f7f93cba700 1 -- 172.21.15.46:0/595892290 >> 172.21.15.46:0/595892290 conn(0x7f7f8c0fef30 msgr2=0x7f7f8c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.955+0000 7f7f93cba700 1 -- 172.21.15.46:0/595892290 shutdown_connections 2024-05-09T00:16:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.955+0000 7f7f93cba700 1 -- 172.21.15.46:0/595892290 wait complete. 2024-05-09T00:16:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.956+0000 7f7f93cba700 1 Processor -- start 2024-05-09T00:16:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.956+0000 7f7f93cba700 1 -- start start 2024-05-09T00:16:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.956+0000 7f7f93cba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f8c1034e0 0x7f7f8c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.956+0000 7f7f93cba700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c107e10 0x7f7f8c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.956+0000 7f7f93cba700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c108760 0x7f7f8c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.956+0000 7f7f93cba700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c1133e0 con 0x7f7f8c1034e0 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.956+0000 7f7f93cba700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c113260 con 0x7f7f8c107e10 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.956+0000 7f7f93cba700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c113560 con 0x7f7f8c108760 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f91a56700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f8c1034e0 0x7f7f8c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f91a56700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f8c1034e0 0x7f7f8c19aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45434/0 (socket says 172.21.15.46:45434) 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f91a56700 1 -- 172.21.15.46:0/878622597 learned_addr learned my addr 172.21.15.46:0/878622597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f91255700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c107e10 0x7f7f8c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f92257700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c108760 0x7f7f8c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f91a56700 1 -- 172.21.15.46:0/878622597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c108760 msgr2=0x7f7f8c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f91a56700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c108760 0x7f7f8c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f91a56700 1 -- 172.21.15.46:0/878622597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c107e10 msgr2=0x7f7f8c19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f91a56700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c107e10 0x7f7f8c19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f91a56700 1 -- 172.21.15.46:0/878622597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8c19fd90 con 0x7f7f8c1034e0 2024-05-09T00:16:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f91a56700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f8c1034e0 0x7f7f8c19aec0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7f880060b0 tx=0x7f7f8800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f82ffd700 1 -- 172.21.15.46:0/878622597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8801a070 con 0x7f7f8c1034e0 2024-05-09T00:16:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f82ffd700 1 -- 172.21.15.46:0/878622597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f8800ddd0 con 0x7f7f8c1034e0 2024-05-09T00:16:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f93cba700 1 -- 172.21.15.46:0/878622597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8c1a0080 con 0x7f7f8c1034e0 2024-05-09T00:16:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f82ffd700 1 -- 172.21.15.46:0/878622597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88014720 con 0x7f7f8c1034e0 2024-05-09T00:16:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.957+0000 7f7f93cba700 1 -- 172.21.15.46:0/878622597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f8c1a79a0 con 0x7f7f8c1034e0 2024-05-09T00:16:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.958+0000 7f7f93cba700 1 -- 172.21.15.46:0/878622597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f8c0512c0 con 0x7f7f8c1034e0 2024-05-09T00:16:27.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.961+0000 7f7f82ffd700 1 -- 172.21.15.46:0/878622597 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7f8801d070 con 0x7f7f8c1034e0 2024-05-09T00:16:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.962+0000 7f7f82ffd700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7f780b2900 0x7f7f780b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.962+0000 7f7f91255700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7f780b2900 0x7f7f780b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.962+0000 7f7f82ffd700 1 -- 172.21.15.46:0/878622597 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7f880e1f20 con 0x7f7f8c1034e0 2024-05-09T00:16:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.962+0000 7f7f91255700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7f780b2900 0x7f7f780b4e10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c006010 tx=0x7f7f7c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:27.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:27.966+0000 7f7f82ffd700 1 -- 172.21.15.46:0/878622597 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f88010070 con 0x7f7f8c1034e0 2024-05-09T00:16:28.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.228+0000 7f7f93cba700 1 -- 172.21.15.46:0/878622597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f8c19c220 con 0x7f7f8c1034e0 2024-05-09T00:16:28.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.229+0000 7f7f82ffd700 1 -- 172.21.15.46:0/878622597 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f7f8c19c220 con 0x7f7f8c1034e0 2024-05-09T00:16:28.228 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:16:28.228 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:16:28.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.232+0000 7f7f93cba700 1 -- 172.21.15.46:0/878622597 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7f780b2900 msgr2=0x7f7f780b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:28.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.232+0000 7f7f93cba700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7f780b2900 0x7f7f780b4e10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c006010 tx=0x7f7f7c000bc0 comp rx=0 tx=0).stop 2024-05-09T00:16:28.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.232+0000 7f7f93cba700 1 -- 172.21.15.46:0/878622597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f8c1034e0 msgr2=0x7f7f8c19aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:28.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.232+0000 7f7f93cba700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f8c1034e0 0x7f7f8c19aec0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7f880060b0 tx=0x7f7f8800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:16:28.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.232+0000 7f7f93cba700 1 -- 172.21.15.46:0/878622597 shutdown_connections 2024-05-09T00:16:28.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.232+0000 7f7f93cba700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7f780b2900 0x7f7f780b4e10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:28.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.232+0000 7f7f93cba700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f8c108760 0x7f7f8c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:28.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.232+0000 7f7f93cba700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f8c1034e0 0x7f7f8c19aec0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:28.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.232+0000 7f7f93cba700 1 --2- 172.21.15.46:0/878622597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c107e10 0x7f7f8c19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:28.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.232+0000 7f7f93cba700 1 -- 172.21.15.46:0/878622597 >> 172.21.15.46:0/878622597 conn(0x7f7f8c0fef30 msgr2=0x7f7f8c1096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:28.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.232+0000 7f7f93cba700 1 -- 172.21.15.46:0/878622597 shutdown_connections 2024-05-09T00:16:28.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:28.232+0000 7f7f93cba700 1 -- 172.21.15.46:0/878622597 wait complete. 2024-05-09T00:16:28.233 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:16:28.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:28 smithi046 ceph-mon[30151]: pgmap v529: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:28.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:28 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/878622597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:16:28.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:28 smithi169 ceph-mon[34777]: pgmap v529: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:28.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:28 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/878622597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:16:28.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:16:28.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:28 smithi142 ceph-mon[34533]: pgmap v529: 97 pgs: 97 active+clean; 2.5 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:28.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:28 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/878622597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:16:29.133 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:16:30.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:30 smithi046 ceph-mon[30151]: pgmap v530: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:30.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:30 smithi169 ceph-mon[34777]: pgmap v530: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:30.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:30 smithi142 ceph-mon[34533]: pgmap v530: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.971+0000 7f86545be700 1 -- 172.21.15.46:0/955843008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f864c1034c0 msgr2=0x7f864c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.971+0000 7f86545be700 1 --2- 172.21.15.46:0/955843008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f864c1034c0 0x7f864c1038d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f864800b0d0 tx=0x7f864800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.972+0000 7f86545be700 1 -- 172.21.15.46:0/955843008 shutdown_connections 2024-05-09T00:16:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.972+0000 7f86545be700 1 --2- 172.21.15.46:0/955843008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f864c108740 0x7f864c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.972+0000 7f86545be700 1 --2- 172.21.15.46:0/955843008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f864c1034c0 0x7f864c1038d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.972+0000 7f86545be700 1 --2- 172.21.15.46:0/955843008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c107df0 0x7f864c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.972+0000 7f86545be700 1 -- 172.21.15.46:0/955843008 >> 172.21.15.46:0/955843008 conn(0x7f864c0fedf0 msgr2=0x7f864c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.972+0000 7f86545be700 1 -- 172.21.15.46:0/955843008 shutdown_connections 2024-05-09T00:16:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.972+0000 7f86545be700 1 -- 172.21.15.46:0/955843008 wait complete. 2024-05-09T00:16:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.972+0000 7f86545be700 1 Processor -- start 2024-05-09T00:16:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.972+0000 7f86545be700 1 -- start start 2024-05-09T00:16:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f86545be700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f864c1034c0 0x7f864c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f86545be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f864c107df0 0x7f864c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f86545be700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c108740 0x7f864c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f86545be700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f864c113370 con 0x7f864c107df0 2024-05-09T00:16:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f86545be700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1131f0 con 0x7f864c1034c0 2024-05-09T00:16:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f86545be700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1134f0 con 0x7f864c108740 2024-05-09T00:16:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f8651b59700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f864c107df0 0x7f864c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f8651b59700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f864c107df0 0x7f864c19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45438/0 (socket says 172.21.15.46:45438) 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f8651b59700 1 -- 172.21.15.46:0/2683674056 learned_addr learned my addr 172.21.15.46:0/2683674056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f8652b5b700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c108740 0x7f864c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f865235a700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f864c1034c0 0x7f864c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f8651b59700 1 -- 172.21.15.46:0/2683674056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c108740 msgr2=0x7f864c19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f8651b59700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c108740 0x7f864c19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f8651b59700 1 -- 172.21.15.46:0/2683674056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f864c1034c0 msgr2=0x7f864c19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f8651b59700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f864c1034c0 0x7f864c19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f8651b59700 1 -- 172.21.15.46:0/2683674056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864c19fc50 con 0x7f864c107df0 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f8652b5b700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c108740 0x7f864c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f865235a700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f864c1034c0 0x7f864c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f8651b59700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f864c107df0 0x7f864c19b3d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f863c00c9e0 tx=0x7f863c00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.973+0000 7f86437fe700 1 -- 172.21.15.46:0/2683674056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c007aa0 con 0x7f864c107df0 2024-05-09T00:16:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.974+0000 7f86437fe700 1 -- 172.21.15.46:0/2683674056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f863c007c40 con 0x7f864c107df0 2024-05-09T00:16:30.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.974+0000 7f86437fe700 1 -- 172.21.15.46:0/2683674056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c018660 con 0x7f864c107df0 2024-05-09T00:16:30.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.974+0000 7f86545be700 1 -- 172.21.15.46:0/2683674056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864c19ff40 con 0x7f864c107df0 2024-05-09T00:16:30.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.974+0000 7f86545be700 1 -- 172.21.15.46:0/2683674056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864c1a0480 con 0x7f864c107df0 2024-05-09T00:16:30.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.977+0000 7f86437fe700 1 -- 172.21.15.46:0/2683674056 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f863c018800 con 0x7f864c107df0 2024-05-09T00:16:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.977+0000 7f86437fe700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f86380b2880 0x7f86380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:30.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.977+0000 7f8662fd4b80 1 Processor -- start 2024-05-09T00:16:30.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f865235a700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f86380b2880 0x7f86380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:30.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f86437fe700 1 -- 172.21.15.46:0/2683674056 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f863c0e2200 con 0x7f864c107df0 2024-05-09T00:16:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8662fd4b80 1 -- start start 2024-05-09T00:16:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8662fd4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc09f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8662fd4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 0x55ffc09fa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8662fd4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 0x55ffc09fe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8662fd4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ffc09180a0 con 0x55ffc09fe1e0 2024-05-09T00:16:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8662fd4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ffc08bd9a0 con 0x55ffc08f2d00 2024-05-09T00:16:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8662fd4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ffc0914060 con 0x55ffc08784a0 2024-05-09T00:16:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8652b5b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 0x55ffc09fe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8652b5b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 0x55ffc09fe5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45448/0 (socket says 172.21.15.46:45448) 2024-05-09T00:16:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8652b5b700 1 -- 172.21.15.46:0/2639006615 learned_addr learned my addr 172.21.15.46:0/2639006615 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f865235a700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f86380b2880 0x7f86380b4d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8648000f80 tx=0x7f864800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:30.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f865235a700 1 --2- 172.21.15.46:0/2639006615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc09f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8651b59700 1 --2- 172.21.15.46:0/2639006615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 0x55ffc09fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8652b5b700 1 -- 172.21.15.46:0/2639006615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 msgr2=0x55ffc09fa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8652b5b700 1 --2- 172.21.15.46:0/2639006615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 0x55ffc09fa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8652b5b700 1 -- 172.21.15.46:0/2639006615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 msgr2=0x55ffc09f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.978+0000 7f8652b5b700 1 --2- 172.21.15.46:0/2639006615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc09f9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.979+0000 7f8652b5b700 1 -- 172.21.15.46:0/2639006615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc08ae500 con 0x55ffc09fe1e0 2024-05-09T00:16:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.979+0000 7f8652b5b700 1 --2- 172.21.15.46:0/2639006615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 0x55ffc09fe5f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f864400cc20 tx=0x7f864400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ede741a81f0d0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.979+0000 7f8640ff9700 1 -- 172.21.15.46:0/2639006615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864400ec50 con 0x55ffc09fe1e0 2024-05-09T00:16:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.979+0000 7f8640ff9700 1 -- 172.21.15.46:0/2639006615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8644016070 con 0x55ffc09fe1e0 2024-05-09T00:16:30.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.979+0000 7f8640ff9700 1 -- 172.21.15.46:0/2639006615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644005020 con 0x55ffc09fe1e0 2024-05-09T00:16:30.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.980+0000 7f8662fd4b80 1 -- 172.21.15.46:0/2639006615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 msgr2=0x55ffc09fe5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:30.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.980+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/2639006615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 0x55ffc09fe5f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f864400cc20 tx=0x7f864400cf30 comp rx=0 tx=0).stop 2024-05-09T00:16:30.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.980+0000 7f8662fd4b80 1 -- 172.21.15.46:0/2639006615 shutdown_connections 2024-05-09T00:16:30.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.980+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/2639006615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 0x55ffc09fe5f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.980+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/2639006615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc09f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.980+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/2639006615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 0x55ffc09fa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.980+0000 7f8662fd4b80 1 -- 172.21.15.46:0/2639006615 >> 172.21.15.46:0/2639006615 conn(0x55ffc087bf70 msgr2=0x55ffc09f0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.980+0000 7f8662fd4b80 1 -- 172.21.15.46:0/2639006615 shutdown_connections 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.980+0000 7f8662fd4b80 1 -- 172.21.15.46:0/2639006615 wait complete. 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.980+0000 7f8662fd4b80 1 -- start start 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f8662fd4b80 1 Processor -- start 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f8662fd4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08784a0 0x55ffc0a03390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f8662fd4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc0a038d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f8662fd4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc09fe1e0 0x55ffc0a03e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f8662fd4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ffc09fef00 con 0x55ffc08784a0 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f8662fd4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ffc08b2b20 con 0x55ffc08f2d00 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f8662fd4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ffc0a06470 con 0x55ffc09fe1e0 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f8662fd4b80 10 client.0 _ll_register_callbacks cb 0x55ffc07896c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:30.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f865235a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08784a0 0x55ffc0a03390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f865235a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08784a0 0x55ffc0a03390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45464/0 (socket says 172.21.15.46:45464) 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f865235a700 1 -- 172.21.15.46:0/671436000 learned_addr learned my addr 172.21.15.46:0/671436000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f8652b5b700 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc09fe1e0 0x55ffc0a03e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f8651b59700 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc0a038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f865235a700 1 -- 172.21.15.46:0/671436000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc09fe1e0 msgr2=0x55ffc0a03e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f865235a700 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc09fe1e0 0x55ffc0a03e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f865235a700 1 -- 172.21.15.46:0/671436000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 msgr2=0x55ffc0a038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f865235a700 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc0a038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f865235a700 1 -- 172.21.15.46:0/671436000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc0793490 con 0x55ffc08784a0 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.981+0000 7f8651b59700 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc0a038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f865235a700 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08784a0 0x55ffc0a03390 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8648009e60 tx=0x7f86480099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb3ac17a705153 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f8635ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f8635ffb700 1 -- 172.21.15.46:0/671436000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864802d070 con 0x55ffc08784a0 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f8635ffb700 1 -- 172.21.15.46:0/671436000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8648005240 con 0x55ffc08784a0 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f8662fd4b80 1 -- 172.21.15.46:0/671436000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8644016070 con 0x55ffc08784a0 2024-05-09T00:16:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f8635ffb700 1 -- 172.21.15.46:0/671436000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8648005440 con 0x55ffc08784a0 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f8662fd4b80 10 client.16053 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f8662fd4b80 1 -- 172.21.15.46:0/671436000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ffc0a10d20 con 0x55ffc08784a0 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f8635ffb700 1 -- 172.21.15.46:0/671436000 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8644016070 con 0x55ffc08784a0 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f8635ffb700 1 -- 172.21.15.46:0/671436000 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8648011070 con 0x55ffc08784a0 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f8662fd4b80 10 client.16053 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f8662fd4b80 10 client.16053 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.982+0000 7f8662fd4b80 20 client.16053 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.983+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ffc0a13300 0x55ffc0a15780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.983+0000 7f8662fd4b80 4 client.16053 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.983+0000 7f8662fd4b80 1 -- 172.21.15.46:0/671436000 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffc0925a80 con 0x55ffc0a13300 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.983+0000 7f8651b59700 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ffc0a13300 0x55ffc0a15780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.984+0000 7f8635ffb700 10 client.16053 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:16:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.983+0000 7f8651b59700 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ffc0a13300 0x55ffc0a15780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=737e1916e94bc45c server_cookie=5b509af424994c71 in_seq=0 out_seq=0 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.984+0000 7f8635ffb700 1 -- 172.21.15.46:0/671436000 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ffc0925a80 con 0x55ffc0a13300 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.984+0000 7f8635ffb700 10 client.16053 handle_command_reply: tid=0 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.984+0000 7f8662fd4b80 1 client.16053 shutdown 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.984+0000 7f8662fd4b80 1 -- 172.21.15.46:0/671436000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08784a0 msgr2=0x55ffc0a03390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.985+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08784a0 0x55ffc0a03390 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8648009e60 tx=0x7f86480099e0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.985+0000 7f8662fd4b80 1 -- 172.21.15.46:0/671436000 shutdown_connections 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.985+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ffc0a13300 0x55ffc0a15780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.985+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc09fe1e0 0x55ffc0a03e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.985+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08784a0 0x55ffc0a03390 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.985+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/671436000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc0a038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.985+0000 7f8662fd4b80 1 -- 172.21.15.46:0/671436000 >> 172.21.15.46:0/671436000 conn(0x55ffc087bf70 msgr2=0x55ffc09e9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.985+0000 7f8635ffb700 0 client.16053 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:16:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.985+0000 7f8662fd4b80 1 -- 172.21.15.46:0/671436000 shutdown_connections 2024-05-09T00:16:30.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.985+0000 7f8662fd4b80 1 -- 172.21.15.46:0/671436000 wait complete. 2024-05-09T00:16:30.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:30.985+0000 7f8662fd4b80 20 client.16053 trim_cache size 0 max 16384 2024-05-09T00:16:31.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.000+0000 7f8662fd4b80 1 Processor -- start 2024-05-09T00:16:31.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.000+0000 7f8662fd4b80 1 -- start start 2024-05-09T00:16:31.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.000+0000 7f8662fd4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 0x55ffc0a03c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:31.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.000+0000 7f8662fd4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08f2d00 0x55ffc0a023b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:31.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.000+0000 7f8662fd4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc09fe1e0 0x55ffc0a0f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.000+0000 7f8662fd4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1af360 con 0x55ffc08f2d00 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.000+0000 7f8662fd4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1acbf0 con 0x55ffc09fe1e0 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.000+0000 7f8662fd4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1aaa40 con 0x55ffc08784a0 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.000+0000 7f8651b59700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08f2d00 0x55ffc0a023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.000+0000 7f8651b59700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08f2d00 0x55ffc0a023b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45472/0 (socket says 172.21.15.46:45472) 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.000+0000 7f8651b59700 1 -- 172.21.15.46:0/3440555892 learned_addr learned my addr 172.21.15.46:0/3440555892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.001+0000 7f865235a700 1 --2- 172.21.15.46:0/3440555892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 0x55ffc0a03c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.001+0000 7f8651b59700 1 -- 172.21.15.46:0/3440555892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 msgr2=0x55ffc0a03c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.001+0000 7f8651b59700 1 --2- 172.21.15.46:0/3440555892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 0x55ffc0a03c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.001+0000 7f8651b59700 1 -- 172.21.15.46:0/3440555892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc09fe1e0 msgr2=0x55ffc0a0f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.001+0000 7f8651b59700 1 --2- 172.21.15.46:0/3440555892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc09fe1e0 0x55ffc0a0f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.001+0000 7f8651b59700 1 -- 172.21.15.46:0/3440555892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc09ec9d0 con 0x55ffc08f2d00 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.001+0000 7f8651b59700 1 --2- 172.21.15.46:0/3440555892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08f2d00 0x55ffc0a023b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f863c0a9520 tx=0x7f863c0cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c617b529672024 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.001+0000 7f8637fff700 1 -- 172.21.15.46:0/3440555892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c0adbd0 con 0x55ffc08f2d00 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.001+0000 7f8637fff700 1 -- 172.21.15.46:0/3440555892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f863c0b2680 con 0x55ffc08f2d00 2024-05-09T00:16:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.001+0000 7f8637fff700 1 -- 172.21.15.46:0/3440555892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c0a9ce0 con 0x55ffc08f2d00 2024-05-09T00:16:31.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.002+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3440555892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08f2d00 msgr2=0x55ffc0a023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:31.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.002+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/3440555892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08f2d00 0x55ffc0a023b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f863c0a9520 tx=0x7f863c0cc040 comp rx=0 tx=0).stop 2024-05-09T00:16:31.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.002+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3440555892 shutdown_connections 2024-05-09T00:16:31.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.002+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/3440555892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc09fe1e0 0x55ffc0a0f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.002+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/3440555892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 0x55ffc0a03c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.002+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/3440555892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc08f2d00 0x55ffc0a023b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.002+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3440555892 >> 172.21.15.46:0/3440555892 conn(0x55ffc087bf70 msgr2=0x55ffc0a01ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.002+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3440555892 shutdown_connections 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.002+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3440555892 wait complete. 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.002+0000 7f8662fd4b80 1 -- start start 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.002+0000 7f8662fd4b80 1 Processor -- start 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8662fd4b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 0x55ffc0a0b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8662fd4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc0a0bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8662fd4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 0x55ffc09fc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8662fd4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1ab0b0 con 0x55ffc09fe1e0 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8662fd4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1ac520 con 0x55ffc08f2d00 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8662fd4b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1ab780 con 0x55ffc08784a0 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8662fd4b80 10 client.0 _ll_register_callbacks cb 0x55ffc07896c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8652b5b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 0x55ffc09fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8652b5b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 0x55ffc09fc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45478/0 (socket says 172.21.15.46:45478) 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8652b5b700 1 -- 172.21.15.46:0/3602333322 learned_addr learned my addr 172.21.15.46:0/3602333322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f865235a700 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 0x55ffc0a0b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8651b59700 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc0a0bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8652b5b700 1 -- 172.21.15.46:0/3602333322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 msgr2=0x55ffc0a0b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8652b5b700 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 0x55ffc0a0b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8652b5b700 1 -- 172.21.15.46:0/3602333322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 msgr2=0x55ffc0a0bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8652b5b700 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc0a0bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8652b5b700 1 -- 172.21.15.46:0/3602333322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc09fc550 con 0x55ffc09fe1e0 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8652b5b700 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 0x55ffc09fc010 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f864400e630 tx=0x7f864400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cca55225c47d8e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8635ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.003+0000 7f8635ffb700 1 -- 172.21.15.46:0/3602333322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644004020 con 0x55ffc09fe1e0 2024-05-09T00:16:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8635ffb700 1 -- 172.21.15.46:0/3602333322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f86440041c0 con 0x55ffc09fe1e0 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3602333322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f863c0b2680 con 0x55ffc09fe1e0 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8635ffb700 1 -- 172.21.15.46:0/3602333322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f86440043c0 con 0x55ffc09fe1e0 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8662fd4b80 10 client.16059 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3602333322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ffc0a1a720 con 0x55ffc09fe1e0 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8635ffb700 1 -- 172.21.15.46:0/3602333322 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f863c0b2680 con 0x55ffc09fe1e0 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8635ffb700 1 -- 172.21.15.46:0/3602333322 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8644026470 con 0x55ffc09fe1e0 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8662fd4b80 10 client.16059 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8662fd4b80 10 client.16059 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8662fd4b80 20 client.16059 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ffc0a1ab50 0x55ffc0a7fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8662fd4b80 4 client.16059 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3602333322 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffc091fb20 con 0x55ffc0a1ab50 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.004+0000 7f865235a700 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ffc0a1ab50 0x55ffc0a7fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f865235a700 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ffc0a1ab50 0x55ffc0a7fb00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=671bdc128facc6db server_cookie=5189b0c307423b9a in_seq=0 out_seq=0 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8635ffb700 10 client.16059 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8635ffb700 1 -- 172.21.15.46:0/3602333322 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffc091fb20 con 0x55ffc0a1ab50 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8635ffb700 10 client.16059 handle_command_reply: tid=0 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8662fd4b80 1 client.16059 shutdown 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3602333322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 msgr2=0x55ffc09fc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 0x55ffc09fc010 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f864400e630 tx=0x7f864400ec80 comp rx=0 tx=0).stop 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3602333322 shutdown_connections 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ffc0a1ab50 0x55ffc0a7fb00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ffc09fe1e0 0x55ffc09fc010 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffc08784a0 0x55ffc0a0b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8635ffb700 0 client.16059 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8662fd4b80 1 --2- 172.21.15.46:0/3602333322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffc08f2d00 0x55ffc0a0bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3602333322 >> 172.21.15.46:0/3602333322 conn(0x55ffc087bf70 msgr2=0x55ffc09e9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.005+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3602333322 shutdown_connections 2024-05-09T00:16:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f8662fd4b80 1 -- 172.21.15.46:0/3602333322 wait complete. 2024-05-09T00:16:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f8662fd4b80 20 client.16059 trim_cache size 0 max 16384 2024-05-09T00:16:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:16:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f86545be700 1 -- 172.21.15.46:0/2683674056 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f86380b2880 msgr2=0x7f86380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f86545be700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f86380b2880 0x7f86380b4d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8648000f80 tx=0x7f864800b040 comp rx=0 tx=0).stop 2024-05-09T00:16:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f86545be700 1 -- 172.21.15.46:0/2683674056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f864c107df0 msgr2=0x7f864c19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f86545be700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f864c107df0 0x7f864c19b3d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f863c00c9e0 tx=0x7f863c00cda0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f86545be700 1 -- 172.21.15.46:0/2683674056 shutdown_connections 2024-05-09T00:16:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f86545be700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f86380b2880 0x7f86380b4d90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f86545be700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c108740 0x7f864c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f86545be700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f864c1034c0 0x7f864c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f86545be700 1 --2- 172.21.15.46:0/2683674056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f864c107df0 0x7f864c19b3d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f86545be700 1 -- 172.21.15.46:0/2683674056 >> 172.21.15.46:0/2683674056 conn(0x7f864c0fedf0 msgr2=0x7f864c1063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f86545be700 1 -- 172.21.15.46:0/2683674056 shutdown_connections 2024-05-09T00:16:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:31.006+0000 7f86545be700 1 -- 172.21.15.46:0/2683674056 wait complete. 2024-05-09T00:16:31.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:16:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:16:32.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:32 smithi046 ceph-mon[30151]: pgmap v531: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:32.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:32 smithi169 ceph-mon[34777]: pgmap v531: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:32.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:32 smithi142 ceph-mon[34533]: pgmap v531: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.812+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/3503264923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd0108740 msgr2=0x7fbcd01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.812+0000 7fbcd7cc6700 1 --2- 172.21.15.46:0/3503264923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd0108740 0x7fbcd01100a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80099c0 tx=0x7fbcc8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.813+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/3503264923 shutdown_connections 2024-05-09T00:16:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.813+0000 7fbcd7cc6700 1 --2- 172.21.15.46:0/3503264923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd0108740 0x7fbcd01100a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.813+0000 7fbcd7cc6700 1 --2- 172.21.15.46:0/3503264923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcd01034c0 0x7fbcd01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.813+0000 7fbcd7cc6700 1 --2- 172.21.15.46:0/3503264923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcd0107df0 0x7fbcd0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.813+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/3503264923 >> 172.21.15.46:0/3503264923 conn(0x7fbcd00fedc0 msgr2=0x7fbcd01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.813+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/3503264923 shutdown_connections 2024-05-09T00:16:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.813+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/3503264923 wait complete. 2024-05-09T00:16:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.813+0000 7fbcd7cc6700 1 Processor -- start 2024-05-09T00:16:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd7cc6700 1 -- start start 2024-05-09T00:16:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd7cc6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcd01034c0 0x7fbcd019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd7cc6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcd0107df0 0x7fbcd019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd7cc6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd0108740 0x7fbcd019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd7cc6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0113410 con 0x7fbcd0108740 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd7cc6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0113290 con 0x7fbcd0107df0 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd7cc6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0113590 con 0x7fbcd01034c0 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd6263700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd0108740 0x7fbcd019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd6263700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd0108740 0x7fbcd019f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45490/0 (socket says 172.21.15.46:45490) 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd6263700 1 -- 172.21.15.46:0/4205110496 learned_addr learned my addr 172.21.15.46:0/4205110496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd5261700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcd0107df0 0x7fbcd019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcd01034c0 0x7fbcd019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd6263700 1 -- 172.21.15.46:0/4205110496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcd01034c0 msgr2=0x7fbcd019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd6263700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcd01034c0 0x7fbcd019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd6263700 1 -- 172.21.15.46:0/4205110496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcd0107df0 msgr2=0x7fbcd019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.814+0000 7fbcd6263700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcd0107df0 0x7fbcd019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.815+0000 7fbcd6263700 1 -- 172.21.15.46:0/4205110496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd019fd20 con 0x7fbcd0108740 2024-05-09T00:16:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.815+0000 7fbcd5261700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcd0107df0 0x7fbcd019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.815+0000 7fbcd6263700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd0108740 0x7fbcd019f680 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005910 tx=0x7fbcc8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.815+0000 7fbcc6ffd700 1 -- 172.21.15.46:0/4205110496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8027070 con 0x7fbcd0108740 2024-05-09T00:16:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.815+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/4205110496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd019ffb0 con 0x7fbcd0108740 2024-05-09T00:16:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.815+0000 7fbcc6ffd700 1 -- 172.21.15.46:0/4205110496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcc8003fa0 con 0x7fbcd0108740 2024-05-09T00:16:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.815+0000 7fbcc6ffd700 1 -- 172.21.15.46:0/4205110496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8022d10 con 0x7fbcd0108740 2024-05-09T00:16:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.815+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/4205110496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd01a0490 con 0x7fbcd0108740 2024-05-09T00:16:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.818+0000 7fbcc6ffd700 1 -- 172.21.15.46:0/4205110496 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbcc802a070 con 0x7fbcd0108740 2024-05-09T00:16:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.818+0000 7fbcc6ffd700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbcbc0b2880 0x7fbcbc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.818+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbcbc0b2880 0x7fbcbc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbcc6ffd700 1 -- 172.21.15.46:0/4205110496 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbcc80e32a0 con 0x7fbcd0108740 2024-05-09T00:16:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbce66dcb80 1 Processor -- start 2024-05-09T00:16:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbce66dcb80 1 -- start start 2024-05-09T00:16:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbce66dcb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4659fd00 0x562d466a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbce66dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 0x562d466a7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbcbc0b2880 0x7fbcbc0b4d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009f20 tx=0x7fbccc0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbce66dcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbce66dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d465c50a0 con 0x562d465254a0 2024-05-09T00:16:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbce66dcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562d4656a9a0 con 0x562d466ab1e0 2024-05-09T00:16:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbce66dcb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562d465c1060 con 0x562d4659fd00 2024-05-09T00:16:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbcd5261700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 0x562d466a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbcd5261700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 0x562d466a7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45502/0 (socket says 172.21.15.46:45502) 2024-05-09T00:16:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbcd5261700 1 -- 172.21.15.46:0/728517861 learned_addr learned my addr 172.21.15.46:0/728517861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbcd6263700 1 --2- 172.21.15.46:0/728517861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.819+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/728517861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4659fd00 0x562d466a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.820+0000 7fbcd5261700 1 -- 172.21.15.46:0/728517861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4659fd00 msgr2=0x562d466a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.820+0000 7fbcd5261700 1 --2- 172.21.15.46:0/728517861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4659fd00 0x562d466a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.820+0000 7fbcd5261700 1 -- 172.21.15.46:0/728517861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 msgr2=0x562d466ab5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.820+0000 7fbcd5261700 1 --2- 172.21.15.46:0/728517861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466ab5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.820+0000 7fbcd5261700 1 -- 172.21.15.46:0/728517861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4655b500 con 0x562d465254a0 2024-05-09T00:16:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.820+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/728517861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4659fd00 0x562d466a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.820+0000 7fbcd5261700 1 --2- 172.21.15.46:0/728517861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 0x562d466a7030 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000bd90 tx=0x7fbcc000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ad44b3a241762a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.820+0000 7fbcbbfff700 1 -- 172.21.15.46:0/728517861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0015070 con 0x562d465254a0 2024-05-09T00:16:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.820+0000 7fbcbbfff700 1 -- 172.21.15.46:0/728517861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcc000fda0 con 0x562d465254a0 2024-05-09T00:16:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.820+0000 7fbcbbfff700 1 -- 172.21.15.46:0/728517861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0010630 con 0x562d465254a0 2024-05-09T00:16:33.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.821+0000 7fbce66dcb80 1 -- 172.21.15.46:0/728517861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 msgr2=0x562d466a7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:33.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.821+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/728517861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 0x562d466a7030 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000bd90 tx=0x7fbcc000f640 comp rx=0 tx=0).stop 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.821+0000 7fbce66dcb80 1 -- 172.21.15.46:0/728517861 shutdown_connections 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.821+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/728517861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466ab5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.821+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/728517861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4659fd00 0x562d466a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.821+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/728517861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 0x562d466a7030 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.821+0000 7fbce66dcb80 1 -- 172.21.15.46:0/728517861 >> 172.21.15.46:0/728517861 conn(0x562d46528f70 msgr2=0x562d4669d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.821+0000 7fbce66dcb80 1 -- 172.21.15.46:0/728517861 shutdown_connections 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.821+0000 7fbce66dcb80 1 -- 172.21.15.46:0/728517861 wait complete. 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.822+0000 7fbce66dcb80 1 -- start start 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.822+0000 7fbce66dcb80 1 Processor -- start 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.822+0000 7fbce66dcb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d465254a0 0x562d466b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.822+0000 7fbce66dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 0x562d466b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.822+0000 7fbce66dcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.822+0000 7fbce66dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d466abf00 con 0x562d4659fd00 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.822+0000 7fbce66dcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562d4655fb20 con 0x562d466ab1e0 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.822+0000 7fbce66dcb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562d466b3470 con 0x562d465254a0 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.822+0000 7fbce66dcb80 10 client.0 _ll_register_callbacks cb 0x562d464366c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.822+0000 7fbcd5261700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 0x562d466b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcd5261700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 0x562d466b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45504/0 (socket says 172.21.15.46:45504) 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcd5261700 1 -- 172.21.15.46:0/1614159044 learned_addr learned my addr 172.21.15.46:0/1614159044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d465254a0 0x562d466b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcd6263700 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcd5261700 1 -- 172.21.15.46:0/1614159044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d465254a0 msgr2=0x562d466b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcd5261700 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d465254a0 0x562d466b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcd5261700 1 -- 172.21.15.46:0/1614159044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 msgr2=0x562d466b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcd5261700 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466b0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcd5261700 1 -- 172.21.15.46:0/1614159044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d46440490 con 0x562d4659fd00 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcd6263700 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcd5261700 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 0x562d466b0920 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0011670 tx=0x7fbcc00116a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7218a8cb198268f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcb97fa700 1 -- 172.21.15.46:0/1614159044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0028060 con 0x562d4659fd00 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcb97fa700 1 -- 172.21.15.46:0/1614159044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbcc000fba0 con 0x562d4659fd00 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbce66dcb80 1 -- 172.21.15.46:0/1614159044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc000fda0 con 0x562d4659fd00 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.823+0000 7fbcb97fa700 1 -- 172.21.15.46:0/1614159044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbcc0015070 con 0x562d4659fd00 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.824+0000 7fbce66dcb80 10 client.16071 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.824+0000 7fbce66dcb80 1 -- 172.21.15.46:0/1614159044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d466bdd70 con 0x562d4659fd00 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.824+0000 7fbcb97fa700 1 -- 172.21.15.46:0/1614159044 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcc000fda0 con 0x562d4659fd00 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.824+0000 7fbcb97fa700 1 -- 172.21.15.46:0/1614159044 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbcc002d070 con 0x562d4659fd00 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.824+0000 7fbce66dcb80 10 client.16071 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.824+0000 7fbce66dcb80 10 client.16071 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:16:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.824+0000 7fbce66dcb80 20 client.16071 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.824+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d466c02f0 0x562d466c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.824+0000 7fbce66dcb80 4 client.16071 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:16:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.824+0000 7fbce66dcb80 1 -- 172.21.15.46:0/1614159044 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d465d2a80 con 0x562d466c02f0 2024-05-09T00:16:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.824+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d466c02f0 0x562d466c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.825+0000 7fbcb97fa700 10 client.16071 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:16:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.825+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d466c02f0 0x562d466c2710 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=300f8ee25a9de992 server_cookie=dc37aa69244eb160 in_seq=0 out_seq=0 2024-05-09T00:16:33.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbcb97fa700 1 -- 172.21.15.46:0/1614159044 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562d465d2a80 con 0x562d466c02f0 2024-05-09T00:16:33.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbcb97fa700 10 client.16071 handle_command_reply: tid=0 2024-05-09T00:16:33.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbce66dcb80 1 client.16071 shutdown 2024-05-09T00:16:33.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbce66dcb80 1 -- 172.21.15.46:0/1614159044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 msgr2=0x562d466b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:33.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 0x562d466b0920 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0011670 tx=0x7fbcc00116a0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbce66dcb80 1 -- 172.21.15.46:0/1614159044 shutdown_connections 2024-05-09T00:16:33.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d466c02f0 0x562d466c2710 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d465254a0 0x562d466b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/1614159044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 0x562d466b0920 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbcb97fa700 0 client.16071 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:16:33.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbce66dcb80 1 -- 172.21.15.46:0/1614159044 >> 172.21.15.46:0/1614159044 conn(0x562d46528f70 msgr2=0x562d46696ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:33.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbce66dcb80 1 -- 172.21.15.46:0/1614159044 shutdown_connections 2024-05-09T00:16:33.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbce66dcb80 1 -- 172.21.15.46:0/1614159044 wait complete. 2024-05-09T00:16:33.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.826+0000 7fbce66dcb80 20 client.16071 trim_cache size 0 max 16384 2024-05-09T00:16:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.841+0000 7fbce66dcb80 1 Processor -- start 2024-05-09T00:16:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.841+0000 7fbce66dcb80 1 -- start start 2024-05-09T00:16:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.841+0000 7fbce66dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 0x562d4669d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.841+0000 7fbce66dcb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4659fd00 0x562d466b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbce66dcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d46697700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbce66dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01af240 con 0x562d465254a0 2024-05-09T00:16:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbce66dcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01acad0 con 0x562d466ab1e0 2024-05-09T00:16:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbce66dcb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01aa900 con 0x562d4659fd00 2024-05-09T00:16:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbcd5a62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 0x562d4669d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbcd5a62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 0x562d4669d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45518/0 (socket says 172.21.15.46:45518) 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbcd5a62700 1 -- 172.21.15.46:0/2012562204 learned_addr learned my addr 172.21.15.46:0/2012562204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbcd5261700 1 --2- 172.21.15.46:0/2012562204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4659fd00 0x562d466b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbcd6263700 1 --2- 172.21.15.46:0/2012562204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d46697700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbcd5a62700 1 -- 172.21.15.46:0/2012562204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4659fd00 msgr2=0x562d466b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/2012562204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4659fd00 0x562d466b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbcd5a62700 1 -- 172.21.15.46:0/2012562204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 msgr2=0x562d46697700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/2012562204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d46697700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbcd5a62700 1 -- 172.21.15.46:0/2012562204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d466b2c80 con 0x562d465254a0 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/2012562204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 0x562d4669d290 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009d00 tx=0x7fbccc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a97b8cd536df76d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.842+0000 7fbcbb7fe700 1 -- 172.21.15.46:0/2012562204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc017d10 con 0x562d465254a0 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.843+0000 7fbcbb7fe700 1 -- 172.21.15.46:0/2012562204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbccc029680 con 0x562d465254a0 2024-05-09T00:16:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.843+0000 7fbcbb7fe700 1 -- 172.21.15.46:0/2012562204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc024720 con 0x562d465254a0 2024-05-09T00:16:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.843+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2012562204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 msgr2=0x562d4669d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.843+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/2012562204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 0x562d4669d290 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009d00 tx=0x7fbccc015000 comp rx=0 tx=0).stop 2024-05-09T00:16:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.844+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2012562204 shutdown_connections 2024-05-09T00:16:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.844+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/2012562204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d46697700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.844+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/2012562204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d465254a0 0x562d4669d290 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.844+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/2012562204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d4659fd00 0x562d466b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.844+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2012562204 >> 172.21.15.46:0/2012562204 conn(0x562d46528f70 msgr2=0x562d46696ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.844+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2012562204 shutdown_connections 2024-05-09T00:16:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.844+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2012562204 wait complete. 2024-05-09T00:16:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.844+0000 7fbce66dcb80 1 -- start start 2024-05-09T00:16:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.844+0000 7fbce66dcb80 1 Processor -- start 2024-05-09T00:16:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbce66dcb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d465254a0 0x562d466b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbce66dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 0x562d466a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbce66dcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbce66dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01aaf90 con 0x562d4659fd00 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbce66dcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01ac400 con 0x562d466ab1e0 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbce66dcb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01ab660 con 0x562d465254a0 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbce66dcb80 10 client.0 _ll_register_callbacks cb 0x562d464366c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbcd5261700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 0x562d466a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbcd6263700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbcd5261700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 0x562d466a8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45530/0 (socket says 172.21.15.46:45530) 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbcd5261700 1 -- 172.21.15.46:0/2705265167 learned_addr learned my addr 172.21.15.46:0/2705265167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d465254a0 0x562d466b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbcd6263700 1 -- 172.21.15.46:0/2705265167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d465254a0 msgr2=0x562d466b8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbcd6263700 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d465254a0 0x562d466b8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbcd6263700 1 -- 172.21.15.46:0/2705265167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 msgr2=0x562d466a8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbcd6263700 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 0x562d466a8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbcd6263700 1 -- 172.21.15.46:0/2705265167 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d466a9470 con 0x562d466ab1e0 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.845+0000 7fbcd5261700 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 0x562d466a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.846+0000 7fbcd6263700 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466a8d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005f50 tx=0x7fbcc80227a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ff63daf5ec05e52 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.846+0000 7fbcb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.846+0000 7fbcb97fa700 1 -- 172.21.15.46:0/2705265167 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc80b3060 con 0x562d466ab1e0 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.846+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2705265167 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbccc029680 con 0x562d466ab1e0 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.846+0000 7fbcb97fa700 1 -- 172.21.15.46:0/2705265167 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbcc8004150 con 0x562d466ab1e0 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.846+0000 7fbcb97fa700 1 -- 172.21.15.46:0/2705265167 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbcc80aa460 con 0x562d466ab1e0 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.846+0000 7fbce66dcb80 10 client.25664 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.846+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2705265167 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d4672a790 con 0x562d466ab1e0 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.846+0000 7fbcb97fa700 1 -- 172.21.15.46:0/2705265167 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbccc029680 con 0x562d466ab1e0 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.846+0000 7fbcb97fa700 1 -- 172.21.15.46:0/2705265167 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbcc8070220 con 0x562d466ab1e0 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.847+0000 7fbce66dcb80 10 client.25664 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.847+0000 7fbce66dcb80 10 client.25664 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.847+0000 7fbce66dcb80 20 client.25664 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.847+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d4672adf0 0x562d4672d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.847+0000 7fbce66dcb80 4 client.25664 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.847+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2705265167 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d465ccb20 con 0x562d4672adf0 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.847+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d4672adf0 0x562d4672d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.847+0000 7fbcd5a62700 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d4672adf0 0x562d4672d210 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=949c1f79593ea99a server_cookie=fd14f66ce22cd83e in_seq=0 out_seq=0 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.847+0000 7fbcb97fa700 10 client.25664 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbcb97fa700 1 -- 172.21.15.46:0/2705265167 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d465ccb20 con 0x562d4672adf0 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbcb97fa700 10 client.25664 handle_command_reply: tid=0 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbce66dcb80 1 client.25664 shutdown 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2705265167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 msgr2=0x562d466a8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466a8d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005f50 tx=0x7fbcc80227a0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2705265167 shutdown_connections 2024-05-09T00:16:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d4672adf0 0x562d4672d210 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d466ab1e0 0x562d466a8d90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbcb97fa700 0 client.25664 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:16:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d465254a0 0x562d466b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbce66dcb80 1 --2- 172.21.15.46:0/2705265167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d4659fd00 0x562d466a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2705265167 >> 172.21.15.46:0/2705265167 conn(0x562d46528f70 msgr2=0x562d466a13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2705265167 shutdown_connections 2024-05-09T00:16:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbce66dcb80 1 -- 172.21.15.46:0/2705265167 wait complete. 2024-05-09T00:16:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.848+0000 7fbce66dcb80 20 client.25664 trim_cache size 0 max 16384 2024-05-09T00:16:33.855 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:16:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.849+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/4205110496 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbcbc0b2880 msgr2=0x7fbcbc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.849+0000 7fbcd7cc6700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbcbc0b2880 0x7fbcbc0b4d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009f20 tx=0x7fbccc0094f0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.849+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/4205110496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd0108740 msgr2=0x7fbcd019f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.849+0000 7fbcd7cc6700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd0108740 0x7fbcd019f680 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005910 tx=0x7fbcc8009580 comp rx=0 tx=0).stop 2024-05-09T00:16:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.849+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/4205110496 shutdown_connections 2024-05-09T00:16:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.849+0000 7fbcd7cc6700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbcbc0b2880 0x7fbcbc0b4d90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.849+0000 7fbcd7cc6700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd0108740 0x7fbcd019f680 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.849+0000 7fbcd7cc6700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcd01034c0 0x7fbcd019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.849+0000 7fbcd7cc6700 1 --2- 172.21.15.46:0/4205110496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcd0107df0 0x7fbcd019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.849+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/4205110496 >> 172.21.15.46:0/4205110496 conn(0x7fbcd00fedc0 msgr2=0x7fbcd006b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.849+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/4205110496 shutdown_connections 2024-05-09T00:16:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:33.849+0000 7fbcd7cc6700 1 -- 172.21.15.46:0/4205110496 wait complete. 2024-05-09T00:16:34.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:16:34.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:34 smithi169 ceph-mon[34777]: pgmap v532: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:16:34.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:34 smithi142 ceph-mon[34533]: pgmap v532: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:35.014 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:34 smithi046 ceph-mon[30151]: pgmap v532: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:36.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.708+0000 7fb13034f700 1 -- 172.21.15.46:0/3978771633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1281034c0 msgr2=0x7fb1281038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:36.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.708+0000 7fb13034f700 1 --2- 172.21.15.46:0/3978771633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1281034c0 0x7fb1281038d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb1240099c0 tx=0x7fb124009cd0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.709+0000 7fb13034f700 1 -- 172.21.15.46:0/3978771633 shutdown_connections 2024-05-09T00:16:36.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.709+0000 7fb13034f700 1 --2- 172.21.15.46:0/3978771633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb128108740 0x7fb1281100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.709+0000 7fb13034f700 1 --2- 172.21.15.46:0/3978771633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1281034c0 0x7fb1281038d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.709+0000 7fb13034f700 1 --2- 172.21.15.46:0/3978771633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb128107df0 0x7fb128108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.709+0000 7fb13034f700 1 -- 172.21.15.46:0/3978771633 >> 172.21.15.46:0/3978771633 conn(0x7fb1280fedc0 msgr2=0x7fb1281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:36.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.709+0000 7fb13034f700 1 -- 172.21.15.46:0/3978771633 shutdown_connections 2024-05-09T00:16:36.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.709+0000 7fb13034f700 1 -- 172.21.15.46:0/3978771633 wait complete. 2024-05-09T00:16:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb13034f700 1 Processor -- start 2024-05-09T00:16:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb13034f700 1 -- start start 2024-05-09T00:16:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb13034f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1281034c0 0x7fb12819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb13034f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb128107df0 0x7fb12819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb13034f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb128108740 0x7fb12819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb13034f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281144c0 con 0x7fb128108740 2024-05-09T00:16:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb13034f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb128114340 con 0x7fb1281034c0 2024-05-09T00:16:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb13034f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb128114640 con 0x7fb128107df0 2024-05-09T00:16:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb12e8ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb128108740 0x7fb12819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb12e8ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb128108740 0x7fb12819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32996/0 (socket says 172.21.15.46:32996) 2024-05-09T00:16:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb12e8ec700 1 -- 172.21.15.46:0/3444001875 learned_addr learned my addr 172.21.15.46:0/3444001875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb12d8ea700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb128107df0 0x7fb12819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb12e0eb700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1281034c0 0x7fb12819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb12e8ec700 1 -- 172.21.15.46:0/3444001875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb128107df0 msgr2=0x7fb12819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.710+0000 7fb12e8ec700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb128107df0 0x7fb12819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.711+0000 7fb12e8ec700 1 -- 172.21.15.46:0/3444001875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1281034c0 msgr2=0x7fb12819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.711+0000 7fb12e8ec700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1281034c0 0x7fb12819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.711+0000 7fb12e8ec700 1 -- 172.21.15.46:0/3444001875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12819fca0 con 0x7fb128108740 2024-05-09T00:16:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.711+0000 7fb12e8ec700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb128108740 0x7fb12819f600 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb120007ee0 tx=0x7fb12000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.711+0000 7fb11f7fe700 1 -- 172.21.15.46:0/3444001875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb120004bb0 con 0x7fb128108740 2024-05-09T00:16:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.711+0000 7fb11f7fe700 1 -- 172.21.15.46:0/3444001875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb120004d50 con 0x7fb128108740 2024-05-09T00:16:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.711+0000 7fb13034f700 1 -- 172.21.15.46:0/3444001875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12819ff90 con 0x7fb128108740 2024-05-09T00:16:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.711+0000 7fb11f7fe700 1 -- 172.21.15.46:0/3444001875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb120005920 con 0x7fb128108740 2024-05-09T00:16:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.711+0000 7fb13034f700 1 -- 172.21.15.46:0/3444001875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1281a78b0 con 0x7fb128108740 2024-05-09T00:16:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.714+0000 7fb11f7fe700 1 -- 172.21.15.46:0/3444001875 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb12001a020 con 0x7fb128108740 2024-05-09T00:16:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.715+0000 7fb11f7fe700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb1140b2880 0x7fb1140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.715+0000 7fb12e0eb700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb1140b2880 0x7fb1140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.715+0000 7fb11f7fe700 1 -- 172.21.15.46:0/3444001875 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb1200e1870 con 0x7fb128108740 2024-05-09T00:16:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.715+0000 7fb13ed65b80 1 Processor -- start 2024-05-09T00:16:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.715+0000 7fb13ed65b80 1 -- start start 2024-05-09T00:16:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.715+0000 7fb12e0eb700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb1140b2880 0x7fb1140b4d90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb124009990 tx=0x7fb124000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.715+0000 7fb13ed65b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec03644160 0x55ec036dbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.715+0000 7fb13ed65b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 0x55ec036dc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.715+0000 7fb13ed65b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 0x55ec036e0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.715+0000 7fb13ed65b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec035f9ff0 con 0x55ec036e0400 2024-05-09T00:16:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb13ed65b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ec035987f0 con 0x55ec03644160 2024-05-09T00:16:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb13ed65b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ec035f5fb0 con 0x55ec0355a4a0 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb12e8ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 0x55ec036e0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb12e8ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 0x55ec036e0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33006/0 (socket says 172.21.15.46:33006) 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb12e8ec700 1 -- 172.21.15.46:0/2569891307 learned_addr learned my addr 172.21.15.46:0/2569891307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb12e0eb700 1 --2- 172.21.15.46:0/2569891307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec03644160 0x55ec036dbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb12e8ec700 1 -- 172.21.15.46:0/2569891307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 msgr2=0x55ec036dc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb12e8ec700 1 --2- 172.21.15.46:0/2569891307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 0x55ec036dc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb12e8ec700 1 -- 172.21.15.46:0/2569891307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec03644160 msgr2=0x55ec036dbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb12e8ec700 1 --2- 172.21.15.46:0/2569891307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec03644160 0x55ec036dbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb12e8ec700 1 -- 172.21.15.46:0/2569891307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec03590370 con 0x55ec036e0400 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb12e8ec700 1 --2- 172.21.15.46:0/2569891307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 0x55ec036e0810 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb12000d700 tx=0x7fb1200a8820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fa688e95e5e871 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb11cd47700 1 -- 172.21.15.46:0/2569891307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200a9550 con 0x55ec036e0400 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb11cd47700 1 -- 172.21.15.46:0/2569891307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1200ab070 con 0x55ec036e0400 2024-05-09T00:16:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.716+0000 7fb11cd47700 1 -- 172.21.15.46:0/2569891307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200a9310 con 0x55ec036e0400 2024-05-09T00:16:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.717+0000 7fb13ed65b80 1 -- 172.21.15.46:0/2569891307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 msgr2=0x55ec036e0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.717+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/2569891307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 0x55ec036e0810 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb12000d700 tx=0x7fb1200a8820 comp rx=0 tx=0).stop 2024-05-09T00:16:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.717+0000 7fb13ed65b80 1 -- 172.21.15.46:0/2569891307 shutdown_connections 2024-05-09T00:16:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.717+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/2569891307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 0x55ec036e0810 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.717+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/2569891307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec03644160 0x55ec036dbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.717+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/2569891307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 0x55ec036dc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.717+0000 7fb13ed65b80 1 -- 172.21.15.46:0/2569891307 >> 172.21.15.46:0/2569891307 conn(0x55ec035ba6a0 msgr2=0x55ec036d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.717+0000 7fb13ed65b80 1 -- 172.21.15.46:0/2569891307 shutdown_connections 2024-05-09T00:16:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.717+0000 7fb13ed65b80 1 -- 172.21.15.46:0/2569891307 wait complete. 2024-05-09T00:16:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb13ed65b80 1 -- start start 2024-05-09T00:16:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb13ed65b80 1 Processor -- start 2024-05-09T00:16:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb13ed65b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 0x55ec036e5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb13ed65b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec03644160 0x55ec036e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb13ed65b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 0x55ec036e6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb13ed65b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec03475ba0 con 0x55ec036e0400 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb13ed65b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ec035b4150 con 0x55ec03644160 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb13ed65b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ec036e11b0 con 0x55ec0355a4a0 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb12e8ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 0x55ec036e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb13ed65b80 10 client.0 _ll_register_callbacks cb 0x55ec0346b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb12e8ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 0x55ec036e6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33016/0 (socket says 172.21.15.46:33016) 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb12e8ec700 1 -- 172.21.15.46:0/189902618 learned_addr learned my addr 172.21.15.46:0/189902618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb12e0eb700 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 0x55ec036e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.718+0000 7fb12d8ea700 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec03644160 0x55ec036e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb12e8ec700 1 -- 172.21.15.46:0/189902618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 msgr2=0x55ec036e5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb12e8ec700 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 0x55ec036e5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb12e8ec700 1 -- 172.21.15.46:0/189902618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec03644160 msgr2=0x55ec036e5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb12e8ec700 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec03644160 0x55ec036e5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb12e8ec700 1 -- 172.21.15.46:0/189902618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec03475490 con 0x55ec036e0400 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb12e8ec700 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 0x55ec036e6110 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb1200a92e0 tx=0x7fb1200a9310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e915c23e82b3e2ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb111ffb700 1 -- 172.21.15.46:0/189902618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12006c070 con 0x55ec036e0400 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb111ffb700 1 -- 172.21.15.46:0/189902618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb1200b1a30 con 0x55ec036e0400 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb13ed65b80 1 -- 172.21.15.46:0/189902618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1200ab070 con 0x55ec036e0400 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb111ffb700 1 -- 172.21.15.46:0/189902618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb1200ad200 con 0x55ec036e0400 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb13ed65b80 10 client.16089 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb13ed65b80 1 -- 172.21.15.46:0/189902618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec036f2f80 con 0x55ec036e0400 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.719+0000 7fb111ffb700 1 -- 172.21.15.46:0/189902618 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1200ab070 con 0x55ec036e0400 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.720+0000 7fb111ffb700 1 -- 172.21.15.46:0/189902618 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb1200a47c0 con 0x55ec036e0400 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.720+0000 7fb13ed65b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.720+0000 7fb13ed65b80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.720+0000 7fb13ed65b80 20 client.16089 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.720+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ec036f5450 0x55ec036f7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.720+0000 7fb13ed65b80 4 client.16089 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.720+0000 7fb13ed65b80 1 -- 172.21.15.46:0/189902618 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec036079d0 con 0x55ec036f5450 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.720+0000 7fb12e0eb700 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ec036f5450 0x55ec036f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.721+0000 7fb111ffb700 10 client.16089 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.721+0000 7fb12e0eb700 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ec036f5450 0x55ec036f7870 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5c96277199d29a6 server_cookie=186ab044a70a9da5 in_seq=0 out_seq=0 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.721+0000 7fb111ffb700 1 -- 172.21.15.46:0/189902618 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec036079d0 con 0x55ec036f5450 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.721+0000 7fb111ffb700 10 client.16089 handle_command_reply: tid=0 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.721+0000 7fb13ed65b80 1 client.16089 shutdown 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.722+0000 7fb13ed65b80 1 -- 172.21.15.46:0/189902618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 msgr2=0x55ec036e6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.722+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 0x55ec036e6110 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb1200a92e0 tx=0x7fb1200a9310 comp rx=0 tx=0).stop 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.722+0000 7fb13ed65b80 1 -- 172.21.15.46:0/189902618 shutdown_connections 2024-05-09T00:16:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.722+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ec036f5450 0x55ec036f7870 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.722+0000 7fb111ffb700 0 client.16089 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.722+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec036e0400 0x55ec036e6110 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.722+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 0x55ec036e5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.722+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/189902618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec03644160 0x55ec036e5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.722+0000 7fb13ed65b80 1 -- 172.21.15.46:0/189902618 >> 172.21.15.46:0/189902618 conn(0x55ec035ba6a0 msgr2=0x55ec036d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.722+0000 7fb13ed65b80 1 -- 172.21.15.46:0/189902618 shutdown_connections 2024-05-09T00:16:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.722+0000 7fb13ed65b80 1 -- 172.21.15.46:0/189902618 wait complete. 2024-05-09T00:16:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.722+0000 7fb13ed65b80 20 client.16089 trim_cache size 0 max 16384 2024-05-09T00:16:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.737+0000 7fb13ed65b80 1 Processor -- start 2024-05-09T00:16:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.737+0000 7fb13ed65b80 1 -- start start 2024-05-09T00:16:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb13ed65b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 0x55ec036d70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb13ed65b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec03644160 0x55ec036f1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb13ed65b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec036e0400 0x55ec036de300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb13ed65b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281af350 con 0x55ec03644160 2024-05-09T00:16:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb13ed65b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281acbe0 con 0x55ec036e0400 2024-05-09T00:16:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb13ed65b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281aaa10 con 0x55ec0355a4a0 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb12d8ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec03644160 0x55ec036f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb12d8ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec03644160 0x55ec036f1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33020/0 (socket says 172.21.15.46:33020) 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb12d8ea700 1 -- 172.21.15.46:0/3220513526 learned_addr learned my addr 172.21.15.46:0/3220513526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb12e0eb700 1 --2- 172.21.15.46:0/3220513526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 0x55ec036d70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb12d8ea700 1 -- 172.21.15.46:0/3220513526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 msgr2=0x55ec036d70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb12d8ea700 1 --2- 172.21.15.46:0/3220513526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 0x55ec036d70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb12d8ea700 1 -- 172.21.15.46:0/3220513526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec036e0400 msgr2=0x55ec036de300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb12d8ea700 1 --2- 172.21.15.46:0/3220513526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec036e0400 0x55ec036de300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb12d8ea700 1 -- 172.21.15.46:0/3220513526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec036e20a0 con 0x55ec03644160 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb12e0eb700 1 --2- 172.21.15.46:0/3220513526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 0x55ec036d70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb12d8ea700 1 --2- 172.21.15.46:0/3220513526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec03644160 0x55ec036f1790 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb118005950 tx=0x7fb11800bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbf3533dd816855 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb113fff700 1 -- 172.21.15.46:0/3220513526 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118004830 con 0x55ec03644160 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb113fff700 1 -- 172.21.15.46:0/3220513526 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb118005020 con 0x55ec03644160 2024-05-09T00:16:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.738+0000 7fb113fff700 1 -- 172.21.15.46:0/3220513526 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118003ac0 con 0x55ec03644160 2024-05-09T00:16:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.739+0000 7fb13ed65b80 1 -- 172.21.15.46:0/3220513526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec03644160 msgr2=0x55ec036f1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.739+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/3220513526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec03644160 0x55ec036f1790 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb118005950 tx=0x7fb11800bae0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.739+0000 7fb13ed65b80 1 -- 172.21.15.46:0/3220513526 shutdown_connections 2024-05-09T00:16:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.739+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/3220513526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec036e0400 0x55ec036de300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.739+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/3220513526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0355a4a0 0x55ec036d70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.739+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/3220513526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec03644160 0x55ec036f1790 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.739+0000 7fb13ed65b80 1 -- 172.21.15.46:0/3220513526 >> 172.21.15.46:0/3220513526 conn(0x55ec035ba6a0 msgr2=0x55ec036dfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.739+0000 7fb13ed65b80 1 -- 172.21.15.46:0/3220513526 shutdown_connections 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.739+0000 7fb13ed65b80 1 -- 172.21.15.46:0/3220513526 wait complete. 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb13ed65b80 1 -- start start 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb13ed65b80 1 Processor -- start 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb13ed65b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0355a4a0 0x55ec036f8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb13ed65b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec03644160 0x55ec036f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb13ed65b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec036e0400 0x55ec036f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb13ed65b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281ab0a0 con 0x55ec0355a4a0 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb13ed65b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281ac510 con 0x55ec036e0400 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb13ed65b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281ab770 con 0x55ec03644160 2024-05-09T00:16:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb12e0eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0355a4a0 0x55ec036f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb12e0eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0355a4a0 0x55ec036f8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33028/0 (socket says 172.21.15.46:33028) 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb12e0eb700 1 -- 172.21.15.46:0/766667477 learned_addr learned my addr 172.21.15.46:0/766667477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb13ed65b80 10 client.0 _ll_register_callbacks cb 0x55ec0346b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb12e8ec700 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec036e0400 0x55ec036f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.740+0000 7fb12d8ea700 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec03644160 0x55ec036f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb12e0eb700 1 -- 172.21.15.46:0/766667477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec03644160 msgr2=0x55ec036f88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb12e0eb700 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec03644160 0x55ec036f88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb12e0eb700 1 -- 172.21.15.46:0/766667477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec036e0400 msgr2=0x55ec036f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb12e0eb700 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec036e0400 0x55ec036f8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb12e0eb700 1 -- 172.21.15.46:0/766667477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec0375e9a0 con 0x55ec0355a4a0 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb12e0eb700 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0355a4a0 0x55ec036f8390 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb1240099c0 tx=0x7fb124009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301abd4b1698cd52 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb111ffb700 1 -- 172.21.15.46:0/766667477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124005260 con 0x55ec0355a4a0 2024-05-09T00:16:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb13ed65b80 1 -- 172.21.15.46:0/766667477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb118005020 con 0x55ec0355a4a0 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb111ffb700 1 -- 172.21.15.46:0/766667477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb124005400 con 0x55ec0355a4a0 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb111ffb700 1 -- 172.21.15.46:0/766667477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb124043050 con 0x55ec0355a4a0 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb13ed65b80 10 client.16095 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb13ed65b80 1 -- 172.21.15.46:0/766667477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec0375fa80 con 0x55ec0355a4a0 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.741+0000 7fb111ffb700 1 -- 172.21.15.46:0/766667477 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb118005020 con 0x55ec0355a4a0 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.742+0000 7fb111ffb700 1 -- 172.21.15.46:0/766667477 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb124022070 con 0x55ec0355a4a0 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.742+0000 7fb13ed65b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.742+0000 7fb13ed65b80 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.742+0000 7fb13ed65b80 20 client.16095 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.742+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ec036f5450 0x55ec037621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.742+0000 7fb13ed65b80 4 client.16095 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.742+0000 7fb13ed65b80 1 -- 172.21.15.46:0/766667477 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec03601a70 con 0x55ec036f5450 2024-05-09T00:16:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.742+0000 7fb12d8ea700 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ec036f5450 0x55ec037621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb12d8ea700 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ec036f5450 0x55ec037621e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edbd6f7c28d273d7 server_cookie=84ffaa9a4636727b in_seq=0 out_seq=0 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb111ffb700 10 client.16095 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb111ffb700 1 -- 172.21.15.46:0/766667477 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec03601a70 con 0x55ec036f5450 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb111ffb700 10 client.16095 handle_command_reply: tid=0 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb13ed65b80 1 client.16095 shutdown 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb13ed65b80 1 -- 172.21.15.46:0/766667477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0355a4a0 msgr2=0x55ec036f8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0355a4a0 0x55ec036f8390 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb1240099c0 tx=0x7fb124009cd0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb13ed65b80 1 -- 172.21.15.46:0/766667477 shutdown_connections 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ec036f5450 0x55ec037621e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec036e0400 0x55ec036f8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0355a4a0 0x55ec036f8390 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb13ed65b80 1 --2- 172.21.15.46:0/766667477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec03644160 0x55ec036f88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb13ed65b80 1 -- 172.21.15.46:0/766667477 >> 172.21.15.46:0/766667477 conn(0x55ec035ba6a0 msgr2=0x55ec036dfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb111ffb700 0 client.16095 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb13ed65b80 1 -- 172.21.15.46:0/766667477 shutdown_connections 2024-05-09T00:16:36.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.743+0000 7fb13ed65b80 1 -- 172.21.15.46:0/766667477 wait complete. 2024-05-09T00:16:36.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13ed65b80 20 client.16095 trim_cache size 0 max 16384 2024-05-09T00:16:36.748 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:16:36.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13034f700 1 -- 172.21.15.46:0/3444001875 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb1140b2880 msgr2=0x7fb1140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13034f700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb1140b2880 0x7fb1140b4d90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb124009990 tx=0x7fb124000bc0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13034f700 1 -- 172.21.15.46:0/3444001875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb128108740 msgr2=0x7fb12819f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13034f700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb128108740 0x7fb12819f600 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb120007ee0 tx=0x7fb12000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13034f700 1 -- 172.21.15.46:0/3444001875 shutdown_connections 2024-05-09T00:16:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13034f700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb1140b2880 0x7fb1140b4d90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13034f700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb128108740 0x7fb12819f600 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13034f700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1281034c0 0x7fb12819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13034f700 1 --2- 172.21.15.46:0/3444001875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb128107df0 0x7fb12819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13034f700 1 -- 172.21.15.46:0/3444001875 >> 172.21.15.46:0/3444001875 conn(0x7fb1280fedc0 msgr2=0x7fb128101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13034f700 1 -- 172.21.15.46:0/3444001875 shutdown_connections 2024-05-09T00:16:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:36.744+0000 7fb13034f700 1 -- 172.21.15.46:0/3444001875 wait complete. 2024-05-09T00:16:36.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:36 smithi169 ceph-mon[34777]: pgmap v533: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:16:36.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:36 smithi142 ceph-mon[34533]: pgmap v533: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:16:37.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:36 smithi046 ceph-mon[30151]: pgmap v533: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:16:38.583 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:16:38.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:16:38.853 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:16:38.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:38 smithi169 ceph-mon[34777]: pgmap v534: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:16:38.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:38 smithi142 ceph-mon[34533]: pgmap v534: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:16:38.977 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:38 smithi046 ceph-mon[30151]: pgmap v534: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:16:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.744+0000 7f5fea840700 1 -- 172.21.15.46:0/2876460908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe41045d0 msgr2=0x7f5fe41049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.744+0000 7f5fea840700 1 --2- 172.21.15.46:0/2876460908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe41045d0 0x7f5fe41049e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40099c0 tx=0x7f5fd4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.744+0000 7f5fea840700 1 -- 172.21.15.46:0/2876460908 shutdown_connections 2024-05-09T00:16:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.744+0000 7f5fea840700 1 --2- 172.21.15.46:0/2876460908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe4109850 0x7f5fe41111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.744+0000 7f5fea840700 1 --2- 172.21.15.46:0/2876460908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe41045d0 0x7f5fe41049e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.744+0000 7f5fea840700 1 --2- 172.21.15.46:0/2876460908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4108f00 0x7f5fe4109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.744+0000 7f5fea840700 1 -- 172.21.15.46:0/2876460908 >> 172.21.15.46:0/2876460908 conn(0x7f5fe40ffea0 msgr2=0x7f5fe41022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.744+0000 7f5fea840700 1 -- 172.21.15.46:0/2876460908 shutdown_connections 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.744+0000 7f5fea840700 1 -- 172.21.15.46:0/2876460908 wait complete. 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.745+0000 7f5fea840700 1 Processor -- start 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.745+0000 7f5fea840700 1 -- start start 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fea840700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe41045d0 0x7f5fe419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fea840700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe4108f00 0x7f5fe419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fea840700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4109850 0x7f5fe419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fea840700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4114480 con 0x7f5fe4108f00 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fea840700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4114300 con 0x7f5fe41045d0 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fea840700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4114600 con 0x7f5fe4109850 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe4108f00 0x7f5fe419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe4108f00 0x7f5fe419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33040/0 (socket says 172.21.15.46:33040) 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fdbfff700 1 -- 172.21.15.46:0/1893051031 learned_addr learned my addr 172.21.15.46:0/1893051031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe41045d0 0x7f5fe419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fe8ddd700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4109850 0x7f5fe419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fdbfff700 1 -- 172.21.15.46:0/1893051031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4109850 msgr2=0x7f5fe419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fdbfff700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4109850 0x7f5fe419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fdbfff700 1 -- 172.21.15.46:0/1893051031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe41045d0 msgr2=0x7f5fe419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fdbfff700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe41045d0 0x7f5fe419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fdbfff700 1 -- 172.21.15.46:0/1893051031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe419fd40 con 0x7f5fe4108f00 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe41045d0 0x7f5fe419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fdbfff700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe4108f00 0x7f5fe419b410 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00c8f0 tx=0x7f5fcc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.746+0000 7f5fe1ffb700 1 -- 172.21.15.46:0/1893051031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc007b00 con 0x7f5fe4108f00 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.747+0000 7f5fea840700 1 -- 172.21.15.46:0/1893051031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe41a0030 con 0x7f5fe4108f00 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.747+0000 7f5fe1ffb700 1 -- 172.21.15.46:0/1893051031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fcc007ca0 con 0x7f5fe4108f00 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.747+0000 7f5fe1ffb700 1 -- 172.21.15.46:0/1893051031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc018660 con 0x7f5fe4108f00 2024-05-09T00:16:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.747+0000 7f5fea840700 1 -- 172.21.15.46:0/1893051031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe41a7950 con 0x7f5fe4108f00 2024-05-09T00:16:40.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.749+0000 7f5fe1ffb700 1 -- 172.21.15.46:0/1893051031 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5fcc018800 con 0x7f5fe4108f00 2024-05-09T00:16:40.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.750+0000 7f5fe1ffb700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5fd00b2900 0x7f5fd00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.750+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5fd00b2900 0x7f5fd00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.750+0000 7f5fe1ffb700 1 -- 172.21.15.46:0/1893051031 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f5fcc0e21c0 con 0x7f5fe4108f00 2024-05-09T00:16:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5ff9256b80 1 Processor -- start 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5ff9256b80 1 -- start start 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5fd00b2900 0x7f5fd00b4e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009fd0 tx=0x7f5fd401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5ff9256b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb957d80 0x555bdba5eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5ff9256b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdb8dd520 0x555bdba5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5ff9256b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdba63260 0x555bdba63670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5ff9256b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555bdb97d120 con 0x555bdba63260 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5ff9256b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555bdb922a20 con 0x555bdb957d80 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5ff9256b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555bdb9790e0 con 0x555bdb8dd520 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5fe8ddd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdba63260 0x555bdba63670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdb8dd520 0x555bdba5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5fe8ddd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdba63260 0x555bdba63670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33046/0 (socket says 172.21.15.46:33046) 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5fe8ddd700 1 -- 172.21.15.46:0/3498344635 learned_addr learned my addr 172.21.15.46:0/3498344635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:40.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/3498344635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb957d80 0x555bdba5eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5fe8ddd700 1 -- 172.21.15.46:0/3498344635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdb8dd520 msgr2=0x555bdba5f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:40.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5fe8ddd700 1 --2- 172.21.15.46:0/3498344635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdb8dd520 0x555bdba5f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5fe8ddd700 1 -- 172.21.15.46:0/3498344635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb957d80 msgr2=0x555bdba5eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5fe8ddd700 1 --2- 172.21.15.46:0/3498344635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb957d80 0x555bdba5eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5fe8ddd700 1 -- 172.21.15.46:0/3498344635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bdb913580 con 0x555bdba63260 2024-05-09T00:16:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.751+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/3498344635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb957d80 0x555bdba5eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.752+0000 7f5fe8ddd700 1 --2- 172.21.15.46:0/3498344635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdba63260 0x555bdba63670 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0084f0 tx=0x7f5fdc008800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9708b3ff9ca22cd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.752+0000 7f5fda7fc700 1 -- 172.21.15.46:0/3498344635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc003c00 con 0x555bdba63260 2024-05-09T00:16:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.752+0000 7f5fda7fc700 1 -- 172.21.15.46:0/3498344635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fdc003da0 con 0x555bdba63260 2024-05-09T00:16:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.752+0000 7f5fda7fc700 1 -- 172.21.15.46:0/3498344635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc012820 con 0x555bdba63260 2024-05-09T00:16:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.753+0000 7f5ff9256b80 1 -- 172.21.15.46:0/3498344635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdba63260 msgr2=0x555bdba63670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.753+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/3498344635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdba63260 0x555bdba63670 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0084f0 tx=0x7f5fdc008800 comp rx=0 tx=0).stop 2024-05-09T00:16:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.753+0000 7f5ff9256b80 1 -- 172.21.15.46:0/3498344635 shutdown_connections 2024-05-09T00:16:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.753+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/3498344635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdba63260 0x555bdba63670 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.753+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/3498344635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb957d80 0x555bdba5eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.753+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/3498344635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdb8dd520 0x555bdba5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.753+0000 7f5ff9256b80 1 -- 172.21.15.46:0/3498344635 >> 172.21.15.46:0/3498344635 conn(0x555bdb8e0ff0 msgr2=0x555bdba552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.753+0000 7f5ff9256b80 1 -- 172.21.15.46:0/3498344635 shutdown_connections 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.753+0000 7f5ff9256b80 1 -- 172.21.15.46:0/3498344635 wait complete. 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.753+0000 7f5ff9256b80 1 -- start start 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5ff9256b80 1 Processor -- start 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5ff9256b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb8dd520 0x555bdba68460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5ff9256b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb957d80 0x555bdba689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5ff9256b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdba63260 0x555bdba68ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5ff9256b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555bdba63f80 con 0x555bdb8dd520 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5ff9256b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555bdb917ba0 con 0x555bdb957d80 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5ff9256b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555bdba6b4f0 con 0x555bdba63260 2024-05-09T00:16:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5ff9256b80 10 client.0 _ll_register_callbacks cb 0x555bdb7ee740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5fe3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb8dd520 0x555bdba68460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5fe3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb8dd520 0x555bdba68460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33056/0 (socket says 172.21.15.46:33056) 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5fe3fff700 1 -- 172.21.15.46:0/2171486181 learned_addr learned my addr 172.21.15.46:0/2171486181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5fdbfff700 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb957d80 0x555bdba689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5fe8ddd700 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdba63260 0x555bdba68ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5fe3fff700 1 -- 172.21.15.46:0/2171486181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdba63260 msgr2=0x555bdba68ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdba63260 0x555bdba68ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5fe3fff700 1 -- 172.21.15.46:0/2171486181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb957d80 msgr2=0x555bdba689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb957d80 0x555bdba689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.754+0000 7f5fe3fff700 1 -- 172.21.15.46:0/2171486181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bdb7f8510 con 0x555bdb8dd520 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.755+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb8dd520 0x555bdba68460 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40099c0 tx=0x7f5fd4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc128e46dc2e43a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.755+0000 7f5fc37fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.755+0000 7f5fc37fe700 1 -- 172.21.15.46:0/2171486181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4033060 con 0x555bdb8dd520 2024-05-09T00:16:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.755+0000 7f5fc37fe700 1 -- 172.21.15.46:0/2171486181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f5fd400f660 con 0x555bdb8dd520 2024-05-09T00:16:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.755+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2171486181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fdc003da0 con 0x555bdb8dd520 2024-05-09T00:16:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.755+0000 7f5fc37fe700 1 -- 172.21.15.46:0/2171486181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5fd40056c0 con 0x555bdb8dd520 2024-05-09T00:16:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.755+0000 7f5ff9256b80 10 client.16107 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.755+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2171486181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555bdba75df0 con 0x555bdb8dd520 2024-05-09T00:16:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.755+0000 7f5fc37fe700 1 -- 172.21.15.46:0/2171486181 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fdc003da0 con 0x555bdb8dd520 2024-05-09T00:16:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.756+0000 7f5fc37fe700 1 -- 172.21.15.46:0/2171486181 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5fd402d070 con 0x555bdb8dd520 2024-05-09T00:16:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.756+0000 7f5ff9256b80 10 client.16107 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.756+0000 7f5ff9256b80 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:16:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.756+0000 7f5ff9256b80 20 client.16107 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.756+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555bdba78480 0x555bdba7a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.756+0000 7f5ff9256b80 4 client.16107 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:16:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.756+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2171486181 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bdb98ab00 con 0x555bdba78480 2024-05-09T00:16:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.756+0000 7f5fdbfff700 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555bdba78480 0x555bdba7a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.756+0000 7f5fdbfff700 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555bdba78480 0x555bdba7a8c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce92a6c3c1bda364 server_cookie=ee6e8a8ffcdf8455 in_seq=0 out_seq=0 2024-05-09T00:16:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.756+0000 7f5fc37fe700 10 client.16107 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.757+0000 7f5fc37fe700 1 -- 172.21.15.46:0/2171486181 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555bdb98ab00 con 0x555bdba78480 2024-05-09T00:16:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.757+0000 7f5fc37fe700 10 client.16107 handle_command_reply: tid=0 2024-05-09T00:16:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.757+0000 7f5ff9256b80 1 client.16107 shutdown 2024-05-09T00:16:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.757+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2171486181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb8dd520 msgr2=0x555bdba68460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.757+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb8dd520 0x555bdba68460 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40099c0 tx=0x7f5fd4009310 comp rx=0 tx=0).stop 2024-05-09T00:16:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.757+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2171486181 shutdown_connections 2024-05-09T00:16:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.757+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555bdba78480 0x555bdba7a8c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.757+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdba63260 0x555bdba68ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.757+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb8dd520 0x555bdba68460 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.757+0000 7f5fc37fe700 0 client.16107 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.757+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/2171486181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb957d80 0x555bdba689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.757+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2171486181 >> 172.21.15.46:0/2171486181 conn(0x555bdb8e0ff0 msgr2=0x555bdba59470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.758+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2171486181 shutdown_connections 2024-05-09T00:16:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.758+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2171486181 wait complete. 2024-05-09T00:16:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.758+0000 7f5ff9256b80 20 client.16107 trim_cache size 0 max 16384 2024-05-09T00:16:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.778+0000 7f5ff9256b80 1 Processor -- start 2024-05-09T00:16:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.778+0000 7f5ff9256b80 1 -- start start 2024-05-09T00:16:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5ff9256b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdb8dd520 0x555bdba4faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5ff9256b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 0x555bdba68d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5ff9256b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdba63260 0x555bdba742a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5ff9256b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe41af390 con 0x555bdb957d80 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5ff9256b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe41acc20 con 0x555bdba63260 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5ff9256b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe41aaa50 con 0x555bdb8dd520 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 0x555bdba68d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 0x555bdba68d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33062/0 (socket says 172.21.15.46:33062) 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fdbfff700 1 -- 172.21.15.46:0/2148989052 learned_addr learned my addr 172.21.15.46:0/2148989052 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/2148989052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdb8dd520 0x555bdba4faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fdbfff700 1 -- 172.21.15.46:0/2148989052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdb8dd520 msgr2=0x555bdba4faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fe8ddd700 1 --2- 172.21.15.46:0/2148989052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdba63260 0x555bdba742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fdbfff700 1 --2- 172.21.15.46:0/2148989052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdb8dd520 0x555bdba4faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fdbfff700 1 -- 172.21.15.46:0/2148989052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdba63260 msgr2=0x555bdba742a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fdbfff700 1 --2- 172.21.15.46:0/2148989052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdba63260 0x555bdba742a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fdbfff700 1 -- 172.21.15.46:0/2148989052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bdba69260 con 0x555bdb957d80 2024-05-09T00:16:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fe8ddd700 1 --2- 172.21.15.46:0/2148989052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdba63260 0x555bdba742a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:16:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fdbfff700 1 --2- 172.21.15.46:0/2148989052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 0x555bdba68d20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0dfcc0 tx=0x7f5fcc092040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48872e1223a7d832 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.779+0000 7f5fd9ffb700 1 -- 172.21.15.46:0/2148989052 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0adb90 con 0x555bdb957d80 2024-05-09T00:16:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.780+0000 7f5fd9ffb700 1 -- 172.21.15.46:0/2148989052 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fcc0e1040 con 0x555bdb957d80 2024-05-09T00:16:40.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.780+0000 7f5fd9ffb700 1 -- 172.21.15.46:0/2148989052 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0920b0 con 0x555bdb957d80 2024-05-09T00:16:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.780+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2148989052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 msgr2=0x555bdba68d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.780+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/2148989052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 0x555bdba68d20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0dfcc0 tx=0x7f5fcc092040 comp rx=0 tx=0).stop 2024-05-09T00:16:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.780+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2148989052 shutdown_connections 2024-05-09T00:16:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.780+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/2148989052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdba63260 0x555bdba742a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.780+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/2148989052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdb8dd520 0x555bdba4faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.780+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/2148989052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 0x555bdba68d20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.780+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2148989052 >> 172.21.15.46:0/2148989052 conn(0x555bdb8e0ff0 msgr2=0x555bdba62ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.780+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2148989052 shutdown_connections 2024-05-09T00:16:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.780+0000 7f5ff9256b80 1 -- 172.21.15.46:0/2148989052 wait complete. 2024-05-09T00:16:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.781+0000 7f5ff9256b80 1 -- start start 2024-05-09T00:16:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.781+0000 7f5ff9256b80 1 Processor -- start 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.781+0000 7f5ff9256b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb8dd520 0x555bdba70ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5ff9256b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 0x555bdba6d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5ff9256b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdba63260 0x555bdba6db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5ff9256b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe41ab0e0 con 0x555bdb957d80 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5ff9256b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe41ac550 con 0x555bdb8dd520 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5ff9256b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe41ab7b0 con 0x555bdba63260 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5ff9256b80 10 client.0 _ll_register_callbacks cb 0x555bdba678d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 0x555bdba6d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 0x555bdba6d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33066/0 (socket says 172.21.15.46:33066) 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fdbfff700 1 -- 172.21.15.46:0/1494603913 learned_addr learned my addr 172.21.15.46:0/1494603913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb8dd520 0x555bdba70ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fe8ddd700 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdba63260 0x555bdba6db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fdbfff700 1 -- 172.21.15.46:0/1494603913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdba63260 msgr2=0x555bdba6db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fdbfff700 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdba63260 0x555bdba6db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fdbfff700 1 -- 172.21.15.46:0/1494603913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb8dd520 msgr2=0x555bdba70ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fdbfff700 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb8dd520 0x555bdba70ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fdbfff700 1 -- 172.21.15.46:0/1494603913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bdba6e1d0 con 0x555bdb957d80 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fdbfff700 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 0x555bdba6d5f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0a9770 tx=0x7f5fcc00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fbc877ea607082 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fc37fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fc37fe700 1 -- 172.21.15.46:0/1494603913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0adb90 con 0x555bdb957d80 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5fc37fe700 1 -- 172.21.15.46:0/1494603913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f5fcc0a5dd0 con 0x555bdb957d80 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.782+0000 7f5ff9256b80 1 -- 172.21.15.46:0/1494603913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fcc0e1040 con 0x555bdb957d80 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.783+0000 7f5fc37fe700 1 -- 172.21.15.46:0/1494603913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5fcc017360 con 0x555bdb957d80 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.783+0000 7f5ff9256b80 10 client.16113 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.783+0000 7f5ff9256b80 1 -- 172.21.15.46:0/1494603913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555bdbae3a40 con 0x555bdb957d80 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.783+0000 7f5fc37fe700 1 -- 172.21.15.46:0/1494603913 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fcc0e1040 con 0x555bdb957d80 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.783+0000 7f5fc37fe700 1 -- 172.21.15.46:0/1494603913 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5fcc0b21b0 con 0x555bdb957d80 2024-05-09T00:16:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.783+0000 7f5ff9256b80 10 client.16113 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.783+0000 7f5ff9256b80 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:16:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.783+0000 7f5ff9256b80 20 client.16113 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.783+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555bdbae4130 0x555bdbae6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.783+0000 7f5ff9256b80 4 client.16113 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:16:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.783+0000 7f5ff9256b80 1 -- 172.21.15.46:0/1494603913 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555bdb984ba0 con 0x555bdbae4130 2024-05-09T00:16:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.783+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555bdbae4130 0x555bdbae6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.784+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555bdbae4130 0x555bdbae6550 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c2dd0e2d724523d server_cookie=75d776aa52034de8 in_seq=0 out_seq=0 2024-05-09T00:16:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.784+0000 7f5fc37fe700 10 client.16113 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.785+0000 7f5fc37fe700 1 -- 172.21.15.46:0/1494603913 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555bdb984ba0 con 0x555bdbae4130 2024-05-09T00:16:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.785+0000 7f5fc37fe700 10 client.16113 handle_command_reply: tid=0 2024-05-09T00:16:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.785+0000 7f5ff9256b80 1 client.16113 shutdown 2024-05-09T00:16:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.785+0000 7f5ff9256b80 1 -- 172.21.15.46:0/1494603913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 msgr2=0x555bdba6d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.785+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 0x555bdba6d5f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0a9770 tx=0x7f5fcc00fc30 comp rx=0 tx=0).stop 2024-05-09T00:16:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.785+0000 7f5ff9256b80 1 -- 172.21.15.46:0/1494603913 shutdown_connections 2024-05-09T00:16:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.785+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555bdbae4130 0x555bdbae6550 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.786+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555bdba63260 0x555bdba6db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.786+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bdb8dd520 0x555bdba70ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.786+0000 7f5fc37fe700 0 client.16113 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.786+0000 7f5ff9256b80 1 --2- 172.21.15.46:0/1494603913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bdb957d80 0x555bdba6d5f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.786+0000 7f5ff9256b80 1 -- 172.21.15.46:0/1494603913 >> 172.21.15.46:0/1494603913 conn(0x555bdb8e0ff0 msgr2=0x555bdba62ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.786+0000 7f5ff9256b80 1 -- 172.21.15.46:0/1494603913 shutdown_connections 2024-05-09T00:16:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.786+0000 7f5ff9256b80 1 -- 172.21.15.46:0/1494603913 wait complete. 2024-05-09T00:16:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.786+0000 7f5ff9256b80 20 client.16113 trim_cache size 0 max 16384 2024-05-09T00:16:40.789 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:16:40.790 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:16:40.790 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "5a766121-abfc-4438-8c69-094af5943b05", 2024-05-09T00:16:40.790 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:16:40.790 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:16:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.786+0000 7f5fea840700 1 -- 172.21.15.46:0/1893051031 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5fd00b2900 msgr2=0x7f5fd00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.786+0000 7f5fea840700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5fd00b2900 0x7f5fd00b4e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009fd0 tx=0x7f5fd401a040 comp rx=0 tx=0).stop 2024-05-09T00:16:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.786+0000 7f5fea840700 1 -- 172.21.15.46:0/1893051031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe4108f00 msgr2=0x7f5fe419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.786+0000 7f5fea840700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe4108f00 0x7f5fe419b410 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00c8f0 tx=0x7f5fcc00ccb0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.787+0000 7f5fea840700 1 -- 172.21.15.46:0/1893051031 shutdown_connections 2024-05-09T00:16:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.787+0000 7f5fea840700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5fd00b2900 0x7f5fd00b4e10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.787+0000 7f5fea840700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe4109850 0x7f5fe419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.787+0000 7f5fea840700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fe41045d0 0x7f5fe419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.787+0000 7f5fea840700 1 --2- 172.21.15.46:0/1893051031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fe4108f00 0x7f5fe419b410 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.787+0000 7f5fea840700 1 -- 172.21.15.46:0/1893051031 >> 172.21.15.46:0/1893051031 conn(0x7f5fe40ffea0 msgr2=0x7f5fe4102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.787+0000 7f5fea840700 1 -- 172.21.15.46:0/1893051031 shutdown_connections 2024-05-09T00:16:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:40.787+0000 7f5fea840700 1 -- 172.21.15.46:0/1893051031 wait complete. 2024-05-09T00:16:40.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:40 smithi169 ceph-mon[34777]: pgmap v535: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:16:40.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:40 smithi142 ceph-mon[34533]: pgmap v535: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:16:41.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:40 smithi046 ceph-mon[30151]: pgmap v535: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:16:41.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:16:41.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:41 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:16:41.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:41 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:16:41.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:41 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:16:41.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:41 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:16:41.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:41 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:16:41.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:41 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:16:41.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:41 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:16:41.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:41 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:16:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:16:41.986 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:41 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:16:41.986 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:41 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:16:41.986 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:41 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:16:41.986 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:41 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:16:42.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:42 smithi046 ceph-mon[30151]: pgmap v536: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:16:42.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:42 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:16:42.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:42 smithi169 ceph-mon[34777]: pgmap v536: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:16:42.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:42 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:16:42.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:42 smithi142 ceph-mon[34533]: pgmap v536: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:16:42.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:42 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:16:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.882+0000 7fe8bfd67700 1 -- 172.21.15.46:0/1468329097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b8108740 msgr2=0x7fe8b81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.882+0000 7fe8bfd67700 1 --2- 172.21.15.46:0/1468329097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b8108740 0x7fe8b81100a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00099c0 tx=0x7fe8b0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.882+0000 7fe8bfd67700 1 -- 172.21.15.46:0/1468329097 shutdown_connections 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.882+0000 7fe8bfd67700 1 --2- 172.21.15.46:0/1468329097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b8108740 0x7fe8b81100a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.882+0000 7fe8bfd67700 1 --2- 172.21.15.46:0/1468329097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b81034c0 0x7fe8b81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.882+0000 7fe8bfd67700 1 --2- 172.21.15.46:0/1468329097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b8107df0 0x7fe8b8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.882+0000 7fe8bfd67700 1 -- 172.21.15.46:0/1468329097 >> 172.21.15.46:0/1468329097 conn(0x7fe8b80fedf0 msgr2=0x7fe8b8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.883+0000 7fe8bfd67700 1 -- 172.21.15.46:0/1468329097 shutdown_connections 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.883+0000 7fe8bfd67700 1 -- 172.21.15.46:0/1468329097 wait complete. 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.883+0000 7fe8bfd67700 1 Processor -- start 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.883+0000 7fe8bfd67700 1 -- start start 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bfd67700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b81034c0 0x7fe8b819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bfd67700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b8107df0 0x7fe8b819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bfd67700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b8108740 0x7fe8b819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bfd67700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b8113370 con 0x7fe8b8107df0 2024-05-09T00:16:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bfd67700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81131f0 con 0x7fe8b8108740 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bfd67700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81134f0 con 0x7fe8b81034c0 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bd302700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b8107df0 0x7fe8b819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bd302700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b8107df0 0x7fe8b819c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33082/0 (socket says 172.21.15.46:33082) 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bd302700 1 -- 172.21.15.46:0/2443385306 learned_addr learned my addr 172.21.15.46:0/2443385306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b81034c0 0x7fe8b819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8be304700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b8108740 0x7fe8b819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bd302700 1 -- 172.21.15.46:0/2443385306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b81034c0 msgr2=0x7fe8b819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bd302700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b81034c0 0x7fe8b819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bd302700 1 -- 172.21.15.46:0/2443385306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b8108740 msgr2=0x7fe8b819c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bd302700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b8108740 0x7fe8b819c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bd302700 1 -- 172.21.15.46:0/2443385306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b81a6fe0 con 0x7fe8b8107df0 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b81034c0 0x7fe8b819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8be304700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b8108740 0x7fe8b819c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:16:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.884+0000 7fe8bd302700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b8107df0 0x7fe8b819c1d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800d980 tx=0x7fe8a800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:43.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.885+0000 7fe8aeffd700 1 -- 172.21.15.46:0/2443385306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8017070 con 0x7fe8b8107df0 2024-05-09T00:16:43.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.885+0000 7fe8bfd67700 1 -- 172.21.15.46:0/2443385306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b81a72d0 con 0x7fe8b8107df0 2024-05-09T00:16:43.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.885+0000 7fe8aeffd700 1 -- 172.21.15.46:0/2443385306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8a8010460 con 0x7fe8b8107df0 2024-05-09T00:16:43.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.885+0000 7fe8aeffd700 1 -- 172.21.15.46:0/2443385306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a800de60 con 0x7fe8b8107df0 2024-05-09T00:16:43.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.885+0000 7fe8bfd67700 1 -- 172.21.15.46:0/2443385306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b81a7860 con 0x7fe8b8107df0 2024-05-09T00:16:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.888+0000 7fe8aeffd700 1 -- 172.21.15.46:0/2443385306 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe8a8010610 con 0x7fe8b8107df0 2024-05-09T00:16:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.888+0000 7fe8aeffd700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe8a40b2880 0x7fe8a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.888+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe8a40b2880 0x7fe8a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.888+0000 7fe8aeffd700 1 -- 172.21.15.46:0/2443385306 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe8a80e1d80 con 0x7fe8b8107df0 2024-05-09T00:16:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.889+0000 7fe8ce77db80 1 Processor -- start 2024-05-09T00:16:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.889+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe8a40b2880 0x7fe8a40b4d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4007fd0 tx=0x7fe8b4007830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.889+0000 7fe8ce77db80 1 -- start start 2024-05-09T00:16:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.889+0000 7fe8ce77db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3506ed00 0x55ce35175ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.889+0000 7fe8ce77db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce34ff44a0 0x55ce35176030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.889+0000 7fe8ce77db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3517a1e0 0x55ce3517a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.889+0000 7fe8ce77db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ce350940a0 con 0x55ce3506ed00 2024-05-09T00:16:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.889+0000 7fe8ce77db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce350399a0 con 0x55ce34ff44a0 2024-05-09T00:16:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.889+0000 7fe8ce77db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ce35090060 con 0x55ce3517a1e0 2024-05-09T00:16:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.889+0000 7fe8bdb03700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3506ed00 0x55ce35175ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.889+0000 7fe8bdb03700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3506ed00 0x55ce35175ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33094/0 (socket says 172.21.15.46:33094) 2024-05-09T00:16:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.889+0000 7fe8bdb03700 1 -- 172.21.15.46:0/3571301941 learned_addr learned my addr 172.21.15.46:0/3571301941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.890+0000 7fe8bd302700 1 --2- 172.21.15.46:0/3571301941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce34ff44a0 0x55ce35176030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.890+0000 7fe8be304700 1 --2- 172.21.15.46:0/3571301941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3517a1e0 0x55ce3517a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.890+0000 7fe8bdb03700 1 -- 172.21.15.46:0/3571301941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3517a1e0 msgr2=0x55ce3517a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.890+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/3571301941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3517a1e0 0x55ce3517a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.890+0000 7fe8bdb03700 1 -- 172.21.15.46:0/3571301941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce34ff44a0 msgr2=0x55ce35176030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.890+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/3571301941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce34ff44a0 0x55ce35176030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.890+0000 7fe8bdb03700 1 -- 172.21.15.46:0/3571301941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce3502a500 con 0x55ce3506ed00 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.890+0000 7fe8bd302700 1 --2- 172.21.15.46:0/3571301941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce34ff44a0 0x55ce35176030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.890+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/3571301941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3506ed00 0x55ce35175ad0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4018840 tx=0x7fe8b4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfddf21349428cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.890+0000 7fe8a3fff700 1 -- 172.21.15.46:0/3571301941 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b40180d0 con 0x55ce3506ed00 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.890+0000 7fe8a3fff700 1 -- 172.21.15.46:0/3571301941 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b4018270 con 0x55ce3506ed00 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.890+0000 7fe8a3fff700 1 -- 172.21.15.46:0/3571301941 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4016410 con 0x55ce3506ed00 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.891+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3571301941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3506ed00 msgr2=0x55ce35175ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.891+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/3571301941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3506ed00 0x55ce35175ad0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4018840 tx=0x7fe8b4011000 comp rx=0 tx=0).stop 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.891+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3571301941 shutdown_connections 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.891+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/3571301941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3517a1e0 0x55ce3517a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.891+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/3571301941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3506ed00 0x55ce35175ad0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.891+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/3571301941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce34ff44a0 0x55ce35176030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.891+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3571301941 >> 172.21.15.46:0/3571301941 conn(0x55ce34ff7f70 msgr2=0x55ce3516c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.891+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3571301941 shutdown_connections 2024-05-09T00:16:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.891+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3571301941 wait complete. 2024-05-09T00:16:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.892+0000 7fe8ce77db80 1 -- start start 2024-05-09T00:16:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.892+0000 7fe8ce77db80 1 Processor -- start 2024-05-09T00:16:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.892+0000 7fe8ce77db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce34ff44a0 0x55ce3517f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.892+0000 7fe8ce77db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3506ed00 0x55ce3517f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.892+0000 7fe8ce77db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 0x55ce3517fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.892+0000 7fe8ce77db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ce3517aea0 con 0x55ce3517a1e0 2024-05-09T00:16:43.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.892+0000 7fe8ce77db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce3502eb20 con 0x55ce34ff44a0 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.892+0000 7fe8ce77db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ce3517b020 con 0x55ce3506ed00 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.892+0000 7fe8ce77db80 10 client.0 _ll_register_callbacks cb 0x55ce34f056c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8be304700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 0x55ce3517fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8be304700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 0x55ce3517fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33102/0 (socket says 172.21.15.46:33102) 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8be304700 1 -- 172.21.15.46:0/3322182884 learned_addr learned my addr 172.21.15.46:0/3322182884 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8bd302700 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3506ed00 0x55ce3517f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce34ff44a0 0x55ce3517f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8be304700 1 -- 172.21.15.46:0/3322182884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3506ed00 msgr2=0x55ce3517f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8be304700 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3506ed00 0x55ce3517f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8be304700 1 -- 172.21.15.46:0/3322182884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce34ff44a0 msgr2=0x55ce3517f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8be304700 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce34ff44a0 0x55ce3517f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8be304700 1 -- 172.21.15.46:0/3322182884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce34f0f490 con 0x55ce3517a1e0 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8be304700 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 0x55ce3517fea0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00099c0 tx=0x7fe8b000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda66b86db24efee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8a17fa700 1 -- 172.21.15.46:0/3322182884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0027060 con 0x55ce3517a1e0 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8a17fa700 1 -- 172.21.15.46:0/3322182884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe8b0003a60 con 0x55ce3517a1e0 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3322182884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b4018270 con 0x55ce3517a1e0 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8a17fa700 1 -- 172.21.15.46:0/3322182884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe8b0003c60 con 0x55ce3517a1e0 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8ce77db80 10 client.16125 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.893+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3322182884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ce3518cd70 con 0x55ce3517a1e0 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.894+0000 7fe8a17fa700 1 -- 172.21.15.46:0/3322182884 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b4018270 con 0x55ce3517a1e0 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.894+0000 7fe8a17fa700 1 -- 172.21.15.46:0/3322182884 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe8b002c070 con 0x55ce3517a1e0 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.894+0000 7fe8ce77db80 10 client.16125 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.894+0000 7fe8ce77db80 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.894+0000 7fe8ce77db80 20 client.16125 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.894+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ce3518f3b0 0x55ce351917d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.894+0000 7fe8ce77db80 4 client.16125 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.894+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3322182884 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce350a1a80 con 0x55ce3518f3b0 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.894+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ce3518f3b0 0x55ce351917d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.895+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ce3518f3b0 0x55ce351917d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6796f50e31368c7 server_cookie=761586d12637acc8 in_seq=0 out_seq=0 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.895+0000 7fe8a17fa700 10 client.16125 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8a17fa700 1 -- 172.21.15.46:0/3322182884 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ce350a1a80 con 0x55ce3518f3b0 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8a17fa700 10 client.16125 handle_command_reply: tid=0 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8ce77db80 1 client.16125 shutdown 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3322182884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 msgr2=0x55ce3517fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 0x55ce3517fea0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00099c0 tx=0x7fe8b000bea0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3322182884 shutdown_connections 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ce3518f3b0 0x55ce351917d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 0x55ce3517fea0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8a17fa700 0 client.16125 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce34ff44a0 0x55ce3517f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/3322182884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3506ed00 0x55ce3517f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3322182884 >> 172.21.15.46:0/3322182884 conn(0x55ce34ff7f70 msgr2=0x55ce351703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:43.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3322182884 shutdown_connections 2024-05-09T00:16:43.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8ce77db80 1 -- 172.21.15.46:0/3322182884 wait complete. 2024-05-09T00:16:43.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.896+0000 7fe8ce77db80 20 client.16125 trim_cache size 0 max 16384 2024-05-09T00:16:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.917+0000 7fe8ce77db80 1 Processor -- start 2024-05-09T00:16:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.917+0000 7fe8ce77db80 1 -- start start 2024-05-09T00:16:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.917+0000 7fe8ce77db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce34ff44a0 0x55ce35170e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.917+0000 7fe8ce77db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce3506ed00 0x55ce3516c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.917+0000 7fe8ce77db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 0x55ce3518b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.917+0000 7fe8ce77db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81af440 con 0x55ce3517a1e0 2024-05-09T00:16:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.917+0000 7fe8ce77db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81accd0 con 0x55ce3506ed00 2024-05-09T00:16:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.917+0000 7fe8ce77db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81aab00 con 0x55ce34ff44a0 2024-05-09T00:16:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8be304700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 0x55ce3518b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8be304700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 0x55ce3518b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33116/0 (socket says 172.21.15.46:33116) 2024-05-09T00:16:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8be304700 1 -- 172.21.15.46:0/2504840918 learned_addr learned my addr 172.21.15.46:0/2504840918 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/2504840918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce34ff44a0 0x55ce35170e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8bd302700 1 --2- 172.21.15.46:0/2504840918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce3506ed00 0x55ce3516c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8be304700 1 -- 172.21.15.46:0/2504840918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce34ff44a0 msgr2=0x55ce35170e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8be304700 1 --2- 172.21.15.46:0/2504840918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce34ff44a0 0x55ce35170e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8be304700 1 -- 172.21.15.46:0/2504840918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce3506ed00 msgr2=0x55ce3516c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8be304700 1 --2- 172.21.15.46:0/2504840918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce3506ed00 0x55ce3516c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8be304700 1 -- 172.21.15.46:0/2504840918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce3516cd90 con 0x55ce3517a1e0 2024-05-09T00:16:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8be304700 1 --2- 172.21.15.46:0/2504840918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 0x55ce3518b1c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009350 tx=0x7fe8b0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dea34731e8ab4a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8a37fe700 1 -- 172.21.15.46:0/2504840918 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0032070 con 0x55ce3517a1e0 2024-05-09T00:16:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8a37fe700 1 -- 172.21.15.46:0/2504840918 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b0022500 con 0x55ce3517a1e0 2024-05-09T00:16:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.918+0000 7fe8a37fe700 1 -- 172.21.15.46:0/2504840918 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0027040 con 0x55ce3517a1e0 2024-05-09T00:16:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.919+0000 7fe8ce77db80 1 -- 172.21.15.46:0/2504840918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 msgr2=0x55ce3518b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.919+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/2504840918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 0x55ce3518b1c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009350 tx=0x7fe8b0009f90 comp rx=0 tx=0).stop 2024-05-09T00:16:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.919+0000 7fe8ce77db80 1 -- 172.21.15.46:0/2504840918 shutdown_connections 2024-05-09T00:16:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.919+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/2504840918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3517a1e0 0x55ce3518b1c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.919+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/2504840918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce34ff44a0 0x55ce35170e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.919+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/2504840918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce3506ed00 0x55ce3516c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.919+0000 7fe8ce77db80 1 -- 172.21.15.46:0/2504840918 >> 172.21.15.46:0/2504840918 conn(0x55ce34ff7f70 msgr2=0x55ce35166660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.919+0000 7fe8ce77db80 1 -- 172.21.15.46:0/2504840918 shutdown_connections 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.919+0000 7fe8ce77db80 1 -- 172.21.15.46:0/2504840918 wait complete. 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.920+0000 7fe8ce77db80 1 -- start start 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.920+0000 7fe8ce77db80 1 Processor -- start 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.920+0000 7fe8ce77db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce34ff44a0 0x55ce35177c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.920+0000 7fe8ce77db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3506ed00 0x55ce351781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.920+0000 7fe8ce77db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce3517a1e0 0x55ce351786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.920+0000 7fe8ce77db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81ab190 con 0x55ce34ff44a0 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.920+0000 7fe8ce77db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81ac600 con 0x55ce3517a1e0 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.920+0000 7fe8ce77db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81ab860 con 0x55ce3506ed00 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.920+0000 7fe8ce77db80 10 client.0 _ll_register_callbacks cb 0x55ce34f056c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.920+0000 7fe8bdb03700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce34ff44a0 0x55ce35177c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.920+0000 7fe8bdb03700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce34ff44a0 0x55ce35177c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33128/0 (socket says 172.21.15.46:33128) 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.920+0000 7fe8bdb03700 1 -- 172.21.15.46:0/1247761581 learned_addr learned my addr 172.21.15.46:0/1247761581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8be304700 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce3517a1e0 0x55ce351786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8bd302700 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3506ed00 0x55ce351781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8bdb03700 1 -- 172.21.15.46:0/1247761581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3506ed00 msgr2=0x55ce351781a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3506ed00 0x55ce351781a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8bdb03700 1 -- 172.21.15.46:0/1247761581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce3517a1e0 msgr2=0x55ce351786e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce3517a1e0 0x55ce351786e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8bdb03700 1 -- 172.21.15.46:0/1247761581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce351858d0 con 0x55ce34ff44a0 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8bd302700 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3506ed00 0x55ce351781a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8bdb03700 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce34ff44a0 0x55ce35177c60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40187e0 tx=0x7fe8b4018280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d457e4ac62295 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8a17fa700 1 -- 172.21.15.46:0/1247761581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4004d40 con 0x55ce34ff44a0 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8a17fa700 1 -- 172.21.15.46:0/1247761581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe8b40166d0 con 0x55ce34ff44a0 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8ce77db80 1 -- 172.21.15.46:0/1247761581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b0022500 con 0x55ce34ff44a0 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8a17fa700 1 -- 172.21.15.46:0/1247761581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe8b40168d0 con 0x55ce34ff44a0 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8ce77db80 10 client.16131 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.921+0000 7fe8ce77db80 1 -- 172.21.15.46:0/1247761581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ce351f9d70 con 0x55ce34ff44a0 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.922+0000 7fe8a17fa700 1 -- 172.21.15.46:0/1247761581 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b0022500 con 0x55ce34ff44a0 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.922+0000 7fe8a17fa700 1 -- 172.21.15.46:0/1247761581 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe8b402e6a0 con 0x55ce34ff44a0 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.922+0000 7fe8ce77db80 10 client.16131 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.922+0000 7fe8ce77db80 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.922+0000 7fe8ce77db80 20 client.16131 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.922+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ce3518f3b0 0x55ce351fc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.922+0000 7fe8ce77db80 4 client.16131 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.922+0000 7fe8ce77db80 1 -- 172.21.15.46:0/1247761581 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce3509bb20 con 0x55ce3518f3b0 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.922+0000 7fe8bd302700 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ce3518f3b0 0x55ce351fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8bd302700 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ce3518f3b0 0x55ce351fc100 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=113c7fae70d38746 server_cookie=193f7aa1c81ab461 in_seq=0 out_seq=0 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8a17fa700 10 client.16131 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8a17fa700 1 -- 172.21.15.46:0/1247761581 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ce3509bb20 con 0x55ce3518f3b0 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8a17fa700 10 client.16131 handle_command_reply: tid=0 2024-05-09T00:16:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8ce77db80 1 client.16131 shutdown 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8ce77db80 1 -- 172.21.15.46:0/1247761581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce34ff44a0 msgr2=0x55ce35177c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce34ff44a0 0x55ce35177c60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40187e0 tx=0x7fe8b4018280 comp rx=0 tx=0).stop 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8ce77db80 1 -- 172.21.15.46:0/1247761581 shutdown_connections 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ce3518f3b0 0x55ce351fc100 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce3517a1e0 0x55ce351786e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce34ff44a0 0x55ce35177c60 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8ce77db80 1 --2- 172.21.15.46:0/1247761581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce3506ed00 0x55ce351781a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8a17fa700 0 client.16131 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8ce77db80 1 -- 172.21.15.46:0/1247761581 >> 172.21.15.46:0/1247761581 conn(0x55ce34ff7f70 msgr2=0x55ce35175ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8ce77db80 1 -- 172.21.15.46:0/1247761581 shutdown_connections 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8ce77db80 1 -- 172.21.15.46:0/1247761581 wait complete. 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.923+0000 7fe8ce77db80 20 client.16131 trim_cache size 0 max 16384 2024-05-09T00:16:43.927 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:16:43.928 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:16:43.928 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:16:43.928 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:16:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.924+0000 7fe8bfd67700 1 -- 172.21.15.46:0/2443385306 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe8a40b2880 msgr2=0x7fe8a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.924+0000 7fe8bfd67700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe8a40b2880 0x7fe8a40b4d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4007fd0 tx=0x7fe8b4007830 comp rx=0 tx=0).stop 2024-05-09T00:16:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.924+0000 7fe8bfd67700 1 -- 172.21.15.46:0/2443385306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b8107df0 msgr2=0x7fe8b819c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.924+0000 7fe8bfd67700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b8107df0 0x7fe8b819c1d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800d980 tx=0x7fe8a800dd40 comp rx=0 tx=0).stop 2024-05-09T00:16:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.924+0000 7fe8bfd67700 1 -- 172.21.15.46:0/2443385306 shutdown_connections 2024-05-09T00:16:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.924+0000 7fe8bfd67700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe8a40b2880 0x7fe8a40b4d90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.924+0000 7fe8bfd67700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b8108740 0x7fe8b819c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.924+0000 7fe8bfd67700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b81034c0 0x7fe8b819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.924+0000 7fe8bfd67700 1 --2- 172.21.15.46:0/2443385306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b8107df0 0x7fe8b819c1d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.924+0000 7fe8bfd67700 1 -- 172.21.15.46:0/2443385306 >> 172.21.15.46:0/2443385306 conn(0x7fe8b80fedf0 msgr2=0x7fe8b8107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.924+0000 7fe8bfd67700 1 -- 172.21.15.46:0/2443385306 shutdown_connections 2024-05-09T00:16:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:43.924+0000 7fe8bfd67700 1 -- 172.21.15.46:0/2443385306 wait complete. 2024-05-09T00:16:44.802 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:16:44.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:16:44.854 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:44 smithi169 ceph-mon[34777]: pgmap v537: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:16:44.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:44 smithi142 ceph-mon[34533]: pgmap v537: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:16:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:16:45.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:44 smithi046 ceph-mon[30151]: pgmap v537: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:16:46.842 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.842 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.842 INFO:tasks.workunit.client.0.smithi046.stdout:127/600: open fd = 3 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:128/600: open fd = 3 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:129/600: open fd = 3 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:130/600: open fd = 3 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.843 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:131/600: open fd = 3 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:132/600: open fd = 3 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:133/600: open fd = 3 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.844 INFO:tasks.workunit.client.0.smithi046.stdout:134/600: open fd = 3 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:135/600: open fd = 3 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:136/600: open fd = 3 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:137/600: open fd = 3 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.845 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:138/600: open fd = 3 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:139/600: open fd = 3 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:140/600: open fd = 3 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.846 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:141/600: open fd = 3 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:142/600: open fd = 3 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:143/600: open fd = 3 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.847 INFO:tasks.workunit.client.0.smithi046.stdout:144/600: open fd = 3 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:145/600: open fd = 3 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:146/600: open fd = 3 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:147/600: open fd = 3 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.848 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:148/600: open fd = 3 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:149/600: open fd = 3 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:150/600: open fd = 3 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:151/600: open fd = 3 2024-05-09T00:16:46.849 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:152/600: open fd = 3 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:153/600: open fd = 3 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:154/600: open fd = 3 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.850 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:155/600: open fd = 3 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:156/600: open fd = 3 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:157/600: open fd = 3 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.851 INFO:tasks.workunit.client.0.smithi046.stdout:158/600: open fd = 3 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:159/600: open fd = 3 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:160/600: open fd = 3 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:161/600: open fd = 3 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.852 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:162/600: open fd = 3 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:163/600: open fd = 3 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:164/600: open fd = 3 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.853 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:165/600: open fd = 3 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:166/600: open fd = 3 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:167/600: open fd = 3 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:168/600: open fd = 3 2024-05-09T00:16:46.854 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:169/600: open fd = 3 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:170/600: open fd = 3 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:171/600: open fd = 3 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.855 INFO:tasks.workunit.client.0.smithi046.stdout:172/600: open fd = 3 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:173/600: open fd = 3 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:174/600: open fd = 3 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:175/600: open fd = 3 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.856 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:176/600: open fd = 3 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:177/600: open fd = 3 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:178/600: open fd = 3 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:179/600: open fd = 3 2024-05-09T00:16:46.857 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:180/600: open fd = 3 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:181/600: open fd = 3 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:182/600: open fd = 3 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.858 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:183/600: open fd = 3 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:184/600: open fd = 3 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:185/600: open fd = 3 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:186/600: open fd = 3 2024-05-09T00:16:46.859 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.860 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.860 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.860 INFO:tasks.workunit.client.0.smithi046.stdout:187/600: open fd = 3 2024-05-09T00:16:46.860 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.860 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.860 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:16:46.860 INFO:tasks.workunit.client.0.smithi046.stdout:188/600: open fd = 3 2024-05-09T00:16:46.860 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.860 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:16:46.870 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:46 smithi046 ceph-mon[30151]: pgmap v538: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:16:46.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.879+0000 7ff4deb8e700 1 -- 172.21.15.46:0/71767864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8107e10 msgr2=0x7ff4d8108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:46.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.879+0000 7ff4deb8e700 1 --2- 172.21.15.46:0/71767864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8107e10 0x7ff4d8108220 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40099c0 tx=0x7ff4c4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:16:46.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.879+0000 7ff4deb8e700 1 -- 172.21.15.46:0/71767864 shutdown_connections 2024-05-09T00:16:46.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.879+0000 7ff4deb8e700 1 --2- 172.21.15.46:0/71767864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8108760 0x7ff4d81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:46.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.879+0000 7ff4deb8e700 1 --2- 172.21.15.46:0/71767864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4d81034e0 0x7ff4d81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:46.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.879+0000 7ff4deb8e700 1 --2- 172.21.15.46:0/71767864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8107e10 0x7ff4d8108220 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.879+0000 7ff4deb8e700 1 -- 172.21.15.46:0/71767864 >> 172.21.15.46:0/71767864 conn(0x7ff4d80fef30 msgr2=0x7ff4d8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.879+0000 7ff4deb8e700 1 -- 172.21.15.46:0/71767864 shutdown_connections 2024-05-09T00:16:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.879+0000 7ff4deb8e700 1 -- 172.21.15.46:0/71767864 wait complete. 2024-05-09T00:16:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.880+0000 7ff4deb8e700 1 Processor -- start 2024-05-09T00:16:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.880+0000 7ff4deb8e700 1 -- start start 2024-05-09T00:16:46.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.880+0000 7ff4deb8e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4d81034e0 0x7ff4d819bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:46.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.880+0000 7ff4deb8e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8107e10 0x7ff4d819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:46.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.880+0000 7ff4deb8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8108760 0x7ff4d81a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:46.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.880+0000 7ff4deb8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d8113370 con 0x7ff4d8108760 2024-05-09T00:16:46.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.880+0000 7ff4deb8e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81131f0 con 0x7ff4d8107e10 2024-05-09T00:16:46.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.880+0000 7ff4deb8e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81134f0 con 0x7ff4d81034e0 2024-05-09T00:16:46.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.880+0000 7ff4dd12b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8108760 0x7ff4d81a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:46.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.880+0000 7ff4dd12b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8108760 0x7ff4d81a6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42158/0 (socket says 172.21.15.46:42158) 2024-05-09T00:16:46.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.880+0000 7ff4dd12b700 1 -- 172.21.15.46:0/478765844 learned_addr learned my addr 172.21.15.46:0/478765844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.880+0000 7ff4d7fff700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8107e10 0x7ff4d819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.881+0000 7ff4dc92a700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4d81034e0 0x7ff4d819bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.881+0000 7ff4dd12b700 1 -- 172.21.15.46:0/478765844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4d81034e0 msgr2=0x7ff4d819bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.881+0000 7ff4dd12b700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4d81034e0 0x7ff4d819bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.881+0000 7ff4dd12b700 1 -- 172.21.15.46:0/478765844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8107e10 msgr2=0x7ff4d819c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.881+0000 7ff4dd12b700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8107e10 0x7ff4d819c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.881+0000 7ff4dd12b700 1 -- 172.21.15.46:0/478765844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d819cac0 con 0x7ff4d8108760 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.881+0000 7ff4dd12b700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8108760 0x7ff4d81a6d50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0007ee0 tx=0x7ff4d000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.881+0000 7ff4d5ffb700 1 -- 172.21.15.46:0/478765844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0004bb0 con 0x7ff4d8108760 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.881+0000 7ff4deb8e700 1 -- 172.21.15.46:0/478765844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d81a73b0 con 0x7ff4d8108760 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.881+0000 7ff4d5ffb700 1 -- 172.21.15.46:0/478765844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d0004d50 con 0x7ff4d8108760 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.881+0000 7ff4d5ffb700 1 -- 172.21.15.46:0/478765844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0005920 con 0x7ff4d8108760 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.881+0000 7ff4deb8e700 1 -- 172.21.15.46:0/478765844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d81a7870 con 0x7ff4d8108760 2024-05-09T00:16:46.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.882+0000 7ff4deb8e700 1 -- 172.21.15.46:0/478765844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d80512c0 con 0x7ff4d8108760 2024-05-09T00:16:46.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.884+0000 7ff4d5ffb700 1 -- 172.21.15.46:0/478765844 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff4d001a020 con 0x7ff4d8108760 2024-05-09T00:16:46.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.885+0000 7ff4d5ffb700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff4c80b2880 0x7ff4c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:46.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.885+0000 7ff4dc92a700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff4c80b2880 0x7ff4c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:46.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.885+0000 7ff4d5ffb700 1 -- 172.21.15.46:0/478765844 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff4d00e1870 con 0x7ff4d8108760 2024-05-09T00:16:46.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.885+0000 7ff4dc92a700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff4c80b2880 0x7ff4c80b4d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc007be0 tx=0x7ff4cc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:46.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:46.890+0000 7ff4d5ffb700 1 -- 172.21.15.46:0/478765844 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff4d0010070 con 0x7ff4d8108760 2024-05-09T00:16:46.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:46 smithi142 ceph-mon[34533]: pgmap v538: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:16:47.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:46 smithi169 ceph-mon[34777]: pgmap v538: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:16:47.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.152+0000 7ff4deb8e700 1 -- 172.21.15.46:0/478765844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4d802e7b0 con 0x7ff4d8108760 2024-05-09T00:16:47.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.152+0000 7ff4d5ffb700 1 -- 172.21.15.46:0/478765844 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7ff4d00a9550 con 0x7ff4d8108760 2024-05-09T00:16:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:16:47.152 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:16:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.155+0000 7ff4deb8e700 1 -- 172.21.15.46:0/478765844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff4c80b2880 msgr2=0x7ff4c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.155+0000 7ff4deb8e700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff4c80b2880 0x7ff4c80b4d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc007be0 tx=0x7ff4cc0073d0 comp rx=0 tx=0).stop 2024-05-09T00:16:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.155+0000 7ff4deb8e700 1 -- 172.21.15.46:0/478765844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8108760 msgr2=0x7ff4d81a6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.155+0000 7ff4deb8e700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8108760 0x7ff4d81a6d50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0007ee0 tx=0x7ff4d000d3b0 comp rx=0 tx=0).stop 2024-05-09T00:16:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.155+0000 7ff4deb8e700 1 -- 172.21.15.46:0/478765844 shutdown_connections 2024-05-09T00:16:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.155+0000 7ff4deb8e700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff4c80b2880 0x7ff4c80b4d90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.155+0000 7ff4deb8e700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8108760 0x7ff4d81a6d50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.155+0000 7ff4deb8e700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4d81034e0 0x7ff4d819bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.155+0000 7ff4deb8e700 1 --2- 172.21.15.46:0/478765844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8107e10 0x7ff4d819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.155+0000 7ff4deb8e700 1 -- 172.21.15.46:0/478765844 >> 172.21.15.46:0/478765844 conn(0x7ff4d80fef30 msgr2=0x7ff4d81075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.155+0000 7ff4deb8e700 1 -- 172.21.15.46:0/478765844 shutdown_connections 2024-05-09T00:16:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:47.155+0000 7ff4deb8e700 1 -- 172.21.15.46:0/478765844 wait complete. 2024-05-09T00:16:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:16:47.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:47 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/478765844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:16:47.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:16:47.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:47 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/478765844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:16:48.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:47 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/478765844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:16:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:16:48.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:48 smithi046 ceph-mon[30151]: pgmap v539: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:48.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:48 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:16:48.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:48 smithi142 ceph-mon[34533]: pgmap v539: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:48.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:48 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:16:49.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:48 smithi169 ceph-mon[34777]: pgmap v539: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:49.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:48 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:16:49.723 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:16:49.723 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:16:49.723 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:16:49.723 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:16:49.723 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:16:49.723 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:16:49.723 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:16:49.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:16:49.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:16:49.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:16:49.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:16:49.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:16:49.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:16:49.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:16:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.974+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/3383661154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf04109860 msgr2=0x7fbf041111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.974+0000 7fbf0a9c0700 1 --2- 172.21.15.46:0/3383661154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf04109860 0x7fbf041111c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00b0d0 tx=0x7fbefc00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.975+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/3383661154 shutdown_connections 2024-05-09T00:16:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.975+0000 7fbf0a9c0700 1 --2- 172.21.15.46:0/3383661154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf04109860 0x7fbf041111c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.975+0000 7fbf0a9c0700 1 --2- 172.21.15.46:0/3383661154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf041045e0 0x7fbf041049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.975+0000 7fbf0a9c0700 1 --2- 172.21.15.46:0/3383661154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf04108f10 0x7fbf04109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.975+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/3383661154 >> 172.21.15.46:0/3383661154 conn(0x7fbf040ffe70 msgr2=0x7fbf041022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.975+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/3383661154 shutdown_connections 2024-05-09T00:16:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.975+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/3383661154 wait complete. 2024-05-09T00:16:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.975+0000 7fbf0a9c0700 1 Processor -- start 2024-05-09T00:16:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbf0a9c0700 1 -- start start 2024-05-09T00:16:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbf0a9c0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf041045e0 0x7fbf0419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbf0a9c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf04108f10 0x7fbf0419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbf0a9c0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf04109860 0x7fbf0419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbf0a9c0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04114490 con 0x7fbf04108f10 2024-05-09T00:16:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbf0a9c0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04114310 con 0x7fbf041045e0 2024-05-09T00:16:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbf0a9c0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04114610 con 0x7fbf04109860 2024-05-09T00:16:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbefbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf04108f10 0x7fbf0419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:49.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbefbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf04108f10 0x7fbf0419b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42178/0 (socket says 172.21.15.46:42178) 2024-05-09T00:16:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbefbfff700 1 -- 172.21.15.46:0/381716703 learned_addr learned my addr 172.21.15.46:0/381716703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbf03fff700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf041045e0 0x7fbf0419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbf08f5d700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf04109860 0x7fbf0419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbefbfff700 1 -- 172.21.15.46:0/381716703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf04109860 msgr2=0x7fbf0419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.976+0000 7fbefbfff700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf04109860 0x7fbf0419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.977+0000 7fbefbfff700 1 -- 172.21.15.46:0/381716703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf041045e0 msgr2=0x7fbf0419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.977+0000 7fbefbfff700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf041045e0 0x7fbf0419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.977+0000 7fbefbfff700 1 -- 172.21.15.46:0/381716703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0419fc40 con 0x7fbf04108f10 2024-05-09T00:16:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.977+0000 7fbf03fff700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf041045e0 0x7fbf0419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.977+0000 7fbefbfff700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf04108f10 0x7fbf0419b3c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00c9e0 tx=0x7fbeec00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.977+0000 7fbf01ffb700 1 -- 172.21.15.46:0/381716703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec007aa0 con 0x7fbf04108f10 2024-05-09T00:16:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.977+0000 7fbf01ffb700 1 -- 172.21.15.46:0/381716703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeec007c40 con 0x7fbf04108f10 2024-05-09T00:16:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.977+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/381716703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0419ff30 con 0x7fbf04108f10 2024-05-09T00:16:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.977+0000 7fbf01ffb700 1 -- 172.21.15.46:0/381716703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec018660 con 0x7fbf04108f10 2024-05-09T00:16:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.977+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/381716703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf041a0470 con 0x7fbf04108f10 2024-05-09T00:16:49.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.980+0000 7fbf01ffb700 1 -- 172.21.15.46:0/381716703 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbeec018800 con 0x7fbf04108f10 2024-05-09T00:16:49.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.980+0000 7fbf01ffb700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbef00b2880 0x7fbef00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:49.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.981+0000 7fbf03fff700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbef00b2880 0x7fbef00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:49.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.981+0000 7fbf01ffb700 1 -- 172.21.15.46:0/381716703 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbeec0e21a0 con 0x7fbf04108f10 2024-05-09T00:16:49.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.981+0000 7fbf03fff700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbef00b2880 0x7fbef00b4d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbef4005950 tx=0x7fbef40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:49.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.981+0000 7fbf193d6b80 1 Processor -- start 2024-05-09T00:16:49.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.981+0000 7fbf193d6b80 1 -- start start 2024-05-09T00:16:49.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbf193d6b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9ce1d00 0x558ac9dea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:49.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbf193d6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9c674a0 0x558ac9dea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbf193d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9de4400 0x558ac9de4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbf193d6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558ac9d070a0 con 0x558ac9c674a0 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbf193d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558ac9cac9a0 con 0x558ac9de4400 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbf193d6b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558ac9d03060 con 0x558ac9ce1d00 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbefbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9c674a0 0x558ac9dea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbefbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9c674a0 0x558ac9dea780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42182/0 (socket says 172.21.15.46:42182) 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbefbfff700 1 -- 172.21.15.46:0/1163885208 learned_addr learned my addr 172.21.15.46:0/1163885208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbf08f5d700 1 --2- 172.21.15.46:0/1163885208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9de4400 0x558ac9de4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbf03fff700 1 --2- 172.21.15.46:0/1163885208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9ce1d00 0x558ac9dea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbefbfff700 1 -- 172.21.15.46:0/1163885208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9ce1d00 msgr2=0x558ac9dea240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbefbfff700 1 --2- 172.21.15.46:0/1163885208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9ce1d00 0x558ac9dea240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbefbfff700 1 -- 172.21.15.46:0/1163885208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9de4400 msgr2=0x558ac9de4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbefbfff700 1 --2- 172.21.15.46:0/1163885208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9de4400 0x558ac9de4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbefbfff700 1 -- 172.21.15.46:0/1163885208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ac9c9d500 con 0x558ac9c674a0 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbefbfff700 1 --2- 172.21.15.46:0/1163885208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9c674a0 0x558ac9dea780 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0a5800 tx=0x7fbeec0a93e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7955efac0cb532e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.982+0000 7fbefa7fc700 1 -- 172.21.15.46:0/1163885208 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0cc0b0 con 0x558ac9c674a0 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.983+0000 7fbefa7fc700 1 -- 172.21.15.46:0/1163885208 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeec0a9780 con 0x558ac9c674a0 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.983+0000 7fbefa7fc700 1 -- 172.21.15.46:0/1163885208 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0a9be0 con 0x558ac9c674a0 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.983+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1163885208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9c674a0 msgr2=0x558ac9dea780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.983+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1163885208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9c674a0 0x558ac9dea780 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0a5800 tx=0x7fbeec0a93e0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.983+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1163885208 shutdown_connections 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.983+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1163885208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9de4400 0x558ac9de4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.983+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1163885208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9ce1d00 0x558ac9dea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.984+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1163885208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9c674a0 0x558ac9dea780 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.984+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1163885208 >> 172.21.15.46:0/1163885208 conn(0x558ac9c6af70 msgr2=0x558ac9ddf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.984+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1163885208 shutdown_connections 2024-05-09T00:16:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.984+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1163885208 wait complete. 2024-05-09T00:16:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.984+0000 7fbf193d6b80 1 -- start start 2024-05-09T00:16:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.984+0000 7fbf193d6b80 1 Processor -- start 2024-05-09T00:16:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.984+0000 7fbf193d6b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9c674a0 0x558ac9dee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf193d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9ce1d00 0x558ac9dee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf193d6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9deeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf193d6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558ac9de5160 con 0x558ac9de4400 2024-05-09T00:16:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf193d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558ac9ca1b20 con 0x558ac9ce1d00 2024-05-09T00:16:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf193d6b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558ac9df1020 con 0x558ac9c674a0 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf08f5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9deeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf03fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9c674a0 0x558ac9dee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf03fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9c674a0 0x558ac9dee080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:34996/0 (socket says 172.21.15.46:34996) 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf03fff700 1 -- 172.21.15.46:0/1644981240 learned_addr learned my addr 172.21.15.46:0/1644981240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf193d6b80 10 client.0 _ll_register_callbacks cb 0x558ac9b786c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbefbfff700 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9ce1d00 0x558ac9dee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf08f5d700 1 -- 172.21.15.46:0/1644981240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9c674a0 msgr2=0x558ac9dee080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf08f5d700 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9c674a0 0x558ac9dee080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf08f5d700 1 -- 172.21.15.46:0/1644981240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9ce1d00 msgr2=0x558ac9dee5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf08f5d700 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9ce1d00 0x558ac9dee5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf08f5d700 1 -- 172.21.15.46:0/1644981240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ac9b82490 con 0x558ac9de4400 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf03fff700 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9c674a0 0x558ac9dee080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbf08f5d700 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9deeb00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00b820 tx=0x7fbefc004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5842e518be93655c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbee37fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.985+0000 7fbee37fe700 1 -- 172.21.15.46:0/1644981240 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc009cf0 con 0x558ac9de4400 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbee37fe700 1 -- 172.21.15.46:0/1644981240 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbefc0077c0 con 0x558ac9de4400 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1644981240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeec0a9780 con 0x558ac9de4400 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbee37fe700 1 -- 172.21.15.46:0/1644981240 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbefc0079c0 con 0x558ac9de4400 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbf193d6b80 10 client.16149 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1644981240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558ac9dfba40 con 0x558ac9de4400 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbee37fe700 1 -- 172.21.15.46:0/1644981240 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeec0a9780 con 0x558ac9de4400 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbee37fe700 1 -- 172.21.15.46:0/1644981240 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbefc011070 con 0x558ac9de4400 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbf193d6b80 10 client.16149 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbf193d6b80 10 client.16149 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbf193d6b80 20 client.16149 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x558ac9dfe080 0x558ac9e004c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbf193d6b80 4 client.16149 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.986+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1644981240 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ac9d14a80 con 0x558ac9dfe080 2024-05-09T00:16:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.987+0000 7fbf03fff700 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x558ac9dfe080 0x558ac9e004c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.987+0000 7fbee37fe700 10 client.16149 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.987+0000 7fbf03fff700 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x558ac9dfe080 0x558ac9e004c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=808c601cfdc4ad23 server_cookie=5aea7884437407a8 in_seq=0 out_seq=0 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbee37fe700 1 -- 172.21.15.46:0/1644981240 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ac9d14a80 con 0x558ac9dfe080 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbee37fe700 10 client.16149 handle_command_reply: tid=0 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbf193d6b80 1 client.16149 shutdown 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1644981240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 msgr2=0x558ac9deeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9deeb00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00b820 tx=0x7fbefc004c80 comp rx=0 tx=0).stop 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1644981240 shutdown_connections 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x558ac9dfe080 0x558ac9e004c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9deeb00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9c674a0 0x558ac9dee080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1644981240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9ce1d00 0x558ac9dee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbee37fe700 0 client.16149 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:16:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1644981240 >> 172.21.15.46:0/1644981240 conn(0x558ac9c6af70 msgr2=0x558ac9dd8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1644981240 shutdown_connections 2024-05-09T00:16:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1644981240 wait complete. 2024-05-09T00:16:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:49.988+0000 7fbf193d6b80 20 client.16149 trim_cache size 0 max 16384 2024-05-09T00:16:50.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf193d6b80 1 Processor -- start 2024-05-09T00:16:50.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf193d6b80 1 -- start start 2024-05-09T00:16:50.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf193d6b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9c674a0 0x558ac9debcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:50.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf193d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9ce1d00 0x558ac9dec970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:50.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf193d6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9deceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:50.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf193d6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041b14b0 con 0x558ac9de4400 2024-05-09T00:16:50.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf193d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041aed40 con 0x558ac9ce1d00 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf193d6b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041acb70 con 0x558ac9c674a0 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf08f5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9deceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf08f5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9deceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42198/0 (socket says 172.21.15.46:42198) 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf08f5d700 1 -- 172.21.15.46:0/2741286017 learned_addr learned my addr 172.21.15.46:0/2741286017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf03fff700 1 --2- 172.21.15.46:0/2741286017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9c674a0 0x558ac9debcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbefbfff700 1 --2- 172.21.15.46:0/2741286017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9ce1d00 0x558ac9dec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf08f5d700 1 -- 172.21.15.46:0/2741286017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9c674a0 msgr2=0x558ac9debcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf08f5d700 1 --2- 172.21.15.46:0/2741286017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9c674a0 0x558ac9debcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf08f5d700 1 -- 172.21.15.46:0/2741286017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9ce1d00 msgr2=0x558ac9dec970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf08f5d700 1 --2- 172.21.15.46:0/2741286017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9ce1d00 0x558ac9dec970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.004+0000 7fbf08f5d700 1 -- 172.21.15.46:0/2741286017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ac9de0aa0 con 0x558ac9de4400 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.005+0000 7fbf08f5d700 1 --2- 172.21.15.46:0/2741286017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9deceb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00b0d0 tx=0x7fbefc0098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7e9ec3c7dc05ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.005+0000 7fbef9ffb700 1 -- 172.21.15.46:0/2741286017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0077f0 con 0x558ac9de4400 2024-05-09T00:16:50.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.005+0000 7fbef9ffb700 1 -- 172.21.15.46:0/2741286017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbefc035020 con 0x558ac9de4400 2024-05-09T00:16:50.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.005+0000 7fbef9ffb700 1 -- 172.21.15.46:0/2741286017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc01d620 con 0x558ac9de4400 2024-05-09T00:16:50.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.005+0000 7fbf193d6b80 1 -- 172.21.15.46:0/2741286017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 msgr2=0x558ac9deceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:50.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.005+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/2741286017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9deceb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00b0d0 tx=0x7fbefc0098c0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.005+0000 7fbf193d6b80 1 -- 172.21.15.46:0/2741286017 shutdown_connections 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.005+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/2741286017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9deceb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.006+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/2741286017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9c674a0 0x558ac9debcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.006+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/2741286017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9ce1d00 0x558ac9dec970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.006+0000 7fbf193d6b80 1 -- 172.21.15.46:0/2741286017 >> 172.21.15.46:0/2741286017 conn(0x558ac9c6af70 msgr2=0x558ac9de91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.006+0000 7fbf193d6b80 1 -- 172.21.15.46:0/2741286017 shutdown_connections 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.006+0000 7fbf193d6b80 1 -- 172.21.15.46:0/2741286017 wait complete. 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.006+0000 7fbf193d6b80 1 -- start start 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.006+0000 7fbf193d6b80 1 Processor -- start 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbf193d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9c674a0 0x558ac9df60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbf193d6b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9ce1d00 0x558ac9df65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbf193d6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9e04c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbf193d6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041ad200 con 0x558ac9de4400 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbf193d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041ae670 con 0x558ac9c674a0 2024-05-09T00:16:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbf193d6b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041ad8d0 con 0x558ac9ce1d00 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbf193d6b80 10 client.0 _ll_register_callbacks cb 0x558ac9de4dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbf08f5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9e04c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbefbfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9ce1d00 0x558ac9df65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbf08f5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9e04c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42200/0 (socket says 172.21.15.46:42200) 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbf08f5d700 1 -- 172.21.15.46:0/1653443225 learned_addr learned my addr 172.21.15.46:0/1653443225 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbf03fff700 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9c674a0 0x558ac9df60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbefbfff700 1 -- 172.21.15.46:0/1653443225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9c674a0 msgr2=0x558ac9df60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbefbfff700 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9c674a0 0x558ac9df60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbefbfff700 1 -- 172.21.15.46:0/1653443225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 msgr2=0x558ac9e04c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbefbfff700 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9e04c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbefbfff700 1 -- 172.21.15.46:0/1653443225 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ac9e051b0 con 0x558ac9ce1d00 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbf08f5d700 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9e04c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbefbfff700 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9ce1d00 0x558ac9df65f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x558ac9de0c90 tx=0x7fbeec01f520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dc5a508de970872 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.007+0000 7fbee37fe700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:16:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbee37fe700 1 -- 172.21.15.46:0/1653443225 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0cc0b0 con 0x558ac9ce1d00 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1653443225 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbefc035020 con 0x558ac9ce1d00 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbee37fe700 1 -- 172.21.15.46:0/1653443225 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbeec091320 con 0x558ac9ce1d00 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbee37fe700 1 -- 172.21.15.46:0/1653443225 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbeec0a9780 con 0x558ac9ce1d00 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbf193d6b80 10 client.25786 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1653443225 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558ac9e67dd0 con 0x558ac9ce1d00 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbee37fe700 1 -- 172.21.15.46:0/1653443225 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbefc035020 con 0x558ac9ce1d00 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbee37fe700 1 -- 172.21.15.46:0/1653443225 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbeec020030 con 0x558ac9ce1d00 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbf193d6b80 10 client.25786 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbf193d6b80 10 client.25786 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbf193d6b80 20 client.25786 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x558ac9e680b0 0x558ac9e6a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbf193d6b80 4 client.25786 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:16:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1653443225 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ac9d0eb20 con 0x558ac9e680b0 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.008+0000 7fbf03fff700 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x558ac9e680b0 0x558ac9e6a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbf03fff700 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x558ac9e680b0 0x558ac9e6a4d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=860a32adf6e75ad server_cookie=f692585da989743b in_seq=0 out_seq=0 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbee37fe700 10 client.25786 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbee37fe700 1 -- 172.21.15.46:0/1653443225 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ac9d0eb20 con 0x558ac9e680b0 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbee37fe700 10 client.25786 handle_command_reply: tid=0 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbf193d6b80 1 client.25786 shutdown 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1653443225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9ce1d00 msgr2=0x558ac9df65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9ce1d00 0x558ac9df65f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x558ac9de0c90 tx=0x7fbeec01f520 comp rx=0 tx=0).stop 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1653443225 shutdown_connections 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x558ac9e680b0 0x558ac9e6a4d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ac9de4400 0x558ac9e04c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbee37fe700 0 client.25786 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558ac9c674a0 0x558ac9df60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbf193d6b80 1 --2- 172.21.15.46:0/1653443225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ac9ce1d00 0x558ac9df65f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.009+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1653443225 >> 172.21.15.46:0/1653443225 conn(0x558ac9c6af70 msgr2=0x558ac9de91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1653443225 shutdown_connections 2024-05-09T00:16:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf193d6b80 1 -- 172.21.15.46:0/1653443225 wait complete. 2024-05-09T00:16:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf193d6b80 20 client.25786 trim_cache size 0 max 16384 2024-05-09T00:16:50.014 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:16:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/381716703 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbef00b2880 msgr2=0x7fbef00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf0a9c0700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbef00b2880 0x7fbef00b4d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbef4005950 tx=0x7fbef40058e0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/381716703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf04108f10 msgr2=0x7fbf0419b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf0a9c0700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf04108f10 0x7fbf0419b3c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00c9e0 tx=0x7fbeec00cda0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/381716703 shutdown_connections 2024-05-09T00:16:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf0a9c0700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbef00b2880 0x7fbef00b4d90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf0a9c0700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf04109860 0x7fbf0419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf0a9c0700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf041045e0 0x7fbf0419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf0a9c0700 1 --2- 172.21.15.46:0/381716703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf04108f10 0x7fbf0419b3c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/381716703 >> 172.21.15.46:0/381716703 conn(0x7fbf040ffe70 msgr2=0x7fbf04106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/381716703 shutdown_connections 2024-05-09T00:16:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:50.010+0000 7fbf0a9c0700 1 -- 172.21.15.46:0/381716703 wait complete. 2024-05-09T00:16:50.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:16:50.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:16:50.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:16:50.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:16:50.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:16:50.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:16:50.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:16:50.784 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:16:50.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:50 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-05-09T00:16:50.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:50 smithi142 ceph-mon[34533]: Unable to set osd_memory_target on smithi046 to 185879859: error parsing value: Value '185879859' is below minimum 939524096 2024-05-09T00:16:50.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:50 smithi142 ceph-mon[34533]: pgmap v540: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:16:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:16:51.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:50 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-05-09T00:16:51.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:50 smithi046 ceph-mon[30151]: Unable to set osd_memory_target on smithi046 to 185879859: error parsing value: Value '185879859' is below minimum 939524096 2024-05-09T00:16:51.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:50 smithi046 ceph-mon[30151]: pgmap v540: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:16:51.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:50 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-05-09T00:16:51.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:50 smithi169 ceph-mon[34777]: Unable to set osd_memory_target on smithi046 to 185879859: error parsing value: Value '185879859' is below minimum 939524096 2024-05-09T00:16:51.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:50 smithi169 ceph-mon[34777]: pgmap v540: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:16:52.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.827+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2547509181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c107df0 msgr2=0x7f9c8c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:52.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.827+0000 7f9c91c9d700 1 --2- 172.21.15.46:0/2547509181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c107df0 0x7f9c8c108200 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f9c740099c0 tx=0x7f9c74009cd0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.827+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2547509181 shutdown_connections 2024-05-09T00:16:52.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.827+0000 7f9c91c9d700 1 --2- 172.21.15.46:0/2547509181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c8c108740 0x7f9c8c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.827+0000 7f9c91c9d700 1 --2- 172.21.15.46:0/2547509181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c8c1034c0 0x7f9c8c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.827+0000 7f9c91c9d700 1 --2- 172.21.15.46:0/2547509181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c107df0 0x7f9c8c108200 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.827+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2547509181 >> 172.21.15.46:0/2547509181 conn(0x7f9c8c0fedf0 msgr2=0x7f9c8c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:52.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.827+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2547509181 shutdown_connections 2024-05-09T00:16:52.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.827+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2547509181 wait complete. 2024-05-09T00:16:52.828 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:52 smithi046 ceph-mon[30151]: pgmap v541: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.828+0000 7f9c91c9d700 1 Processor -- start 2024-05-09T00:16:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.828+0000 7f9c91c9d700 1 -- start start 2024-05-09T00:16:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.828+0000 7f9c91c9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c1034c0 0x7f9c8c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.828+0000 7f9c91c9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c8c107df0 0x7f9c8c19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.828+0000 7f9c91c9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c8c108740 0x7f9c8c1a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.828+0000 7f9c91c9d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c113350 con 0x7f9c8c1034c0 2024-05-09T00:16:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.828+0000 7f9c91c9d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c1131d0 con 0x7f9c8c108740 2024-05-09T00:16:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.828+0000 7f9c91c9d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c1134d0 con 0x7f9c8c107df0 2024-05-09T00:16:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.828+0000 7f9c8b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c1034c0 0x7f9c8c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.828+0000 7f9c8b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c1034c0 0x7f9c8c19d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42210/0 (socket says 172.21.15.46:42210) 2024-05-09T00:16:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.828+0000 7f9c8b7fe700 1 -- 172.21.15.46:0/2132665102 learned_addr learned my addr 172.21.15.46:0/2132665102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.828+0000 7f9c8bfff700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c8c108740 0x7f9c8c1a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.829+0000 7f9c82ffd700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c8c107df0 0x7f9c8c19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.829+0000 7f9c8b7fe700 1 -- 172.21.15.46:0/2132665102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c8c107df0 msgr2=0x7f9c8c19d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.829+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c8c107df0 0x7f9c8c19d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.829+0000 7f9c8b7fe700 1 -- 172.21.15.46:0/2132665102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c8c108740 msgr2=0x7f9c8c1a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.829+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c8c108740 0x7f9c8c1a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.829+0000 7f9c8b7fe700 1 -- 172.21.15.46:0/2132665102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8c1a1ee0 con 0x7f9c8c1034c0 2024-05-09T00:16:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.829+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c1034c0 0x7f9c8c19d010 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c920 tx=0x7f9c7c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.829+0000 7f9c897fa700 1 -- 172.21.15.46:0/2132665102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c007b00 con 0x7f9c8c1034c0 2024-05-09T00:16:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.829+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2132665102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8c1a21d0 con 0x7f9c8c1034c0 2024-05-09T00:16:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.829+0000 7f9c897fa700 1 -- 172.21.15.46:0/2132665102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c7c004ce0 con 0x7f9c8c1034c0 2024-05-09T00:16:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.829+0000 7f9c897fa700 1 -- 172.21.15.46:0/2132665102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c005550 con 0x7f9c8c1034c0 2024-05-09T00:16:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.829+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2132665102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8c1a9af0 con 0x7f9c8c1034c0 2024-05-09T00:16:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.833+0000 7f9c897fa700 1 -- 172.21.15.46:0/2132665102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9c7c00f450 con 0x7f9c8c1034c0 2024-05-09T00:16:52.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9ca06b3b80 1 Processor -- start 2024-05-09T00:16:52.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9ca06b3b80 1 -- start start 2024-05-09T00:16:52.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9c897fa700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9c780b2900 0x7f9c780b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9ca06b3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7ce9d00 0x5569a7df2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9ca06b3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7c6f4a0 0x5569a7df2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9c8bfff700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9c780b2900 0x7f9c780b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9ca06b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7dec400 0x5569a7dec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9ca06b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5569a7d0f0a0 con 0x5569a7dec400 2024-05-09T00:16:52.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9ca06b3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5569a7cb49a0 con 0x5569a7c6f4a0 2024-05-09T00:16:52.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9ca06b3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5569a7d0b060 con 0x5569a7ce9d00 2024-05-09T00:16:52.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9c897fa700 1 -- 172.21.15.46:0/2132665102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9c7c0e20f0 con 0x7f9c8c1034c0 2024-05-09T00:16:52.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9c8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7dec400 0x5569a7dec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.834+0000 7f9c8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7dec400 0x5569a7dec810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42222/0 (socket says 172.21.15.46:42222) 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c8bfff700 1 -- 172.21.15.46:0/79175230 learned_addr learned my addr 172.21.15.46:0/79175230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/79175230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7ce9d00 0x5569a7df2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c82ffd700 1 --2- 172.21.15.46:0/79175230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7c6f4a0 0x5569a7df2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c8bfff700 1 -- 172.21.15.46:0/79175230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7ce9d00 msgr2=0x5569a7df2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c8bfff700 1 --2- 172.21.15.46:0/79175230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7ce9d00 0x5569a7df2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c8bfff700 1 -- 172.21.15.46:0/79175230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7c6f4a0 msgr2=0x5569a7df2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c8bfff700 1 --2- 172.21.15.46:0/79175230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7c6f4a0 0x5569a7df2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c8bfff700 1 -- 172.21.15.46:0/79175230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a7ca5500 con 0x5569a7dec400 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c8bfff700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9c780b2900 0x7f9c780b4e10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9c84004230 tx=0x7f9c84003ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c8bfff700 1 --2- 172.21.15.46:0/79175230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7dec400 0x5569a7dec810 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9c84000b60 tx=0x7f9c8401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d613b8c9f946b7d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c81ffb700 1 -- 172.21.15.46:0/79175230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84004bb0 con 0x5569a7dec400 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c81ffb700 1 -- 172.21.15.46:0/79175230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c84004d50 con 0x5569a7dec400 2024-05-09T00:16:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.835+0000 7f9c81ffb700 1 -- 172.21.15.46:0/79175230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84025b50 con 0x5569a7dec400 2024-05-09T00:16:52.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.836+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/79175230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7dec400 msgr2=0x5569a7dec810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:52.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.836+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/79175230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7dec400 0x5569a7dec810 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9c84000b60 tx=0x7f9c8401c000 comp rx=0 tx=0).stop 2024-05-09T00:16:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.836+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/79175230 shutdown_connections 2024-05-09T00:16:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.836+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/79175230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7dec400 0x5569a7dec810 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.836+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/79175230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7ce9d00 0x5569a7df2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.836+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/79175230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7c6f4a0 0x5569a7df2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.836+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/79175230 >> 172.21.15.46:0/79175230 conn(0x5569a7c72f70 msgr2=0x5569a7de7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.837+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/79175230 shutdown_connections 2024-05-09T00:16:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.837+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/79175230 wait complete. 2024-05-09T00:16:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.837+0000 7f9ca06b3b80 1 -- start start 2024-05-09T00:16:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.837+0000 7f9ca06b3b80 1 Processor -- start 2024-05-09T00:16:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.837+0000 7f9ca06b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c6f4a0 0x5569a7df6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.837+0000 7f9ca06b3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7ce9d00 0x5569a7df6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9ca06b3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7dec400 0x5569a7df6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9ca06b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5569a7ded100 con 0x5569a7c6f4a0 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9ca06b3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5569a7ca9b20 con 0x5569a7dec400 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9ca06b3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5569a7e021d0 con 0x5569a7ce9d00 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9ca06b3b80 10 client.0 _ll_register_callbacks cb 0x5569a7b806c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c8b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c6f4a0 0x5569a7df6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c8b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c6f4a0 0x5569a7df6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42234/0 (socket says 172.21.15.46:42234) 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c8b7fe700 1 -- 172.21.15.46:0/1423363378 learned_addr learned my addr 172.21.15.46:0/1423363378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c82ffd700 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7ce9d00 0x5569a7df6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c8b7fe700 1 -- 172.21.15.46:0/1423363378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7ce9d00 msgr2=0x5569a7df6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7ce9d00 0x5569a7df6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c8b7fe700 1 -- 172.21.15.46:0/1423363378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7dec400 msgr2=0x5569a7df6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7dec400 0x5569a7df6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c8b7fe700 1 -- 172.21.15.46:0/1423363378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a7b8a490 con 0x5569a7c6f4a0 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c82ffd700 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7ce9d00 0x5569a7df6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c6f4a0 0x5569a7df6040 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0a9190 tx=0x7f9c7c0a9780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1deda12743c66f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c6affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c6affd700 1 -- 172.21.15.46:0/1423363378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0c3060 con 0x5569a7c6f4a0 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9c6affd700 1 -- 172.21.15.46:0/1423363378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9c7c0a5cc0 con 0x5569a7c6f4a0 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.838+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1423363378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c84004d50 con 0x5569a7c6f4a0 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.839+0000 7f9c6affd700 1 -- 172.21.15.46:0/1423363378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c7c0ca050 con 0x5569a7c6f4a0 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.839+0000 7f9ca06b3b80 10 client.16167 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.839+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1423363378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5569a7e03b80 con 0x5569a7c6f4a0 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.839+0000 7f9c6affd700 1 -- 172.21.15.46:0/1423363378 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c84004d50 con 0x5569a7c6f4a0 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.839+0000 7f9c6affd700 1 -- 172.21.15.46:0/1423363378 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9c7c0a9930 con 0x5569a7c6f4a0 2024-05-09T00:16:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.839+0000 7f9ca06b3b80 10 client.16167 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:52.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.839+0000 7f9ca06b3b80 10 client.16167 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:16:52.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.839+0000 7f9ca06b3b80 20 client.16167 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:52.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.839+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5569a7e06110 0x5569a7e08530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.839+0000 7f9ca06b3b80 4 client.16167 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:16:52.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.839+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1423363378 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569a7d1ca80 con 0x5569a7e06110 2024-05-09T00:16:52.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.840+0000 7f9c82ffd700 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5569a7e06110 0x5569a7e08530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.840+0000 7f9c82ffd700 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5569a7e06110 0x5569a7e08530 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d25dffb7690a39d1 server_cookie=776e379bec52a9d8 in_seq=0 out_seq=0 2024-05-09T00:16:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.840+0000 7f9c6affd700 10 client.16167 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:16:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9c6affd700 1 -- 172.21.15.46:0/1423363378 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5569a7d1ca80 con 0x5569a7e06110 2024-05-09T00:16:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9c6affd700 10 client.16167 handle_command_reply: tid=0 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9ca06b3b80 1 client.16167 shutdown 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1423363378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c6f4a0 msgr2=0x5569a7df6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c6f4a0 0x5569a7df6040 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0a9190 tx=0x7f9c7c0a9780 comp rx=0 tx=0).stop 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1423363378 shutdown_connections 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5569a7e06110 0x5569a7e08530 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7dec400 0x5569a7df6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c6f4a0 0x5569a7df6040 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1423363378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7ce9d00 0x5569a7df6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1423363378 >> 172.21.15.46:0/1423363378 conn(0x5569a7c72f70 msgr2=0x5569a7de0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9c6affd700 0 client.16167 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1423363378 shutdown_connections 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.842+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1423363378 wait complete. 2024-05-09T00:16:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.843+0000 7f9ca06b3b80 20 client.16167 trim_cache size 0 max 16384 2024-05-09T00:16:52.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.858+0000 7f9ca06b3b80 1 Processor -- start 2024-05-09T00:16:52.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.858+0000 7f9ca06b3b80 1 -- start start 2024-05-09T00:16:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.858+0000 7f9ca06b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c72f70 0x5569a7df5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.858+0000 7f9ca06b3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7c6f4a0 0x5569a7df6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.858+0000 7f9ca06b3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7ce9d00 0x5569a7e0dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.858+0000 7f9ca06b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c1b3720 con 0x5569a7c72f70 2024-05-09T00:16:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.858+0000 7f9ca06b3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c1b0fb0 con 0x5569a7ce9d00 2024-05-09T00:16:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.858+0000 7f9ca06b3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c1aede0 con 0x5569a7c6f4a0 2024-05-09T00:16:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.858+0000 7f9c8b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c72f70 0x5569a7df5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.858+0000 7f9c8b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c72f70 0x5569a7df5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42242/0 (socket says 172.21.15.46:42242) 2024-05-09T00:16:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.858+0000 7f9c8b7fe700 1 -- 172.21.15.46:0/1074216505 learned_addr learned my addr 172.21.15.46:0/1074216505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.858+0000 7f9c82ffd700 1 --2- 172.21.15.46:0/1074216505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7c6f4a0 0x5569a7df6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.859+0000 7f9c8b7fe700 1 -- 172.21.15.46:0/1074216505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7c6f4a0 msgr2=0x5569a7df6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:52.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.859+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/1074216505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7c6f4a0 0x5569a7df6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.859+0000 7f9c8b7fe700 1 -- 172.21.15.46:0/1074216505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7ce9d00 msgr2=0x5569a7e0dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:52.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.859+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/1074216505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7ce9d00 0x5569a7e0dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.859+0000 7f9c8b7fe700 1 -- 172.21.15.46:0/1074216505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a7de39d0 con 0x5569a7c72f70 2024-05-09T00:16:52.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.859+0000 7f9c82ffd700 1 --2- 172.21.15.46:0/1074216505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7c6f4a0 0x5569a7df6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:16:52.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.859+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/1074216505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c72f70 0x5569a7df5360 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0a5750 tx=0x7f9c7c0b23a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0afadb5b8af7d83 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:52.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.859+0000 7f9c817fa700 1 -- 172.21.15.46:0/1074216505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c087070 con 0x5569a7c72f70 2024-05-09T00:16:52.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.859+0000 7f9c817fa700 1 -- 172.21.15.46:0/1074216505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c7c0056f0 con 0x5569a7c72f70 2024-05-09T00:16:52.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.859+0000 7f9c817fa700 1 -- 172.21.15.46:0/1074216505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c01d040 con 0x5569a7c72f70 2024-05-09T00:16:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.860+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1074216505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c72f70 msgr2=0x5569a7df5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.860+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1074216505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c72f70 0x5569a7df5360 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0a5750 tx=0x7f9c7c0b23a0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.860+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1074216505 shutdown_connections 2024-05-09T00:16:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.860+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1074216505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7ce9d00 0x5569a7e0dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.860+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1074216505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7c72f70 0x5569a7df5360 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.860+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1074216505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7c6f4a0 0x5569a7df6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.860+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1074216505 >> 172.21.15.46:0/1074216505 conn(0x5569a7de7520 msgr2=0x5569a7df1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.860+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1074216505 shutdown_connections 2024-05-09T00:16:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.860+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1074216505 wait complete. 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.860+0000 7f9ca06b3b80 1 -- start start 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.860+0000 7f9ca06b3b80 1 Processor -- start 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9ca06b3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7c6f4a0 0x5569a7dfe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9ca06b3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7c72f70 0x5569a7e0cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9ca06b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7ce9d00 0x5569a7e0d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9ca06b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c1af470 con 0x5569a7ce9d00 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9ca06b3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c1b08e0 con 0x5569a7c6f4a0 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9ca06b3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c1afb40 con 0x5569a7c72f70 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9ca06b3b80 10 client.0 _ll_register_callbacks cb 0x5569a7b806c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9c8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7ce9d00 0x5569a7e0d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9c8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7ce9d00 0x5569a7e0d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42256/0 (socket says 172.21.15.46:42256) 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9c8bfff700 1 -- 172.21.15.46:0/1743100249 learned_addr learned my addr 172.21.15.46:0/1743100249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9c82ffd700 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7c72f70 0x5569a7e0cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7c6f4a0 0x5569a7dfe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9c8bfff700 1 -- 172.21.15.46:0/1743100249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7c72f70 msgr2=0x5569a7e0cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9c8bfff700 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7c72f70 0x5569a7e0cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9c8bfff700 1 -- 172.21.15.46:0/1743100249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7c6f4a0 msgr2=0x5569a7dfe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9c8bfff700 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7c6f4a0 0x5569a7dfe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9c8bfff700 1 -- 172.21.15.46:0/1743100249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a7de0cc0 con 0x5569a7ce9d00 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9c8bfff700 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7ce9d00 0x5569a7e0d2c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9c84003cb0 tx=0x7f9c84007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc74fe19981314f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.861+0000 7f9c6affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9c6affd700 1 -- 172.21.15.46:0/1743100249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84004a90 con 0x5569a7ce9d00 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9c6affd700 1 -- 172.21.15.46:0/1743100249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9c84025e00 con 0x5569a7ce9d00 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1743100249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c7c0056f0 con 0x5569a7ce9d00 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9c6affd700 1 -- 172.21.15.46:0/1743100249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c84004c30 con 0x5569a7ce9d00 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9ca06b3b80 10 client.16173 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1743100249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5569a7e70050 con 0x5569a7ce9d00 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9c6affd700 1 -- 172.21.15.46:0/1743100249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c7c0056f0 con 0x5569a7ce9d00 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9c6affd700 1 -- 172.21.15.46:0/1743100249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9c84015030 con 0x5569a7ce9d00 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9ca06b3b80 10 client.16173 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9ca06b3b80 10 client.16173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9ca06b3b80 20 client.16173 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5569a7dec400 0x5569a7e726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9ca06b3b80 4 client.16173 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.862+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1743100249 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569a7d16b20 con 0x5569a7dec400 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.863+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5569a7dec400 0x5569a7e726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.863+0000 7f9c6affd700 10 client.16173 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.863+0000 7f9c8b7fe700 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5569a7dec400 0x5569a7e726c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ba4485aae83e9b9 server_cookie=96572de791b3212c in_seq=0 out_seq=0 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.863+0000 7f9c6affd700 1 -- 172.21.15.46:0/1743100249 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569a7d16b20 con 0x5569a7dec400 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.863+0000 7f9c6affd700 10 client.16173 handle_command_reply: tid=0 2024-05-09T00:16:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.863+0000 7f9ca06b3b80 1 client.16173 shutdown 2024-05-09T00:16:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.863+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1743100249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7ce9d00 msgr2=0x5569a7e0d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.863+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7ce9d00 0x5569a7e0d2c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9c84003cb0 tx=0x7f9c84007f90 comp rx=0 tx=0).stop 2024-05-09T00:16:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.863+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1743100249 shutdown_connections 2024-05-09T00:16:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.863+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x5569a7dec400 0x5569a7e726c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.863+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a7ce9d00 0x5569a7e0d2c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5569a7c6f4a0 0x5569a7dfe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9ca06b3b80 1 --2- 172.21.15.46:0/1743100249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569a7c72f70 0x5569a7e0cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9c6affd700 0 client.16173 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:16:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1743100249 >> 172.21.15.46:0/1743100249 conn(0x5569a7de7520 msgr2=0x5569a7de0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1743100249 shutdown_connections 2024-05-09T00:16:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9ca06b3b80 1 -- 172.21.15.46:0/1743100249 wait complete. 2024-05-09T00:16:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9ca06b3b80 20 client.16173 trim_cache size 0 max 16384 2024-05-09T00:16:52.869 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:16:52.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2132665102 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9c780b2900 msgr2=0x7f9c780b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:52.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9c91c9d700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9c780b2900 0x7f9c780b4e10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9c84004230 tx=0x7f9c84003ed0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2132665102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c1034c0 msgr2=0x7f9c8c19d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:52.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9c91c9d700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c1034c0 0x7f9c8c19d010 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c920 tx=0x7f9c7c00cc30 comp rx=0 tx=0).stop 2024-05-09T00:16:52.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2132665102 shutdown_connections 2024-05-09T00:16:52.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9c91c9d700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9c780b2900 0x7f9c780b4e10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9c91c9d700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c8c108740 0x7f9c8c1a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9c91c9d700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c1034c0 0x7f9c8c19d010 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9c91c9d700 1 --2- 172.21.15.46:0/2132665102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c8c107df0 0x7f9c8c19d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:52.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.864+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2132665102 >> 172.21.15.46:0/2132665102 conn(0x7f9c8c0fedf0 msgr2=0x7f9c8c1076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:52.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.865+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2132665102 shutdown_connections 2024-05-09T00:16:52.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:52.865+0000 7f9c91c9d700 1 -- 172.21.15.46:0/2132665102 wait complete. 2024-05-09T00:16:52.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:52 smithi142 ceph-mon[34533]: pgmap v541: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:53.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:52 smithi169 ceph-mon[34777]: pgmap v541: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:16:53.748 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:16:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:16:54.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:54 smithi046 ceph-mon[30151]: pgmap v542: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:54.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:54 smithi142 ceph-mon[34533]: pgmap v542: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:55.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:54 smithi169 ceph-mon[34777]: pgmap v542: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:16:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.916+0000 7f6dad20b700 1 -- 172.21.15.46:0/1412735843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da81034c0 msgr2=0x7f6da81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:55.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.916+0000 7f6dad20b700 1 --2- 172.21.15.46:0/1412735843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da81034c0 0x7f6da81038d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6d980099c0 tx=0x7f6d98009cd0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.916+0000 7f6dad20b700 1 -- 172.21.15.46:0/1412735843 shutdown_connections 2024-05-09T00:16:55.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.916+0000 7f6dad20b700 1 --2- 172.21.15.46:0/1412735843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da8108740 0x7f6da81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.916+0000 7f6dad20b700 1 --2- 172.21.15.46:0/1412735843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da81034c0 0x7f6da81038d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.916+0000 7f6dad20b700 1 --2- 172.21.15.46:0/1412735843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6da8107df0 0x7f6da8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.916+0000 7f6dad20b700 1 -- 172.21.15.46:0/1412735843 >> 172.21.15.46:0/1412735843 conn(0x7f6da80fedc0 msgr2=0x7f6da81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:55.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.916+0000 7f6dad20b700 1 -- 172.21.15.46:0/1412735843 shutdown_connections 2024-05-09T00:16:55.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.916+0000 7f6dad20b700 1 -- 172.21.15.46:0/1412735843 wait complete. 2024-05-09T00:16:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.917+0000 7f6dad20b700 1 Processor -- start 2024-05-09T00:16:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.917+0000 7f6dad20b700 1 -- start start 2024-05-09T00:16:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.917+0000 7f6dad20b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da81034c0 0x7f6da80ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.917+0000 7f6dad20b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da8107df0 0x7f6da80ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.917+0000 7f6dad20b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6da8108740 0x7f6da80ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.917+0000 7f6dad20b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6da8113370 con 0x7f6da81034c0 2024-05-09T00:16:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.917+0000 7f6dad20b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81131f0 con 0x7f6da8108740 2024-05-09T00:16:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.917+0000 7f6dad20b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81134f0 con 0x7f6da8107df0 2024-05-09T00:16:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6da6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da81034c0 0x7f6da80ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6da6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da81034c0 0x7f6da80ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42274/0 (socket says 172.21.15.46:42274) 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6da6d9d700 1 -- 172.21.15.46:0/3301218430 learned_addr learned my addr 172.21.15.46:0/3301218430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6da759e700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6da8108740 0x7f6da80ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6da659c700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da8107df0 0x7f6da80ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6da6d9d700 1 -- 172.21.15.46:0/3301218430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da8107df0 msgr2=0x7f6da80ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6da6d9d700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da8107df0 0x7f6da80ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6da6d9d700 1 -- 172.21.15.46:0/3301218430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6da8108740 msgr2=0x7f6da80ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6da6d9d700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6da8108740 0x7f6da80ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6da6d9d700 1 -- 172.21.15.46:0/3301218430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da81a9130 con 0x7f6da81034c0 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6da659c700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da8107df0 0x7f6da80ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6da6d9d700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da81034c0 0x7f6da80ff2a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800b580 tx=0x7f6d980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6d8ffff700 1 -- 172.21.15.46:0/3301218430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98027070 con 0x7f6da81034c0 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6dad20b700 1 -- 172.21.15.46:0/3301218430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da81a93c0 con 0x7f6da81034c0 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6d8ffff700 1 -- 172.21.15.46:0/3301218430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d98003f20 con 0x7f6da81034c0 2024-05-09T00:16:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.918+0000 7f6d8ffff700 1 -- 172.21.15.46:0/3301218430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98022d70 con 0x7f6da81034c0 2024-05-09T00:16:55.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.919+0000 7f6dad20b700 1 -- 172.21.15.46:0/3301218430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da81a9810 con 0x7f6da81034c0 2024-05-09T00:16:55.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.921+0000 7f6d8ffff700 1 -- 172.21.15.46:0/3301218430 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6d98022500 con 0x7f6da81034c0 2024-05-09T00:16:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.922+0000 7f6d8ffff700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d900b2880 0x7f6d900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.922+0000 7f6da659c700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d900b2880 0x7f6d900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.922+0000 7f6d8ffff700 1 -- 172.21.15.46:0/3301218430 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6d980e2fb0 con 0x7f6da81034c0 2024-05-09T00:16:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.922+0000 7f6dbbc21b80 1 Processor -- start 2024-05-09T00:16:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.922+0000 7f6dbbc21b80 1 -- start start 2024-05-09T00:16:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6da659c700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d900b2880 0x7f6d900b4d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c005950 tx=0x7f6d9c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:55.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6dbbc21b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b835bd00 0x5580b8464240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6dbbc21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b82e14a0 0x5580b8464780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6dbbc21b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b845e400 0x5580b845e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6dbbc21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5580b83810a0 con 0x5580b82e14a0 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6dbbc21b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5580b83269a0 con 0x5580b835bd00 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6dbbc21b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5580b837d060 con 0x5580b845e400 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6da659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b82e14a0 0x5580b8464780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6da659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b82e14a0 0x5580b8464780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42284/0 (socket says 172.21.15.46:42284) 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6da6d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b835bd00 0x5580b8464240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6da659c700 1 -- 172.21.15.46:0/3666255470 learned_addr learned my addr 172.21.15.46:0/3666255470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6da759e700 1 --2- 172.21.15.46:0/3666255470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b845e400 0x5580b845e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6da659c700 1 -- 172.21.15.46:0/3666255470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b845e400 msgr2=0x5580b845e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6da659c700 1 --2- 172.21.15.46:0/3666255470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b845e400 0x5580b845e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6da659c700 1 -- 172.21.15.46:0/3666255470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b835bd00 msgr2=0x5580b8464240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6da659c700 1 --2- 172.21.15.46:0/3666255470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b835bd00 0x5580b8464240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6da659c700 1 -- 172.21.15.46:0/3666255470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b8317500 con 0x5580b82e14a0 2024-05-09T00:16:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.923+0000 7f6da6d9d700 1 --2- 172.21.15.46:0/3666255470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b835bd00 0x5580b8464240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:16:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.924+0000 7f6da659c700 1 --2- 172.21.15.46:0/3666255470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b82e14a0 0x5580b8464780 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00b5d0 tx=0x7f6d9c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6386b52633a47d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.924+0000 7f6d8d7fa700 1 -- 172.21.15.46:0/3666255470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c003880 con 0x5580b82e14a0 2024-05-09T00:16:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.924+0000 7f6d8d7fa700 1 -- 172.21.15.46:0/3666255470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c024070 con 0x5580b82e14a0 2024-05-09T00:16:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.924+0000 7f6d8d7fa700 1 -- 172.21.15.46:0/3666255470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c01fc30 con 0x5580b82e14a0 2024-05-09T00:16:55.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.925+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/3666255470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b82e14a0 msgr2=0x5580b8464780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.925+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/3666255470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b82e14a0 0x5580b8464780 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00b5d0 tx=0x7f6d9c012000 comp rx=0 tx=0).stop 2024-05-09T00:16:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.925+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/3666255470 shutdown_connections 2024-05-09T00:16:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.925+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/3666255470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b845e400 0x5580b845e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.925+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/3666255470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b835bd00 0x5580b8464240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.925+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/3666255470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b82e14a0 0x5580b8464780 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.925+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/3666255470 >> 172.21.15.46:0/3666255470 conn(0x5580b82e4f70 msgr2=0x5580b8459220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.925+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/3666255470 shutdown_connections 2024-05-09T00:16:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.925+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/3666255470 wait complete. 2024-05-09T00:16:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.925+0000 7f6dbbc21b80 1 -- start start 2024-05-09T00:16:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.925+0000 7f6dbbc21b80 1 Processor -- start 2024-05-09T00:16:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6dbbc21b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8468020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6dbbc21b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b835bd00 0x5580b8468560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6dbbc21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b845e400 0x5580b8468aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6dbbc21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5580b845f160 con 0x5580b845e400 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6dbbc21b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5580b831bb20 con 0x5580b82e14a0 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6dbbc21b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5580b846b020 con 0x5580b835bd00 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6dbbc21b80 10 client.0 _ll_register_callbacks cb 0x5580b81f26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6da759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b845e400 0x5580b8468aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6da759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b845e400 0x5580b8468aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42286/0 (socket says 172.21.15.46:42286) 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6da759e700 1 -- 172.21.15.46:0/841463579 learned_addr learned my addr 172.21.15.46:0/841463579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6da659c700 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b835bd00 0x5580b8468560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6da6d9d700 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8468020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6da759e700 1 -- 172.21.15.46:0/841463579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b835bd00 msgr2=0x5580b8468560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6da759e700 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b835bd00 0x5580b8468560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6da759e700 1 -- 172.21.15.46:0/841463579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 msgr2=0x5580b8468020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6da759e700 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8468020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6da759e700 1 -- 172.21.15.46:0/841463579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b81fc490 con 0x5580b845e400 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6da6d9d700 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8468020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6da759e700 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b845e400 0x5580b8468aa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6da0011cc0 tx=0x7f6da0007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455e71ea6198f964 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6d8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.926+0000 7f6d8aff5700 1 -- 172.21.15.46:0/841463579 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da001b020 con 0x5580b845e400 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6d8aff5700 1 -- 172.21.15.46:0/841463579 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6da000f9f0 con 0x5580b845e400 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/841463579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d9c024070 con 0x5580b845e400 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6d8aff5700 1 -- 172.21.15.46:0/841463579 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6da000fbf0 con 0x5580b845e400 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6dbbc21b80 10 client.16185 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/841463579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580b8475a50 con 0x5580b845e400 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6d8aff5700 1 -- 172.21.15.46:0/841463579 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c024070 con 0x5580b845e400 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6d8aff5700 1 -- 172.21.15.46:0/841463579 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6da0022430 con 0x5580b845e400 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6dbbc21b80 10 client.16185 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6dbbc21b80 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:16:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6dbbc21b80 20 client.16185 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:55.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580b8478090 0x5580b847a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6dbbc21b80 4 client.16185 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:16:55.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.927+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/841463579 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580b838ea80 con 0x5580b8478090 2024-05-09T00:16:55.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.928+0000 7f6da6d9d700 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580b8478090 0x5580b847a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.928+0000 7f6da6d9d700 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580b8478090 0x5580b847a4f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47cb5a1dd01e1b0 server_cookie=7a71788e757f8a88 in_seq=0 out_seq=0 2024-05-09T00:16:55.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.928+0000 7f6d8aff5700 10 client.16185 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:55.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.930+0000 7f6d8aff5700 1 -- 172.21.15.46:0/841463579 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5580b838ea80 con 0x5580b8478090 2024-05-09T00:16:55.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.930+0000 7f6d8aff5700 10 client.16185 handle_command_reply: tid=0 2024-05-09T00:16:55.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.930+0000 7f6dbbc21b80 1 client.16185 shutdown 2024-05-09T00:16:55.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.931+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/841463579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b845e400 msgr2=0x5580b8468aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:55.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.931+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b845e400 0x5580b8468aa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6da0011cc0 tx=0x7f6da0007f10 comp rx=0 tx=0).stop 2024-05-09T00:16:55.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.931+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/841463579 shutdown_connections 2024-05-09T00:16:55.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.931+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580b8478090 0x5580b847a4f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.931+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b845e400 0x5580b8468aa0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.931+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8468020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.931+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/841463579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b835bd00 0x5580b8468560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.931+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/841463579 >> 172.21.15.46:0/841463579 conn(0x5580b82e4f70 msgr2=0x5580b8452ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:55.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.931+0000 7f6d8aff5700 0 client.16185 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:55.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.931+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/841463579 shutdown_connections 2024-05-09T00:16:55.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.931+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/841463579 wait complete. 2024-05-09T00:16:55.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.931+0000 7f6dbbc21b80 20 client.16185 trim_cache size 0 max 16384 2024-05-09T00:16:55.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.946+0000 7f6dbbc21b80 1 Processor -- start 2024-05-09T00:16:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.946+0000 7f6dbbc21b80 1 -- start start 2024-05-09T00:16:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.946+0000 7f6dbbc21b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b82e4f70 0x5580b8467020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.946+0000 7f6dbbc21b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8473e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.946+0000 7f6dbbc21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 0x5580b8474360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.946+0000 7f6dbbc21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b35b0 con 0x5580b835bd00 2024-05-09T00:16:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.946+0000 7f6dbbc21b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b0e40 con 0x5580b82e14a0 2024-05-09T00:16:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.946+0000 7f6dbbc21b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81aec70 con 0x5580b82e4f70 2024-05-09T00:16:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.946+0000 7f6da759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 0x5580b8474360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.946+0000 7f6da759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 0x5580b8474360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42294/0 (socket says 172.21.15.46:42294) 2024-05-09T00:16:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.946+0000 7f6da759e700 1 -- 172.21.15.46:0/1986922527 learned_addr learned my addr 172.21.15.46:0/1986922527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.946+0000 7f6da6d9d700 1 --2- 172.21.15.46:0/1986922527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b82e4f70 0x5580b8467020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.947+0000 7f6da659c700 1 --2- 172.21.15.46:0/1986922527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8473e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.947+0000 7f6da759e700 1 -- 172.21.15.46:0/1986922527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b82e4f70 msgr2=0x5580b8467020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.947+0000 7f6da759e700 1 --2- 172.21.15.46:0/1986922527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b82e4f70 0x5580b8467020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.947+0000 7f6da759e700 1 -- 172.21.15.46:0/1986922527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 msgr2=0x5580b8473e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.947+0000 7f6da759e700 1 --2- 172.21.15.46:0/1986922527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8473e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.947+0000 7f6da759e700 1 -- 172.21.15.46:0/1986922527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b846a8c0 con 0x5580b835bd00 2024-05-09T00:16:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.947+0000 7f6da759e700 1 --2- 172.21.15.46:0/1986922527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 0x5580b8474360 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6da00228a0 tx=0x7f6da000fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e9e90e9de3242a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.947+0000 7f6d8cff9700 1 -- 172.21.15.46:0/1986922527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da001b040 con 0x5580b835bd00 2024-05-09T00:16:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.947+0000 7f6d8cff9700 1 -- 172.21.15.46:0/1986922527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da0022b10 con 0x5580b835bd00 2024-05-09T00:16:55.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.947+0000 7f6d8cff9700 1 -- 172.21.15.46:0/1986922527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da00176e0 con 0x5580b835bd00 2024-05-09T00:16:55.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.948+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1986922527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 msgr2=0x5580b8474360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:55.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.948+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/1986922527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 0x5580b8474360 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6da00228a0 tx=0x7f6da000fb80 comp rx=0 tx=0).stop 2024-05-09T00:16:55.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.948+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1986922527 shutdown_connections 2024-05-09T00:16:55.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.948+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/1986922527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 0x5580b8474360 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.948+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/1986922527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b82e4f70 0x5580b8467020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.948+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/1986922527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8473e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.948+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1986922527 >> 172.21.15.46:0/1986922527 conn(0x5580b8459520 msgr2=0x5580b8466910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:55.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.948+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1986922527 shutdown_connections 2024-05-09T00:16:55.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.948+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1986922527 wait complete. 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.948+0000 7f6dbbc21b80 1 -- start start 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.948+0000 7f6dbbc21b80 1 Processor -- start 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6dbbc21b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8470300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6dbbc21b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b82e4f70 0x5580b8470840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6dbbc21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 0x5580b847ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6dbbc21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81af300 con 0x5580b835bd00 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6dbbc21b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81b0770 con 0x5580b82e14a0 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6dbbc21b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6da81af9d0 con 0x5580b82e4f70 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6da759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 0x5580b847ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6dbbc21b80 10 client.0 _ll_register_callbacks cb 0x5580b81f26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6da759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 0x5580b847ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42306/0 (socket says 172.21.15.46:42306) 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6da759e700 1 -- 172.21.15.46:0/1314109877 learned_addr learned my addr 172.21.15.46:0/1314109877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6da659c700 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b82e4f70 0x5580b8470840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6da6d9d700 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8470300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6da759e700 1 -- 172.21.15.46:0/1314109877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b82e4f70 msgr2=0x5580b8470840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6da759e700 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b82e4f70 0x5580b8470840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6da759e700 1 -- 172.21.15.46:0/1314109877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 msgr2=0x5580b8470300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6da759e700 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8470300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6da759e700 1 -- 172.21.15.46:0/1314109877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b847f4e0 con 0x5580b835bd00 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6da759e700 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 0x5580b847ed60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6da0012010 tx=0x7f6da0007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacb68bb3567578f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.949+0000 7f6d8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6d8aff5700 1 -- 172.21.15.46:0/1314109877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0031020 con 0x5580b835bd00 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6d8aff5700 1 -- 172.21.15.46:0/1314109877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6da0017480 con 0x5580b835bd00 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1314109877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da0022b10 con 0x5580b835bd00 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6d8aff5700 1 -- 172.21.15.46:0/1314109877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6da001b040 con 0x5580b835bd00 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6dbbc21b80 10 client.16191 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1314109877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580b84e2240 con 0x5580b835bd00 2024-05-09T00:16:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6d8aff5700 1 -- 172.21.15.46:0/1314109877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da0022b10 con 0x5580b835bd00 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6d8aff5700 1 -- 172.21.15.46:0/1314109877 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6da002b440 con 0x5580b835bd00 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6dbbc21b80 10 client.16191 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6dbbc21b80 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6dbbc21b80 20 client.16191 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580b845e400 0x5580b84e4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6dbbc21b80 4 client.16191 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.950+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1314109877 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580b8388b20 con 0x5580b845e400 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6da6d9d700 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580b845e400 0x5580b84e4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6da6d9d700 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580b845e400 0x5580b84e4870 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b816dfb0eb93a8 server_cookie=b35c6ce3a472a96e in_seq=0 out_seq=0 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6d8aff5700 10 client.16191 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6d8aff5700 1 -- 172.21.15.46:0/1314109877 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580b8388b20 con 0x5580b845e400 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6d8aff5700 10 client.16191 handle_command_reply: tid=0 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6dbbc21b80 1 client.16191 shutdown 2024-05-09T00:16:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1314109877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 msgr2=0x5580b847ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 0x5580b847ed60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6da0012010 tx=0x7f6da0007a30 comp rx=0 tx=0).stop 2024-05-09T00:16:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1314109877 shutdown_connections 2024-05-09T00:16:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580b845e400 0x5580b84e4870 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580b835bd00 0x5580b847ed60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580b82e14a0 0x5580b8470300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6dbbc21b80 1 --2- 172.21.15.46:0/1314109877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580b82e4f70 0x5580b8470840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6d8aff5700 0 client.16191 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.951+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1314109877 >> 172.21.15.46:0/1314109877 conn(0x5580b8459520 msgr2=0x5580b8466910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1314109877 shutdown_connections 2024-05-09T00:16:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dbbc21b80 1 -- 172.21.15.46:0/1314109877 wait complete. 2024-05-09T00:16:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dbbc21b80 20 client.16191 trim_cache size 0 max 16384 2024-05-09T00:16:55.957 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:16:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dad20b700 1 -- 172.21.15.46:0/3301218430 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d900b2880 msgr2=0x7f6d900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dad20b700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d900b2880 0x7f6d900b4d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c005950 tx=0x7f6d9c0058e0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dad20b700 1 -- 172.21.15.46:0/3301218430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da81034c0 msgr2=0x7f6da80ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dad20b700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da81034c0 0x7f6da80ff2a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800b580 tx=0x7f6d980047b0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dad20b700 1 -- 172.21.15.46:0/3301218430 shutdown_connections 2024-05-09T00:16:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dad20b700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6d900b2880 0x7f6d900b4d90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dad20b700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6da8108740 0x7f6da80ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dad20b700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da81034c0 0x7f6da80ff2a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dad20b700 1 --2- 172.21.15.46:0/3301218430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da8107df0 0x7f6da80ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dad20b700 1 -- 172.21.15.46:0/3301218430 >> 172.21.15.46:0/3301218430 conn(0x7f6da80fedc0 msgr2=0x7f6da806ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dad20b700 1 -- 172.21.15.46:0/3301218430 shutdown_connections 2024-05-09T00:16:55.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:55.952+0000 7f6dad20b700 1 -- 172.21.15.46:0/3301218430 wait complete. 2024-05-09T00:16:56.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:56 smithi142 ceph-mon[34533]: pgmap v543: 97 pgs: 97 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:56.952 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:56 smithi046 ceph-mon[30151]: pgmap v543: 97 pgs: 97 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:57.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:56 smithi169 ceph-mon[34777]: pgmap v543: 97 pgs: 97 active+clean; 2.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:16:57.675 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:16:57.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:16:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:16:58.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:16:58 smithi142 ceph-mon[34533]: pgmap v544: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:59.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:16:58 smithi046 ceph-mon[30151]: pgmap v544: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:59.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:16:58 smithi169 ceph-mon[34777]: pgmap v544: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:16:59.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.821+0000 7fbb28df8700 1 -- 172.21.15.46:0/1741697345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb24108770 msgr2=0x7fbb241100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.821+0000 7fbb28df8700 1 --2- 172.21.15.46:0/1741697345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb24108770 0x7fbb241100d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00b0d0 tx=0x7fbb1c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.821+0000 7fbb28df8700 1 -- 172.21.15.46:0/1741697345 shutdown_connections 2024-05-09T00:16:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.821+0000 7fbb28df8700 1 --2- 172.21.15.46:0/1741697345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb24108770 0x7fbb241100d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.821+0000 7fbb28df8700 1 --2- 172.21.15.46:0/1741697345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb24103540 0x7fbb24103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.821+0000 7fbb28df8700 1 --2- 172.21.15.46:0/1741697345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb24107e20 0x7fbb24108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.821+0000 7fbb28df8700 1 -- 172.21.15.46:0/1741697345 >> 172.21.15.46:0/1741697345 conn(0x7fbb24078e50 msgr2=0x7fbb24079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.821+0000 7fbb28df8700 1 -- 172.21.15.46:0/1741697345 shutdown_connections 2024-05-09T00:16:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.821+0000 7fbb28df8700 1 -- 172.21.15.46:0/1741697345 wait complete. 2024-05-09T00:16:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.822+0000 7fbb28df8700 1 Processor -- start 2024-05-09T00:16:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.822+0000 7fbb28df8700 1 -- start start 2024-05-09T00:16:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.822+0000 7fbb28df8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb24103540 0x7fbb24100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.822+0000 7fbb28df8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb24107e20 0x7fbb24100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.822+0000 7fbb28df8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb24108770 0x7fbb24101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.822+0000 7fbb28df8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241133a0 con 0x7fbb24108770 2024-05-09T00:16:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.822+0000 7fbb28df8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbb24113220 con 0x7fbb24107e20 2024-05-09T00:16:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.822+0000 7fbb28df8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb24113520 con 0x7fbb24103540 2024-05-09T00:16:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.822+0000 7fbb22d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb24108770 0x7fbb24101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.822+0000 7fbb22d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb24108770 0x7fbb24101010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33002/0 (socket says 172.21.15.46:33002) 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.822+0000 7fbb22d9d700 1 -- 172.21.15.46:0/575832668 learned_addr learned my addr 172.21.15.46:0/575832668 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb2259c700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb24103540 0x7fbb24100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb21d9b700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb24107e20 0x7fbb24100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb22d9d700 1 -- 172.21.15.46:0/575832668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb24103540 msgr2=0x7fbb24100590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb22d9d700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb24103540 0x7fbb24100590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb22d9d700 1 -- 172.21.15.46:0/575832668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb24107e20 msgr2=0x7fbb24100ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb22d9d700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb24107e20 0x7fbb24100ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb22d9d700 1 -- 172.21.15.46:0/575832668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb241a9180 con 0x7fbb24108770 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb22d9d700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb24108770 0x7fbb24101010 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009850 tx=0x7fbb1c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb137fe700 1 -- 172.21.15.46:0/575832668 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c01f070 con 0x7fbb24108770 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb137fe700 1 -- 172.21.15.46:0/575832668 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1c004c50 con 0x7fbb24108770 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb28df8700 1 -- 172.21.15.46:0/575832668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb241a9410 con 0x7fbb24108770 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb137fe700 1 -- 172.21.15.46:0/575832668 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c01b690 con 0x7fbb24108770 2024-05-09T00:16:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.823+0000 7fbb28df8700 1 -- 172.21.15.46:0/575832668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb241a9860 con 0x7fbb24108770 2024-05-09T00:16:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.826+0000 7fbb137fe700 1 -- 172.21.15.46:0/575832668 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbb1c022070 con 0x7fbb24108770 2024-05-09T00:16:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.827+0000 7fbb137fe700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbb0c0b2880 0x7fbb0c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.827+0000 7fbb2259c700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbb0c0b2880 0x7fbb0c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.827+0000 7fbb137fe700 1 -- 172.21.15.46:0/575832668 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbb1c00f020 con 0x7fbb24108770 2024-05-09T00:16:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.827+0000 7fbb3780eb80 1 Processor -- start 2024-05-09T00:16:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.827+0000 7fbb2259c700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbb0c0b2880 0x7fbb0c0b4d90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009c60 tx=0x7fbb14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.827+0000 7fbb3780eb80 1 -- start start 2024-05-09T00:16:59.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb3780eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72ac2ad80 0x55e72ad332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb3780eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 0x55e72ad33800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb3780eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ad2d480 0x55e72ad2d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb3780eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e72ac50120 con 0x55e72abb0520 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb3780eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e72abf5a20 con 0x55e72ac2ad80 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb3780eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e72ac4c0e0 con 0x55e72ad2d480 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb21d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 0x55e72ad33800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb21d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 0x55e72ad33800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33008/0 (socket says 172.21.15.46:33008) 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb21d9b700 1 -- 172.21.15.46:0/2721310703 learned_addr learned my addr 172.21.15.46:0/2721310703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb22d9d700 1 --2- 172.21.15.46:0/2721310703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ad2d480 0x55e72ad2d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb21d9b700 1 -- 172.21.15.46:0/2721310703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ad2d480 msgr2=0x55e72ad2d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb21d9b700 1 --2- 172.21.15.46:0/2721310703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ad2d480 0x55e72ad2d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb2259c700 1 --2- 172.21.15.46:0/2721310703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72ac2ad80 0x55e72ad332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb21d9b700 1 -- 172.21.15.46:0/2721310703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72ac2ad80 msgr2=0x55e72ad332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb21d9b700 1 --2- 172.21.15.46:0/2721310703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72ac2ad80 0x55e72ad332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb21d9b700 1 -- 172.21.15.46:0/2721310703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e72abe6580 con 0x55e72abb0520 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb2259c700 1 --2- 172.21.15.46:0/2721310703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72ac2ad80 0x55e72ad332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:16:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb21d9b700 1 --2- 172.21.15.46:0/2721310703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 0x55e72ad33800 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbb18007be0 tx=0x7fbb18007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bfd1b047c369a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:59.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.828+0000 7fbb10ff9700 1 -- 172.21.15.46:0/2721310703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18010cf0 con 0x55e72abb0520 2024-05-09T00:16:59.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.829+0000 7fbb10ff9700 1 -- 172.21.15.46:0/2721310703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb18015070 con 0x55e72abb0520 2024-05-09T00:16:59.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.829+0000 7fbb10ff9700 1 -- 172.21.15.46:0/2721310703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18003e00 con 0x55e72abb0520 2024-05-09T00:16:59.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.829+0000 7fbb3780eb80 1 -- 172.21.15.46:0/2721310703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 msgr2=0x55e72ad33800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:59.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.829+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/2721310703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 0x55e72ad33800 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbb18007be0 tx=0x7fbb18007ef0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.830+0000 7fbb3780eb80 1 -- 172.21.15.46:0/2721310703 shutdown_connections 2024-05-09T00:16:59.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.830+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/2721310703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ad2d480 0x55e72ad2d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.830+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/2721310703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72ac2ad80 0x55e72ad332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.830+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/2721310703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 0x55e72ad33800 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.830+0000 7fbb3780eb80 1 -- 172.21.15.46:0/2721310703 >> 172.21.15.46:0/2721310703 conn(0x55e72abb3ff0 msgr2=0x55e72ad282a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.830+0000 7fbb3780eb80 1 -- 172.21.15.46:0/2721310703 shutdown_connections 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.830+0000 7fbb3780eb80 1 -- 172.21.15.46:0/2721310703 wait complete. 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.830+0000 7fbb3780eb80 1 -- start start 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.830+0000 7fbb3780eb80 1 Processor -- start 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb3780eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72abb0520 0x55e72ad37190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb3780eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 0x55e72ad376d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb3780eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 0x55e72ad37c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb3780eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e72ad2e1e0 con 0x55e72ad2d480 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb3780eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e72abeaba0 con 0x55e72abb0520 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb3780eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e72ad3a0a0 con 0x55e72ac2ad80 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb3780eb80 10 client.0 _ll_register_callbacks cb 0x55e72aac1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb22d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 0x55e72ad37c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb22d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 0x55e72ad37c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33018/0 (socket says 172.21.15.46:33018) 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb22d9d700 1 -- 172.21.15.46:0/475557756 learned_addr learned my addr 172.21.15.46:0/475557756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb21d9b700 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 0x55e72ad376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb2259c700 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72abb0520 0x55e72ad37190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb22d9d700 1 -- 172.21.15.46:0/475557756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 msgr2=0x55e72ad376d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb22d9d700 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 0x55e72ad376d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb22d9d700 1 -- 172.21.15.46:0/475557756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72abb0520 msgr2=0x55e72ad37190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb22d9d700 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72abb0520 0x55e72ad37190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb22d9d700 1 -- 172.21.15.46:0/475557756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e72aacb510 con 0x55e72ad2d480 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb22d9d700 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 0x55e72ad37c10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009fd0 tx=0x7fbb1c0aaea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bcc71ecc79e921 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.831+0000 7fbb09ffb700 1 -- 172.21.15.46:0/475557756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0a61a0 con 0x55e72ad2d480 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb09ffb700 1 -- 172.21.15.46:0/475557756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbb1c0b3560 con 0x55e72ad2d480 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb3780eb80 1 -- 172.21.15.46:0/475557756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb18015070 con 0x55e72ad2d480 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb09ffb700 1 -- 172.21.15.46:0/475557756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbb1c004e00 con 0x55e72ad2d480 2024-05-09T00:16:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb3780eb80 10 client.16203 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb3780eb80 1 -- 172.21.15.46:0/475557756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e72ad44b10 con 0x55e72ad2d480 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb09ffb700 1 -- 172.21.15.46:0/475557756 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb18015070 con 0x55e72ad2d480 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb09ffb700 1 -- 172.21.15.46:0/475557756 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbb1c011070 con 0x55e72ad2d480 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb3780eb80 10 client.16203 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb3780eb80 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb3780eb80 20 client.16203 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e72ad470d0 0x55e72ad49510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb3780eb80 4 client.16203 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb3780eb80 1 -- 172.21.15.46:0/475557756 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e72ac5db00 con 0x55e72ad470d0 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.832+0000 7fbb2259c700 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e72ad470d0 0x55e72ad49510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.833+0000 7fbb2259c700 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e72ad470d0 0x55e72ad49510 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3390d99c07725ab server_cookie=93870c3d404f6526 in_seq=0 out_seq=0 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.833+0000 7fbb09ffb700 10 client.16203 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb09ffb700 1 -- 172.21.15.46:0/475557756 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e72ac5db00 con 0x55e72ad470d0 2024-05-09T00:16:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb09ffb700 10 client.16203 handle_command_reply: tid=0 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb3780eb80 1 client.16203 shutdown 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb3780eb80 1 -- 172.21.15.46:0/475557756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 msgr2=0x55e72ad37c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 0x55e72ad37c10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009fd0 tx=0x7fbb1c0aaea0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb3780eb80 1 -- 172.21.15.46:0/475557756 shutdown_connections 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e72ad470d0 0x55e72ad49510 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 0x55e72ad37c10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72abb0520 0x55e72ad37190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/475557756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 0x55e72ad376d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb3780eb80 1 -- 172.21.15.46:0/475557756 >> 172.21.15.46:0/475557756 conn(0x55e72abb3ff0 msgr2=0x55e72ad22070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb09ffb700 0 client.16203 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb3780eb80 1 -- 172.21.15.46:0/475557756 shutdown_connections 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb3780eb80 1 -- 172.21.15.46:0/475557756 wait complete. 2024-05-09T00:16:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.834+0000 7fbb3780eb80 20 client.16203 trim_cache size 0 max 16384 2024-05-09T00:16:59.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.855+0000 7fbb3780eb80 1 Processor -- start 2024-05-09T00:16:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.855+0000 7fbb3780eb80 1 -- start start 2024-05-09T00:16:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.855+0000 7fbb3780eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 0x55e72ad22aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.855+0000 7fbb3780eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 0x55e72ad36340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.855+0000 7fbb3780eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72ad2d480 0x55e72ad37860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.855+0000 7fbb3780eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241b35b0 con 0x55e72abb0520 2024-05-09T00:16:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.855+0000 7fbb3780eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241b0e40 con 0x55e72ad2d480 2024-05-09T00:16:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.855+0000 7fbb3780eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241aec70 con 0x55e72ac2ad80 2024-05-09T00:16:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.855+0000 7fbb2259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 0x55e72ad22aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.855+0000 7fbb2259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 0x55e72ad22aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33026/0 (socket says 172.21.15.46:33026) 2024-05-09T00:16:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.855+0000 7fbb2259c700 1 -- 172.21.15.46:0/3082302862 learned_addr learned my addr 172.21.15.46:0/3082302862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.855+0000 7fbb21d9b700 1 --2- 172.21.15.46:0/3082302862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 0x55e72ad36340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.856+0000 7fbb2259c700 1 -- 172.21.15.46:0/3082302862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 msgr2=0x55e72ad36340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:59.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.856+0000 7fbb2259c700 1 --2- 172.21.15.46:0/3082302862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 0x55e72ad36340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.856+0000 7fbb22d9d700 1 --2- 172.21.15.46:0/3082302862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72ad2d480 0x55e72ad37860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.856+0000 7fbb2259c700 1 -- 172.21.15.46:0/3082302862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72ad2d480 msgr2=0x55e72ad37860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:59.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.856+0000 7fbb2259c700 1 --2- 172.21.15.46:0/3082302862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72ad2d480 0x55e72ad37860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.856+0000 7fbb2259c700 1 -- 172.21.15.46:0/3082302862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e72ad24a50 con 0x55e72abb0520 2024-05-09T00:16:59.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.856+0000 7fbb22d9d700 1 --2- 172.21.15.46:0/3082302862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72ad2d480 0x55e72ad37860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:16:59.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.856+0000 7fbb2259c700 1 --2- 172.21.15.46:0/3082302862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 0x55e72ad22aa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x55e72ad36a10 tx=0x7fbb14014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb56473ded9344a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:59.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.856+0000 7fbb0bfff700 1 -- 172.21.15.46:0/3082302862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1401dc10 con 0x55e72abb0520 2024-05-09T00:16:59.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.856+0000 7fbb0bfff700 1 -- 172.21.15.46:0/3082302862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1401ddb0 con 0x55e72abb0520 2024-05-09T00:16:59.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.856+0000 7fbb0bfff700 1 -- 172.21.15.46:0/3082302862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1401cb60 con 0x55e72abb0520 2024-05-09T00:16:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.857+0000 7fbb3780eb80 1 -- 172.21.15.46:0/3082302862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 msgr2=0x55e72ad22aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.857+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/3082302862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 0x55e72ad22aa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x55e72ad36a10 tx=0x7fbb14014000 comp rx=0 tx=0).stop 2024-05-09T00:16:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.857+0000 7fbb3780eb80 1 -- 172.21.15.46:0/3082302862 shutdown_connections 2024-05-09T00:16:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.857+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/3082302862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72ad2d480 0x55e72ad37860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.857+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/3082302862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72abb0520 0x55e72ad22aa0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.857+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/3082302862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 0x55e72ad36340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.857+0000 7fbb3780eb80 1 -- 172.21.15.46:0/3082302862 >> 172.21.15.46:0/3082302862 conn(0x55e72abb3ff0 msgr2=0x55e72ad37150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.857+0000 7fbb3780eb80 1 -- 172.21.15.46:0/3082302862 shutdown_connections 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.857+0000 7fbb3780eb80 1 -- 172.21.15.46:0/3082302862 wait complete. 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.857+0000 7fbb3780eb80 1 -- start start 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.857+0000 7fbb3780eb80 1 Processor -- start 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb3780eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72abb0520 0x55e72ad3c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb3780eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 0x55e72ad3c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb3780eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 0x55e72ad3cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb3780eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241af300 con 0x55e72ad2d480 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb3780eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241b0770 con 0x55e72abb0520 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb3780eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241af9d0 con 0x55e72ac2ad80 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb3780eb80 10 client.0 _ll_register_callbacks cb 0x55e72aac1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:16:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb22d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 0x55e72ad3cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb22d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 0x55e72ad3cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33030/0 (socket says 172.21.15.46:33030) 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb22d9d700 1 -- 172.21.15.46:0/1884131449 learned_addr learned my addr 172.21.15.46:0/1884131449 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb21d9b700 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 0x55e72ad3c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb2259c700 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72abb0520 0x55e72ad3c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb22d9d700 1 -- 172.21.15.46:0/1884131449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 msgr2=0x55e72ad3c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb22d9d700 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 0x55e72ad3c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb22d9d700 1 -- 172.21.15.46:0/1884131449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72abb0520 msgr2=0x55e72ad3c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb22d9d700 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72abb0520 0x55e72ad3c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb22d9d700 1 -- 172.21.15.46:0/1884131449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e72adb1890 con 0x55e72ad2d480 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb2259c700 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72abb0520 0x55e72ad3c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb22d9d700 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 0x55e72ad3cb50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0e3400 tx=0x7fbb1c0aa410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126de2eb0ee22d7c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.858+0000 7fbb09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb09ffb700 1 -- 172.21.15.46:0/1884131449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0a61a0 con 0x55e72ad2d480 2024-05-09T00:16:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb3780eb80 1 -- 172.21.15.46:0/1884131449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1401ddb0 con 0x55e72ad2d480 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb09ffb700 1 -- 172.21.15.46:0/1884131449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbb1c0b3560 con 0x55e72ad2d480 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb09ffb700 1 -- 172.21.15.46:0/1884131449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbb1c0a68d0 con 0x55e72ad2d480 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb3780eb80 10 client.16209 fetch_fsmap learned FSMap version 26 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb3780eb80 1 -- 172.21.15.46:0/1884131449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e72adb2920 con 0x55e72ad2d480 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb09ffb700 1 -- 172.21.15.46:0/1884131449 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1401ddb0 con 0x55e72ad2d480 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb09ffb700 1 -- 172.21.15.46:0/1884131449 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbb1c011070 con 0x55e72ad2d480 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb3780eb80 10 client.16209 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb3780eb80 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb3780eb80 20 client.16209 populate_metadata read hostname 'smithi046' 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e72adb2e40 0x55e72adb5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb3780eb80 4 client.16209 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.859+0000 7fbb3780eb80 1 -- 172.21.15.46:0/1884131449 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e72ac57ba0 con 0x55e72adb2e40 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.860+0000 7fbb2259c700 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e72adb2e40 0x55e72adb5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.860+0000 7fbb09ffb700 10 client.16209 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.860+0000 7fbb2259c700 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e72adb2e40 0x55e72adb5260 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a19f38b533f262 server_cookie=9ba50a90f21fac1c in_seq=0 out_seq=0 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.861+0000 7fbb09ffb700 1 -- 172.21.15.46:0/1884131449 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e72ac57ba0 con 0x55e72adb2e40 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.861+0000 7fbb09ffb700 10 client.16209 handle_command_reply: tid=0 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.861+0000 7fbb3780eb80 1 client.16209 shutdown 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.861+0000 7fbb3780eb80 1 -- 172.21.15.46:0/1884131449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 msgr2=0x55e72ad3cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.861+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 0x55e72ad3cb50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0e3400 tx=0x7fbb1c0aa410 comp rx=0 tx=0).stop 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.861+0000 7fbb3780eb80 1 -- 172.21.15.46:0/1884131449 shutdown_connections 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.861+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e72adb2e40 0x55e72adb5260 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.861+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e72ad2d480 0x55e72ad3cb50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.861+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e72abb0520 0x55e72ad3c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.861+0000 7fbb3780eb80 1 --2- 172.21.15.46:0/1884131449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e72ac2ad80 0x55e72ad3c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.861+0000 7fbb3780eb80 1 -- 172.21.15.46:0/1884131449 >> 172.21.15.46:0/1884131449 conn(0x55e72abb3ff0 msgr2=0x55e72ad22800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb09ffb700 0 client.16209 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:16:59.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb3780eb80 1 -- 172.21.15.46:0/1884131449 shutdown_connections 2024-05-09T00:16:59.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb3780eb80 1 -- 172.21.15.46:0/1884131449 wait complete. 2024-05-09T00:16:59.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb3780eb80 20 client.16209 trim_cache size 0 max 16384 2024-05-09T00:16:59.866 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:16:59.866 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:16:59.866 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "2a594a29-d60d-416e-b91e-c9d588120937", 2024-05-09T00:16:59.866 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:16:59.866 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:16:59.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb28df8700 1 -- 172.21.15.46:0/575832668 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbb0c0b2880 msgr2=0x7fbb0c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:59.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb28df8700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbb0c0b2880 0x7fbb0c0b4d90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009c60 tx=0x7fbb14009260 comp rx=0 tx=0).stop 2024-05-09T00:16:59.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb28df8700 1 -- 172.21.15.46:0/575832668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb24108770 msgr2=0x7fbb24101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:16:59.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb28df8700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb24108770 0x7fbb24101010 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009850 tx=0x7fbb1c007a20 comp rx=0 tx=0).stop 2024-05-09T00:16:59.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb28df8700 1 -- 172.21.15.46:0/575832668 shutdown_connections 2024-05-09T00:16:59.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb28df8700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbb0c0b2880 0x7fbb0c0b4d90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb28df8700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb24108770 0x7fbb24101010 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb28df8700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb24103540 0x7fbb24100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb28df8700 1 --2- 172.21.15.46:0/575832668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb24107e20 0x7fbb24100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:16:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.862+0000 7fbb28df8700 1 -- 172.21.15.46:0/575832668 >> 172.21.15.46:0/575832668 conn(0x7fbb24078e50 msgr2=0x7fbb24075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:16:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.863+0000 7fbb28df8700 1 -- 172.21.15.46:0/575832668 shutdown_connections 2024-05-09T00:16:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:16:59.863+0000 7fbb28df8700 1 -- 172.21.15.46:0/575832668 wait complete. 2024-05-09T00:17:00.764 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:17:00.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:00 smithi142 ceph-mon[34533]: pgmap v545: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:00.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:00 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:17:00.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:00 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:17:00.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:00 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:17:00.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:00 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:17:01.024 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:17:01.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:00 smithi046 ceph-mon[30151]: pgmap v545: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:01.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:00 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:17:01.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:00 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:17:01.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:00 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:17:01.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:00 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:17:01.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:00 smithi169 ceph-mon[34777]: pgmap v545: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:01.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:00 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:17:01.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:00 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:17:01.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:00 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:17:01.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:00 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:17:02.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.871+0000 7fd7ac952700 1 -- 172.21.15.46:0/3829230129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8109850 msgr2=0x7fd7a81111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:02.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.871+0000 7fd7ac952700 1 --2- 172.21.15.46:0/3829230129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8109850 0x7fd7a81111b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000b0d0 tx=0x7fd7a000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.871+0000 7fd7ac952700 1 -- 172.21.15.46:0/3829230129 shutdown_connections 2024-05-09T00:17:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.871+0000 7fd7ac952700 1 --2- 172.21.15.46:0/3829230129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8109850 0x7fd7a81111b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.871+0000 7fd7ac952700 1 --2- 172.21.15.46:0/3829230129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7a81045d0 0x7fd7a81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.871+0000 7fd7ac952700 1 --2- 172.21.15.46:0/3829230129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a8108f00 0x7fd7a8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.871+0000 7fd7ac952700 1 -- 172.21.15.46:0/3829230129 >> 172.21.15.46:0/3829230129 conn(0x7fd7a80ffea0 msgr2=0x7fd7a81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.871+0000 7fd7ac952700 1 -- 172.21.15.46:0/3829230129 shutdown_connections 2024-05-09T00:17:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.871+0000 7fd7ac952700 1 -- 172.21.15.46:0/3829230129 wait complete. 2024-05-09T00:17:02.872 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:02 smithi046 ceph-mon[30151]: pgmap v546: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:02.872 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:02 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:17:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.872+0000 7fd7ac952700 1 Processor -- start 2024-05-09T00:17:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.872+0000 7fd7ac952700 1 -- start start 2024-05-09T00:17:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.872+0000 7fd7ac952700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a81045d0 0x7fd7a819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7ac952700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a8108f00 0x7fd7a819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7ac952700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7a8109850 0x7fd7a819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7ac952700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8114480 con 0x7fd7a81045d0 2024-05-09T00:17:02.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7ac952700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8114300 con 0x7fd7a8108f00 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7ac952700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8114600 con 0x7fd7a8109850 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7a659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a81045d0 0x7fd7a819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7a659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a81045d0 0x7fd7a819ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33044/0 (socket says 172.21.15.46:33044) 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7a659c700 1 -- 172.21.15.46:0/2383884998 learned_addr learned my addr 172.21.15.46:0/2383884998 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7a5d9b700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a8108f00 0x7fd7a819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7a6d9d700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7a8109850 0x7fd7a819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7a659c700 1 -- 172.21.15.46:0/2383884998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7a8109850 msgr2=0x7fd7a819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7a659c700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7a8109850 0x7fd7a819f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7a659c700 1 -- 172.21.15.46:0/2383884998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a8108f00 msgr2=0x7fd7a819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7a659c700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a8108f00 0x7fd7a819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7a659c700 1 -- 172.21.15.46:0/2383884998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a819fc60 con 0x7fd7a81045d0 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.873+0000 7fd7a659c700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a81045d0 0x7fd7a819ad90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd79800d950 tx=0x7fd79800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.874+0000 7fd7977fe700 1 -- 172.21.15.46:0/2383884998 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798016070 con 0x7fd7a81045d0 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.874+0000 7fd7ac952700 1 -- 172.21.15.46:0/2383884998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a819ff50 con 0x7fd7a81045d0 2024-05-09T00:17:02.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.874+0000 7fd7977fe700 1 -- 172.21.15.46:0/2383884998 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd79800fd40 con 0x7fd7a81045d0 2024-05-09T00:17:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.874+0000 7fd7977fe700 1 -- 172.21.15.46:0/2383884998 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79800e3b0 con 0x7fd7a81045d0 2024-05-09T00:17:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.874+0000 7fd7ac952700 1 -- 172.21.15.46:0/2383884998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a81a7870 con 0x7fd7a81045d0 2024-05-09T00:17:02.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.877+0000 7fd7977fe700 1 -- 172.21.15.46:0/2383884998 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd798019070 con 0x7fd7a81045d0 2024-05-09T00:17:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.878+0000 7fd7977fe700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7900b2880 0x7fd7900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.878+0000 7fd7a5d9b700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7900b2880 0x7fd7900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.878+0000 7fd7977fe700 1 -- 172.21.15.46:0/2383884998 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd7980e1ff0 con 0x7fd7a81045d0 2024-05-09T00:17:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.878+0000 7fd7bb368b80 1 Processor -- start 2024-05-09T00:17:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.878+0000 7fd7bb368b80 1 -- start start 2024-05-09T00:17:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.878+0000 7fd7a5d9b700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7900b2880 0x7fd7900b4d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8107cb0 tx=0x7fd79c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.878+0000 7fd7bb368b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd560d00 0x55c1bd667ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.878+0000 7fd7bb368b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd4e64a0 0x55c1bd668030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7bb368b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd66c1e0 0x55c1bd66c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7bb368b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c1bd5860a0 con 0x55c1bd4e64a0 2024-05-09T00:17:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7bb368b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c1bd52b9a0 con 0x55c1bd560d00 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7bb368b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c1bd582060 con 0x55c1bd66c1e0 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7a5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd4e64a0 0x55c1bd668030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7a5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd4e64a0 0x55c1bd668030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33058/0 (socket says 172.21.15.46:33058) 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7a5d9b700 1 -- 172.21.15.46:0/4066185400 learned_addr learned my addr 172.21.15.46:0/4066185400 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7a659c700 1 --2- 172.21.15.46:0/4066185400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd560d00 0x55c1bd667ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7a6d9d700 1 --2- 172.21.15.46:0/4066185400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd66c1e0 0x55c1bd66c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7a5d9b700 1 -- 172.21.15.46:0/4066185400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd66c1e0 msgr2=0x55c1bd66c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7a5d9b700 1 --2- 172.21.15.46:0/4066185400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd66c1e0 0x55c1bd66c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7a5d9b700 1 -- 172.21.15.46:0/4066185400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd560d00 msgr2=0x55c1bd667ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7a5d9b700 1 --2- 172.21.15.46:0/4066185400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd560d00 0x55c1bd667ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7a5d9b700 1 -- 172.21.15.46:0/4066185400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1bd51c500 con 0x55c1bd4e64a0 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7a659c700 1 --2- 172.21.15.46:0/4066185400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd560d00 0x55c1bd667ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd7a5d9b700 1 --2- 172.21.15.46:0/4066185400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd4e64a0 0x55c1bd668030 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd79c010dc0 tx=0x7fd79c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50292767bb10b33d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd794ff9700 1 -- 172.21.15.46:0/4066185400 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c01e420 con 0x55c1bd4e64a0 2024-05-09T00:17:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.879+0000 7fd794ff9700 1 -- 172.21.15.46:0/4066185400 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd79c01ea70 con 0x55c1bd4e64a0 2024-05-09T00:17:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.880+0000 7fd794ff9700 1 -- 172.21.15.46:0/4066185400 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c01cb00 con 0x55c1bd4e64a0 2024-05-09T00:17:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.880+0000 7fd7bb368b80 1 -- 172.21.15.46:0/4066185400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd4e64a0 msgr2=0x55c1bd668030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.880+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/4066185400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd4e64a0 0x55c1bd668030 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd79c010dc0 tx=0x7fd79c013000 comp rx=0 tx=0).stop 2024-05-09T00:17:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.880+0000 7fd7bb368b80 1 -- 172.21.15.46:0/4066185400 shutdown_connections 2024-05-09T00:17:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.880+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/4066185400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd66c1e0 0x55c1bd66c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.880+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/4066185400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd560d00 0x55c1bd667ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.880+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/4066185400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd4e64a0 0x55c1bd668030 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.881+0000 7fd7bb368b80 1 -- 172.21.15.46:0/4066185400 >> 172.21.15.46:0/4066185400 conn(0x55c1bd4e9f70 msgr2=0x55c1bd65e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.881+0000 7fd7bb368b80 1 -- 172.21.15.46:0/4066185400 shutdown_connections 2024-05-09T00:17:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.881+0000 7fd7bb368b80 1 -- 172.21.15.46:0/4066185400 wait complete. 2024-05-09T00:17:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.881+0000 7fd7bb368b80 1 -- start start 2024-05-09T00:17:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.881+0000 7fd7bb368b80 1 Processor -- start 2024-05-09T00:17:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.881+0000 7fd7bb368b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd4e64a0 0x55c1bd671470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.881+0000 7fd7bb368b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd560d00 0x55c1bd6719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.881+0000 7fd7bb368b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd671ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.881+0000 7fd7bb368b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c1bd66cf00 con 0x55c1bd66c1e0 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.881+0000 7fd7bb368b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c1bd520b20 con 0x55c1bd560d00 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.881+0000 7fd7bb368b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c1bd674470 con 0x55c1bd4e64a0 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7bb368b80 10 client.0 _ll_register_callbacks cb 0x55c1bd3f76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd671ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd671ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33060/0 (socket says 172.21.15.46:33060) 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a6d9d700 1 -- 172.21.15.46:0/1488776438 learned_addr learned my addr 172.21.15.46:0/1488776438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a5d9b700 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd560d00 0x55c1bd6719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a659c700 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd4e64a0 0x55c1bd671470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a6d9d700 1 -- 172.21.15.46:0/1488776438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd4e64a0 msgr2=0x55c1bd671470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a6d9d700 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd4e64a0 0x55c1bd671470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a6d9d700 1 -- 172.21.15.46:0/1488776438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd560d00 msgr2=0x55c1bd6719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a6d9d700 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd560d00 0x55c1bd6719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a6d9d700 1 -- 172.21.15.46:0/1488776438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1bd401490 con 0x55c1bd66c1e0 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a5d9b700 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd560d00 0x55c1bd6719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a659c700 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd4e64a0 0x55c1bd671470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7a6d9d700 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd671ef0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00095b0 tx=0x7fd7a0003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6388e9f9d57a0b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd78dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd78dffb700 1 -- 172.21.15.46:0/1488776438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a00075c0 con 0x55c1bd66c1e0 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd78dffb700 1 -- 172.21.15.46:0/1488776438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd7a0007760 con 0x55c1bd66c1e0 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.882+0000 7fd7bb368b80 1 -- 172.21.15.46:0/1488776438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd79c01ea70 con 0x55c1bd66c1e0 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.883+0000 7fd78dffb700 1 -- 172.21.15.46:0/1488776438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7a0007960 con 0x55c1bd66c1e0 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.883+0000 7fd7bb368b80 10 client.16221 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.883+0000 7fd7bb368b80 1 -- 172.21.15.46:0/1488776438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c1bd67ee50 con 0x55c1bd66c1e0 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.883+0000 7fd78dffb700 1 -- 172.21.15.46:0/1488776438 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd79c01ea70 con 0x55c1bd66c1e0 2024-05-09T00:17:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.883+0000 7fd78dffb700 1 -- 172.21.15.46:0/1488776438 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd7a0011070 con 0x55c1bd66c1e0 2024-05-09T00:17:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.883+0000 7fd7bb368b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.883+0000 7fd7bb368b80 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:17:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.883+0000 7fd7bb368b80 20 client.16221 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.883+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c1bd681490 0x55c1bd6838b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.883+0000 7fd7bb368b80 4 client.16221 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:17:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.883+0000 7fd7bb368b80 1 -- 172.21.15.46:0/1488776438 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1bd593a80 con 0x55c1bd681490 2024-05-09T00:17:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.883+0000 7fd7a659c700 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c1bd681490 0x55c1bd6838b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.884+0000 7fd78dffb700 10 client.16221 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.884+0000 7fd7a659c700 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c1bd681490 0x55c1bd6838b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2cf7004d64bb50d server_cookie=87c45def3ad6cae4 in_seq=0 out_seq=0 2024-05-09T00:17:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd78dffb700 1 -- 172.21.15.46:0/1488776438 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c1bd593a80 con 0x55c1bd681490 2024-05-09T00:17:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd78dffb700 10 client.16221 handle_command_reply: tid=0 2024-05-09T00:17:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd7bb368b80 1 client.16221 shutdown 2024-05-09T00:17:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd7bb368b80 1 -- 172.21.15.46:0/1488776438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 msgr2=0x55c1bd671ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd671ef0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00095b0 tx=0x7fd7a0003ab0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd7bb368b80 1 -- 172.21.15.46:0/1488776438 shutdown_connections 2024-05-09T00:17:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c1bd681490 0x55c1bd6838b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd671ef0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd4e64a0 0x55c1bd671470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/1488776438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd560d00 0x55c1bd6719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd7bb368b80 1 -- 172.21.15.46:0/1488776438 >> 172.21.15.46:0/1488776438 conn(0x55c1bd4e9f70 msgr2=0x55c1bd657ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd78dffb700 0 client.16221 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd7bb368b80 1 -- 172.21.15.46:0/1488776438 shutdown_connections 2024-05-09T00:17:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd7bb368b80 1 -- 172.21.15.46:0/1488776438 wait complete. 2024-05-09T00:17:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.885+0000 7fd7bb368b80 20 client.16221 trim_cache size 0 max 16384 2024-05-09T00:17:02.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.905+0000 7fd7bb368b80 1 Processor -- start 2024-05-09T00:17:02.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.905+0000 7fd7bb368b80 1 -- start start 2024-05-09T00:17:02.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7bb368b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd4e64a0 0x55c1bd65e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7bb368b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd560d00 0x55c1bd671d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7bb368b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd658700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7bb368b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81af2a0 con 0x55c1bd66c1e0 2024-05-09T00:17:02.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7bb368b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81acb30 con 0x55c1bd4e64a0 2024-05-09T00:17:02.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7bb368b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81aa980 con 0x55c1bd560d00 2024-05-09T00:17:02.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd658700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd658700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33064/0 (socket says 172.21.15.46:33064) 2024-05-09T00:17:02.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a6d9d700 1 -- 172.21.15.46:0/3388153632 learned_addr learned my addr 172.21.15.46:0/3388153632 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:02.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a659c700 1 --2- 172.21.15.46:0/3388153632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd4e64a0 0x55c1bd65e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a5d9b700 1 --2- 172.21.15.46:0/3388153632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd560d00 0x55c1bd671d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a6d9d700 1 -- 172.21.15.46:0/3388153632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd560d00 msgr2=0x55c1bd671d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:02.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a6d9d700 1 --2- 172.21.15.46:0/3388153632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd560d00 0x55c1bd671d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a6d9d700 1 -- 172.21.15.46:0/3388153632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd4e64a0 msgr2=0x55c1bd65e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:02.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a6d9d700 1 --2- 172.21.15.46:0/3388153632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd4e64a0 0x55c1bd65e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a6d9d700 1 -- 172.21.15.46:0/3388153632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1bd65ecf0 con 0x55c1bd66c1e0 2024-05-09T00:17:02.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a5d9b700 1 --2- 172.21.15.46:0/3388153632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd560d00 0x55c1bd671d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:02.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a659c700 1 --2- 172.21.15.46:0/3388153632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd4e64a0 0x55c1bd65e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:02.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.906+0000 7fd7a6d9d700 1 --2- 172.21.15.46:0/3388153632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd658700 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000b820 tx=0x7fd7a0007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6761cc086daf063f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:02.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.907+0000 7fd78ffff700 1 -- 172.21.15.46:0/3388153632 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0034030 con 0x55c1bd66c1e0 2024-05-09T00:17:02.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.907+0000 7fd78ffff700 1 -- 172.21.15.46:0/3388153632 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a0012c80 con 0x55c1bd66c1e0 2024-05-09T00:17:02.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.907+0000 7fd78ffff700 1 -- 172.21.15.46:0/3388153632 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0012550 con 0x55c1bd66c1e0 2024-05-09T00:17:02.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.907+0000 7fd7bb368b80 1 -- 172.21.15.46:0/3388153632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 msgr2=0x55c1bd658700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:02.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.907+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/3388153632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd658700 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000b820 tx=0x7fd7a0007870 comp rx=0 tx=0).stop 2024-05-09T00:17:02.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.907+0000 7fd7bb368b80 1 -- 172.21.15.46:0/3388153632 shutdown_connections 2024-05-09T00:17:02.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.907+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/3388153632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd658700 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.907+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/3388153632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd4e64a0 0x55c1bd65e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.907+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/3388153632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd560d00 0x55c1bd671d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.907+0000 7fd7bb368b80 1 -- 172.21.15.46:0/3388153632 >> 172.21.15.46:0/3388153632 conn(0x55c1bd4e9f70 msgr2=0x55c1bd657ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:02.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.907+0000 7fd7bb368b80 1 -- 172.21.15.46:0/3388153632 shutdown_connections 2024-05-09T00:17:02.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.908+0000 7fd7bb368b80 1 -- 172.21.15.46:0/3388153632 wait complete. 2024-05-09T00:17:02.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.908+0000 7fd7bb368b80 1 -- start start 2024-05-09T00:17:02.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.908+0000 7fd7bb368b80 1 Processor -- start 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.908+0000 7fd7bb368b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd4e64a0 0x55c1bd679c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.908+0000 7fd7bb368b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd560d00 0x55c1bd669aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.908+0000 7fd7bb368b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd66a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7bb368b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81aaff0 con 0x55c1bd66c1e0 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7bb368b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81ac460 con 0x55c1bd4e64a0 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7bb368b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81ab6c0 con 0x55c1bd560d00 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7a6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd66a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7a6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd66a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33068/0 (socket says 172.21.15.46:33068) 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7a6d9d700 1 -- 172.21.15.46:0/2523631593 learned_addr learned my addr 172.21.15.46:0/2523631593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7a659c700 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd4e64a0 0x55c1bd679c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7bb368b80 10 client.0 _ll_register_callbacks cb 0x55c1bd3f76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7a5d9b700 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd560d00 0x55c1bd669aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7a6d9d700 1 -- 172.21.15.46:0/2523631593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd560d00 msgr2=0x55c1bd669aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7a6d9d700 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd560d00 0x55c1bd669aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7a6d9d700 1 -- 172.21.15.46:0/2523631593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd4e64a0 msgr2=0x55c1bd679c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7a6d9d700 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd4e64a0 0x55c1bd679c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7a6d9d700 1 -- 172.21.15.46:0/2523631593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1bd66a890 con 0x55c1bd66c1e0 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7a6d9d700 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd66a0f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009900 tx=0x7fd7a0009180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0db47f7d55da0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd78dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd78dffb700 1 -- 172.21.15.46:0/2523631593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a003d060 con 0x55c1bd66c1e0 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd78dffb700 1 -- 172.21.15.46:0/2523631593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd7a00123f0 con 0x55c1bd66c1e0 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.909+0000 7fd7bb368b80 1 -- 172.21.15.46:0/2523631593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7a0012c80 con 0x55c1bd66c1e0 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.910+0000 7fd78dffb700 1 -- 172.21.15.46:0/2523631593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7a0034040 con 0x55c1bd66c1e0 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.910+0000 7fd7bb368b80 10 client.16227 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.910+0000 7fd7bb368b80 1 -- 172.21.15.46:0/2523631593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c1bd6ebc40 con 0x55c1bd66c1e0 2024-05-09T00:17:02.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.910+0000 7fd78dffb700 1 -- 172.21.15.46:0/2523631593 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a0012c80 con 0x55c1bd66c1e0 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.910+0000 7fd78dffb700 1 -- 172.21.15.46:0/2523631593 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd7a0042070 con 0x55c1bd66c1e0 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.910+0000 7fd7bb368b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.910+0000 7fd7bb368b80 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.910+0000 7fd7bb368b80 20 client.16227 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.910+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c1bd6ec0e0 0x55c1bd6ee500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.910+0000 7fd7bb368b80 4 client.16227 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.910+0000 7fd7bb368b80 1 -- 172.21.15.46:0/2523631593 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c1bd58db20 con 0x55c1bd6ec0e0 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.910+0000 7fd7a659c700 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c1bd6ec0e0 0x55c1bd6ee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7a659c700 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c1bd6ec0e0 0x55c1bd6ee500 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ad424e35033716f server_cookie=438229171ea7f17e in_seq=0 out_seq=0 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd78dffb700 10 client.16227 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd78dffb700 1 -- 172.21.15.46:0/2523631593 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c1bd58db20 con 0x55c1bd6ec0e0 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd78dffb700 10 client.16227 handle_command_reply: tid=0 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7bb368b80 1 client.16227 shutdown 2024-05-09T00:17:02.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7bb368b80 1 -- 172.21.15.46:0/2523631593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 msgr2=0x55c1bd66a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:02.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd66a0f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009900 tx=0x7fd7a0009180 comp rx=0 tx=0).stop 2024-05-09T00:17:02.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7bb368b80 1 -- 172.21.15.46:0/2523631593 shutdown_connections 2024-05-09T00:17:02.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c1bd6ec0e0 0x55c1bd6ee500 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1bd66c1e0 0x55c1bd66a0f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1bd4e64a0 0x55c1bd679c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd78dffb700 0 client.16227 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:02.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7bb368b80 1 --2- 172.21.15.46:0/2523631593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c1bd560d00 0x55c1bd669aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7bb368b80 1 -- 172.21.15.46:0/2523631593 >> 172.21.15.46:0/2523631593 conn(0x55c1bd4e9f70 msgr2=0x55c1bd6623f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:02.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7bb368b80 1 -- 172.21.15.46:0/2523631593 shutdown_connections 2024-05-09T00:17:02.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7bb368b80 1 -- 172.21.15.46:0/2523631593 wait complete. 2024-05-09T00:17:02.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.911+0000 7fd7bb368b80 20 client.16227 trim_cache size 0 max 16384 2024-05-09T00:17:02.917 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:17:02.918 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:17:02.918 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:17:02.918 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:17:02.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.912+0000 7fd7ac952700 1 -- 172.21.15.46:0/2383884998 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7900b2880 msgr2=0x7fd7900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:02.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.912+0000 7fd7ac952700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7900b2880 0x7fd7900b4d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8107cb0 tx=0x7fd79c00b410 comp rx=0 tx=0).stop 2024-05-09T00:17:02.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.912+0000 7fd7ac952700 1 -- 172.21.15.46:0/2383884998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a81045d0 msgr2=0x7fd7a819ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:02.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.912+0000 7fd7ac952700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a81045d0 0x7fd7a819ad90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd79800d950 tx=0x7fd79800dc60 comp rx=0 tx=0).stop 2024-05-09T00:17:02.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.912+0000 7fd7ac952700 1 -- 172.21.15.46:0/2383884998 shutdown_connections 2024-05-09T00:17:02.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.912+0000 7fd7ac952700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd7900b2880 0x7fd7900b4d90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.912+0000 7fd7ac952700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7a8109850 0x7fd7a819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.912+0000 7fd7ac952700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a81045d0 0x7fd7a819ad90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.912+0000 7fd7ac952700 1 --2- 172.21.15.46:0/2383884998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a8108f00 0x7fd7a819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:02.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.912+0000 7fd7ac952700 1 -- 172.21.15.46:0/2383884998 >> 172.21.15.46:0/2383884998 conn(0x7fd7a80ffea0 msgr2=0x7fd7a8108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:02.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.912+0000 7fd7ac952700 1 -- 172.21.15.46:0/2383884998 shutdown_connections 2024-05-09T00:17:02.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:02.912+0000 7fd7ac952700 1 -- 172.21.15.46:0/2383884998 wait complete. 2024-05-09T00:17:02.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:02 smithi142 ceph-mon[34533]: pgmap v546: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:02.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:02 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:17:03.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:02 smithi169 ceph-mon[34777]: pgmap v546: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:03.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:02 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:17:03.764 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:17:03.764 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:17:04.019 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:17:04.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:04 smithi046 ceph-mon[30151]: pgmap v547: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:04.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:04 smithi142 ceph-mon[34533]: pgmap v547: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:05.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:04 smithi169 ceph-mon[34777]: pgmap v547: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:05.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.880+0000 7fbb318ee700 1 -- 172.21.15.46:0/1817902493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c104650 msgr2=0x7fbb2c104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.880+0000 7fbb318ee700 1 --2- 172.21.15.46:0/1817902493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c104650 0x7fbb2c104a60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0099c0 tx=0x7fbb1c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:17:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.880+0000 7fbb318ee700 1 -- 172.21.15.46:0/1817902493 shutdown_connections 2024-05-09T00:17:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.880+0000 7fbb318ee700 1 --2- 172.21.15.46:0/1817902493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2c109880 0x7fbb2c1111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.880+0000 7fbb318ee700 1 --2- 172.21.15.46:0/1817902493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c104650 0x7fbb2c104a60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.880+0000 7fbb318ee700 1 --2- 172.21.15.46:0/1817902493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb2c108f30 0x7fbb2c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.880+0000 7fbb318ee700 1 -- 172.21.15.46:0/1817902493 >> 172.21.15.46:0/1817902493 conn(0x7fbb2c079f00 msgr2=0x7fbb2c07a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.880+0000 7fbb318ee700 1 -- 172.21.15.46:0/1817902493 shutdown_connections 2024-05-09T00:17:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.880+0000 7fbb318ee700 1 -- 172.21.15.46:0/1817902493 wait complete. 2024-05-09T00:17:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.881+0000 7fbb318ee700 1 Processor -- start 2024-05-09T00:17:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.881+0000 7fbb318ee700 1 -- start start 2024-05-09T00:17:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.881+0000 7fbb318ee700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb2c104650 0x7fbb2c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.881+0000 7fbb318ee700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c108f30 0x7fbb2c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb318ee700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2c109880 0x7fbb2c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb318ee700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c1144b0 con 0x7fbb2c108f30 2024-05-09T00:17:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb318ee700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c114330 con 0x7fbb2c104650 2024-05-09T00:17:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb318ee700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c114630 con 0x7fbb2c109880 2024-05-09T00:17:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb2a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c108f30 0x7fbb2c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb2a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c108f30 0x7fbb2c19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33090/0 (socket says 172.21.15.46:33090) 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb2a7fc700 1 -- 172.21.15.46:0/2977198140 learned_addr learned my addr 172.21.15.46:0/2977198140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb2b7fe700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2c109880 0x7fbb2c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb2affd700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb2c104650 0x7fbb2c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb2a7fc700 1 -- 172.21.15.46:0/2977198140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2c109880 msgr2=0x7fbb2c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb2a7fc700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2c109880 0x7fbb2c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb2a7fc700 1 -- 172.21.15.46:0/2977198140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb2c104650 msgr2=0x7fbb2c19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb2a7fc700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb2c104650 0x7fbb2c19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb2a7fc700 1 -- 172.21.15.46:0/2977198140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c19fcf0 con 0x7fbb2c108f30 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb2b7fe700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2c109880 0x7fbb2c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb2a7fc700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c108f30 0x7fbb2c19b3c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400d980 tx=0x7fbb1400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb308ec700 1 -- 172.21.15.46:0/2977198140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14017070 con 0x7fbb2c108f30 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.882+0000 7fbb308ec700 1 -- 172.21.15.46:0/2977198140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb14010460 con 0x7fbb2c108f30 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.883+0000 7fbb318ee700 1 -- 172.21.15.46:0/2977198140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2c19ffe0 con 0x7fbb2c108f30 2024-05-09T00:17:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.883+0000 7fbb308ec700 1 -- 172.21.15.46:0/2977198140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400de60 con 0x7fbb2c108f30 2024-05-09T00:17:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.883+0000 7fbb318ee700 1 -- 172.21.15.46:0/2977198140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2c1a7900 con 0x7fbb2c108f30 2024-05-09T00:17:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.884+0000 7fbb318ee700 1 -- 172.21.15.46:0/2977198140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb2c0512c0 con 0x7fbb2c108f30 2024-05-09T00:17:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.886+0000 7fbb308ec700 1 -- 172.21.15.46:0/2977198140 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbb14010610 con 0x7fbb2c108f30 2024-05-09T00:17:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.887+0000 7fbb308ec700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbb180b2900 0x7fbb180b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.887+0000 7fbb2affd700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbb180b2900 0x7fbb180b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.887+0000 7fbb308ec700 1 -- 172.21.15.46:0/2977198140 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbb140e1d20 con 0x7fbb2c108f30 2024-05-09T00:17:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.888+0000 7fbb2affd700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbb180b2900 0x7fbb180b4e10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c003800 tx=0x7fbb1c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:05.891+0000 7fbb308ec700 1 -- 172.21.15.46:0/2977198140 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb140a9ac0 con 0x7fbb2c108f30 2024-05-09T00:17:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.157+0000 7fbb318ee700 1 -- 172.21.15.46:0/2977198140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb2c19c180 con 0x7fbb2c108f30 2024-05-09T00:17:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.158+0000 7fbb308ec700 1 -- 172.21.15.46:0/2977198140 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7fbb140a93c0 con 0x7fbb2c108f30 2024-05-09T00:17:06.157 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:17:06.157 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:17:06.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.160+0000 7fbb318ee700 1 -- 172.21.15.46:0/2977198140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbb180b2900 msgr2=0x7fbb180b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.160+0000 7fbb318ee700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbb180b2900 0x7fbb180b4e10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c003800 tx=0x7fbb1c01a040 comp rx=0 tx=0).stop 2024-05-09T00:17:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.160+0000 7fbb318ee700 1 -- 172.21.15.46:0/2977198140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c108f30 msgr2=0x7fbb2c19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.160+0000 7fbb318ee700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c108f30 0x7fbb2c19b3c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400d980 tx=0x7fbb1400dd40 comp rx=0 tx=0).stop 2024-05-09T00:17:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.160+0000 7fbb318ee700 1 -- 172.21.15.46:0/2977198140 shutdown_connections 2024-05-09T00:17:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.160+0000 7fbb318ee700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbb180b2900 0x7fbb180b4e10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.160+0000 7fbb318ee700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2c109880 0x7fbb2c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.160+0000 7fbb318ee700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb2c104650 0x7fbb2c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.160+0000 7fbb318ee700 1 --2- 172.21.15.46:0/2977198140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c108f30 0x7fbb2c19b3c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.160+0000 7fbb318ee700 1 -- 172.21.15.46:0/2977198140 >> 172.21.15.46:0/2977198140 conn(0x7fbb2c079f00 msgr2=0x7fbb2c1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.160+0000 7fbb318ee700 1 -- 172.21.15.46:0/2977198140 shutdown_connections 2024-05-09T00:17:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:06.160+0000 7fbb318ee700 1 -- 172.21.15.46:0/2977198140 wait complete. 2024-05-09T00:17:06.161 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:17:06.813 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:06 smithi046 ceph-mon[30151]: pgmap v548: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:06.813 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:06 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2977198140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:17:06.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:06 smithi142 ceph-mon[34533]: pgmap v548: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:06.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:06 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2977198140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:17:06.965 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:17:07.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:06 smithi169 ceph-mon[34777]: pgmap v548: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:07.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:06 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2977198140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:17:07.225 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:17:08.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:08 smithi169 ceph-mon[34777]: pgmap v549: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:08.916 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:08 smithi046 ceph-mon[30151]: pgmap v549: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:08.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:08 smithi142 ceph-mon[34533]: pgmap v549: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:09.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.039+0000 7f4fca4dd700 1 -- 172.21.15.46:0/371672814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4107df0 msgr2=0x7f4fc4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.039+0000 7f4fca4dd700 1 --2- 172.21.15.46:0/371672814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4107df0 0x7f4fc4108200 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0099c0 tx=0x7f4fac009cd0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.039+0000 7f4fca4dd700 1 -- 172.21.15.46:0/371672814 shutdown_connections 2024-05-09T00:17:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.039+0000 7f4fca4dd700 1 --2- 172.21.15.46:0/371672814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fc4108740 0x7f4fc41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.039+0000 7f4fca4dd700 1 --2- 172.21.15.46:0/371672814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc41034c0 0x7f4fc41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.039+0000 7f4fca4dd700 1 --2- 172.21.15.46:0/371672814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4107df0 0x7f4fc4108200 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.039+0000 7f4fca4dd700 1 -- 172.21.15.46:0/371672814 >> 172.21.15.46:0/371672814 conn(0x7f4fc40fedc0 msgr2=0x7f4fc41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.039+0000 7f4fca4dd700 1 -- 172.21.15.46:0/371672814 shutdown_connections 2024-05-09T00:17:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.039+0000 7f4fca4dd700 1 -- 172.21.15.46:0/371672814 wait complete. 2024-05-09T00:17:09.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fca4dd700 1 Processor -- start 2024-05-09T00:17:09.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fca4dd700 1 -- start start 2024-05-09T00:17:09.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fca4dd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc41034c0 0x7f4fc419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fca4dd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fc4107df0 0x7f4fc419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fca4dd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4108740 0x7f4fc419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fca4dd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41133c0 con 0x7f4fc4108740 2024-05-09T00:17:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fca4dd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4113240 con 0x7f4fc41034c0 2024-05-09T00:17:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fca4dd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4113540 con 0x7f4fc4107df0 2024-05-09T00:17:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fbbfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fc4107df0 0x7f4fc419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fc8a7a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4108740 0x7f4fc419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fbbfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fc4107df0 0x7f4fc419c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:54178/0 (socket says 172.21.15.46:54178) 2024-05-09T00:17:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fbbfff700 1 -- 172.21.15.46:0/1443432919 learned_addr learned my addr 172.21.15.46:0/1443432919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.040+0000 7f4fc3fff700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc41034c0 0x7f4fc419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.041+0000 7f4fc8a7a700 1 -- 172.21.15.46:0/1443432919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fc4107df0 msgr2=0x7f4fc419c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.041+0000 7f4fc8a7a700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fc4107df0 0x7f4fc419c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.041+0000 7f4fc8a7a700 1 -- 172.21.15.46:0/1443432919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc41034c0 msgr2=0x7f4fc419bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.041+0000 7f4fc8a7a700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc41034c0 0x7f4fc419bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.041+0000 7f4fc8a7a700 1 -- 172.21.15.46:0/1443432919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc41a6f80 con 0x7f4fc4108740 2024-05-09T00:17:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.041+0000 7f4fc3fff700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc41034c0 0x7f4fc419bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.041+0000 7f4fc8a7a700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4108740 0x7f4fc419c760 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00be20 tx=0x7f4fbc009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.041+0000 7f4fc1ffb700 1 -- 172.21.15.46:0/1443432919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc019070 con 0x7f4fc4108740 2024-05-09T00:17:09.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.041+0000 7f4fca4dd700 1 -- 172.21.15.46:0/1443432919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc41a7270 con 0x7f4fc4108740 2024-05-09T00:17:09.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.041+0000 7f4fc1ffb700 1 -- 172.21.15.46:0/1443432919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fbc00cc10 con 0x7f4fc4108740 2024-05-09T00:17:09.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.041+0000 7f4fc1ffb700 1 -- 172.21.15.46:0/1443432919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc0125a0 con 0x7f4fc4108740 2024-05-09T00:17:09.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.041+0000 7f4fca4dd700 1 -- 172.21.15.46:0/1443432919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc41a7760 con 0x7f4fc4108740 2024-05-09T00:17:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.044+0000 7f4fc1ffb700 1 -- 172.21.15.46:0/1443432919 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4fbc01c070 con 0x7f4fc4108740 2024-05-09T00:17:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.044+0000 7f4fc1ffb700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4fb00b2880 0x7f4fb00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fc3fff700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4fb00b2880 0x7f4fb00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fc1ffb700 1 -- 172.21.15.46:0/1443432919 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4fbc0e1830 con 0x7f4fc4108740 2024-05-09T00:17:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fd8ef3b80 1 Processor -- start 2024-05-09T00:17:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fd8ef3b80 1 -- start start 2024-05-09T00:17:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fd8ef3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454bfd00 0x55ac455c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fd8ef3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454454a0 0x55ac455c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fd8ef3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 0x55ac455cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fd8ef3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ac454e50a0 con 0x55ac455cb1e0 2024-05-09T00:17:09.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fd8ef3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ac4548a9a0 con 0x55ac454454a0 2024-05-09T00:17:09.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fd8ef3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ac454e1060 con 0x55ac454bfd00 2024-05-09T00:17:09.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fc3fff700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4fb00b2880 0x7f4fb00b4d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4005fd0 tx=0x7f4fb4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:09.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fc8a7a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 0x55ac455cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fc3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454bfd00 0x55ac455c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fbbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454454a0 0x55ac455c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fc8a7a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 0x55ac455cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57328/0 (socket says 172.21.15.46:57328) 2024-05-09T00:17:09.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fc8a7a700 1 -- 172.21.15.46:0/2319168301 learned_addr learned my addr 172.21.15.46:0/2319168301 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.045+0000 7f4fbbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454454a0 0x55ac455c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:51386/0 (socket says 172.21.15.46:51386) 2024-05-09T00:17:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.046+0000 7f4fc8a7a700 1 -- 172.21.15.46:0/2319168301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454bfd00 msgr2=0x55ac455c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.046+0000 7f4fc8a7a700 1 --2- 172.21.15.46:0/2319168301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454bfd00 0x55ac455c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.046+0000 7f4fc8a7a700 1 -- 172.21.15.46:0/2319168301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454454a0 msgr2=0x55ac455c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.046+0000 7f4fc8a7a700 1 --2- 172.21.15.46:0/2319168301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454454a0 0x55ac455c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.046+0000 7f4fc8a7a700 1 -- 172.21.15.46:0/2319168301 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac4547b500 con 0x55ac455cb1e0 2024-05-09T00:17:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.046+0000 7f4fc3fff700 1 --2- 172.21.15.46:0/2319168301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454bfd00 0x55ac455c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:17:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.046+0000 7f4fbbfff700 1 --2- 172.21.15.46:0/2319168301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454454a0 0x55ac455c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:17:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.046+0000 7f4fc8a7a700 1 --2- 172.21.15.46:0/2319168301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 0x55ac455cb5f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0a60c0 tx=0x7f4fbc014cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ce56cca6137765 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.046+0000 7f4fba7fc700 1 -- 172.21.15.46:0/2319168301 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc0a8e50 con 0x55ac455cb1e0 2024-05-09T00:17:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.046+0000 7f4fba7fc700 1 -- 172.21.15.46:0/2319168301 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fbc0ab030 con 0x55ac455cb1e0 2024-05-09T00:17:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.046+0000 7f4fba7fc700 1 -- 172.21.15.46:0/2319168301 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc0b1a20 con 0x55ac455cb1e0 2024-05-09T00:17:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.047+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/2319168301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 msgr2=0x55ac455cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.047+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/2319168301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 0x55ac455cb5f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0a60c0 tx=0x7f4fbc014cc0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.047+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/2319168301 shutdown_connections 2024-05-09T00:17:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.047+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/2319168301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 0x55ac455cb5f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.047+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/2319168301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454bfd00 0x55ac455c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.047+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/2319168301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454454a0 0x55ac455c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.047+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/2319168301 >> 172.21.15.46:0/2319168301 conn(0x55ac45448f70 msgr2=0x55ac455bd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.047+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/2319168301 shutdown_connections 2024-05-09T00:17:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.047+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/2319168301 wait complete. 2024-05-09T00:17:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.047+0000 7f4fd8ef3b80 1 -- start start 2024-05-09T00:17:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fd8ef3b80 1 Processor -- start 2024-05-09T00:17:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fd8ef3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fd8ef3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 0x55ac455d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fd8ef3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac455cb1e0 0x55ac455d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fd8ef3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ac455cbf00 con 0x55ac454bfd00 2024-05-09T00:17:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fd8ef3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ac4547fb20 con 0x55ac455cb1e0 2024-05-09T00:17:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fd8ef3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ac455d3470 con 0x55ac454454a0 2024-05-09T00:17:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fd8ef3b80 10 client.0 _ll_register_callbacks cb 0x55ac453566c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fbbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 0x55ac455d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fbbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 0x55ac455d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57342/0 (socket says 172.21.15.46:57342) 2024-05-09T00:17:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fc3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fbbfff700 1 -- 172.21.15.46:0/3748560660 learned_addr learned my addr 172.21.15.46:0/3748560660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.048+0000 7f4fc8a7a700 1 --2- 172.21.15.46:0/3748560660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac455cb1e0 0x55ac455d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fbbfff700 1 -- 172.21.15.46:0/3748560660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 msgr2=0x55ac455d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fbbfff700 1 --2- 172.21.15.46:0/3748560660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fbbfff700 1 -- 172.21.15.46:0/3748560660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac455cb1e0 msgr2=0x55ac455d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fbbfff700 1 --2- 172.21.15.46:0/3748560660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac455cb1e0 0x55ac455d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fbbfff700 1 -- 172.21.15.46:0/3748560660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac45360490 con 0x55ac454bfd00 2024-05-09T00:17:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fbbfff700 1 --2- 172.21.15.46:0/3748560660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 0x55ac455d0a10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000c00 tx=0x7f4fac00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992ce342bf476308 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fa37fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fa37fe700 1 -- 172.21.15.46:0/3748560660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac003b60 con 0x55ac454bfd00 2024-05-09T00:17:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fa37fe700 1 -- 172.21.15.46:0/3748560660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4fac003d00 con 0x55ac454bfd00 2024-05-09T00:17:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/3748560660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fbc0ab030 con 0x55ac454bfd00 2024-05-09T00:17:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fa37fe700 1 -- 172.21.15.46:0/3748560660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4fac0050d0 con 0x55ac454bfd00 2024-05-09T00:17:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fd8ef3b80 10 client.16245 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.049+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/3748560660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ac455dde20 con 0x55ac454bfd00 2024-05-09T00:17:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.050+0000 7f4fa37fe700 1 -- 172.21.15.46:0/3748560660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fbc0ab030 con 0x55ac454bfd00 2024-05-09T00:17:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.050+0000 7f4fa37fe700 1 -- 172.21.15.46:0/3748560660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4fac02c030 con 0x55ac454bfd00 2024-05-09T00:17:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.050+0000 7f4fd8ef3b80 10 client.16245 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.050+0000 7f4fd8ef3b80 10 client.16245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:17:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.050+0000 7f4fd8ef3b80 20 client.16245 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:09.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.050+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/3748560660 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ac455e0500 0x55ac455e2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.050+0000 7f4fd8ef3b80 4 client.16245 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:17:09.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.050+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/3748560660 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac454f2a80 con 0x55ac455e0500 2024-05-09T00:17:09.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.050+0000 7f4fc3fff700 1 --2- 172.21.15.46:0/3748560660 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ac455e0500 0x55ac455e2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.051+0000 7f4fc3fff700 1 --2- 172.21.15.46:0/3748560660 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ac455e0500 0x55ac455e2920 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f18d3da7996452d3 server_cookie=edfdad3d9b00e31 in_seq=0 out_seq=0 2024-05-09T00:17:09.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.051+0000 7f4fa37fe700 10 client.16245 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:17:09.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.051+0000 7f4fa37fe700 1 -- 172.21.15.46:0/3748560660 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac454f2a80 con 0x55ac455e0500 2024-05-09T00:17:09.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.051+0000 7f4fa37fe700 10 client.16245 handle_command_reply: tid=0 2024-05-09T00:17:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.051+0000 7f4fd8ef3b80 1 client.16245 shutdown 2024-05-09T00:17:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.052+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/3748560660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 msgr2=0x55ac455d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.052+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/3748560660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 0x55ac455d0a10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000c00 tx=0x7f4fac00bee0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.052+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/3748560660 shutdown_connections 2024-05-09T00:17:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.052+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/3748560660 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ac455e0500 0x55ac455e2920 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.052+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/3748560660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac455cb1e0 0x55ac455d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.052+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/3748560660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.052+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/3748560660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 0x55ac455d0a10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.052+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/3748560660 >> 172.21.15.46:0/3748560660 conn(0x55ac45448f70 msgr2=0x55ac455b6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:09.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.052+0000 7f4fa37fe700 0 client.16245 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:17:09.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.052+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/3748560660 shutdown_connections 2024-05-09T00:17:09.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.052+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/3748560660 wait complete. 2024-05-09T00:17:09.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.052+0000 7f4fd8ef3b80 20 client.16245 trim_cache size 0 max 16384 2024-05-09T00:17:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.067+0000 7f4fd8ef3b80 1 Processor -- start 2024-05-09T00:17:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.067+0000 7f4fd8ef3b80 1 -- start start 2024-05-09T00:17:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.067+0000 7f4fd8ef3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.067+0000 7f4fd8ef3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 0x55ac455b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.067+0000 7f4fd8ef3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac455cb1e0 0x55ac455bd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.067+0000 7f4fd8ef3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41af250 con 0x55ac454bfd00 2024-05-09T00:17:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.067+0000 7f4fd8ef3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41acae0 con 0x55ac455cb1e0 2024-05-09T00:17:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.067+0000 7f4fd8ef3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41aa930 con 0x55ac454454a0 2024-05-09T00:17:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.067+0000 7f4fbbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 0x55ac455b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.067+0000 7f4fbbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 0x55ac455b7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57346/0 (socket says 172.21.15.46:57346) 2024-05-09T00:17:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.067+0000 7f4fbbfff700 1 -- 172.21.15.46:0/514747832 learned_addr learned my addr 172.21.15.46:0/514747832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.068+0000 7f4fc3fff700 1 --2- 172.21.15.46:0/514747832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.068+0000 7f4fbbfff700 1 -- 172.21.15.46:0/514747832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 msgr2=0x55ac455ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.068+0000 7f4fbbfff700 1 --2- 172.21.15.46:0/514747832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.068+0000 7f4fc8a7a700 1 --2- 172.21.15.46:0/514747832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac455cb1e0 0x55ac455bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.068+0000 7f4fbbfff700 1 -- 172.21.15.46:0/514747832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac455cb1e0 msgr2=0x55ac455bd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.068+0000 7f4fbbfff700 1 --2- 172.21.15.46:0/514747832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac455cb1e0 0x55ac455bd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.068+0000 7f4fbbfff700 1 -- 172.21.15.46:0/514747832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac455cea40 con 0x55ac454bfd00 2024-05-09T00:17:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.068+0000 7f4fc3fff700 1 --2- 172.21.15.46:0/514747832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.068+0000 7f4fbbfff700 1 --2- 172.21.15.46:0/514747832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 0x55ac455b7a20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0098e0 tx=0x7f4fac009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcc9daf45d9c8df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.068+0000 7f4fb9ffb700 1 -- 172.21.15.46:0/514747832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac03d070 con 0x55ac454bfd00 2024-05-09T00:17:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.068+0000 7f4fb9ffb700 1 -- 172.21.15.46:0/514747832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac003e10 con 0x55ac454bfd00 2024-05-09T00:17:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.068+0000 7f4fb9ffb700 1 -- 172.21.15.46:0/514747832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac024710 con 0x55ac454bfd00 2024-05-09T00:17:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.069+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/514747832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 msgr2=0x55ac455b7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.069+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/514747832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 0x55ac455b7a20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0098e0 tx=0x7f4fac009310 comp rx=0 tx=0).stop 2024-05-09T00:17:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.069+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/514747832 shutdown_connections 2024-05-09T00:17:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.069+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/514747832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac455cb1e0 0x55ac455bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.069+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/514747832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.069+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/514747832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac454bfd00 0x55ac455b7a20 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.069+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/514747832 >> 172.21.15.46:0/514747832 conn(0x55ac45448f70 msgr2=0x55ac455c13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.069+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/514747832 shutdown_connections 2024-05-09T00:17:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.069+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/514747832 wait complete. 2024-05-09T00:17:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.069+0000 7f4fd8ef3b80 1 -- start start 2024-05-09T00:17:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fd8ef3b80 1 Processor -- start 2024-05-09T00:17:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fd8ef3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455d85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fd8ef3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454bfd00 0x55ac455d8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fd8ef3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 0x55ac455e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fd8ef3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41aafa0 con 0x55ac455cb1e0 2024-05-09T00:17:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fd8ef3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41ac410 con 0x55ac454bfd00 2024-05-09T00:17:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fd8ef3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41ab670 con 0x55ac454454a0 2024-05-09T00:17:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fd8ef3b80 10 client.0 _ll_register_callbacks cb 0x55ac453566c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fc8a7a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 0x55ac455e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fc8a7a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 0x55ac455e70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57350/0 (socket says 172.21.15.46:57350) 2024-05-09T00:17:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fc8a7a700 1 -- 172.21.15.46:0/1389651575 learned_addr learned my addr 172.21.15.46:0/1389651575 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fc3fff700 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455d85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fbbfff700 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454bfd00 0x55ac455d8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fc8a7a700 1 -- 172.21.15.46:0/1389651575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 msgr2=0x55ac455d85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fc8a7a700 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455d85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fc8a7a700 1 -- 172.21.15.46:0/1389651575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454bfd00 msgr2=0x55ac455d8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:09.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fc8a7a700 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454bfd00 0x55ac455d8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fc8a7a700 1 -- 172.21.15.46:0/1389651575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac455e77f0 con 0x55ac455cb1e0 2024-05-09T00:17:09.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.070+0000 7f4fbbfff700 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454bfd00 0x55ac455d8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:17:09.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fc8a7a700 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 0x55ac455e70c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0a8b50 tx=0x7f4fbc0097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d58d0bece8070be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fa37fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fa37fe700 1 -- 172.21.15.46:0/1389651575 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc0a8e50 con 0x55ac455cb1e0 2024-05-09T00:17:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fa37fe700 1 -- 172.21.15.46:0/1389651575 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4fbc0a5440 con 0x55ac455cb1e0 2024-05-09T00:17:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/1389651575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fac003e10 con 0x55ac455cb1e0 2024-05-09T00:17:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fa37fe700 1 -- 172.21.15.46:0/1389651575 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4fbc0b1cb0 con 0x55ac455cb1e0 2024-05-09T00:17:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fd8ef3b80 10 client.16251 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/1389651575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ac4564a600 con 0x55ac455cb1e0 2024-05-09T00:17:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fa37fe700 1 -- 172.21.15.46:0/1389651575 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac003e10 con 0x55ac455cb1e0 2024-05-09T00:17:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fa37fe700 1 -- 172.21.15.46:0/1389651575 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4fbc0c0030 con 0x55ac455cb1e0 2024-05-09T00:17:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fd8ef3b80 10 client.16251 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fd8ef3b80 10 client.16251 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:17:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fd8ef3b80 20 client.16251 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ac4564a8e0 0x55ac4564cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fd8ef3b80 4 client.16251 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:17:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.071+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/1389651575 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac454ecb20 con 0x55ac4564a8e0 2024-05-09T00:17:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.072+0000 7f4fc3fff700 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ac4564a8e0 0x55ac4564cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.072+0000 7f4fc3fff700 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ac4564a8e0 0x55ac4564cd00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fcf561eab02df5f server_cookie=ebec00597669aa81 in_seq=0 out_seq=0 2024-05-09T00:17:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.072+0000 7f4fa37fe700 10 client.16251 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:17:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.072+0000 7f4fa37fe700 1 -- 172.21.15.46:0/1389651575 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac454ecb20 con 0x55ac4564a8e0 2024-05-09T00:17:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.072+0000 7f4fa37fe700 10 client.16251 handle_command_reply: tid=0 2024-05-09T00:17:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.072+0000 7f4fd8ef3b80 1 client.16251 shutdown 2024-05-09T00:17:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/1389651575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 msgr2=0x55ac455e70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:09.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 0x55ac455e70c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0a8b50 tx=0x7f4fbc0097f0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/1389651575 shutdown_connections 2024-05-09T00:17:09.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ac4564a8e0 0x55ac4564cd00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fa37fe700 0 client.16251 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:17:09.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac455cb1e0 0x55ac455e70c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ac454454a0 0x55ac455d85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fd8ef3b80 1 --2- 172.21.15.46:0/1389651575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac454bfd00 0x55ac455d8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/1389651575 >> 172.21.15.46:0/1389651575 conn(0x55ac45448f70 msgr2=0x55ac455b7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/1389651575 shutdown_connections 2024-05-09T00:17:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fd8ef3b80 1 -- 172.21.15.46:0/1389651575 wait complete. 2024-05-09T00:17:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fd8ef3b80 20 client.16251 trim_cache size 0 max 16384 2024-05-09T00:17:09.092 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:17:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fca4dd700 1 -- 172.21.15.46:0/1443432919 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4fb00b2880 msgr2=0x7f4fb00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fca4dd700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4fb00b2880 0x7f4fb00b4d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4005fd0 tx=0x7f4fb4005e30 comp rx=0 tx=0).stop 2024-05-09T00:17:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fca4dd700 1 -- 172.21.15.46:0/1443432919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4108740 msgr2=0x7f4fc419c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.073+0000 7f4fca4dd700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4108740 0x7f4fc419c760 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00be20 tx=0x7f4fbc009ed0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.074+0000 7f4fca4dd700 1 -- 172.21.15.46:0/1443432919 shutdown_connections 2024-05-09T00:17:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.074+0000 7f4fca4dd700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4fb00b2880 0x7f4fb00b4d90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.074+0000 7f4fca4dd700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4108740 0x7f4fc419c760 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.074+0000 7f4fca4dd700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc41034c0 0x7f4fc419bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.074+0000 7f4fca4dd700 1 --2- 172.21.15.46:0/1443432919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fc4107df0 0x7f4fc419c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.074+0000 7f4fca4dd700 1 -- 172.21.15.46:0/1443432919 >> 172.21.15.46:0/1443432919 conn(0x7f4fc40fedc0 msgr2=0x7f4fc41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.074+0000 7f4fca4dd700 1 -- 172.21.15.46:0/1443432919 shutdown_connections 2024-05-09T00:17:09.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:09.074+0000 7f4fca4dd700 1 -- 172.21.15.46:0/1443432919 wait complete. 2024-05-09T00:17:09.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:17:10.043 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:17:10.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:10 smithi142 ceph-mon[34533]: pgmap v550: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:17:11.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:10 smithi046 ceph-mon[30151]: pgmap v550: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:17:11.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:10 smithi169 ceph-mon[34777]: pgmap v550: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:17:11.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.717+0000 7f582f6a0700 1 -- 172.21.15.46:0/188770949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5828109850 msgr2=0x7f58281111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:11.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.717+0000 7f582f6a0700 1 --2- 172.21.15.46:0/188770949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5828109850 0x7f58281111b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f582000b0d0 tx=0x7f582000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.717+0000 7f582f6a0700 1 -- 172.21.15.46:0/188770949 shutdown_connections 2024-05-09T00:17:11.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.717+0000 7f582f6a0700 1 --2- 172.21.15.46:0/188770949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5828109850 0x7f58281111b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.717+0000 7f582f6a0700 1 --2- 172.21.15.46:0/188770949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58281045d0 0x7f58281049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.717+0000 7f582f6a0700 1 --2- 172.21.15.46:0/188770949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5828108f00 0x7f5828109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.717+0000 7f582f6a0700 1 -- 172.21.15.46:0/188770949 >> 172.21.15.46:0/188770949 conn(0x7f58280ffe80 msgr2=0x7f58281022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:11.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.717+0000 7f582f6a0700 1 -- 172.21.15.46:0/188770949 shutdown_connections 2024-05-09T00:17:11.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.717+0000 7f582f6a0700 1 -- 172.21.15.46:0/188770949 wait complete. 2024-05-09T00:17:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.718+0000 7f582f6a0700 1 Processor -- start 2024-05-09T00:17:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582f6a0700 1 -- start start 2024-05-09T00:17:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582f6a0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58281045d0 0x7f582819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582f6a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5828108f00 0x7f582819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582f6a0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5828109850 0x7f582819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582f6a0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5828114480 con 0x7f5828108f00 2024-05-09T00:17:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582f6a0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5828114300 con 0x7f5828109850 2024-05-09T00:17:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582f6a0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5828114600 con 0x7f58281045d0 2024-05-09T00:17:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582cc3b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5828108f00 0x7f582819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582cc3b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5828108f00 0x7f582819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57370/0 (socket says 172.21.15.46:57370) 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582cc3b700 1 -- 172.21.15.46:0/3019635505 learned_addr learned my addr 172.21.15.46:0/3019635505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582d43c700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58281045d0 0x7f582819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582dc3d700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5828109850 0x7f582819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582cc3b700 1 -- 172.21.15.46:0/3019635505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58281045d0 msgr2=0x7f582819adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582cc3b700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58281045d0 0x7f582819adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582cc3b700 1 -- 172.21.15.46:0/3019635505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5828109850 msgr2=0x7f582819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582cc3b700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5828109850 0x7f582819f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.719+0000 7f582cc3b700 1 -- 172.21.15.46:0/3019635505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582819fc80 con 0x7f5828108f00 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.720+0000 7f582cc3b700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5828108f00 0x7f582819b350 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f581800e970 tx=0x7f581800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.720+0000 7f581e7fc700 1 -- 172.21.15.46:0/3019635505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581800cbd0 con 0x7f5828108f00 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.720+0000 7f581e7fc700 1 -- 172.21.15.46:0/3019635505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f581800cd70 con 0x7f5828108f00 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.720+0000 7f582f6a0700 1 -- 172.21.15.46:0/3019635505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582819ff70 con 0x7f5828108f00 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.720+0000 7f581e7fc700 1 -- 172.21.15.46:0/3019635505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818005680 con 0x7f5828108f00 2024-05-09T00:17:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.720+0000 7f582f6a0700 1 -- 172.21.15.46:0/3019635505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58281a7890 con 0x7f5828108f00 2024-05-09T00:17:11.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.723+0000 7f581e7fc700 1 -- 172.21.15.46:0/3019635505 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5818010430 con 0x7f5828108f00 2024-05-09T00:17:11.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f583e0b6b80 1 Processor -- start 2024-05-09T00:17:11.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f583e0b6b80 1 -- start start 2024-05-09T00:17:11.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f583e0b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c0e8d00 0x56260c1f1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f583e0b6b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c06e4a0 0x56260c1f1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f583e0b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c1eb400 0x56260c1eb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f583e0b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56260c10e0a0 con 0x56260c1eb400 2024-05-09T00:17:11.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f583e0b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56260c0b39a0 con 0x56260c0e8d00 2024-05-09T00:17:11.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f583e0b6b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56260c10a060 con 0x56260c06e4a0 2024-05-09T00:17:11.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f581e7fc700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58140b2900 0x7f58140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f582dc3d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c0e8d00 0x56260c1f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f582dc3d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c0e8d00 0x56260c1f1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:51434/0 (socket says 172.21.15.46:51434) 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f582dc3d700 1 -- 172.21.15.46:0/3095000366 learned_addr learned my addr 172.21.15.46:0/3095000366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f582cc3b700 1 --2- 172.21.15.46:0/3095000366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c1eb400 0x56260c1eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.724+0000 7f582dc3d700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58140b2900 0x7f58140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.725+0000 7f582d43c700 1 --2- 172.21.15.46:0/3095000366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c06e4a0 0x56260c1f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.725+0000 7f581e7fc700 1 -- 172.21.15.46:0/3019635505 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f58180e24d0 con 0x7f5828108f00 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.725+0000 7f582d43c700 1 -- 172.21.15.46:0/3095000366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c0e8d00 msgr2=0x56260c1f1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.725+0000 7f582d43c700 1 --2- 172.21.15.46:0/3095000366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c0e8d00 0x56260c1f1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.725+0000 7f582d43c700 1 -- 172.21.15.46:0/3095000366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c1eb400 msgr2=0x56260c1eb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.725+0000 7f582d43c700 1 --2- 172.21.15.46:0/3095000366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c1eb400 0x56260c1eb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.725+0000 7f582d43c700 1 -- 172.21.15.46:0/3095000366 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260c0a4500 con 0x56260c06e4a0 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.725+0000 7f582dc3d700 1 --2- 172.21.15.46:0/3095000366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c0e8d00 0x56260c1f1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.725+0000 7f582dc3d700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58140b2900 0x7f58140b4e10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5820009040 tx=0x7f5820005ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.725+0000 7f582d43c700 1 --2- 172.21.15.46:0/3095000366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c06e4a0 0x56260c1f1780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5824007bf0 tx=0x7f5824007f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cf441d95c4fc1e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.726+0000 7f58137fe700 1 -- 172.21.15.46:0/3095000366 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58240042f0 con 0x56260c06e4a0 2024-05-09T00:17:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.726+0000 7f58137fe700 1 -- 172.21.15.46:0/3095000366 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5824015070 con 0x56260c06e4a0 2024-05-09T00:17:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.726+0000 7f58137fe700 1 -- 172.21.15.46:0/3095000366 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824004db0 con 0x56260c06e4a0 2024-05-09T00:17:11.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.726+0000 7f583e0b6b80 1 -- 172.21.15.46:0/3095000366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c06e4a0 msgr2=0x56260c1f1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.726+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/3095000366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c06e4a0 0x56260c1f1780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5824007bf0 tx=0x7f5824007f00 comp rx=0 tx=0).stop 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.726+0000 7f583e0b6b80 1 -- 172.21.15.46:0/3095000366 shutdown_connections 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.726+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/3095000366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c1eb400 0x56260c1eb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.726+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/3095000366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c0e8d00 0x56260c1f1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.727+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/3095000366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c06e4a0 0x56260c1f1780 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.727+0000 7f583e0b6b80 1 -- 172.21.15.46:0/3095000366 >> 172.21.15.46:0/3095000366 conn(0x56260c071f70 msgr2=0x56260c1e6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.727+0000 7f583e0b6b80 1 -- 172.21.15.46:0/3095000366 shutdown_connections 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.727+0000 7f583e0b6b80 1 -- 172.21.15.46:0/3095000366 wait complete. 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.727+0000 7f583e0b6b80 1 -- start start 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.727+0000 7f583e0b6b80 1 Processor -- start 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f583e0b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 0x56260c1f5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f583e0b6b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c0e8d00 0x56260c1f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f583e0b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c1eb400 0x56260c1f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f583e0b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56260c1ec160 con 0x56260c06e4a0 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f583e0b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56260c0a8b20 con 0x56260c1eb400 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f583e0b6b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56260c1f8020 con 0x56260c0e8d00 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f583e0b6b80 10 client.0 _ll_register_callbacks cb 0x56260bf7f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f582d43c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 0x56260c1f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f582d43c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 0x56260c1f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57402/0 (socket says 172.21.15.46:57402) 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f582d43c700 1 -- 172.21.15.46:0/1455804710 learned_addr learned my addr 172.21.15.46:0/1455804710 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f582dc3d700 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c1eb400 0x56260c1f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f582cc3b700 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c0e8d00 0x56260c1f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f582d43c700 1 -- 172.21.15.46:0/1455804710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c0e8d00 msgr2=0x56260c1f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f582d43c700 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c0e8d00 0x56260c1f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f582d43c700 1 -- 172.21.15.46:0/1455804710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c1eb400 msgr2=0x56260c1f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f582d43c700 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c1eb400 0x56260c1f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f582d43c700 1 -- 172.21.15.46:0/1455804710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260bf89490 con 0x56260c06e4a0 2024-05-09T00:17:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f582d43c700 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 0x56260c1f5080 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5824007670 tx=0x7f5824004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8757cc170d0e56d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.728+0000 7f5810ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.729+0000 7f5810ff9700 1 -- 172.21.15.46:0/1455804710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824028060 con 0x56260c06e4a0 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.729+0000 7f5810ff9700 1 -- 172.21.15.46:0/1455804710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f5824010d90 con 0x56260c06e4a0 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.729+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1455804710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5824015070 con 0x56260c06e4a0 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.729+0000 7f5810ff9700 1 -- 172.21.15.46:0/1455804710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5824005030 con 0x56260c06e4a0 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.729+0000 7f583e0b6b80 10 client.16263 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.729+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1455804710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56260c202a50 con 0x56260c06e4a0 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.729+0000 7f5810ff9700 1 -- 172.21.15.46:0/1455804710 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5824015070 con 0x56260c06e4a0 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.729+0000 7f5810ff9700 1 -- 172.21.15.46:0/1455804710 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5824022030 con 0x56260c06e4a0 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.729+0000 7f583e0b6b80 10 client.16263 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.730+0000 7f583e0b6b80 10 client.16263 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.730+0000 7f583e0b6b80 20 client.16263 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.730+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56260c204f20 0x56260c207340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.730+0000 7f583e0b6b80 4 client.16263 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:17:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.730+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1455804710 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56260c11ba80 con 0x56260c204f20 2024-05-09T00:17:11.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.730+0000 7f582cc3b700 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56260c204f20 0x56260c207340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.730+0000 7f5810ff9700 10 client.16263 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:17:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.730+0000 7f582cc3b700 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56260c204f20 0x56260c207340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=198c4efaaef9a16a server_cookie=e755ee44d7c7acba in_seq=0 out_seq=0 2024-05-09T00:17:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.731+0000 7f5810ff9700 1 -- 172.21.15.46:0/1455804710 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56260c11ba80 con 0x56260c204f20 2024-05-09T00:17:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.731+0000 7f5810ff9700 10 client.16263 handle_command_reply: tid=0 2024-05-09T00:17:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.731+0000 7f583e0b6b80 1 client.16263 shutdown 2024-05-09T00:17:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.731+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1455804710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 msgr2=0x56260c1f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.731+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 0x56260c1f5080 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5824007670 tx=0x7f5824004460 comp rx=0 tx=0).stop 2024-05-09T00:17:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.732+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1455804710 shutdown_connections 2024-05-09T00:17:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.732+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56260c204f20 0x56260c207340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.732+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c1eb400 0x56260c1f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.732+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 0x56260c1f5080 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.732+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/1455804710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c0e8d00 0x56260c1f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.732+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1455804710 >> 172.21.15.46:0/1455804710 conn(0x56260c071f70 msgr2=0x56260c1dfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.732+0000 7f5810ff9700 0 client.16263 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:17:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.732+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1455804710 shutdown_connections 2024-05-09T00:17:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.732+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1455804710 wait complete. 2024-05-09T00:17:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.732+0000 7f583e0b6b80 20 client.16263 trim_cache size 0 max 16384 2024-05-09T00:17:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.747+0000 7f583e0b6b80 1 Processor -- start 2024-05-09T00:17:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.747+0000 7f583e0b6b80 1 -- start start 2024-05-09T00:17:11.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f583e0b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c06e4a0 0x56260c1f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f583e0b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c0e8d00 0x56260c1e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f583e0b6b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c1eb400 0x56260c1f5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f583e0b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58281af360 con 0x56260c0e8d00 2024-05-09T00:17:11.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f583e0b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f58281acbf0 con 0x56260c06e4a0 2024-05-09T00:17:11.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f583e0b6b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f58281aaa20 con 0x56260c1eb400 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f582cc3b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c0e8d00 0x56260c1e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f582cc3b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c0e8d00 0x56260c1e0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57406/0 (socket says 172.21.15.46:57406) 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f582cc3b700 1 -- 172.21.15.46:0/423036221 learned_addr learned my addr 172.21.15.46:0/423036221 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f582d43c700 1 --2- 172.21.15.46:0/423036221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c06e4a0 0x56260c1f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f582dc3d700 1 --2- 172.21.15.46:0/423036221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c1eb400 0x56260c1f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f582cc3b700 1 -- 172.21.15.46:0/423036221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c1eb400 msgr2=0x56260c1f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f582cc3b700 1 --2- 172.21.15.46:0/423036221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c1eb400 0x56260c1f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f582cc3b700 1 -- 172.21.15.46:0/423036221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c06e4a0 msgr2=0x56260c1f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f582cc3b700 1 --2- 172.21.15.46:0/423036221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c06e4a0 0x56260c1f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f582cc3b700 1 -- 172.21.15.46:0/423036221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260c1e6f10 con 0x56260c0e8d00 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.748+0000 7f582cc3b700 1 --2- 172.21.15.46:0/423036221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c0e8d00 0x56260c1e0a20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f58180050d0 tx=0x7f581801bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4eb11a7d6189f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.749+0000 7f5812ffd700 1 -- 172.21.15.46:0/423036221 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180a59b0 con 0x56260c0e8d00 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.749+0000 7f5812ffd700 1 -- 172.21.15.46:0/423036221 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58180b2290 con 0x56260c0e8d00 2024-05-09T00:17:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.749+0000 7f5812ffd700 1 -- 172.21.15.46:0/423036221 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581801bd90 con 0x56260c0e8d00 2024-05-09T00:17:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.750+0000 7f583e0b6b80 1 -- 172.21.15.46:0/423036221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c0e8d00 msgr2=0x56260c1e0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.750+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/423036221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c0e8d00 0x56260c1e0a20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f58180050d0 tx=0x7f581801bd20 comp rx=0 tx=0).stop 2024-05-09T00:17:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.750+0000 7f583e0b6b80 1 -- 172.21.15.46:0/423036221 shutdown_connections 2024-05-09T00:17:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.750+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/423036221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c1eb400 0x56260c1f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.750+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/423036221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c06e4a0 0x56260c1f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.750+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/423036221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c0e8d00 0x56260c1e0a20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.750+0000 7f583e0b6b80 1 -- 172.21.15.46:0/423036221 >> 172.21.15.46:0/423036221 conn(0x56260c071f70 msgr2=0x56260c1eec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.750+0000 7f583e0b6b80 1 -- 172.21.15.46:0/423036221 shutdown_connections 2024-05-09T00:17:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.750+0000 7f583e0b6b80 1 -- 172.21.15.46:0/423036221 wait complete. 2024-05-09T00:17:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.750+0000 7f583e0b6b80 1 -- start start 2024-05-09T00:17:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.750+0000 7f583e0b6b80 1 Processor -- start 2024-05-09T00:17:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f583e0b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 0x56260c2072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f583e0b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c0e8d00 0x56260c207830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f583e0b6b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c1eb400 0x56260c26d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f583e0b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58281ab0b0 con 0x56260c06e4a0 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f583e0b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f58281ac520 con 0x56260c0e8d00 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f583e0b6b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f58281ab780 con 0x56260c1eb400 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f583e0b6b80 10 client.0 _ll_register_callbacks cb 0x56260bf7f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f582d43c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 0x56260c2072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f582d43c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 0x56260c2072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57410/0 (socket says 172.21.15.46:57410) 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f582d43c700 1 -- 172.21.15.46:0/1149424201 learned_addr learned my addr 172.21.15.46:0/1149424201 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f582cc3b700 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c0e8d00 0x56260c207830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f582dc3d700 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c1eb400 0x56260c26d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f582d43c700 1 -- 172.21.15.46:0/1149424201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c1eb400 msgr2=0x56260c26d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:11.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f582d43c700 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c1eb400 0x56260c26d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f582d43c700 1 -- 172.21.15.46:0/1149424201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c0e8d00 msgr2=0x56260c207830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f582d43c700 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c0e8d00 0x56260c207830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f582d43c700 1 -- 172.21.15.46:0/1149424201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260c1e29d0 con 0x56260c06e4a0 2024-05-09T00:17:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.751+0000 7f582dc3d700 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c1eb400 0x56260c26d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f582d43c700 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 0x56260c2072f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5824007f40 tx=0x7f5824007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1bf2ebab661f81 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f5810ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f5810ff9700 1 -- 172.21.15.46:0/1149424201 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824028070 con 0x56260c06e4a0 2024-05-09T00:17:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f5810ff9700 1 -- 172.21.15.46:0/1149424201 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f5824010b90 con 0x56260c06e4a0 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1149424201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56260c115b20 con 0x56260c06e4a0 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f5810ff9700 1 -- 172.21.15.46:0/1149424201 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f582403e050 con 0x56260c06e4a0 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f583e0b6b80 10 client.16269 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1149424201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56260c26ea60 con 0x56260c06e4a0 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f5810ff9700 1 -- 172.21.15.46:0/1149424201 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5824015040 con 0x56260c06e4a0 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f5810ff9700 1 -- 172.21.15.46:0/1149424201 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5824022040 con 0x56260c06e4a0 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f583e0b6b80 10 client.16269 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f583e0b6b80 10 client.16269 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f583e0b6b80 20 client.16269 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56260c26f140 0x56260c271560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f583e0b6b80 4 client.16269 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.752+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1149424201 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56260c115b20 con 0x56260c26f140 2024-05-09T00:17:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.753+0000 7f582cc3b700 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56260c26f140 0x56260c271560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.753+0000 7f5810ff9700 10 client.16269 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:17:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.753+0000 7f582cc3b700 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56260c26f140 0x56260c271560 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a4fd7567645df25 server_cookie=d9a31caa60d4f884 in_seq=0 out_seq=0 2024-05-09T00:17:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f5810ff9700 1 -- 172.21.15.46:0/1149424201 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56260c115b20 con 0x56260c26f140 2024-05-09T00:17:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f5810ff9700 10 client.16269 handle_command_reply: tid=0 2024-05-09T00:17:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f583e0b6b80 1 client.16269 shutdown 2024-05-09T00:17:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1149424201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 msgr2=0x56260c2072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 0x56260c2072f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5824007f40 tx=0x7f5824007520 comp rx=0 tx=0).stop 2024-05-09T00:17:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1149424201 shutdown_connections 2024-05-09T00:17:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x56260c26f140 0x56260c271560 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56260c1eb400 0x56260c26d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56260c06e4a0 0x56260c2072f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f583e0b6b80 1 --2- 172.21.15.46:0/1149424201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56260c0e8d00 0x56260c207830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1149424201 >> 172.21.15.46:0/1149424201 conn(0x56260c071f70 msgr2=0x56260c1e0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f5810ff9700 0 client.16269 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:17:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1149424201 shutdown_connections 2024-05-09T00:17:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f583e0b6b80 1 -- 172.21.15.46:0/1149424201 wait complete. 2024-05-09T00:17:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.754+0000 7f583e0b6b80 20 client.16269 trim_cache size 0 max 16384 2024-05-09T00:17:11.758 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:17:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.755+0000 7f582f6a0700 1 -- 172.21.15.46:0/3019635505 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58140b2900 msgr2=0x7f58140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.755+0000 7f582f6a0700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58140b2900 0x7f58140b4e10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5820009040 tx=0x7f5820005ae0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.755+0000 7f582f6a0700 1 -- 172.21.15.46:0/3019635505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5828108f00 msgr2=0x7f582819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.755+0000 7f582f6a0700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5828108f00 0x7f582819b350 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f581800e970 tx=0x7f581800ec80 comp rx=0 tx=0).stop 2024-05-09T00:17:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.755+0000 7f582f6a0700 1 -- 172.21.15.46:0/3019635505 shutdown_connections 2024-05-09T00:17:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.755+0000 7f582f6a0700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f58140b2900 0x7f58140b4e10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.755+0000 7f582f6a0700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5828109850 0x7f582819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.755+0000 7f582f6a0700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58281045d0 0x7f582819adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.755+0000 7f582f6a0700 1 --2- 172.21.15.46:0/3019635505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5828108f00 0x7f582819b350 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.755+0000 7f582f6a0700 1 -- 172.21.15.46:0/3019635505 >> 172.21.15.46:0/3019635505 conn(0x7f58280ffe80 msgr2=0x7f58281086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.755+0000 7f582f6a0700 1 -- 172.21.15.46:0/3019635505 shutdown_connections 2024-05-09T00:17:11.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:11.755+0000 7f582f6a0700 1 -- 172.21.15.46:0/3019635505 wait complete. 2024-05-09T00:17:12.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:17:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:17:12.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:12 smithi046 ceph-mon[30151]: pgmap v551: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:12.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:12 smithi142 ceph-mon[34533]: pgmap v551: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:13.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:12 smithi169 ceph-mon[34777]: pgmap v551: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:14.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.620+0000 7f15e2e50700 1 -- 172.21.15.46:0/3982287470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15dc108760 msgr2=0x7f15dc1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:14.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.620+0000 7f15e2e50700 1 --2- 172.21.15.46:0/3982287470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15dc108760 0x7f15dc1100c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f15d400b0d0 tx=0x7f15d400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.620+0000 7f15e2e50700 1 -- 172.21.15.46:0/3982287470 shutdown_connections 2024-05-09T00:17:14.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.620+0000 7f15e2e50700 1 --2- 172.21.15.46:0/3982287470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15dc108760 0x7f15dc1100c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.620+0000 7f15e2e50700 1 --2- 172.21.15.46:0/3982287470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15dc1034e0 0x7f15dc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.620+0000 7f15e2e50700 1 --2- 172.21.15.46:0/3982287470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15dc107e10 0x7f15dc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.620+0000 7f15e2e50700 1 -- 172.21.15.46:0/3982287470 >> 172.21.15.46:0/3982287470 conn(0x7f15dc0fef30 msgr2=0x7f15dc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:14.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.620+0000 7f15e2e50700 1 -- 172.21.15.46:0/3982287470 shutdown_connections 2024-05-09T00:17:14.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.620+0000 7f15e2e50700 1 -- 172.21.15.46:0/3982287470 wait complete. 2024-05-09T00:17:14.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.621+0000 7f15e2e50700 1 Processor -- start 2024-05-09T00:17:14.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.621+0000 7f15e2e50700 1 -- start start 2024-05-09T00:17:14.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.621+0000 7f15e2e50700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15dc1034e0 0x7f15dc19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e2e50700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15dc107e10 0x7f15dc19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e2e50700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15dc108760 0x7f15dc19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e2e50700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc113390 con 0x7f15dc108760 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e2e50700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc113210 con 0x7f15dc107e10 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e2e50700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc113510 con 0x7f15dc1034e0 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e13ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15dc108760 0x7f15dc19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e13ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15dc108760 0x7f15dc19f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57434/0 (socket says 172.21.15.46:57434) 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e13ed700 1 -- 172.21.15.46:0/1092025255 learned_addr learned my addr 172.21.15.46:0/1092025255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e0bec700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15dc1034e0 0x7f15dc19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15dbfff700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15dc107e10 0x7f15dc19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e13ed700 1 -- 172.21.15.46:0/1092025255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15dc1034e0 msgr2=0x7f15dc19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e13ed700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15dc1034e0 0x7f15dc19ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e13ed700 1 -- 172.21.15.46:0/1092025255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15dc107e10 msgr2=0x7f15dc19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e13ed700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15dc107e10 0x7f15dc19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e13ed700 1 -- 172.21.15.46:0/1092025255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15dc19fc60 con 0x7f15dc108760 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e13ed700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15dc108760 0x7f15dc19f5c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f15d40095c0 tx=0x7f15d4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:14.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15d9ffb700 1 -- 172.21.15.46:0/1092025255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d401f070 con 0x7f15dc108760 2024-05-09T00:17:14.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15d9ffb700 1 -- 172.21.15.46:0/1092025255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d4007e20 con 0x7f15dc108760 2024-05-09T00:17:14.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15e2e50700 1 -- 172.21.15.46:0/1092025255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15dc19fef0 con 0x7f15dc108760 2024-05-09T00:17:14.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.622+0000 7f15d9ffb700 1 -- 172.21.15.46:0/1092025255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d401b4a0 con 0x7f15dc108760 2024-05-09T00:17:14.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.623+0000 7f15e2e50700 1 -- 172.21.15.46:0/1092025255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15dc1a03d0 con 0x7f15dc108760 2024-05-09T00:17:14.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.626+0000 7f15d9ffb700 1 -- 172.21.15.46:0/1092025255 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f15d4022070 con 0x7f15dc108760 2024-05-09T00:17:14.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.627+0000 7f15d9ffb700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f15c40b2880 0x7f15c40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.627+0000 7f15d9ffb700 1 -- 172.21.15.46:0/1092025255 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f15d400f020 con 0x7f15dc108760 2024-05-09T00:17:14.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.627+0000 7f15e0bec700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f15c40b2880 0x7f15c40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.627+0000 7f15f1866b80 1 Processor -- start 2024-05-09T00:17:14.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.627+0000 7f15f1866b80 1 -- start start 2024-05-09T00:17:14.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15f1866b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0fe160 0x56171b1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15f1866b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0144a0 0x56171b197a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15f1866b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b191ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15e0bec700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f15c40b2880 0x7f15c40b4d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f15cc005950 tx=0x7f15cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15f1866b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56171b0b3ff0 con 0x56171b1916a0 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15f1866b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56171b0527f0 con 0x56171b0144a0 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15f1866b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56171b0affb0 con 0x56171b0fe160 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15e13ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b191ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15e13ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b191ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57448/0 (socket says 172.21.15.46:57448) 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15e13ed700 1 -- 172.21.15.46:0/3001736128 learned_addr learned my addr 172.21.15.46:0/3001736128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15e0bec700 1 --2- 172.21.15.46:0/3001736128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0fe160 0x56171b1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15dbfff700 1 --2- 172.21.15.46:0/3001736128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0144a0 0x56171b197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15e13ed700 1 -- 172.21.15.46:0/3001736128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0fe160 msgr2=0x56171b1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15e13ed700 1 --2- 172.21.15.46:0/3001736128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0fe160 0x56171b1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15e13ed700 1 -- 172.21.15.46:0/3001736128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0144a0 msgr2=0x56171b197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:14.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15e13ed700 1 --2- 172.21.15.46:0/3001736128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0144a0 0x56171b197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15e13ed700 1 -- 172.21.15.46:0/3001736128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56171b04a370 con 0x56171b1916a0 2024-05-09T00:17:14.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15dbfff700 1 --2- 172.21.15.46:0/3001736128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0144a0 0x56171b197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:14.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.628+0000 7f15e13ed700 1 --2- 172.21.15.46:0/3001736128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b191ad0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f15d40aab30 tx=0x7f15d40aa390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abddfdac46fd1533 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:14.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.629+0000 7f15c2ffd700 1 -- 172.21.15.46:0/3001736128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d40b31e0 con 0x56171b1916a0 2024-05-09T00:17:14.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.629+0000 7f15c2ffd700 1 -- 172.21.15.46:0/3001736128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d40aee70 con 0x56171b1916a0 2024-05-09T00:17:14.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.629+0000 7f15c2ffd700 1 -- 172.21.15.46:0/3001736128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4022e40 con 0x56171b1916a0 2024-05-09T00:17:14.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.629+0000 7f15f1866b80 1 -- 172.21.15.46:0/3001736128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 msgr2=0x56171b191ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:14.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.629+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3001736128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b191ad0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f15d40aab30 tx=0x7f15d40aa390 comp rx=0 tx=0).stop 2024-05-09T00:17:14.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.630+0000 7f15f1866b80 1 -- 172.21.15.46:0/3001736128 shutdown_connections 2024-05-09T00:17:14.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.630+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3001736128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b191ad0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.630+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3001736128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0fe160 0x56171b1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.630+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3001736128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0144a0 0x56171b197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.630+0000 7f15f1866b80 1 -- 172.21.15.46:0/3001736128 >> 172.21.15.46:0/3001736128 conn(0x56171b0746a0 msgr2=0x56171b18c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.630+0000 7f15f1866b80 1 -- 172.21.15.46:0/3001736128 shutdown_connections 2024-05-09T00:17:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.630+0000 7f15f1866b80 1 -- 172.21.15.46:0/3001736128 wait complete. 2024-05-09T00:17:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.630+0000 7f15f1866b80 1 -- start start 2024-05-09T00:17:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.630+0000 7f15f1866b80 1 Processor -- start 2024-05-09T00:17:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15f1866b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0144a0 0x56171b1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15f1866b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0fe160 0x56171b1a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15f1866b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b19b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15f1866b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56171af2fba0 con 0x56171b1916a0 2024-05-09T00:17:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15f1866b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56171b06e150 con 0x56171b0144a0 2024-05-09T00:17:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15f1866b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56171b192360 con 0x56171b0fe160 2024-05-09T00:17:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15f1866b80 10 client.0 _ll_register_callbacks cb 0x56171af256c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15e13ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15e13ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b19b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57452/0 (socket says 172.21.15.46:57452) 2024-05-09T00:17:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15e13ed700 1 -- 172.21.15.46:0/3882288957 learned_addr learned my addr 172.21.15.46:0/3882288957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15dbfff700 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0fe160 0x56171b1a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15e0bec700 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0144a0 0x56171b1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15e13ed700 1 -- 172.21.15.46:0/3882288957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0fe160 msgr2=0x56171b1a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15e13ed700 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0fe160 0x56171b1a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15e13ed700 1 -- 172.21.15.46:0/3882288957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0144a0 msgr2=0x56171b1a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15e13ed700 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0144a0 0x56171b1a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15e13ed700 1 -- 172.21.15.46:0/3882288957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171af2f490 con 0x56171b1916a0 2024-05-09T00:17:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15e13ed700 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b19b080 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f15d400b420 tx=0x7f15d40a6af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174993a4e1d2763c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.631+0000 7f15bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15bbfff700 1 -- 172.21.15.46:0/3882288957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d406f070 con 0x56171b1916a0 2024-05-09T00:17:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15bbfff700 1 -- 172.21.15.46:0/3882288957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f15d40712c0 con 0x56171b1916a0 2024-05-09T00:17:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15f1866b80 1 -- 172.21.15.46:0/3882288957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15d40aee70 con 0x56171b1916a0 2024-05-09T00:17:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15bbfff700 1 -- 172.21.15.46:0/3882288957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f15d40b3890 con 0x56171b1916a0 2024-05-09T00:17:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15f1866b80 10 client.16281 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15f1866b80 1 -- 172.21.15.46:0/3882288957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56171b1a88a0 con 0x56171b1916a0 2024-05-09T00:17:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15bbfff700 1 -- 172.21.15.46:0/3882288957 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d40aee70 con 0x56171b1916a0 2024-05-09T00:17:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15bbfff700 1 -- 172.21.15.46:0/3882288957 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f15d40cbce0 con 0x56171b1916a0 2024-05-09T00:17:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15f1866b80 10 client.16281 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15f1866b80 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:17:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15f1866b80 20 client.16281 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56171b1aad70 0x56171b1ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15f1866b80 4 client.16281 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:17:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15f1866b80 1 -- 172.21.15.46:0/3882288957 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56171b0c19d0 con 0x56171b1aad70 2024-05-09T00:17:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.632+0000 7f15e0bec700 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56171b1aad70 0x56171b1ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.633+0000 7f15e0bec700 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56171b1aad70 0x56171b1ad190 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab6c6a6887ccf8bc server_cookie=393a1c2247328779 in_seq=0 out_seq=0 2024-05-09T00:17:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.633+0000 7f15bbfff700 10 client.16281 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15bbfff700 1 -- 172.21.15.46:0/3882288957 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56171b0c19d0 con 0x56171b1aad70 2024-05-09T00:17:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15bbfff700 10 client.16281 handle_command_reply: tid=0 2024-05-09T00:17:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15f1866b80 1 client.16281 shutdown 2024-05-09T00:17:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15f1866b80 1 -- 172.21.15.46:0/3882288957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 msgr2=0x56171b19b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b19b080 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f15d400b420 tx=0x7f15d40a6af0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15f1866b80 1 -- 172.21.15.46:0/3882288957 shutdown_connections 2024-05-09T00:17:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56171b1aad70 0x56171b1ad190 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b19b080 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0144a0 0x56171b1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3882288957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0fe160 0x56171b1a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15f1866b80 1 -- 172.21.15.46:0/3882288957 >> 172.21.15.46:0/3882288957 conn(0x56171b0746a0 msgr2=0x56171b18c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15bbfff700 0 client.16281 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15f1866b80 1 -- 172.21.15.46:0/3882288957 shutdown_connections 2024-05-09T00:17:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15f1866b80 1 -- 172.21.15.46:0/3882288957 wait complete. 2024-05-09T00:17:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.634+0000 7f15f1866b80 20 client.16281 trim_cache size 0 max 16384 2024-05-09T00:17:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.649+0000 7f15f1866b80 1 Processor -- start 2024-05-09T00:17:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15f1866b80 1 -- start start 2024-05-09T00:17:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15f1866b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b0144a0 0x56171b19b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15f1866b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0fe160 0x56171b1b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15f1866b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b1916a0 0x56171b1b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15f1866b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1b14a0 con 0x56171b0144a0 2024-05-09T00:17:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15f1866b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1aed30 con 0x56171b0fe160 2024-05-09T00:17:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15f1866b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1acb60 con 0x56171b1916a0 2024-05-09T00:17:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15e0bec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b0144a0 0x56171b19b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15e0bec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b0144a0 0x56171b19b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57458/0 (socket says 172.21.15.46:57458) 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15e0bec700 1 -- 172.21.15.46:0/1877168439 learned_addr learned my addr 172.21.15.46:0/1877168439 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15dbfff700 1 --2- 172.21.15.46:0/1877168439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0fe160 0x56171b1b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15e13ed700 1 --2- 172.21.15.46:0/1877168439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b1916a0 0x56171b1b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15e0bec700 1 -- 172.21.15.46:0/1877168439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b1916a0 msgr2=0x56171b1b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15e0bec700 1 --2- 172.21.15.46:0/1877168439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b1916a0 0x56171b1b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15e0bec700 1 -- 172.21.15.46:0/1877168439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0fe160 msgr2=0x56171b1b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15e0bec700 1 --2- 172.21.15.46:0/1877168439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0fe160 0x56171b1b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15e0bec700 1 -- 172.21.15.46:0/1877168439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56171b18dd00 con 0x56171b0144a0 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15e13ed700 1 --2- 172.21.15.46:0/1877168439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b1916a0 0x56171b1b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.650+0000 7f15dbfff700 1 --2- 172.21.15.46:0/1877168439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0fe160 0x56171b1b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.651+0000 7f15e0bec700 1 --2- 172.21.15.46:0/1877168439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b0144a0 0x56171b19b040 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f15cc005d90 tx=0x7f15cc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41676f1f55c445a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.651+0000 7f15c27fc700 1 -- 172.21.15.46:0/1877168439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc021ba0 con 0x56171b0144a0 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.651+0000 7f15c27fc700 1 -- 172.21.15.46:0/1877168439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15cc021d40 con 0x56171b0144a0 2024-05-09T00:17:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.651+0000 7f15c27fc700 1 -- 172.21.15.46:0/1877168439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc005500 con 0x56171b0144a0 2024-05-09T00:17:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.652+0000 7f15f1866b80 1 -- 172.21.15.46:0/1877168439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b0144a0 msgr2=0x56171b19b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.652+0000 7f15f1866b80 1 --2- 172.21.15.46:0/1877168439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b0144a0 0x56171b19b040 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f15cc005d90 tx=0x7f15cc013000 comp rx=0 tx=0).stop 2024-05-09T00:17:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.652+0000 7f15f1866b80 1 -- 172.21.15.46:0/1877168439 shutdown_connections 2024-05-09T00:17:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.652+0000 7f15f1866b80 1 --2- 172.21.15.46:0/1877168439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b1916a0 0x56171b1b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.652+0000 7f15f1866b80 1 --2- 172.21.15.46:0/1877168439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b0144a0 0x56171b19b040 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.652+0000 7f15f1866b80 1 --2- 172.21.15.46:0/1877168439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0fe160 0x56171b1b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.652+0000 7f15f1866b80 1 -- 172.21.15.46:0/1877168439 >> 172.21.15.46:0/1877168439 conn(0x56171b0746a0 msgr2=0x56171b19a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.652+0000 7f15f1866b80 1 -- 172.21.15.46:0/1877168439 shutdown_connections 2024-05-09T00:17:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.652+0000 7f15f1866b80 1 -- 172.21.15.46:0/1877168439 wait complete. 2024-05-09T00:17:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.652+0000 7f15f1866b80 1 -- start start 2024-05-09T00:17:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.652+0000 7f15f1866b80 1 Processor -- start 2024-05-09T00:17:14.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15f1866b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0144a0 0x56171b20c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15f1866b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0fe160 0x56171b20cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15f1866b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b217530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15f1866b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1ad1f0 con 0x56171b1916a0 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15f1866b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1ae660 con 0x56171b0fe160 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15f1866b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1ad8c0 con 0x56171b0144a0 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15f1866b80 10 client.0 _ll_register_callbacks cb 0x56171af256c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15e13ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b217530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15e13ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b217530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57468/0 (socket says 172.21.15.46:57468) 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15e13ed700 1 -- 172.21.15.46:0/3931747813 learned_addr learned my addr 172.21.15.46:0/3931747813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15e0bec700 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0144a0 0x56171b20c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15dbfff700 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0fe160 0x56171b20cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15e13ed700 1 -- 172.21.15.46:0/3931747813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0144a0 msgr2=0x56171b20c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:14.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15e13ed700 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0144a0 0x56171b20c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15e13ed700 1 -- 172.21.15.46:0/3931747813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0fe160 msgr2=0x56171b20cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15e13ed700 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0fe160 0x56171b20cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15e13ed700 1 -- 172.21.15.46:0/3931747813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171b1a16b0 con 0x56171b1916a0 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.653+0000 7f15dbfff700 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0fe160 0x56171b20cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.654+0000 7f15e13ed700 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b217530 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f15d40aa850 tx=0x7f15d40714d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee309fd1bbda2d84 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.654+0000 7f15bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.655+0000 7f15bbfff700 1 -- 172.21.15.46:0/3931747813 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d406f040 con 0x56171b1916a0 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.655+0000 7f15bbfff700 1 -- 172.21.15.46:0/3931747813 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f15d4005c60 con 0x56171b1916a0 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.655+0000 7f15f1866b80 1 -- 172.21.15.46:0/3931747813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15cc021d40 con 0x56171b1916a0 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.655+0000 7f15bbfff700 1 -- 172.21.15.46:0/3931747813 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f15d40a6330 con 0x56171b1916a0 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.655+0000 7f15f1866b80 10 client.16287 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.655+0000 7f15f1866b80 1 -- 172.21.15.46:0/3931747813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56171b1a16b0 con 0x56171b1916a0 2024-05-09T00:17:14.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.656+0000 7f15bbfff700 1 -- 172.21.15.46:0/3931747813 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15cc021d40 con 0x56171b1916a0 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.656+0000 7f15bbfff700 1 -- 172.21.15.46:0/3931747813 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f15d4011070 con 0x56171b1916a0 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.656+0000 7f15f1866b80 10 client.16287 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.656+0000 7f15f1866b80 10 client.16287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.656+0000 7f15f1866b80 20 client.16287 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.656+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56171b218c80 0x56171b21b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.656+0000 7f15f1866b80 4 client.16287 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.656+0000 7f15f1866b80 1 -- 172.21.15.46:0/3931747813 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56171b0bba70 con 0x56171b218c80 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.656+0000 7f15e0bec700 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56171b218c80 0x56171b21b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.656+0000 7f15bbfff700 10 client.16287 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15e0bec700 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56171b218c80 0x56171b21b0a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f1ee8f2b025cbf2 server_cookie=65c7e78ab7467e2e in_seq=0 out_seq=0 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15bbfff700 1 -- 172.21.15.46:0/3931747813 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56171b0bba70 con 0x56171b218c80 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15bbfff700 10 client.16287 handle_command_reply: tid=0 2024-05-09T00:17:14.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15f1866b80 1 client.16287 shutdown 2024-05-09T00:17:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15f1866b80 1 -- 172.21.15.46:0/3931747813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 msgr2=0x56171b217530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b217530 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f15d40aa850 tx=0x7f15d40714d0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15f1866b80 1 -- 172.21.15.46:0/3931747813 shutdown_connections 2024-05-09T00:17:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56171b218c80 0x56171b21b0a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56171b1916a0 0x56171b217530 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56171b0144a0 0x56171b20c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15f1866b80 1 --2- 172.21.15.46:0/3931747813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171b0fe160 0x56171b20cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15f1866b80 1 -- 172.21.15.46:0/3931747813 >> 172.21.15.46:0/3931747813 conn(0x56171b0746a0 msgr2=0x56171b196160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15bbfff700 0 client.16287 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15f1866b80 1 -- 172.21.15.46:0/3931747813 shutdown_connections 2024-05-09T00:17:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15f1866b80 1 -- 172.21.15.46:0/3931747813 wait complete. 2024-05-09T00:17:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.657+0000 7f15f1866b80 20 client.16287 trim_cache size 0 max 16384 2024-05-09T00:17:14.661 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:17:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.658+0000 7f15e2e50700 1 -- 172.21.15.46:0/1092025255 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f15c40b2880 msgr2=0x7f15c40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.658+0000 7f15e2e50700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f15c40b2880 0x7f15c40b4d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f15cc005950 tx=0x7f15cc00b410 comp rx=0 tx=0).stop 2024-05-09T00:17:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.658+0000 7f15e2e50700 1 -- 172.21.15.46:0/1092025255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15dc108760 msgr2=0x7f15dc19f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.658+0000 7f15e2e50700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15dc108760 0x7f15dc19f5c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f15d40095c0 tx=0x7f15d4007a20 comp rx=0 tx=0).stop 2024-05-09T00:17:14.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.658+0000 7f15e2e50700 1 -- 172.21.15.46:0/1092025255 shutdown_connections 2024-05-09T00:17:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.658+0000 7f15e2e50700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f15c40b2880 0x7f15c40b4d90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.658+0000 7f15e2e50700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15dc108760 0x7f15dc19f5c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.658+0000 7f15e2e50700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15dc1034e0 0x7f15dc19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.658+0000 7f15e2e50700 1 --2- 172.21.15.46:0/1092025255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15dc107e10 0x7f15dc19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.658+0000 7f15e2e50700 1 -- 172.21.15.46:0/1092025255 >> 172.21.15.46:0/1092025255 conn(0x7f15dc0fef30 msgr2=0x7f15dc06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.658+0000 7f15e2e50700 1 -- 172.21.15.46:0/1092025255 shutdown_connections 2024-05-09T00:17:14.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:14.658+0000 7f15e2e50700 1 -- 172.21.15.46:0/1092025255 wait complete. 2024-05-09T00:17:14.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:14 smithi142 ceph-mon[34533]: pgmap v552: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:14.991 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:14 smithi046 ceph-mon[30151]: pgmap v552: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:15.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:14 smithi169 ceph-mon[34777]: pgmap v552: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:16.421 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:17:16.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:17:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:17:16.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:16 smithi046 ceph-mon[30151]: pgmap v553: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:16.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:16 smithi142 ceph-mon[34533]: pgmap v553: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:17.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:16 smithi169 ceph-mon[34777]: pgmap v553: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.395+0000 7fcd75bca700 1 -- 172.21.15.46:0/3802029364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd701034c0 msgr2=0x7fcd701038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.395+0000 7fcd75bca700 1 --2- 172.21.15.46:0/3802029364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd701034c0 0x7fcd701038d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000b0d0 tx=0x7fcd6000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.395+0000 7fcd75bca700 1 -- 172.21.15.46:0/3802029364 shutdown_connections 2024-05-09T00:17:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.395+0000 7fcd75bca700 1 --2- 172.21.15.46:0/3802029364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd70108740 0x7fcd701100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.395+0000 7fcd75bca700 1 --2- 172.21.15.46:0/3802029364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd701034c0 0x7fcd701038d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.395+0000 7fcd75bca700 1 --2- 172.21.15.46:0/3802029364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd70107df0 0x7fcd70108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.395+0000 7fcd75bca700 1 -- 172.21.15.46:0/3802029364 >> 172.21.15.46:0/3802029364 conn(0x7fcd700fedf0 msgr2=0x7fcd70101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.395+0000 7fcd75bca700 1 -- 172.21.15.46:0/3802029364 shutdown_connections 2024-05-09T00:17:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.395+0000 7fcd75bca700 1 -- 172.21.15.46:0/3802029364 wait complete. 2024-05-09T00:17:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.396+0000 7fcd75bca700 1 Processor -- start 2024-05-09T00:17:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.396+0000 7fcd75bca700 1 -- start start 2024-05-09T00:17:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd75bca700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd701034c0 0x7fcd7019f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd75bca700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd70107df0 0x7fcd7019fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd75bca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd70108740 0x7fcd70199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd75bca700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70113370 con 0x7fcd70108740 2024-05-09T00:17:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd75bca700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701131f0 con 0x7fcd70107df0 2024-05-09T00:17:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd75bca700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701134f0 con 0x7fcd701034c0 2024-05-09T00:17:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd6ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd70108740 0x7fcd70199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd6ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd70108740 0x7fcd70199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50016/0 (socket says 172.21.15.46:50016) 2024-05-09T00:17:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd6ffff700 1 -- 172.21.15.46:0/3200850388 learned_addr learned my addr 172.21.15.46:0/3200850388 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd6effd700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd70107df0 0x7fcd7019fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd6f7fe700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd701034c0 0x7fcd7019f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd6ffff700 1 -- 172.21.15.46:0/3200850388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd701034c0 msgr2=0x7fcd7019f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd701034c0 0x7fcd7019f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd6ffff700 1 -- 172.21.15.46:0/3200850388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd70107df0 msgr2=0x7fcd7019fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd70107df0 0x7fcd7019fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.397+0000 7fcd6ffff700 1 -- 172.21.15.46:0/3200850388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7019a1d0 con 0x7fcd70108740 2024-05-09T00:17:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.398+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd70108740 0x7fcd70199b30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800c900 tx=0x7fcd6800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:18.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.398+0000 7fcd6cff9700 1 -- 172.21.15.46:0/3200850388 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680078d0 con 0x7fcd70108740 2024-05-09T00:17:18.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.398+0000 7fcd75bca700 1 -- 172.21.15.46:0/3200850388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7019a4c0 con 0x7fcd70108740 2024-05-09T00:17:18.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.398+0000 7fcd6cff9700 1 -- 172.21.15.46:0/3200850388 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6800f450 con 0x7fcd70108740 2024-05-09T00:17:18.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.398+0000 7fcd6cff9700 1 -- 172.21.15.46:0/3200850388 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68018620 con 0x7fcd70108740 2024-05-09T00:17:18.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.398+0000 7fcd75bca700 1 -- 172.21.15.46:0/3200850388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd701a6750 con 0x7fcd70108740 2024-05-09T00:17:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.401+0000 7fcd6cff9700 1 -- 172.21.15.46:0/3200850388 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcd680213f0 con 0x7fcd70108740 2024-05-09T00:17:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.401+0000 7fcd6cff9700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcd5c0b2880 0x7fcd5c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.401+0000 7fcd845e0b80 1 Processor -- start 2024-05-09T00:17:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.401+0000 7fcd6f7fe700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcd5c0b2880 0x7fcd5c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd6cff9700 1 -- 172.21.15.46:0/3200850388 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fcd680e1ff0 con 0x7fcd70108740 2024-05-09T00:17:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd845e0b80 1 -- start start 2024-05-09T00:17:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd845e0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178bf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd845e0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773a520 0x55f6178bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd845e0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd6f7fe700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcd5c0b2880 0x7fcd5c0b4d90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009900 tx=0x7fcd6000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd845e0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f6177da120 con 0x55f6178b9690 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd845e0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f61777fa20 con 0x55f6177b4d80 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd845e0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f6177d60e0 con 0x55f61773a520 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd6ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd6ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50026/0 (socket says 172.21.15.46:50026) 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd6ffff700 1 -- 172.21.15.46:0/4172889969 learned_addr learned my addr 172.21.15.46:0/4172889969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd6effd700 1 --2- 172.21.15.46:0/4172889969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773a520 0x55f6178bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.402+0000 7fcd6f7fe700 1 --2- 172.21.15.46:0/4172889969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.403+0000 7fcd6ffff700 1 -- 172.21.15.46:0/4172889969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773a520 msgr2=0x55f6178bfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.403+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/4172889969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773a520 0x55f6178bfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.403+0000 7fcd6ffff700 1 -- 172.21.15.46:0/4172889969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 msgr2=0x55f6178bf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.403+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/4172889969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178bf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.403+0000 7fcd6ffff700 1 -- 172.21.15.46:0/4172889969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f617770580 con 0x55f6178b9690 2024-05-09T00:17:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.403+0000 7fcd6f7fe700 1 --2- 172.21.15.46:0/4172889969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178bf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.403+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/4172889969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178b9aa0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800cfd0 tx=0x7fcd680a9130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980c95bde445ea6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.403+0000 7fcd65ffb700 1 -- 172.21.15.46:0/4172889969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680a96b0 con 0x55f6178b9690 2024-05-09T00:17:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.403+0000 7fcd65ffb700 1 -- 172.21.15.46:0/4172889969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd680ada60 con 0x55f6178b9690 2024-05-09T00:17:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.403+0000 7fcd65ffb700 1 -- 172.21.15.46:0/4172889969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680a9910 con 0x55f6178b9690 2024-05-09T00:17:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.404+0000 7fcd845e0b80 1 -- 172.21.15.46:0/4172889969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 msgr2=0x55f6178b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.404+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/4172889969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178b9aa0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800cfd0 tx=0x7fcd680a9130 comp rx=0 tx=0).stop 2024-05-09T00:17:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.404+0000 7fcd845e0b80 1 -- 172.21.15.46:0/4172889969 shutdown_connections 2024-05-09T00:17:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.404+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/4172889969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178b9aa0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.404+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/4172889969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178bf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.404+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/4172889969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773a520 0x55f6178bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.404+0000 7fcd845e0b80 1 -- 172.21.15.46:0/4172889969 >> 172.21.15.46:0/4172889969 conn(0x55f61773dff0 msgr2=0x55f6178b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.404+0000 7fcd845e0b80 1 -- 172.21.15.46:0/4172889969 shutdown_connections 2024-05-09T00:17:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.404+0000 7fcd845e0b80 1 -- 172.21.15.46:0/4172889969 wait complete. 2024-05-09T00:17:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.404+0000 7fcd845e0b80 1 -- start start 2024-05-09T00:17:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd845e0b80 1 Processor -- start 2024-05-09T00:17:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd845e0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f61773a520 0x55f6178c3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd845e0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd845e0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6178b9690 0x55f6178c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd845e0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f6178ba3e0 con 0x55f61773a520 2024-05-09T00:17:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd845e0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f617774ba0 con 0x55f6177b4d80 2024-05-09T00:17:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd845e0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f6178c62f0 con 0x55f6178b9690 2024-05-09T00:17:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd845e0b80 10 client.0 _ll_register_callbacks cb 0x55f61764b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd6f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f61773a520 0x55f6178c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd6f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f61773a520 0x55f6178c3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50034/0 (socket says 172.21.15.46:50034) 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd6f7fe700 1 -- 172.21.15.46:0/3040569729 learned_addr learned my addr 172.21.15.46:0/3040569729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6178b9690 0x55f6178c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd6effd700 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd6f7fe700 1 -- 172.21.15.46:0/3040569729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6178b9690 msgr2=0x55f6178c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd6f7fe700 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6178b9690 0x55f6178c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd6f7fe700 1 -- 172.21.15.46:0/3040569729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 msgr2=0x55f6178c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.405+0000 7fcd6f7fe700 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.406+0000 7fcd6f7fe700 1 -- 172.21.15.46:0/3040569729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f617655510 con 0x55f61773a520 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.406+0000 7fcd6effd700 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.406+0000 7fcd6f7fe700 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f61773a520 0x55f6178c3320 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fcd60015000 tx=0x7fcd600099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15684e0a90164625 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.406+0000 7fcd4effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.406+0000 7fcd4effd700 1 -- 172.21.15.46:0/3040569729 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6002e070 con 0x55f61773a520 2024-05-09T00:17:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.406+0000 7fcd4effd700 1 -- 172.21.15.46:0/3040569729 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fcd600075c0 con 0x55f61773a520 2024-05-09T00:17:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.406+0000 7fcd845e0b80 1 -- 172.21.15.46:0/3040569729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd680ada60 con 0x55f61773a520 2024-05-09T00:17:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.406+0000 7fcd4effd700 1 -- 172.21.15.46:0/3040569729 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcd6003a050 con 0x55f61773a520 2024-05-09T00:17:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.406+0000 7fcd845e0b80 10 client.16299 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.406+0000 7fcd845e0b80 1 -- 172.21.15.46:0/3040569729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f6178d0d00 con 0x55f61773a520 2024-05-09T00:17:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.406+0000 7fcd4effd700 1 -- 172.21.15.46:0/3040569729 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd680ada60 con 0x55f61773a520 2024-05-09T00:17:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.407+0000 7fcd4effd700 1 -- 172.21.15.46:0/3040569729 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcd60011070 con 0x55f61773a520 2024-05-09T00:17:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.407+0000 7fcd845e0b80 10 client.16299 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.407+0000 7fcd845e0b80 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:17:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.407+0000 7fcd845e0b80 20 client.16299 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.407+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f6178d3330 0x55f6178d5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.407+0000 7fcd845e0b80 4 client.16299 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:17:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.407+0000 7fcd845e0b80 1 -- 172.21.15.46:0/3040569729 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6177e7b00 con 0x55f6178d3330 2024-05-09T00:17:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.407+0000 7fcd6effd700 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f6178d3330 0x55f6178d5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.407+0000 7fcd4effd700 10 client.16299 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.407+0000 7fcd6effd700 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f6178d3330 0x55f6178d5770 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7c758aaa233eda server_cookie=8b8fd6d2f28531b in_seq=0 out_seq=0 2024-05-09T00:17:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.408+0000 7fcd4effd700 1 -- 172.21.15.46:0/3040569729 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f6177e7b00 con 0x55f6178d3330 2024-05-09T00:17:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.408+0000 7fcd4effd700 10 client.16299 handle_command_reply: tid=0 2024-05-09T00:17:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.408+0000 7fcd845e0b80 1 client.16299 shutdown 2024-05-09T00:17:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.408+0000 7fcd845e0b80 1 -- 172.21.15.46:0/3040569729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f61773a520 msgr2=0x55f6178c3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.408+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f61773a520 0x55f6178c3320 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fcd60015000 tx=0x7fcd600099e0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.408+0000 7fcd845e0b80 1 -- 172.21.15.46:0/3040569729 shutdown_connections 2024-05-09T00:17:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.408+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f6178d3330 0x55f6178d5770 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.408+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f6178b9690 0x55f6178c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.408+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f61773a520 0x55f6178c3320 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.409+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/3040569729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.409+0000 7fcd845e0b80 1 -- 172.21.15.46:0/3040569729 >> 172.21.15.46:0/3040569729 conn(0x55f61773dff0 msgr2=0x55f6178ac070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.409+0000 7fcd4effd700 0 client.16299 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.409+0000 7fcd845e0b80 1 -- 172.21.15.46:0/3040569729 shutdown_connections 2024-05-09T00:17:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.409+0000 7fcd845e0b80 1 -- 172.21.15.46:0/3040569729 wait complete. 2024-05-09T00:17:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.409+0000 7fcd845e0b80 20 client.16299 trim_cache size 0 max 16384 2024-05-09T00:17:18.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd845e0b80 1 Processor -- start 2024-05-09T00:17:18.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd845e0b80 1 -- start start 2024-05-09T00:17:18.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd845e0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773dff0 0x55f6178b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd845e0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd845e0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178c5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd845e0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701af230 con 0x55f6178b9690 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd845e0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701acac0 con 0x55f6177b4d80 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd845e0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701aa8f0 con 0x55f61773dff0 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd6ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd6ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50042/0 (socket says 172.21.15.46:50042) 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd6ffff700 1 -- 172.21.15.46:0/389675178 learned_addr learned my addr 172.21.15.46:0/389675178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd6f7fe700 1 --2- 172.21.15.46:0/389675178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773dff0 0x55f6178b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd6effd700 1 --2- 172.21.15.46:0/389675178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd6ffff700 1 -- 172.21.15.46:0/389675178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773dff0 msgr2=0x55f6178b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/389675178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773dff0 0x55f6178b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd6ffff700 1 -- 172.21.15.46:0/389675178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 msgr2=0x55f6178c3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/389675178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd6ffff700 1 -- 172.21.15.46:0/389675178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6178b3cb0 con 0x55f6178b9690 2024-05-09T00:17:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.430+0000 7fcd6effd700 1 --2- 172.21.15.46:0/389675178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.431+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/389675178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178c5680 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800fd20 tx=0x7fcd680cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bf8cc4966b0034 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.431+0000 7fcd657fa700 1 -- 172.21.15.46:0/389675178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680b2510 con 0x55f6178b9690 2024-05-09T00:17:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.431+0000 7fcd657fa700 1 -- 172.21.15.46:0/389675178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6806b070 con 0x55f6178b9690 2024-05-09T00:17:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.431+0000 7fcd657fa700 1 -- 172.21.15.46:0/389675178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680a9910 con 0x55f6178b9690 2024-05-09T00:17:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.432+0000 7fcd845e0b80 1 -- 172.21.15.46:0/389675178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 msgr2=0x55f6178c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.432+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/389675178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178c5680 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800fd20 tx=0x7fcd680cc040 comp rx=0 tx=0).stop 2024-05-09T00:17:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.432+0000 7fcd845e0b80 1 -- 172.21.15.46:0/389675178 shutdown_connections 2024-05-09T00:17:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.432+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/389675178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178c5680 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.432+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/389675178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773dff0 0x55f6178b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.432+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/389675178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.432+0000 7fcd845e0b80 1 -- 172.21.15.46:0/389675178 >> 172.21.15.46:0/389675178 conn(0x55f6178d5b00 msgr2=0x55f6178ac6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.432+0000 7fcd845e0b80 1 -- 172.21.15.46:0/389675178 shutdown_connections 2024-05-09T00:17:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.432+0000 7fcd845e0b80 1 -- 172.21.15.46:0/389675178 wait complete. 2024-05-09T00:17:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.432+0000 7fcd845e0b80 1 -- start start 2024-05-09T00:17:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.432+0000 7fcd845e0b80 1 Processor -- start 2024-05-09T00:17:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd845e0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773dff0 0x55f6178cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd845e0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd845e0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd845e0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701aaf80 con 0x55f6178b9690 2024-05-09T00:17:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd845e0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701ac3f0 con 0x55f6177b4d80 2024-05-09T00:17:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd845e0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701ab650 con 0x55f61773dff0 2024-05-09T00:17:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd845e0b80 10 client.0 _ll_register_callbacks cb 0x55f61764b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6f7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773dff0 0x55f6178cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50054/0 (socket says 172.21.15.46:50054) 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6f7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773dff0 0x55f6178cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:41292/0 (socket says 172.21.15.46:41292) 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6ffff700 1 -- 172.21.15.46:0/1387425438 learned_addr learned my addr 172.21.15.46:0/1387425438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6effd700 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6ffff700 1 -- 172.21.15.46:0/1387425438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773dff0 msgr2=0x55f6178cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773dff0 0x55f6178cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6ffff700 1 -- 172.21.15.46:0/1387425438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 msgr2=0x55f6178c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6ffff700 1 -- 172.21.15.46:0/1387425438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6178c9210 con 0x55f6178b9690 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6effd700 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6f7fe700 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773dff0 0x55f6178cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:17:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd6ffff700 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178c8a40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fcd680a6920 tx=0x7fcd680cc120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1361bff59f7c72ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.433+0000 7fcd4effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd4effd700 1 -- 172.21.15.46:0/1387425438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680912c0 con 0x55f6178b9690 2024-05-09T00:17:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd4effd700 1 -- 172.21.15.46:0/1387425438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fcd680a9910 con 0x55f6178b9690 2024-05-09T00:17:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd845e0b80 1 -- 172.21.15.46:0/1387425438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6806b070 con 0x55f6178b9690 2024-05-09T00:17:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd4effd700 1 -- 172.21.15.46:0/1387425438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcd6808a050 con 0x55f6178b9690 2024-05-09T00:17:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd845e0b80 10 client.16305 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd845e0b80 1 -- 172.21.15.46:0/1387425438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f61793eb90 con 0x55f6178b9690 2024-05-09T00:17:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd4effd700 1 -- 172.21.15.46:0/1387425438 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6806b070 con 0x55f6178b9690 2024-05-09T00:17:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd4effd700 1 -- 172.21.15.46:0/1387425438 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcd680beb00 con 0x55f6178b9690 2024-05-09T00:17:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd845e0b80 10 client.16305 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:18.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd845e0b80 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:17:18.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd845e0b80 20 client.16305 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:18.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f61793ed60 0x55f617941180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:18.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd845e0b80 4 client.16305 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:17:18.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.434+0000 7fcd845e0b80 1 -- 172.21.15.46:0/1387425438 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f6177e1ba0 con 0x55f61793ed60 2024-05-09T00:17:18.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.435+0000 7fcd6f7fe700 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f61793ed60 0x55f617941180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:18.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.435+0000 7fcd4effd700 10 client.16305 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:18.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.435+0000 7fcd6f7fe700 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f61793ed60 0x55f617941180 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a60c377240f6d5 server_cookie=6ef22621fb8425e0 in_seq=0 out_seq=0 2024-05-09T00:17:18.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.436+0000 7fcd4effd700 1 -- 172.21.15.46:0/1387425438 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f6177e1ba0 con 0x55f61793ed60 2024-05-09T00:17:18.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.436+0000 7fcd4effd700 10 client.16305 handle_command_reply: tid=0 2024-05-09T00:17:18.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.436+0000 7fcd845e0b80 1 client.16305 shutdown 2024-05-09T00:17:18.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.436+0000 7fcd845e0b80 1 -- 172.21.15.46:0/1387425438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 msgr2=0x55f6178c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.436+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178c8a40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fcd680a6920 tx=0x7fcd680cc120 comp rx=0 tx=0).stop 2024-05-09T00:17:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.436+0000 7fcd845e0b80 1 -- 172.21.15.46:0/1387425438 shutdown_connections 2024-05-09T00:17:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.436+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f61793ed60 0x55f617941180 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd4effd700 0 client.16305 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6178b9690 0x55f6178c8a40 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f61773dff0 0x55f6178cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd845e0b80 1 --2- 172.21.15.46:0/1387425438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6177b4d80 0x55f6178c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd845e0b80 1 -- 172.21.15.46:0/1387425438 >> 172.21.15.46:0/1387425438 conn(0x55f6178d5b00 msgr2=0x55f6178bcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd845e0b80 1 -- 172.21.15.46:0/1387425438 shutdown_connections 2024-05-09T00:17:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd845e0b80 1 -- 172.21.15.46:0/1387425438 wait complete. 2024-05-09T00:17:18.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd845e0b80 20 client.16305 trim_cache size 0 max 16384 2024-05-09T00:17:18.444 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:17:18.444 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:17:18.444 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "bca115e7-0297-4a9e-9c22-ed55d13e6d6e", 2024-05-09T00:17:18.444 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:17:18.444 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:17:18.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd75bca700 1 -- 172.21.15.46:0/3200850388 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcd5c0b2880 msgr2=0x7fcd5c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:18.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd75bca700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcd5c0b2880 0x7fcd5c0b4d90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009900 tx=0x7fcd6000b040 comp rx=0 tx=0).stop 2024-05-09T00:17:18.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd75bca700 1 -- 172.21.15.46:0/3200850388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd70108740 msgr2=0x7fcd70199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:18.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd75bca700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd70108740 0x7fcd70199b30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800c900 tx=0x7fcd6800cc10 comp rx=0 tx=0).stop 2024-05-09T00:17:18.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd75bca700 1 -- 172.21.15.46:0/3200850388 shutdown_connections 2024-05-09T00:17:18.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd75bca700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcd5c0b2880 0x7fcd5c0b4d90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd75bca700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd70108740 0x7fcd70199b30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd75bca700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd701034c0 0x7fcd7019f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd75bca700 1 --2- 172.21.15.46:0/3200850388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd70107df0 0x7fcd7019fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.437+0000 7fcd75bca700 1 -- 172.21.15.46:0/3200850388 >> 172.21.15.46:0/3200850388 conn(0x7fcd700fedf0 msgr2=0x7fcd701096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.438+0000 7fcd75bca700 1 -- 172.21.15.46:0/3200850388 shutdown_connections 2024-05-09T00:17:18.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:18.438+0000 7fcd75bca700 1 -- 172.21.15.46:0/3200850388 wait complete. 2024-05-09T00:17:18.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:18 smithi142 ceph-mon[34533]: pgmap v554: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:19.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:18 smithi046 ceph-mon[30151]: pgmap v554: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:19.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:18 smithi169 ceph-mon[34777]: pgmap v554: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:19.121 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:17:19.377 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:17:19.775 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:19 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:17:19.775 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:19 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:17:19.775 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:19 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:17:19.775 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:19 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:17:19.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:19 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:17:19.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:19 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:17:19.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:19 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:17:19.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:19 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:17:20.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:19 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:17:20.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:19 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:17:20.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:19 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:17:20.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:19 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:17:20.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:20 smithi046 ceph-mon[30151]: pgmap v555: 97 pgs: 97 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:20.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:20 smithi142 ceph-mon[34533]: pgmap v555: 97 pgs: 97 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:21.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:20 smithi169 ceph-mon[34777]: pgmap v555: 97 pgs: 97 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:21.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.138+0000 7fbd22279700 1 -- 172.21.15.46:0/45251828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd1c1034c0 msgr2=0x7fbd1c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:21.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.138+0000 7fbd22279700 1 --2- 172.21.15.46:0/45251828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd1c1034c0 0x7fbd1c1038d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0099c0 tx=0x7fbd0c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.138+0000 7fbd22279700 1 -- 172.21.15.46:0/45251828 shutdown_connections 2024-05-09T00:17:21.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.138+0000 7fbd22279700 1 --2- 172.21.15.46:0/45251828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c108740 0x7fbd1c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.138+0000 7fbd22279700 1 --2- 172.21.15.46:0/45251828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd1c1034c0 0x7fbd1c1038d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.138+0000 7fbd22279700 1 --2- 172.21.15.46:0/45251828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd1c107df0 0x7fbd1c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.138+0000 7fbd22279700 1 -- 172.21.15.46:0/45251828 >> 172.21.15.46:0/45251828 conn(0x7fbd1c0fedf0 msgr2=0x7fbd1c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.138+0000 7fbd22279700 1 -- 172.21.15.46:0/45251828 shutdown_connections 2024-05-09T00:17:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.138+0000 7fbd22279700 1 -- 172.21.15.46:0/45251828 wait complete. 2024-05-09T00:17:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.139+0000 7fbd22279700 1 Processor -- start 2024-05-09T00:17:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.139+0000 7fbd22279700 1 -- start start 2024-05-09T00:17:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.140+0000 7fbd22279700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd1c1034c0 0x7fbd1c112440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.140+0000 7fbd22279700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd1c107df0 0x7fbd1c1108a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.140+0000 7fbd22279700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c108740 0x7fbd1c110ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.140+0000 7fbd22279700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1144c0 con 0x7fbd1c1034c0 2024-05-09T00:17:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.140+0000 7fbd22279700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c114340 con 0x7fbd1c108740 2024-05-09T00:17:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.140+0000 7fbd22279700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c114640 con 0x7fbd1c107df0 2024-05-09T00:17:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.140+0000 7fbd13fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd1c107df0 0x7fbd1c1108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.140+0000 7fbd1bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd1c1034c0 0x7fbd1c112440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.140+0000 7fbd1bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd1c1034c0 0x7fbd1c112440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50070/0 (socket says 172.21.15.46:50070) 2024-05-09T00:17:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.140+0000 7fbd1bfff700 1 -- 172.21.15.46:0/2738914239 learned_addr learned my addr 172.21.15.46:0/2738914239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.140+0000 7fbd20816700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c108740 0x7fbd1c110ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.141+0000 7fbd1bfff700 1 -- 172.21.15.46:0/2738914239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd1c107df0 msgr2=0x7fbd1c1108a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.141+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd1c107df0 0x7fbd1c1108a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.141+0000 7fbd1bfff700 1 -- 172.21.15.46:0/2738914239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c108740 msgr2=0x7fbd1c110ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.141+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c108740 0x7fbd1c110ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.141+0000 7fbd1bfff700 1 -- 172.21.15.46:0/2738914239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c1ad780 con 0x7fbd1c1034c0 2024-05-09T00:17:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.141+0000 7fbd13fff700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd1c107df0 0x7fbd1c1108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:17:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.141+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd1c1034c0 0x7fbd1c112440 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b580 tx=0x7fbd0c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.141+0000 7fbd19ffb700 1 -- 172.21.15.46:0/2738914239 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c027070 con 0x7fbd1c1034c0 2024-05-09T00:17:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.141+0000 7fbd22279700 1 -- 172.21.15.46:0/2738914239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1c1ad950 con 0x7fbd1c1034c0 2024-05-09T00:17:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.141+0000 7fbd19ffb700 1 -- 172.21.15.46:0/2738914239 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0c003f20 con 0x7fbd1c1034c0 2024-05-09T00:17:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.141+0000 7fbd19ffb700 1 -- 172.21.15.46:0/2738914239 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c022d70 con 0x7fbd1c1034c0 2024-05-09T00:17:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.141+0000 7fbd22279700 1 -- 172.21.15.46:0/2738914239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1c1adcf0 con 0x7fbd1c1034c0 2024-05-09T00:17:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.144+0000 7fbd19ffb700 1 -- 172.21.15.46:0/2738914239 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbd0c022500 con 0x7fbd1c1034c0 2024-05-09T00:17:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.145+0000 7fbd19ffb700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbd080b2880 0x7fbd080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.145+0000 7fbd13fff700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbd080b2880 0x7fbd080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.145+0000 7fbd19ffb700 1 -- 172.21.15.46:0/2738914239 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbd0c0e2ef0 con 0x7fbd1c1034c0 2024-05-09T00:17:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.145+0000 7fbd13fff700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbd080b2880 0x7fbd080b4d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c112060 tx=0x7fbd0400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd30c8fb80 1 Processor -- start 2024-05-09T00:17:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd30c8fb80 1 -- start start 2024-05-09T00:17:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd30c8fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babb74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd30c8fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 0x5646babb7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd30c8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646babb16a0 0x5646babb1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd30c8fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5646baad3ff0 con 0x5646baa344a0 2024-05-09T00:17:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd30c8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5646baa727f0 con 0x5646babb16a0 2024-05-09T00:17:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd30c8fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5646baacffb0 con 0x5646bab1e160 2024-05-09T00:17:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 0x5646babb7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 0x5646babb7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50080/0 (socket says 172.21.15.46:50080) 2024-05-09T00:17:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd13fff700 1 -- 172.21.15.46:0/234940921 learned_addr learned my addr 172.21.15.46:0/234940921 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/234940921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babb74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.146+0000 7fbd20816700 1 --2- 172.21.15.46:0/234940921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646babb16a0 0x5646babb1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.147+0000 7fbd1bfff700 1 -- 172.21.15.46:0/234940921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646babb16a0 msgr2=0x5646babb1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.147+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/234940921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646babb16a0 0x5646babb1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.147+0000 7fbd1bfff700 1 -- 172.21.15.46:0/234940921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 msgr2=0x5646babb7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.147+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/234940921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 0x5646babb7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.147+0000 7fbd1bfff700 1 -- 172.21.15.46:0/234940921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646baa6a370 con 0x5646bab1e160 2024-05-09T00:17:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.147+0000 7fbd13fff700 1 --2- 172.21.15.46:0/234940921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 0x5646babb7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:17:21.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.147+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/234940921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babb74f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0a77d0 tx=0x7fbd0c009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39ad3d88d291a310 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:21.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.147+0000 7fbd127fc700 1 -- 172.21.15.46:0/234940921 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0ae910 con 0x5646bab1e160 2024-05-09T00:17:21.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.147+0000 7fbd127fc700 1 -- 172.21.15.46:0/234940921 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0c0b3330 con 0x5646bab1e160 2024-05-09T00:17:21.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.147+0000 7fbd127fc700 1 -- 172.21.15.46:0/234940921 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0a97c0 con 0x5646bab1e160 2024-05-09T00:17:21.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.148+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/234940921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 msgr2=0x5646babb74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:21.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.148+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/234940921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babb74f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0a77d0 tx=0x7fbd0c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.148+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/234940921 shutdown_connections 2024-05-09T00:17:21.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.148+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/234940921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646babb16a0 0x5646babb1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.148+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/234940921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babb74f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.148+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/234940921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 0x5646babb7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.148+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/234940921 >> 172.21.15.46:0/234940921 conn(0x5646baa946a0 msgr2=0x5646babac400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.148+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/234940921 shutdown_connections 2024-05-09T00:17:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.148+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/234940921 wait complete. 2024-05-09T00:17:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.149+0000 7fbd30c8fb80 1 -- start start 2024-05-09T00:17:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.149+0000 7fbd30c8fb80 1 Processor -- start 2024-05-09T00:17:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.149+0000 7fbd30c8fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646baa344a0 0x5646babc0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.149+0000 7fbd30c8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646bab1e160 0x5646babc11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd30c8fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646babb16a0 0x5646babbb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd30c8fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5646ba94fba0 con 0x5646babb16a0 2024-05-09T00:17:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd30c8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5646baa8e150 con 0x5646bab1e160 2024-05-09T00:17:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd30c8fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5646babb2360 con 0x5646baa344a0 2024-05-09T00:17:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd1bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646baa344a0 0x5646babc0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd1bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646baa344a0 0x5646babc0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:41322/0 (socket says 172.21.15.46:41322) 2024-05-09T00:17:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd1bfff700 1 -- 172.21.15.46:0/3972722126 learned_addr learned my addr 172.21.15.46:0/3972722126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd30c8fb80 10 client.0 _ll_register_callbacks cb 0x5646ba9456c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd13fff700 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646bab1e160 0x5646babc11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd20816700 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646babb16a0 0x5646babbb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd1bfff700 1 -- 172.21.15.46:0/3972722126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646bab1e160 msgr2=0x5646babc11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646bab1e160 0x5646babc11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd1bfff700 1 -- 172.21.15.46:0/3972722126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646babb16a0 msgr2=0x5646babbb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646babb16a0 0x5646babbb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd1bfff700 1 -- 172.21.15.46:0/3972722126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646ba94f490 con 0x5646baa344a0 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646baa344a0 0x5646babc0c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0a65e0 tx=0x7fbd0c0b3260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb9c93f33afbe10e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.150+0000 7fbcfb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.151+0000 7fbcfb7fe700 1 -- 172.21.15.46:0/3972722126 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c06e060 con 0x5646baa344a0 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.151+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3972722126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd0c0b3330 con 0x5646baa344a0 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.151+0000 7fbcfb7fe700 1 -- 172.21.15.46:0/3972722126 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbd0c0a6b50 con 0x5646baa344a0 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.151+0000 7fbcfb7fe700 1 -- 172.21.15.46:0/3972722126 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbd0c0c3030 con 0x5646baa344a0 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.151+0000 7fbd30c8fb80 10 client.25915 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.151+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3972722126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646babc8860 con 0x5646baa344a0 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.151+0000 7fbcfb7fe700 1 -- 172.21.15.46:0/3972722126 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0c0b3330 con 0x5646baa344a0 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.151+0000 7fbcfb7fe700 1 -- 172.21.15.46:0/3972722126 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbd0c0a97c0 con 0x5646baa344a0 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.151+0000 7fbd30c8fb80 10 client.25915 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.151+0000 7fbd30c8fb80 10 client.25915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:17:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.151+0000 7fbd30c8fb80 20 client.25915 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.152+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646babcaf70 0x5646babcd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.152+0000 7fbd30c8fb80 4 client.25915 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:17:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.152+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3972722126 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646baae19d0 con 0x5646babcaf70 2024-05-09T00:17:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.152+0000 7fbd13fff700 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646babcaf70 0x5646babcd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.152+0000 7fbcfb7fe700 10 client.25915 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.152+0000 7fbd13fff700 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646babcaf70 0x5646babcd400 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d5d4ac11a52d65 server_cookie=4cf2b90672d8e35 in_seq=0 out_seq=0 2024-05-09T00:17:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbcfb7fe700 1 -- 172.21.15.46:0/3972722126 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646baae19d0 con 0x5646babcaf70 2024-05-09T00:17:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbcfb7fe700 10 client.25915 handle_command_reply: tid=0 2024-05-09T00:17:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbd30c8fb80 1 client.25915 shutdown 2024-05-09T00:17:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3972722126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646baa344a0 msgr2=0x5646babc0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646baa344a0 0x5646babc0c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0a65e0 tx=0x7fbd0c0b3260 comp rx=0 tx=0).stop 2024-05-09T00:17:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3972722126 shutdown_connections 2024-05-09T00:17:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646babcaf70 0x5646babcd400 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646babb16a0 0x5646babbb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646baa344a0 0x5646babc0c60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbcfb7fe700 0 client.25915 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/3972722126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646bab1e160 0x5646babc11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3972722126 >> 172.21.15.46:0/3972722126 conn(0x5646baa946a0 msgr2=0x5646babac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:21.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3972722126 shutdown_connections 2024-05-09T00:17:21.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3972722126 wait complete. 2024-05-09T00:17:21.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.154+0000 7fbd30c8fb80 20 client.25915 trim_cache size 0 max 16384 2024-05-09T00:17:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.175+0000 7fbd30c8fb80 1 Processor -- start 2024-05-09T00:17:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.175+0000 7fbd30c8fb80 1 -- start start 2024-05-09T00:17:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.175+0000 7fbd30c8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646baa946a0 0x5646babb5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.175+0000 7fbd30c8fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 0x5646babb10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.175+0000 7fbd30c8fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babb3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.175+0000 7fbd30c8fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1b79a0 con 0x5646baa344a0 2024-05-09T00:17:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.175+0000 7fbd30c8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1b5230 con 0x5646baa946a0 2024-05-09T00:17:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.175+0000 7fbd30c8fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1b3060 con 0x5646bab1e160 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.175+0000 7fbd13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 0x5646babb10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.175+0000 7fbd13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 0x5646babb10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50102/0 (socket says 172.21.15.46:50102) 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.175+0000 7fbd13fff700 1 -- 172.21.15.46:0/2230802052 learned_addr learned my addr 172.21.15.46:0/2230802052 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.176+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/2230802052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646baa946a0 0x5646babb5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.176+0000 7fbd20816700 1 --2- 172.21.15.46:0/2230802052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babb3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.176+0000 7fbd13fff700 1 -- 172.21.15.46:0/2230802052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 msgr2=0x5646babb3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.176+0000 7fbd13fff700 1 --2- 172.21.15.46:0/2230802052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babb3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.176+0000 7fbd13fff700 1 -- 172.21.15.46:0/2230802052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646baa946a0 msgr2=0x5646babb5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.176+0000 7fbd13fff700 1 --2- 172.21.15.46:0/2230802052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646baa946a0 0x5646babb5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.176+0000 7fbd13fff700 1 -- 172.21.15.46:0/2230802052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646babadd00 con 0x5646baa344a0 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.176+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/2230802052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646baa946a0 0x5646babb5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.176+0000 7fbd13fff700 1 --2- 172.21.15.46:0/2230802052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 0x5646babb10c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400adc0 tx=0x7fbd04012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4655ee37cdbe9149 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.176+0000 7fbd11ffb700 1 -- 172.21.15.46:0/2230802052 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04016070 con 0x5646baa344a0 2024-05-09T00:17:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.176+0000 7fbd11ffb700 1 -- 172.21.15.46:0/2230802052 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0402d070 con 0x5646baa344a0 2024-05-09T00:17:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.176+0000 7fbd11ffb700 1 -- 172.21.15.46:0/2230802052 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0401cd90 con 0x5646baa344a0 2024-05-09T00:17:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.177+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/2230802052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 msgr2=0x5646babb10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.177+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/2230802052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 0x5646babb10c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400adc0 tx=0x7fbd04012000 comp rx=0 tx=0).stop 2024-05-09T00:17:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.177+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/2230802052 shutdown_connections 2024-05-09T00:17:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.177+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/2230802052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babb3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.177+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/2230802052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646baa946a0 0x5646babb5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.177+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/2230802052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa344a0 0x5646babb10c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.177+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/2230802052 >> 172.21.15.46:0/2230802052 conn(0x5646babb7cb0 msgr2=0x5646babb9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.177+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/2230802052 shutdown_connections 2024-05-09T00:17:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.177+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/2230802052 wait complete. 2024-05-09T00:17:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.177+0000 7fbd30c8fb80 1 -- start start 2024-05-09T00:17:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd30c8fb80 1 Processor -- start 2024-05-09T00:17:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd30c8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646baa344a0 0x5646babc2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd30c8fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa946a0 0x5646babc32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd30c8fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babc37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd30c8fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1b36f0 con 0x5646baa946a0 2024-05-09T00:17:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd30c8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1b4b60 con 0x5646baa344a0 2024-05-09T00:17:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd30c8fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1b3dc0 con 0x5646bab1e160 2024-05-09T00:17:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd30c8fb80 10 client.0 _ll_register_callbacks cb 0x5646ba9456c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa946a0 0x5646babc32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa946a0 0x5646babc32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50104/0 (socket says 172.21.15.46:50104) 2024-05-09T00:17:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd13fff700 1 -- 172.21.15.46:0/3648033656 learned_addr learned my addr 172.21.15.46:0/3648033656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd20816700 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babc37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.178+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646baa344a0 0x5646babc2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbd13fff700 1 -- 172.21.15.46:0/3648033656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 msgr2=0x5646babc37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbd13fff700 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babc37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbd13fff700 1 -- 172.21.15.46:0/3648033656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646baa344a0 msgr2=0x5646babc2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbd13fff700 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646baa344a0 0x5646babc2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbd13fff700 1 -- 172.21.15.46:0/3648033656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646baba5f60 con 0x5646baa946a0 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbd13fff700 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa946a0 0x5646babc32b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbd0401c680 tx=0x7fbd0401c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d179d47de5a218 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbcfb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbcfb7fe700 1 -- 172.21.15.46:0/3648033656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04037020 con 0x5646baa946a0 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbcfb7fe700 1 -- 172.21.15.46:0/3648033656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fbd040284f0 con 0x5646baa946a0 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3648033656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd0402d070 con 0x5646baa946a0 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbcfb7fe700 1 -- 172.21.15.46:0/3648033656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbd0401ce20 con 0x5646baa946a0 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbd30c8fb80 10 client.16323 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3648033656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646bac35870 con 0x5646baa946a0 2024-05-09T00:17:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.179+0000 7fbcfb7fe700 1 -- 172.21.15.46:0/3648033656 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0402d070 con 0x5646baa946a0 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.180+0000 7fbcfb7fe700 1 -- 172.21.15.46:0/3648033656 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbd04033b70 con 0x5646baa946a0 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.180+0000 7fbd30c8fb80 10 client.16323 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.180+0000 7fbd30c8fb80 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.180+0000 7fbd30c8fb80 20 client.16323 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.180+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646babb16a0 0x5646bac37fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.180+0000 7fbd30c8fb80 4 client.16323 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.180+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3648033656 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646baadba70 con 0x5646babb16a0 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.180+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646babb16a0 0x5646bac37fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.180+0000 7fbd1bfff700 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646babb16a0 0x5646bac37fb0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81bf158c49452f19 server_cookie=dccecad0a5930666 in_seq=0 out_seq=0 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.180+0000 7fbcfb7fe700 10 client.16323 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbcfb7fe700 1 -- 172.21.15.46:0/3648033656 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5646baadba70 con 0x5646babb16a0 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbcfb7fe700 10 client.16323 handle_command_reply: tid=0 2024-05-09T00:17:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbd30c8fb80 1 client.16323 shutdown 2024-05-09T00:17:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3648033656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa946a0 msgr2=0x5646babc32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa946a0 0x5646babc32b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbd0401c680 tx=0x7fbd0401c6b0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3648033656 shutdown_connections 2024-05-09T00:17:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646babb16a0 0x5646bac37fb0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bab1e160 0x5646babc37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646baa344a0 0x5646babc2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbcfb7fe700 0 client.16323 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbd30c8fb80 1 --2- 172.21.15.46:0/3648033656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646baa946a0 0x5646babc32b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3648033656 >> 172.21.15.46:0/3648033656 conn(0x5646babb7cb0 msgr2=0x5646babb4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3648033656 shutdown_connections 2024-05-09T00:17:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbd30c8fb80 1 -- 172.21.15.46:0/3648033656 wait complete. 2024-05-09T00:17:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.181+0000 7fbd30c8fb80 20 client.16323 trim_cache size 0 max 16384 2024-05-09T00:17:21.186 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:17:21.186 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:17:21.186 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T00:17:21.186 INFO:teuthology.orchestra.run.smithi046.stdout: "bca115e7-0297-4a9e-9c22-ed55d13e6d6e": { 2024-05-09T00:17:21.186 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T00:17:21.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "bca115e7-0297-4a9e-9c22-ed55d13e6d6e", 2024-05-09T00:17:21.186 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T00:17:21.186 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:17:21.186 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:17:21.187 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:17:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.182+0000 7fbd22279700 1 -- 172.21.15.46:0/2738914239 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbd080b2880 msgr2=0x7fbd080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.182+0000 7fbd22279700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbd080b2880 0x7fbd080b4d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c112060 tx=0x7fbd0400a300 comp rx=0 tx=0).stop 2024-05-09T00:17:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.182+0000 7fbd22279700 1 -- 172.21.15.46:0/2738914239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd1c1034c0 msgr2=0x7fbd1c112440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.182+0000 7fbd22279700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd1c1034c0 0x7fbd1c112440 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b580 tx=0x7fbd0c0047b0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.182+0000 7fbd22279700 1 -- 172.21.15.46:0/2738914239 shutdown_connections 2024-05-09T00:17:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.182+0000 7fbd22279700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fbd080b2880 0x7fbd080b4d90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.182+0000 7fbd22279700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd1c108740 0x7fbd1c110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.182+0000 7fbd22279700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd1c1034c0 0x7fbd1c112440 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.182+0000 7fbd22279700 1 --2- 172.21.15.46:0/2738914239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd1c107df0 0x7fbd1c1108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.182+0000 7fbd22279700 1 -- 172.21.15.46:0/2738914239 >> 172.21.15.46:0/2738914239 conn(0x7fbd1c0fedf0 msgr2=0x7fbd1c1070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.182+0000 7fbd22279700 1 -- 172.21.15.46:0/2738914239 shutdown_connections 2024-05-09T00:17:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:21.182+0000 7fbd22279700 1 -- 172.21.15.46:0/2738914239 wait complete. 2024-05-09T00:17:21.955 INFO:tasks.cephfs.filesystem:scrub status for tag:bca115e7-0297-4a9e-9c22-ed55d13e6d6e - {'path': '/', 'tag': 'bca115e7-0297-4a9e-9c22-ed55d13e6d6e', 'options': 'recursive,force'} 2024-05-09T00:17:22.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:22 smithi142 ceph-mon[34533]: pgmap v556: 97 pgs: 97 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:22.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:22 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:17:23.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:22 smithi046 ceph-mon[30151]: pgmap v556: 97 pgs: 97 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:23.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:22 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:17:23.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:22 smithi169 ceph-mon[34777]: pgmap v556: 97 pgs: 97 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:23.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:22 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:17:24.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:24 smithi142 ceph-mon[34533]: pgmap v557: 97 pgs: 97 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:25.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:24 smithi046 ceph-mon[30151]: pgmap v557: 97 pgs: 97 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:25.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:24 smithi169 ceph-mon[34777]: pgmap v557: 97 pgs: 97 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:26.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:26 smithi142 ceph-mon[34533]: pgmap v558: 97 pgs: 97 active+clean; 3.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:27.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:26 smithi046 ceph-mon[30151]: pgmap v558: 97 pgs: 97 active+clean; 3.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:27.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:26 smithi169 ceph-mon[34777]: pgmap v558: 97 pgs: 97 active+clean; 3.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:28.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:28 smithi142 ceph-mon[34533]: pgmap v559: 97 pgs: 97 active+clean; 3.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:29.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:28 smithi046 ceph-mon[30151]: pgmap v559: 97 pgs: 97 active+clean; 3.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:29.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:28 smithi169 ceph-mon[34777]: pgmap v559: 97 pgs: 97 active+clean; 3.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:30.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:30 smithi142 ceph-mon[34533]: pgmap v560: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:17:31.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:30 smithi046 ceph-mon[30151]: pgmap v560: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:17:31.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:30 smithi169 ceph-mon[34777]: pgmap v560: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:17:32.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:32 smithi142 ceph-mon[34533]: pgmap v561: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:33.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:32 smithi046 ceph-mon[30151]: pgmap v561: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:33.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:32 smithi169 ceph-mon[34777]: pgmap v561: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:34.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:34 smithi142 ceph-mon[34533]: pgmap v562: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:35.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:34 smithi046 ceph-mon[30151]: pgmap v562: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:35.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:34 smithi169 ceph-mon[34777]: pgmap v562: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:36.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:36 smithi142 ceph-mon[34533]: pgmap v563: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:17:37.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:36 smithi046 ceph-mon[30151]: pgmap v563: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:17:37.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:36 smithi169 ceph-mon[34777]: pgmap v563: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:17:38.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:38 smithi142 ceph-mon[34533]: pgmap v564: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:39.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:38 smithi046 ceph-mon[30151]: pgmap v564: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:39.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:38 smithi169 ceph-mon[34777]: pgmap v564: 97 pgs: 97 active+clean; 3.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:41.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:40 smithi046 ceph-mon[30151]: pgmap v565: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:41.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:40 smithi169 ceph-mon[34777]: pgmap v565: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:41.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:40 smithi142 ceph-mon[34533]: pgmap v565: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:43.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:42 smithi046 ceph-mon[30151]: pgmap v566: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:43.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:42 smithi169 ceph-mon[34777]: pgmap v566: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:43.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:42 smithi142 ceph-mon[34533]: pgmap v566: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:45.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:44 smithi046 ceph-mon[30151]: pgmap v567: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:45.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:44 smithi169 ceph-mon[34777]: pgmap v567: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:45.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:44 smithi142 ceph-mon[34533]: pgmap v567: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:47.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:46 smithi046 ceph-mon[30151]: pgmap v568: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:47.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:46 smithi169 ceph-mon[34777]: pgmap v568: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:47.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:46 smithi142 ceph-mon[34533]: pgmap v568: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:49.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:48 smithi046 ceph-mon[30151]: pgmap v569: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:49.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:48 smithi169 ceph-mon[34777]: pgmap v569: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:49.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:48 smithi142 ceph-mon[34533]: pgmap v569: 97 pgs: 97 active+clean; 3.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:49.377 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.377 INFO:tasks.workunit.client.0.smithi046.stdout:189/600: open fd = 3 2024-05-09T00:17:49.377 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.377 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.377 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.377 INFO:tasks.workunit.client.0.smithi046.stdout:190/600: open fd = 3 2024-05-09T00:17:49.377 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.377 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.377 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:191/600: open fd = 3 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:192/600: open fd = 3 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:193/600: open fd = 3 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:194/600: open fd = 3 2024-05-09T00:17:49.378 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.379 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.379 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.379 INFO:tasks.workunit.client.0.smithi046.stdout:195/600: open fd = 3 2024-05-09T00:17:49.379 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.379 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.379 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.379 INFO:tasks.workunit.client.0.smithi046.stdout:196/600: open fd = 3 2024-05-09T00:17:49.379 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.379 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.379 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.380 INFO:tasks.workunit.client.0.smithi046.stdout:197/600: open fd = 3 2024-05-09T00:17:49.380 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.380 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.380 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.380 INFO:tasks.workunit.client.0.smithi046.stdout:198/600: open fd = 3 2024-05-09T00:17:49.380 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.380 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.380 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.380 INFO:tasks.workunit.client.0.smithi046.stdout:199/600: open fd = 3 2024-05-09T00:17:49.380 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.380 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.381 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.381 INFO:tasks.workunit.client.0.smithi046.stdout:200/600: open fd = 3 2024-05-09T00:17:49.381 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.381 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.381 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.381 INFO:tasks.workunit.client.0.smithi046.stdout:201/600: open fd = 3 2024-05-09T00:17:49.381 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.381 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.381 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.381 INFO:tasks.workunit.client.0.smithi046.stdout:202/600: open fd = 3 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:203/600: open fd = 3 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:204/600: open fd = 3 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:205/600: open fd = 3 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.382 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.383 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.383 INFO:tasks.workunit.client.0.smithi046.stdout:206/600: open fd = 3 2024-05-09T00:17:49.383 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.383 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.383 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.383 INFO:tasks.workunit.client.0.smithi046.stdout:207/600: open fd = 3 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:208/600: open fd = 3 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:209/600: open fd = 3 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:210/600: open fd = 3 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.384 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.385 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.385 INFO:tasks.workunit.client.0.smithi046.stdout:211/600: open fd = 3 2024-05-09T00:17:49.385 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.385 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.385 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.385 INFO:tasks.workunit.client.0.smithi046.stdout:212/600: open fd = 3 2024-05-09T00:17:49.385 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.385 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.385 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:213/600: open fd = 3 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:214/600: open fd = 3 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:215/600: open fd = 3 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.386 INFO:tasks.workunit.client.0.smithi046.stdout:216/600: open fd = 3 2024-05-09T00:17:49.387 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.387 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.388 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.388 INFO:tasks.workunit.client.0.smithi046.stdout:217/600: open fd = 3 2024-05-09T00:17:49.388 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.388 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.388 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.388 INFO:tasks.workunit.client.0.smithi046.stdout:218/600: open fd = 3 2024-05-09T00:17:49.388 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.388 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.388 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.388 INFO:tasks.workunit.client.0.smithi046.stdout:219/600: open fd = 3 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:220/600: open fd = 3 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:221/600: open fd = 3 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:222/600: open fd = 3 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.389 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:223/600: open fd = 3 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:224/600: open fd = 3 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:225/600: open fd = 3 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:226/600: open fd = 3 2024-05-09T00:17:49.390 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:227/600: open fd = 3 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:228/600: open fd = 3 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:229/600: open fd = 3 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.391 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:230/600: open fd = 3 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:231/600: open fd = 3 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:232/600: open fd = 3 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:233/600: open fd = 3 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.392 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:234/600: open fd = 3 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:235/600: open fd = 3 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:236/600: open fd = 3 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:237/600: open fd = 3 2024-05-09T00:17:49.393 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:238/600: open fd = 3 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:239/600: open fd = 3 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:240/600: open fd = 3 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.394 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:241/600: open fd = 3 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:242/600: open fd = 3 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:243/600: open fd = 3 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:244/600: open fd = 3 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.395 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:245/600: open fd = 3 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:246/600: open fd = 3 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:247/600: open fd = 3 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.396 INFO:tasks.workunit.client.0.smithi046.stdout:248/600: open fd = 3 2024-05-09T00:17:49.397 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.397 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.397 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.397 INFO:tasks.workunit.client.0.smithi046.stdout:249/600: open fd = 3 2024-05-09T00:17:49.397 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.397 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.397 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:17:49.397 INFO:tasks.workunit.client.0.smithi046.stdout:250/600: open fd = 3 2024-05-09T00:17:49.397 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:49.397 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:17:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:49 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:17:50.104 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:49 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:17:50.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:49 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:17:51.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:50 smithi046 ceph-mon[30151]: pgmap v570: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:51.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:17:51.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:17:51.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:17:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:50 smithi169 ceph-mon[34777]: pgmap v570: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:17:51.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:17:51.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:17:51.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:50 smithi142 ceph-mon[34533]: pgmap v570: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:51.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T00:17:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:17:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:17:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:17:51.956 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:17:51.986 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:51 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-05-09T00:17:51.987 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:51 smithi046 ceph-mon[30151]: Adjusting osd_memory_target on smithi142 to 176.2M 2024-05-09T00:17:51.987 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:51 smithi046 ceph-mon[30151]: Unable to set osd_memory_target on smithi169 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-09T00:17:51.987 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:51 smithi046 ceph-mon[30151]: Unable to set osd_memory_target on smithi142 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-09T00:17:52.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:51 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-05-09T00:17:52.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:51 smithi169 ceph-mon[34777]: Adjusting osd_memory_target on smithi142 to 176.2M 2024-05-09T00:17:52.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:51 smithi169 ceph-mon[34777]: Unable to set osd_memory_target on smithi169 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-09T00:17:52.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:51 smithi169 ceph-mon[34777]: Unable to set osd_memory_target on smithi142 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-09T00:17:52.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:51 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-05-09T00:17:52.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:51 smithi142 ceph-mon[34533]: Adjusting osd_memory_target on smithi142 to 176.2M 2024-05-09T00:17:52.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:51 smithi142 ceph-mon[34533]: Unable to set osd_memory_target on smithi169 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-09T00:17:52.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:51 smithi142 ceph-mon[34533]: Unable to set osd_memory_target on smithi142 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-09T00:17:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:17:53.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:52 smithi046 ceph-mon[30151]: pgmap v571: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:53.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:52 smithi169 ceph-mon[34777]: pgmap v571: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:53.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:52 smithi142 ceph-mon[34533]: pgmap v571: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:17:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.056+0000 7f93fd1d4700 1 -- 172.21.15.46:0/63696797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f8109860 msgr2=0x7f93f81111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.056+0000 7f93fd1d4700 1 --2- 172.21.15.46:0/63696797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f8109860 0x7f93f81111c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f93f000b0d0 tx=0x7f93f000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.056+0000 7f93fd1d4700 1 -- 172.21.15.46:0/63696797 shutdown_connections 2024-05-09T00:17:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.056+0000 7f93fd1d4700 1 --2- 172.21.15.46:0/63696797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f8109860 0x7f93f81111c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.056+0000 7f93fd1d4700 1 --2- 172.21.15.46:0/63696797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93f81045e0 0x7f93f81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.056+0000 7f93fd1d4700 1 --2- 172.21.15.46:0/63696797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f8108f10 0x7f93f8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.056+0000 7f93fd1d4700 1 -- 172.21.15.46:0/63696797 >> 172.21.15.46:0/63696797 conn(0x7f93f80ffed0 msgr2=0x7f93f8102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.056+0000 7f93fd1d4700 1 -- 172.21.15.46:0/63696797 shutdown_connections 2024-05-09T00:17:54.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.056+0000 7f93fd1d4700 1 -- 172.21.15.46:0/63696797 wait complete. 2024-05-09T00:17:54.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.057+0000 7f93fd1d4700 1 Processor -- start 2024-05-09T00:17:54.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.057+0000 7f93fd1d4700 1 -- start start 2024-05-09T00:17:54.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.057+0000 7f93fd1d4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93f81045e0 0x7f93f819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.057+0000 7f93fd1d4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f8108f10 0x7f93f819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.057+0000 7f93fd1d4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f8109860 0x7f93f819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.057+0000 7f93fd1d4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8114490 con 0x7f93f8108f10 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.057+0000 7f93fd1d4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8114310 con 0x7f93f81045e0 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.057+0000 7f93fd1d4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8114610 con 0x7f93f8109860 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93f659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f8108f10 0x7f93f819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93f659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f8108f10 0x7f93f819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40078/0 (socket says 172.21.15.46:40078) 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93f659c700 1 -- 172.21.15.46:0/2948689019 learned_addr learned my addr 172.21.15.46:0/2948689019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93f6d9d700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93f81045e0 0x7f93f819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93f759e700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f8109860 0x7f93f819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93f659c700 1 -- 172.21.15.46:0/2948689019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f8109860 msgr2=0x7f93f819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93f659c700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f8109860 0x7f93f819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93f659c700 1 -- 172.21.15.46:0/2948689019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93f81045e0 msgr2=0x7f93f819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93f659c700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93f81045e0 0x7f93f819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93f659c700 1 -- 172.21.15.46:0/2948689019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f819fba0 con 0x7f93f8108f10 2024-05-09T00:17:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93f659c700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f8108f10 0x7f93f819b320 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00c9e0 tx=0x7f93ec00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93dffff700 1 -- 172.21.15.46:0/2948689019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec007aa0 con 0x7f93f8108f10 2024-05-09T00:17:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93fd1d4700 1 -- 172.21.15.46:0/2948689019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f819fe90 con 0x7f93f8108f10 2024-05-09T00:17:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93dffff700 1 -- 172.21.15.46:0/2948689019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ec007c40 con 0x7f93f8108f10 2024-05-09T00:17:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.058+0000 7f93dffff700 1 -- 172.21.15.46:0/2948689019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec00fe60 con 0x7f93f8108f10 2024-05-09T00:17:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.059+0000 7f93fd1d4700 1 -- 172.21.15.46:0/2948689019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f81a7860 con 0x7f93f8108f10 2024-05-09T00:17:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.061+0000 7f93dffff700 1 -- 172.21.15.46:0/2948689019 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f93ec004870 con 0x7f93f8108f10 2024-05-09T00:17:54.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.062+0000 7f93dffff700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f93e00b2880 0x7f93e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.062+0000 7f93f6d9d700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f93e00b2880 0x7f93e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.062+0000 7f93dffff700 1 -- 172.21.15.46:0/2948689019 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f93ec0e20d0 con 0x7f93f8108f10 2024-05-09T00:17:54.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.063+0000 7f940bbeab80 1 Processor -- start 2024-05-09T00:17:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.063+0000 7f940bbeab80 1 -- start start 2024-05-09T00:17:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.063+0000 7f93f6d9d700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f93e00b2880 0x7f93e00b4d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f93e8005950 tx=0x7f93e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.063+0000 7f940bbeab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112fe01d00 0x56112ff0a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.063+0000 7f940bbeab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 0x56112ff0a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.063+0000 7f940bbeab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112ff04400 0x56112ff04810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.063+0000 7f940bbeab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56112fe270a0 con 0x56112fd874a0 2024-05-09T00:17:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.063+0000 7f940bbeab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56112fdcc9a0 con 0x56112ff04400 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.063+0000 7f940bbeab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56112fe23060 con 0x56112fe01d00 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.063+0000 7f93f6d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112fe01d00 0x56112ff0a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.063+0000 7f93f659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 0x56112ff0a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93f6d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112fe01d00 0x56112ff0a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:33964/0 (socket says 172.21.15.46:33964) 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93f6d9d700 1 -- 172.21.15.46:0/4112834150 learned_addr learned my addr 172.21.15.46:0/4112834150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93f759e700 1 --2- 172.21.15.46:0/4112834150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112ff04400 0x56112ff04810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93f6d9d700 1 -- 172.21.15.46:0/4112834150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112ff04400 msgr2=0x56112ff04810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93f6d9d700 1 --2- 172.21.15.46:0/4112834150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112ff04400 0x56112ff04810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93f6d9d700 1 -- 172.21.15.46:0/4112834150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 msgr2=0x56112ff0a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93f6d9d700 1 --2- 172.21.15.46:0/4112834150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 0x56112ff0a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93f6d9d700 1 -- 172.21.15.46:0/4112834150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56112fdbd500 con 0x56112fe01d00 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93f659c700 1 --2- 172.21.15.46:0/4112834150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 0x56112ff0a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:17:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93f6d9d700 1 --2- 172.21.15.46:0/4112834150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112fe01d00 0x56112ff0a240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f93e8010fa0 tx=0x7f93e8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe16e808ab5f845f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:54.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93dd7fa700 1 -- 172.21.15.46:0/4112834150 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e800be60 con 0x56112fe01d00 2024-05-09T00:17:54.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93dd7fa700 1 -- 172.21.15.46:0/4112834150 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e8011db0 con 0x56112fe01d00 2024-05-09T00:17:54.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.064+0000 7f93dd7fa700 1 -- 172.21.15.46:0/4112834150 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e801c700 con 0x56112fe01d00 2024-05-09T00:17:54.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.065+0000 7f940bbeab80 1 -- 172.21.15.46:0/4112834150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112fe01d00 msgr2=0x56112ff0a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:54.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.065+0000 7f940bbeab80 1 --2- 172.21.15.46:0/4112834150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112fe01d00 0x56112ff0a240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f93e8010fa0 tx=0x7f93e8013000 comp rx=0 tx=0).stop 2024-05-09T00:17:54.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.065+0000 7f940bbeab80 1 -- 172.21.15.46:0/4112834150 shutdown_connections 2024-05-09T00:17:54.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.065+0000 7f940bbeab80 1 --2- 172.21.15.46:0/4112834150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112ff04400 0x56112ff04810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.065+0000 7f940bbeab80 1 --2- 172.21.15.46:0/4112834150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112fe01d00 0x56112ff0a240 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.065+0000 7f940bbeab80 1 --2- 172.21.15.46:0/4112834150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 0x56112ff0a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.065+0000 7f940bbeab80 1 -- 172.21.15.46:0/4112834150 >> 172.21.15.46:0/4112834150 conn(0x56112fd8af70 msgr2=0x56112feff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:54.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.065+0000 7f940bbeab80 1 -- 172.21.15.46:0/4112834150 shutdown_connections 2024-05-09T00:17:54.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.065+0000 7f940bbeab80 1 -- 172.21.15.46:0/4112834150 wait complete. 2024-05-09T00:17:54.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f940bbeab80 1 -- start start 2024-05-09T00:17:54.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f940bbeab80 1 Processor -- start 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f940bbeab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 0x56112ff0dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f940bbeab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112fe01d00 0x56112ff0e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f940bbeab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112ff04400 0x56112ff0ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f940bbeab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56112ff05160 con 0x56112fd874a0 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f940bbeab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56112fdc1b20 con 0x56112fe01d00 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f940bbeab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56112ff11020 con 0x56112ff04400 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f940bbeab80 10 client.0 _ll_register_callbacks cb 0x56112fc986c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f93f6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 0x56112ff0dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f93f6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 0x56112ff0dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40088/0 (socket says 172.21.15.46:40088) 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f93f6d9d700 1 -- 172.21.15.46:0/1624031000 learned_addr learned my addr 172.21.15.46:0/1624031000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f93f759e700 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112ff04400 0x56112ff0ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.066+0000 7f93f659c700 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112fe01d00 0x56112ff0e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.067+0000 7f93f6d9d700 1 -- 172.21.15.46:0/1624031000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112ff04400 msgr2=0x56112ff0ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:54.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.067+0000 7f93f6d9d700 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112ff04400 0x56112ff0ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.067+0000 7f93f6d9d700 1 -- 172.21.15.46:0/1624031000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112fe01d00 msgr2=0x56112ff0e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:54.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.067+0000 7f93f6d9d700 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112fe01d00 0x56112ff0e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.067+0000 7f93f6d9d700 1 -- 172.21.15.46:0/1624031000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56112fca2490 con 0x56112fd874a0 2024-05-09T00:17:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.067+0000 7f93f6d9d700 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 0x56112ff0dfc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f93e8010a40 tx=0x7f93e8005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9594d9e6688d68f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.068+0000 7f93daff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.068+0000 7f93f659c700 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112fe01d00 0x56112ff0e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:17:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.068+0000 7f93daff5700 1 -- 172.21.15.46:0/1624031000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e80056d0 con 0x56112fd874a0 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.069+0000 7f93daff5700 1 -- 172.21.15.46:0/1624031000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f93e8011bb0 con 0x56112fd874a0 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.069+0000 7f940bbeab80 1 -- 172.21.15.46:0/1624031000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e8011db0 con 0x56112fd874a0 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.069+0000 7f93daff5700 1 -- 172.21.15.46:0/1624031000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93e8004710 con 0x56112fd874a0 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.070+0000 7f940bbeab80 10 client.16335 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.070+0000 7f940bbeab80 1 -- 172.21.15.46:0/1624031000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56112ff1b990 con 0x56112fd874a0 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.070+0000 7f93daff5700 1 -- 172.21.15.46:0/1624031000 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e8011db0 con 0x56112fd874a0 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.071+0000 7f93daff5700 1 -- 172.21.15.46:0/1624031000 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f93e801e9c0 con 0x56112fd874a0 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.071+0000 7f940bbeab80 10 client.16335 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.071+0000 7f940bbeab80 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.071+0000 7f940bbeab80 20 client.16335 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.071+0000 7f940bbeab80 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56112ff1df10 0x56112ff20330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.071+0000 7f940bbeab80 4 client.16335 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:17:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.071+0000 7f940bbeab80 1 -- 172.21.15.46:0/1624031000 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56112fe34a80 con 0x56112ff1df10 2024-05-09T00:17:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.071+0000 7f93f659c700 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56112ff1df10 0x56112ff20330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.072+0000 7f93daff5700 10 client.16335 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.072+0000 7f93f659c700 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56112ff1df10 0x56112ff20330 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4be831d9c2dbbcc server_cookie=163a959e6780372 in_seq=0 out_seq=0 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f93daff5700 1 -- 172.21.15.46:0/1624031000 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56112fe34a80 con 0x56112ff1df10 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f93daff5700 10 client.16335 handle_command_reply: tid=0 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f940bbeab80 1 client.16335 shutdown 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f940bbeab80 1 -- 172.21.15.46:0/1624031000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 msgr2=0x56112ff0dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f940bbeab80 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 0x56112ff0dfc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f93e8010a40 tx=0x7f93e8005250 comp rx=0 tx=0).stop 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f940bbeab80 1 -- 172.21.15.46:0/1624031000 shutdown_connections 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f940bbeab80 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56112ff1df10 0x56112ff20330 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f940bbeab80 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112ff04400 0x56112ff0ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f940bbeab80 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fd874a0 0x56112ff0dfc0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f940bbeab80 1 --2- 172.21.15.46:0/1624031000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112fe01d00 0x56112ff0e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f93daff5700 0 client.16335 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f940bbeab80 1 -- 172.21.15.46:0/1624031000 >> 172.21.15.46:0/1624031000 conn(0x56112fd8af70 msgr2=0x56112fef8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:54.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f940bbeab80 1 -- 172.21.15.46:0/1624031000 shutdown_connections 2024-05-09T00:17:54.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f940bbeab80 1 -- 172.21.15.46:0/1624031000 wait complete. 2024-05-09T00:17:54.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.073+0000 7f940bbeab80 20 client.16335 trim_cache size 0 max 16384 2024-05-09T00:17:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f940bbeab80 1 Processor -- start 2024-05-09T00:17:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f940bbeab80 1 -- start start 2024-05-09T00:17:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f940bbeab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112fd874a0 0x56112ff08630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f940bbeab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 0x56112ff0e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f940bbeab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112ff04400 0x56112ff23350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f940bbeab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81af350 con 0x56112fe01d00 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f940bbeab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81acbe0 con 0x56112ff04400 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f940bbeab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81aaa10 con 0x56112fd874a0 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f93f659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 0x56112ff0e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f93f659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 0x56112ff0e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40104/0 (socket says 172.21.15.46:40104) 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f93f659c700 1 -- 172.21.15.46:0/974246299 learned_addr learned my addr 172.21.15.46:0/974246299 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f93f6d9d700 1 --2- 172.21.15.46:0/974246299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112fd874a0 0x56112ff08630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f93f759e700 1 --2- 172.21.15.46:0/974246299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112ff04400 0x56112ff23350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f93f659c700 1 -- 172.21.15.46:0/974246299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112fd874a0 msgr2=0x56112ff08630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f93f659c700 1 --2- 172.21.15.46:0/974246299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112fd874a0 0x56112ff08630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f93f659c700 1 -- 172.21.15.46:0/974246299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112ff04400 msgr2=0x56112ff23350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f93f659c700 1 --2- 172.21.15.46:0/974246299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112ff04400 0x56112ff23350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f93f659c700 1 -- 172.21.15.46:0/974246299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56112fefb9d0 con 0x56112fe01d00 2024-05-09T00:17:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.095+0000 7f93f759e700 1 --2- 172.21.15.46:0/974246299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112ff04400 0x56112ff23350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:17:54.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.096+0000 7f93f659c700 1 --2- 172.21.15.46:0/974246299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 0x56112ff0e880 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0074a0 tx=0x7f93ec0dfaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933aa684122c10b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:54.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.096+0000 7f93dcff9700 1 -- 172.21.15.46:0/974246299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec088070 con 0x56112fe01d00 2024-05-09T00:17:54.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.096+0000 7f93dcff9700 1 -- 172.21.15.46:0/974246299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ec06d070 con 0x56112fe01d00 2024-05-09T00:17:54.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.096+0000 7f93dcff9700 1 -- 172.21.15.46:0/974246299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec0dfd00 con 0x56112fe01d00 2024-05-09T00:17:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.097+0000 7f940bbeab80 1 -- 172.21.15.46:0/974246299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 msgr2=0x56112ff0e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.097+0000 7f940bbeab80 1 --2- 172.21.15.46:0/974246299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 0x56112ff0e880 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0074a0 tx=0x7f93ec0dfaa0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.097+0000 7f940bbeab80 1 -- 172.21.15.46:0/974246299 shutdown_connections 2024-05-09T00:17:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.097+0000 7f940bbeab80 1 --2- 172.21.15.46:0/974246299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112ff04400 0x56112ff23350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.097+0000 7f940bbeab80 1 --2- 172.21.15.46:0/974246299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112fd874a0 0x56112ff08630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.097+0000 7f940bbeab80 1 --2- 172.21.15.46:0/974246299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 0x56112ff0e880 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.097+0000 7f940bbeab80 1 -- 172.21.15.46:0/974246299 >> 172.21.15.46:0/974246299 conn(0x56112fd8af70 msgr2=0x56112fef9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.097+0000 7f940bbeab80 1 -- 172.21.15.46:0/974246299 shutdown_connections 2024-05-09T00:17:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.097+0000 7f940bbeab80 1 -- 172.21.15.46:0/974246299 wait complete. 2024-05-09T00:17:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.097+0000 7f940bbeab80 1 -- start start 2024-05-09T00:17:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.097+0000 7f940bbeab80 1 Processor -- start 2024-05-09T00:17:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f940bbeab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112fd874a0 0x56112ff21640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f940bbeab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 0x56112ff21b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f940bbeab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112ff04400 0x56112ff87460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f940bbeab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81ab0a0 con 0x56112fe01d00 2024-05-09T00:17:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f940bbeab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81ac510 con 0x56112fd874a0 2024-05-09T00:17:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f940bbeab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81ab770 con 0x56112ff04400 2024-05-09T00:17:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f940bbeab80 10 client.0 _ll_register_callbacks cb 0x56112fc986c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:17:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93f659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 0x56112ff21b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93f6d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112fd874a0 0x56112ff21640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93f659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 0x56112ff21b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40114/0 (socket says 172.21.15.46:40114) 2024-05-09T00:17:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93f659c700 1 -- 172.21.15.46:0/2452432841 learned_addr learned my addr 172.21.15.46:0/2452432841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93f759e700 1 --2- 172.21.15.46:0/2452432841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112ff04400 0x56112ff87460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93f659c700 1 -- 172.21.15.46:0/2452432841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112ff04400 msgr2=0x56112ff87460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93f659c700 1 --2- 172.21.15.46:0/2452432841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112ff04400 0x56112ff87460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93f659c700 1 -- 172.21.15.46:0/2452432841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112fd874a0 msgr2=0x56112ff21640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93f659c700 1 --2- 172.21.15.46:0/2452432841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112fd874a0 0x56112ff21640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93f659c700 1 -- 172.21.15.46:0/2452432841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56112ff00bd0 con 0x56112fe01d00 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93f659c700 1 --2- 172.21.15.46:0/2452432841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 0x56112ff21b80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0a9680 tx=0x7f93ec091410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910b33ab28afd0c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93daff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93daff5700 1 -- 172.21.15.46:0/2452432841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec088040 con 0x56112fe01d00 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f93daff5700 1 -- 172.21.15.46:0/2452432841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f93ec083e00 con 0x56112fe01d00 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.098+0000 7f940bbeab80 1 -- 172.21.15.46:0/2452432841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93ec06d070 con 0x56112fe01d00 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.099+0000 7f93daff5700 1 -- 172.21.15.46:0/2452432841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93ec0a96f0 con 0x56112fe01d00 2024-05-09T00:17:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.099+0000 7f940bbeab80 10 client.16341 fetch_fsmap learned FSMap version 26 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.099+0000 7f940bbeab80 1 -- 172.21.15.46:0/2452432841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56112ff88900 con 0x56112fe01d00 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.099+0000 7f93daff5700 1 -- 172.21.15.46:0/2452432841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ec06d070 con 0x56112fe01d00 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.099+0000 7f93daff5700 1 -- 172.21.15.46:0/2452432841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f93ec0a5060 con 0x56112fe01d00 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.099+0000 7f940bbeab80 10 client.16341 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.099+0000 7f940bbeab80 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.099+0000 7f940bbeab80 20 client.16341 populate_metadata read hostname 'smithi046' 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.099+0000 7f940bbeab80 1 --2- 172.21.15.46:0/2452432841 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56112ff88f30 0x56112ff8b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.099+0000 7f940bbeab80 4 client.16341 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.099+0000 7f940bbeab80 1 -- 172.21.15.46:0/2452432841 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56112fe2eb20 con 0x56112ff88f30 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.099+0000 7f93f6d9d700 1 --2- 172.21.15.46:0/2452432841 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56112ff88f30 0x56112ff8b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.100+0000 7f93f6d9d700 1 --2- 172.21.15.46:0/2452432841 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56112ff88f30 0x56112ff8b350 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1a413568f564208 server_cookie=73a5c2a640873680 in_seq=0 out_seq=0 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.100+0000 7f93daff5700 10 client.16341 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.100+0000 7f93daff5700 1 -- 172.21.15.46:0/2452432841 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56112fe2eb20 con 0x56112ff88f30 2024-05-09T00:17:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.100+0000 7f93daff5700 10 client.16341 handle_command_reply: tid=0 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.100+0000 7f940bbeab80 1 client.16341 shutdown 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.100+0000 7f940bbeab80 1 -- 172.21.15.46:0/2452432841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 msgr2=0x56112ff21b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.100+0000 7f940bbeab80 1 --2- 172.21.15.46:0/2452432841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 0x56112ff21b80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0a9680 tx=0x7f93ec091410 comp rx=0 tx=0).stop 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f940bbeab80 1 -- 172.21.15.46:0/2452432841 shutdown_connections 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f940bbeab80 1 --2- 172.21.15.46:0/2452432841 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56112ff88f30 0x56112ff8b350 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f940bbeab80 1 --2- 172.21.15.46:0/2452432841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56112ff04400 0x56112ff87460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f940bbeab80 1 --2- 172.21.15.46:0/2452432841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56112fd874a0 0x56112ff21640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f940bbeab80 1 --2- 172.21.15.46:0/2452432841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56112fe01d00 0x56112ff21b80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f93daff5700 0 client.16341 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f940bbeab80 1 -- 172.21.15.46:0/2452432841 >> 172.21.15.46:0/2452432841 conn(0x56112fd8af70 msgr2=0x56112ff0f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f940bbeab80 1 -- 172.21.15.46:0/2452432841 shutdown_connections 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f940bbeab80 1 -- 172.21.15.46:0/2452432841 wait complete. 2024-05-09T00:17:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f940bbeab80 20 client.16341 trim_cache size 0 max 16384 2024-05-09T00:17:54.105 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:17:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:17:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:17:54.105 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:17:54.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f93fd1d4700 1 -- 172.21.15.46:0/2948689019 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f93e00b2880 msgr2=0x7f93e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f93fd1d4700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f93e00b2880 0x7f93e00b4d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f93e8005950 tx=0x7f93e800b410 comp rx=0 tx=0).stop 2024-05-09T00:17:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f93fd1d4700 1 -- 172.21.15.46:0/2948689019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f8108f10 msgr2=0x7f93f819b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f93fd1d4700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f8108f10 0x7f93f819b320 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00c9e0 tx=0x7f93ec00ccf0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f93fd1d4700 1 -- 172.21.15.46:0/2948689019 shutdown_connections 2024-05-09T00:17:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.101+0000 7f93fd1d4700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f93e00b2880 0x7f93e00b4d90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.102+0000 7f93fd1d4700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f8109860 0x7f93f819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.102+0000 7f93fd1d4700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93f81045e0 0x7f93f819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.102+0000 7f93fd1d4700 1 --2- 172.21.15.46:0/2948689019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f8108f10 0x7f93f819b320 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.102+0000 7f93fd1d4700 1 -- 172.21.15.46:0/2948689019 >> 172.21.15.46:0/2948689019 conn(0x7f93f80ffed0 msgr2=0x7f93f8108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.102+0000 7f93fd1d4700 1 -- 172.21.15.46:0/2948689019 shutdown_connections 2024-05-09T00:17:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:54.102+0000 7f93fd1d4700 1 -- 172.21.15.46:0/2948689019 wait complete. 2024-05-09T00:17:54.799 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:17:54.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:17:55.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:54 smithi046 ceph-mon[30151]: pgmap v572: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:55.098 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:17:55.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:54 smithi169 ceph-mon[34777]: pgmap v572: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:55.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:54 smithi142 ceph-mon[34533]: pgmap v572: 97 pgs: 97 active+clean; 3.5 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:17:56.976 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:56 smithi046 ceph-mon[30151]: pgmap v573: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.985+0000 7f614d418700 1 -- 172.21.15.46:0/2760927940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6148107df0 msgr2=0x7f6148108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.985+0000 7f614d418700 1 --2- 172.21.15.46:0/2760927940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6148107df0 0x7f6148108200 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f61300099c0 tx=0x7f6130009cd0 comp rx=0 tx=0).stop 2024-05-09T00:17:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.985+0000 7f614d418700 1 -- 172.21.15.46:0/2760927940 shutdown_connections 2024-05-09T00:17:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.985+0000 7f614d418700 1 --2- 172.21.15.46:0/2760927940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6148108740 0x7f61481100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.985+0000 7f614d418700 1 --2- 172.21.15.46:0/2760927940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61481034c0 0x7f61481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.985+0000 7f614d418700 1 --2- 172.21.15.46:0/2760927940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6148107df0 0x7f6148108200 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.985+0000 7f614d418700 1 -- 172.21.15.46:0/2760927940 >> 172.21.15.46:0/2760927940 conn(0x7f61480fedc0 msgr2=0x7f61481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.985+0000 7f614d418700 1 -- 172.21.15.46:0/2760927940 shutdown_connections 2024-05-09T00:17:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.985+0000 7f614d418700 1 -- 172.21.15.46:0/2760927940 wait complete. 2024-05-09T00:17:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.985+0000 7f614d418700 1 Processor -- start 2024-05-09T00:17:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f614d418700 1 -- start start 2024-05-09T00:17:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f614d418700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61481034c0 0x7f61480ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f614d418700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6148107df0 0x7f61480ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f614d418700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6148108740 0x7f61480ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f614d418700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6148113350 con 0x7f6148108740 2024-05-09T00:17:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f614d418700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f61481131d0 con 0x7f6148107df0 2024-05-09T00:17:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f614d418700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f61481134d0 con 0x7f61481034c0 2024-05-09T00:17:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f61477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6148108740 0x7f61480ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f61477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6148108740 0x7f61480ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56286/0 (socket says 172.21.15.46:56286) 2024-05-09T00:17:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f61477fe700 1 -- 172.21.15.46:0/750139287 learned_addr learned my addr 172.21.15.46:0/750139287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:17:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f6146ffd700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61481034c0 0x7f61480ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f613e7fc700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6148107df0 0x7f61480ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f61477fe700 1 -- 172.21.15.46:0/750139287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61481034c0 msgr2=0x7f61480ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f61477fe700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61481034c0 0x7f61480ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f61477fe700 1 -- 172.21.15.46:0/750139287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6148107df0 msgr2=0x7f61480ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f61477fe700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6148107df0 0x7f61480ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.986+0000 7f61477fe700 1 -- 172.21.15.46:0/750139287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61481a90a0 con 0x7f6148108740 2024-05-09T00:17:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.987+0000 7f61477fe700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6148108740 0x7f61480ffd20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6140007d70 tx=0x7f614000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.987+0000 7f6144ff9700 1 -- 172.21.15.46:0/750139287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140016020 con 0x7f6148108740 2024-05-09T00:17:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.987+0000 7f614d418700 1 -- 172.21.15.46:0/750139287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61481a9390 con 0x7f6148108740 2024-05-09T00:17:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.987+0000 7f6144ff9700 1 -- 172.21.15.46:0/750139287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f614000ac90 con 0x7f6148108740 2024-05-09T00:17:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.987+0000 7f6144ff9700 1 -- 172.21.15.46:0/750139287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61400115e0 con 0x7f6148108740 2024-05-09T00:17:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.987+0000 7f614d418700 1 -- 172.21.15.46:0/750139287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61481a98a0 con 0x7f6148108740 2024-05-09T00:17:56.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.988+0000 7f614d418700 1 -- 172.21.15.46:0/750139287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61480512c0 con 0x7f6148108740 2024-05-09T00:17:56.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.990+0000 7f6144ff9700 1 -- 172.21.15.46:0/750139287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6140019020 con 0x7f6148108740 2024-05-09T00:17:56.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.990+0000 7f6144ff9700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61340b2880 0x7f61340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:17:56.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.991+0000 7f6146ffd700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61340b2880 0x7f61340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:17:56.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.991+0000 7f6144ff9700 1 -- 172.21.15.46:0/750139287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f61400e25f0 con 0x7f6148108740 2024-05-09T00:17:56.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.991+0000 7f6146ffd700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61340b2880 0x7f61340b4d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f613800ac50 tx=0x7f613800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:17:56.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:56.994+0000 7f6144ff9700 1 -- 172.21.15.46:0/750139287 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6140020020 con 0x7f6148108740 2024-05-09T00:17:57.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:56 smithi169 ceph-mon[34777]: pgmap v573: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:57.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:56 smithi142 ceph-mon[34533]: pgmap v573: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:17:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.257+0000 7f614d418700 1 -- 172.21.15.46:0/750139287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61481007d0 con 0x7f6148108740 2024-05-09T00:17:57.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.258+0000 7f6144ff9700 1 -- 172.21.15.46:0/750139287 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f61400aa310 con 0x7f6148108740 2024-05-09T00:17:57.257 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:17:57.257 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:17:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.260+0000 7f614d418700 1 -- 172.21.15.46:0/750139287 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61340b2880 msgr2=0x7f61340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.260+0000 7f614d418700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61340b2880 0x7f61340b4d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f613800ac50 tx=0x7f613800a380 comp rx=0 tx=0).stop 2024-05-09T00:17:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.260+0000 7f614d418700 1 -- 172.21.15.46:0/750139287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6148108740 msgr2=0x7f61480ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:17:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.260+0000 7f614d418700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6148108740 0x7f61480ffd20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6140007d70 tx=0x7f614000a340 comp rx=0 tx=0).stop 2024-05-09T00:17:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.260+0000 7f614d418700 1 -- 172.21.15.46:0/750139287 shutdown_connections 2024-05-09T00:17:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.260+0000 7f614d418700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f61340b2880 0x7f61340b4d90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.260+0000 7f614d418700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6148108740 0x7f61480ffd20 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.260+0000 7f614d418700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61481034c0 0x7f61480ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.260+0000 7f614d418700 1 --2- 172.21.15.46:0/750139287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6148107df0 0x7f61480ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:17:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.260+0000 7f614d418700 1 -- 172.21.15.46:0/750139287 >> 172.21.15.46:0/750139287 conn(0x7f61480fedc0 msgr2=0x7f6148107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:17:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.260+0000 7f614d418700 1 -- 172.21.15.46:0/750139287 shutdown_connections 2024-05-09T00:17:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:17:57.260+0000 7f614d418700 1 -- 172.21.15.46:0/750139287 wait complete. 2024-05-09T00:17:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:17:57.972 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:57 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/750139287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:17:58.003 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:17:58.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:57 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/750139287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:17:58.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:57 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/750139287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:17:58.256 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:17:59.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:17:58 smithi046 ceph-mon[30151]: pgmap v574: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:59.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:17:58 smithi169 ceph-mon[34777]: pgmap v574: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:17:59.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:17:58 smithi142 ceph-mon[34533]: pgmap v574: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:00.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.113+0000 7fc6995d1700 1 -- 172.21.15.46:0/2333003248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc694108f10 msgr2=0x7fc694109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:00.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.113+0000 7fc6995d1700 1 --2- 172.21.15.46:0/2333003248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc694108f10 0x7fc694109320 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc6880099c0 tx=0x7fc688009cd0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.113+0000 7fc6995d1700 1 -- 172.21.15.46:0/2333003248 shutdown_connections 2024-05-09T00:18:00.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.113+0000 7fc6995d1700 1 --2- 172.21.15.46:0/2333003248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc694109860 0x7fc6941111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.113+0000 7fc6995d1700 1 --2- 172.21.15.46:0/2333003248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6941045e0 0x7fc6941049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.113+0000 7fc6995d1700 1 --2- 172.21.15.46:0/2333003248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc694108f10 0x7fc694109320 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.113+0000 7fc6995d1700 1 -- 172.21.15.46:0/2333003248 >> 172.21.15.46:0/2333003248 conn(0x7fc6940ffef0 msgr2=0x7fc694102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:00.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.113+0000 7fc6995d1700 1 -- 172.21.15.46:0/2333003248 shutdown_connections 2024-05-09T00:18:00.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.113+0000 7fc6995d1700 1 -- 172.21.15.46:0/2333003248 wait complete. 2024-05-09T00:18:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.113+0000 7fc6995d1700 1 Processor -- start 2024-05-09T00:18:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc6995d1700 1 -- start start 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc6995d1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6941045e0 0x7fc69419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc6995d1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc694108f10 0x7fc69419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc6995d1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc694109860 0x7fc69419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc6995d1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc694114470 con 0x7fc6941045e0 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc6995d1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941142f0 con 0x7fc694108f10 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc6995d1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941145f0 con 0x7fc694109860 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc692ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6941045e0 0x7fc69419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc692ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6941045e0 0x7fc69419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56300/0 (socket says 172.21.15.46:56300) 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc692ffd700 1 -- 172.21.15.46:0/807214126 learned_addr learned my addr 172.21.15.46:0/807214126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc6937fe700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc694109860 0x7fc69419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc6927fc700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc694108f10 0x7fc69419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc692ffd700 1 -- 172.21.15.46:0/807214126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc694109860 msgr2=0x7fc69419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc692ffd700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc694109860 0x7fc69419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc692ffd700 1 -- 172.21.15.46:0/807214126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc694108f10 msgr2=0x7fc69419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc692ffd700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc694108f10 0x7fc69419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc692ffd700 1 -- 172.21.15.46:0/807214126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69419fc20 con 0x7fc6941045e0 2024-05-09T00:18:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.114+0000 7fc6927fc700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc694108f10 0x7fc69419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:18:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.115+0000 7fc692ffd700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6941045e0 0x7fc69419ae00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc684007e20 tx=0x7fc68400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.115+0000 7fc67bfff700 1 -- 172.21.15.46:0/807214126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684016020 con 0x7fc6941045e0 2024-05-09T00:18:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.115+0000 7fc67bfff700 1 -- 172.21.15.46:0/807214126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68400ae00 con 0x7fc6941045e0 2024-05-09T00:18:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.115+0000 7fc6995d1700 1 -- 172.21.15.46:0/807214126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69419ff10 con 0x7fc6941045e0 2024-05-09T00:18:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.115+0000 7fc67bfff700 1 -- 172.21.15.46:0/807214126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684011700 con 0x7fc6941045e0 2024-05-09T00:18:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.115+0000 7fc6995d1700 1 -- 172.21.15.46:0/807214126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6941a0450 con 0x7fc6941045e0 2024-05-09T00:18:00.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.117+0000 7fc67bfff700 1 -- 172.21.15.46:0/807214126 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc684019020 con 0x7fc6941045e0 2024-05-09T00:18:00.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.118+0000 7fc67bfff700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc67c0b2880 0x7fc67c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.118+0000 7fc6927fc700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc67c0b2880 0x7fc67c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.118+0000 7fc67bfff700 1 -- 172.21.15.46:0/807214126 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc6840e1ff0 con 0x7fc6941045e0 2024-05-09T00:18:00.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.118+0000 7fc6a7fe7b80 1 Processor -- start 2024-05-09T00:18:00.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc6a7fe7b80 1 -- start start 2024-05-09T00:18:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc6a7fe7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e185ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc6927fc700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc67c0b2880 0x7fc67c0b4d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc688006010 tx=0x7fc688005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc6a7fe7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e186030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc6a7fe7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc6a7fe7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec0e0a40a0 con 0x55ec0e07ed00 2024-05-09T00:18:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc6a7fe7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ec0e0499a0 con 0x55ec0e0044a0 2024-05-09T00:18:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc6a7fe7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ec0e0a0060 con 0x55ec0e18a1e0 2024-05-09T00:18:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc692ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e185ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc692ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e185ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56304/0 (socket says 172.21.15.46:56304) 2024-05-09T00:18:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc692ffd700 1 -- 172.21.15.46:0/953539990 learned_addr learned my addr 172.21.15.46:0/953539990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc6937fe700 1 --2- 172.21.15.46:0/953539990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc6927fc700 1 --2- 172.21.15.46:0/953539990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e186030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc692ffd700 1 -- 172.21.15.46:0/953539990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e18a1e0 msgr2=0x55ec0e18a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc692ffd700 1 --2- 172.21.15.46:0/953539990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc692ffd700 1 -- 172.21.15.46:0/953539990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 msgr2=0x55ec0e186030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc692ffd700 1 --2- 172.21.15.46:0/953539990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e186030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc692ffd700 1 -- 172.21.15.46:0/953539990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0e03a500 con 0x55ec0e07ed00 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.119+0000 7fc6927fc700 1 --2- 172.21.15.46:0/953539990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e186030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.120+0000 7fc692ffd700 1 --2- 172.21.15.46:0/953539990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e185ad0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc6840a9610 tx=0x7fc6840a92e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66017330ba5ece65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.120+0000 7fc6797fa700 1 -- 172.21.15.46:0/953539990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6840b2470 con 0x55ec0e07ed00 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.120+0000 7fc6797fa700 1 -- 172.21.15.46:0/953539990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6840b1fe0 con 0x55ec0e07ed00 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.120+0000 7fc6797fa700 1 -- 172.21.15.46:0/953539990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6840a8c10 con 0x55ec0e07ed00 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.121+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/953539990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 msgr2=0x55ec0e185ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.121+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/953539990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e185ad0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc6840a9610 tx=0x7fc6840a92e0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.121+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/953539990 shutdown_connections 2024-05-09T00:18:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.121+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/953539990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.121+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/953539990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e185ad0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.121+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/953539990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e186030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.121+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/953539990 >> 172.21.15.46:0/953539990 conn(0x55ec0e007f70 msgr2=0x55ec0e17c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.121+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/953539990 shutdown_connections 2024-05-09T00:18:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.121+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/953539990 wait complete. 2024-05-09T00:18:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.121+0000 7fc6a7fe7b80 1 -- start start 2024-05-09T00:18:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.121+0000 7fc6a7fe7b80 1 Processor -- start 2024-05-09T00:18:00.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6a7fe7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e18f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6a7fe7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e07ed00 0x55ec0e18f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6a7fe7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6a7fe7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec0e18adb0 con 0x55ec0e18a1e0 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6a7fe7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ec0e03eb20 con 0x55ec0e0044a0 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6a7fe7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ec0e18af30 con 0x55ec0e07ed00 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6a7fe7b80 10 client.0 _ll_register_callbacks cb 0x55ec0df156c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56308/0 (socket says 172.21.15.46:56308) 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6937fe700 1 -- 172.21.15.46:0/3466454029 learned_addr learned my addr 172.21.15.46:0/3466454029 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6927fc700 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e07ed00 0x55ec0e18f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc692ffd700 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e18f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6937fe700 1 -- 172.21.15.46:0/3466454029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e07ed00 msgr2=0x55ec0e18f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6937fe700 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e07ed00 0x55ec0e18f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6937fe700 1 -- 172.21.15.46:0/3466454029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 msgr2=0x55ec0e18f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6937fe700 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e18f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6937fe700 1 -- 172.21.15.46:0/3466454029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec0df1f490 con 0x55ec0e18a1e0 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6927fc700 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e07ed00 0x55ec0e18f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc6937fe700 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18fec0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00dfa0 tx=0x7fc68c0127e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132a54f7b3b7cd00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc676ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.122+0000 7fc676ff5700 1 -- 172.21.15.46:0/3466454029 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c01f060 con 0x55ec0e18a1e0 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc676ff5700 1 -- 172.21.15.46:0/3466454029 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc68c0107e0 con 0x55ec0e18a1e0 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/3466454029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6840b1fe0 con 0x55ec0e18a1e0 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc676ff5700 1 -- 172.21.15.46:0/3466454029 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc68c0109e0 con 0x55ec0e18a1e0 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc6a7fe7b80 10 client.16359 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/3466454029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec0e19ccf0 con 0x55ec0e18a1e0 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc676ff5700 1 -- 172.21.15.46:0/3466454029 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6840b1fe0 con 0x55ec0e18a1e0 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc676ff5700 1 -- 172.21.15.46:0/3466454029 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc68c024070 con 0x55ec0e18a1e0 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc6a7fe7b80 10 client.16359 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc6a7fe7b80 10 client.16359 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc6a7fe7b80 20 client.16359 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ec0e19f280 0x55ec0e1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc6a7fe7b80 4 client.16359 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.123+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/3466454029 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec0e0b1a80 con 0x55ec0e19f280 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.124+0000 7fc692ffd700 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ec0e19f280 0x55ec0e1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.124+0000 7fc692ffd700 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ec0e19f280 0x55ec0e1a1700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b729606e1e50f2dd server_cookie=e178aec66e5590a5 in_seq=0 out_seq=0 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.124+0000 7fc676ff5700 10 client.16359 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc676ff5700 1 -- 172.21.15.46:0/3466454029 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec0e0b1a80 con 0x55ec0e19f280 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc676ff5700 10 client.16359 handle_command_reply: tid=0 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc6a7fe7b80 1 client.16359 shutdown 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/3466454029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e18a1e0 msgr2=0x55ec0e18fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18fec0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00dfa0 tx=0x7fc68c0127e0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/3466454029 shutdown_connections 2024-05-09T00:18:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ec0e19f280 0x55ec0e1a1700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc676ff5700 0 client.16359 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:18:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18fec0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e18f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/3466454029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e07ed00 0x55ec0e18f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/3466454029 >> 172.21.15.46:0/3466454029 conn(0x55ec0e007f70 msgr2=0x55ec0e1803f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/3466454029 shutdown_connections 2024-05-09T00:18:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/3466454029 wait complete. 2024-05-09T00:18:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.125+0000 7fc6a7fe7b80 20 client.16359 trim_cache size 0 max 16384 2024-05-09T00:18:00.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.140+0000 7fc6a7fe7b80 1 Processor -- start 2024-05-09T00:18:00.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.140+0000 7fc6a7fe7b80 1 -- start start 2024-05-09T00:18:00.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6a7fe7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e0044a0 0x55ec0e176a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6a7fe7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e189ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6a7fe7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6a7fe7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941af3d0 con 0x55ec0e07ed00 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6a7fe7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941acc60 con 0x55ec0e18a1e0 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6a7fe7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941aaa90 con 0x55ec0e0044a0 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6927fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e189ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6927fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e189ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56310/0 (socket says 172.21.15.46:56310) 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6927fc700 1 -- 172.21.15.46:0/263574038 learned_addr learned my addr 172.21.15.46:0/263574038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc692ffd700 1 --2- 172.21.15.46:0/263574038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e0044a0 0x55ec0e176a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6937fe700 1 --2- 172.21.15.46:0/263574038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6927fc700 1 -- 172.21.15.46:0/263574038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e0044a0 msgr2=0x55ec0e176a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6927fc700 1 --2- 172.21.15.46:0/263574038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e0044a0 0x55ec0e176a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6927fc700 1 -- 172.21.15.46:0/263574038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e18a1e0 msgr2=0x55ec0e18fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6927fc700 1 --2- 172.21.15.46:0/263574038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6927fc700 1 -- 172.21.15.46:0/263574038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0e191ce0 con 0x55ec0e07ed00 2024-05-09T00:18:00.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.141+0000 7fc6927fc700 1 --2- 172.21.15.46:0/263574038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e189ab0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc69406c1b0 tx=0x7fc68801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6984ab0063dc6420 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:00.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.142+0000 7fc678ff9700 1 -- 172.21.15.46:0/263574038 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688034070 con 0x55ec0e07ed00 2024-05-09T00:18:00.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.142+0000 7fc678ff9700 1 -- 172.21.15.46:0/263574038 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68800bc40 con 0x55ec0e07ed00 2024-05-09T00:18:00.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.142+0000 7fc678ff9700 1 -- 172.21.15.46:0/263574038 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68800fd70 con 0x55ec0e07ed00 2024-05-09T00:18:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.142+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/263574038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 msgr2=0x55ec0e189ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.142+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/263574038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e189ab0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc69406c1b0 tx=0x7fc68801d000 comp rx=0 tx=0).stop 2024-05-09T00:18:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.142+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/263574038 shutdown_connections 2024-05-09T00:18:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.142+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/263574038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.142+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/263574038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e0044a0 0x55ec0e176a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.142+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/263574038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e189ab0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.142+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/263574038 >> 172.21.15.46:0/263574038 conn(0x55ec0e007f70 msgr2=0x55ec0e18dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.142+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/263574038 shutdown_connections 2024-05-09T00:18:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.142+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/263574038 wait complete. 2024-05-09T00:18:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc6a7fe7b80 1 -- start start 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc6a7fe7b80 1 Processor -- start 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc6a7fe7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e197a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc6a7fe7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e18f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc6a7fe7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc6a7fe7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941ab120 con 0x55ec0e07ed00 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc6a7fe7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941ac590 con 0x55ec0e0044a0 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc6a7fe7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941ab7f0 con 0x55ec0e18a1e0 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc6a7fe7b80 10 client.0 _ll_register_callbacks cb 0x55ec0df156c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc6927fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e18f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc6927fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e18f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56318/0 (socket says 172.21.15.46:56318) 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc6927fc700 1 -- 172.21.15.46:0/1613098973 learned_addr learned my addr 172.21.15.46:0/1613098973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.143+0000 7fc692ffd700 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc6927fc700 1 -- 172.21.15.46:0/1613098973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e18a1e0 msgr2=0x55ec0e18f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc6937fe700 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc6927fc700 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc6927fc700 1 -- 172.21.15.46:0/1613098973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 msgr2=0x55ec0e197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc6927fc700 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc6927fc700 1 -- 172.21.15.46:0/1613098973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec0e18fc90 con 0x55ec0e07ed00 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc692ffd700 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc6937fe700 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc6927fc700 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e18f020 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc688032040 tx=0x7fc68800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d672d44cbfd831b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc676ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc676ff5700 1 -- 172.21.15.46:0/1613098973 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688034030 con 0x55ec0e07ed00 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc676ff5700 1 -- 172.21.15.46:0/1613098973 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc68801e070 con 0x55ec0e07ed00 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/1613098973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc68800bc40 con 0x55ec0e07ed00 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc676ff5700 1 -- 172.21.15.46:0/1613098973 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc68800baa0 con 0x55ec0e07ed00 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc6a7fe7b80 10 client.16365 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.144+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/1613098973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec0e209a70 con 0x55ec0e07ed00 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.145+0000 7fc676ff5700 1 -- 172.21.15.46:0/1613098973 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68800bc40 con 0x55ec0e07ed00 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.145+0000 7fc676ff5700 1 -- 172.21.15.46:0/1613098973 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc68802e070 con 0x55ec0e07ed00 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.145+0000 7fc6a7fe7b80 10 client.16365 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.145+0000 7fc6a7fe7b80 10 client.16365 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.145+0000 7fc6a7fe7b80 20 client.16365 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.145+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ec0e209fc0 0x55ec0e20c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.145+0000 7fc6a7fe7b80 4 client.16365 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.145+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/1613098973 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec0e17df00 con 0x55ec0e209fc0 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.145+0000 7fc692ffd700 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ec0e209fc0 0x55ec0e20c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.145+0000 7fc676ff5700 10 client.16365 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.145+0000 7fc692ffd700 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ec0e209fc0 0x55ec0e20c3e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ac3357200c922d5 server_cookie=62937298f244d6f0 in_seq=0 out_seq=0 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc676ff5700 1 -- 172.21.15.46:0/1613098973 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec0e17df00 con 0x55ec0e209fc0 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc676ff5700 10 client.16365 handle_command_reply: tid=0 2024-05-09T00:18:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc6a7fe7b80 1 client.16365 shutdown 2024-05-09T00:18:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/1613098973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 msgr2=0x55ec0e18f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e18f020 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc688032040 tx=0x7fc68800bdf0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/1613098973 shutdown_connections 2024-05-09T00:18:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ec0e209fc0 0x55ec0e20c3e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc676ff5700 0 client.16365 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:18:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ec0e18a1e0 0x55ec0e18f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec0e0044a0 0x55ec0e197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc6a7fe7b80 1 --2- 172.21.15.46:0/1613098973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec0e07ed00 0x55ec0e18f020 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/1613098973 >> 172.21.15.46:0/1613098973 conn(0x55ec0e007f70 msgr2=0x55ec0e176780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/1613098973 shutdown_connections 2024-05-09T00:18:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc6a7fe7b80 1 -- 172.21.15.46:0/1613098973 wait complete. 2024-05-09T00:18:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.146+0000 7fc6a7fe7b80 20 client.16365 trim_cache size 0 max 16384 2024-05-09T00:18:00.151 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:18:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.147+0000 7fc6995d1700 1 -- 172.21.15.46:0/807214126 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc67c0b2880 msgr2=0x7fc67c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.147+0000 7fc6995d1700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc67c0b2880 0x7fc67c0b4d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc688006010 tx=0x7fc688005e80 comp rx=0 tx=0).stop 2024-05-09T00:18:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.147+0000 7fc6995d1700 1 -- 172.21.15.46:0/807214126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6941045e0 msgr2=0x7fc69419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.147+0000 7fc6995d1700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6941045e0 0x7fc69419ae00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc684007e20 tx=0x7fc68400a340 comp rx=0 tx=0).stop 2024-05-09T00:18:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.147+0000 7fc6995d1700 1 -- 172.21.15.46:0/807214126 shutdown_connections 2024-05-09T00:18:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.147+0000 7fc6995d1700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc67c0b2880 0x7fc67c0b4d90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.147+0000 7fc6995d1700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc694109860 0x7fc69419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.147+0000 7fc6995d1700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6941045e0 0x7fc69419ae00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.147+0000 7fc6995d1700 1 --2- 172.21.15.46:0/807214126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc694108f10 0x7fc69419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.147+0000 7fc6995d1700 1 -- 172.21.15.46:0/807214126 >> 172.21.15.46:0/807214126 conn(0x7fc6940ffef0 msgr2=0x7fc69410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.147+0000 7fc6995d1700 1 -- 172.21.15.46:0/807214126 shutdown_connections 2024-05-09T00:18:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:00.147+0000 7fc6995d1700 1 -- 172.21.15.46:0/807214126 wait complete. 2024-05-09T00:18:00.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:00 smithi046 ceph-mon[30151]: pgmap v575: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:00.976 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:18:01.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:00 smithi169 ceph-mon[34777]: pgmap v575: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:01.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:00 smithi142 ceph-mon[34533]: pgmap v575: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:01.226 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:18:02.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:02 smithi142 ceph-mon[34533]: pgmap v576: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:18:03.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:02 smithi046 ceph-mon[30151]: pgmap v576: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:18:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.086+0000 7f4521ce7700 1 -- 172.21.15.46:0/2121035671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f451c108740 msgr2=0x7f451c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.086+0000 7f4521ce7700 1 --2- 172.21.15.46:0/2121035671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f451c108740 0x7f451c1100a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f451400b0d0 tx=0x7f451400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.087+0000 7f4521ce7700 1 -- 172.21.15.46:0/2121035671 shutdown_connections 2024-05-09T00:18:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.087+0000 7f4521ce7700 1 --2- 172.21.15.46:0/2121035671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f451c108740 0x7f451c1100a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.087+0000 7f4521ce7700 1 --2- 172.21.15.46:0/2121035671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f451c1034c0 0x7f451c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.087+0000 7f4521ce7700 1 --2- 172.21.15.46:0/2121035671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f451c107df0 0x7f451c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.087+0000 7f4521ce7700 1 -- 172.21.15.46:0/2121035671 >> 172.21.15.46:0/2121035671 conn(0x7f451c0fedc0 msgr2=0x7f451c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.087+0000 7f4521ce7700 1 -- 172.21.15.46:0/2121035671 shutdown_connections 2024-05-09T00:18:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.087+0000 7f4521ce7700 1 -- 172.21.15.46:0/2121035671 wait complete. 2024-05-09T00:18:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.087+0000 7f4521ce7700 1 Processor -- start 2024-05-09T00:18:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.087+0000 7f4521ce7700 1 -- start start 2024-05-09T00:18:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4521ce7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f451c1034c0 0x7f451c199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4521ce7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f451c107df0 0x7f451c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4521ce7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f451c108740 0x7f451c19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4521ce7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f451c1133c0 con 0x7f451c107df0 2024-05-09T00:18:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4521ce7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f451c113240 con 0x7f451c1034c0 2024-05-09T00:18:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4521ce7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f451c113540 con 0x7f451c108740 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4512ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f451c107df0 0x7f451c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4512ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f451c107df0 0x7f451c19a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56334/0 (socket says 172.21.15.46:56334) 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4512ffd700 1 -- 172.21.15.46:0/2805252593 learned_addr learned my addr 172.21.15.46:0/2805252593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f451b7fe700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f451c1034c0 0x7f451c199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f451bfff700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f451c108740 0x7f451c19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4512ffd700 1 -- 172.21.15.46:0/2805252593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f451c108740 msgr2=0x7f451c19e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4512ffd700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f451c108740 0x7f451c19e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4512ffd700 1 -- 172.21.15.46:0/2805252593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f451c1034c0 msgr2=0x7f451c199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4512ffd700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f451c1034c0 0x7f451c199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.088+0000 7f4512ffd700 1 -- 172.21.15.46:0/2805252593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c19ebe0 con 0x7f451c107df0 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.089+0000 7f4512ffd700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f451c107df0 0x7f451c19a2b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f450400d980 tx=0x7f450400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.089+0000 7f45197fa700 1 -- 172.21.15.46:0/2805252593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504017070 con 0x7f451c107df0 2024-05-09T00:18:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.089+0000 7f4521ce7700 1 -- 172.21.15.46:0/2805252593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c19eed0 con 0x7f451c107df0 2024-05-09T00:18:03.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.089+0000 7f45197fa700 1 -- 172.21.15.46:0/2805252593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f450400fd40 con 0x7f451c107df0 2024-05-09T00:18:03.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.089+0000 7f45197fa700 1 -- 172.21.15.46:0/2805252593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450400e3b0 con 0x7f451c107df0 2024-05-09T00:18:03.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.089+0000 7f4521ce7700 1 -- 172.21.15.46:0/2805252593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c1a67f0 con 0x7f451c107df0 2024-05-09T00:18:03.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.092+0000 7f45197fa700 1 -- 172.21.15.46:0/2805252593 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4504010610 con 0x7f451c107df0 2024-05-09T00:18:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f45197fa700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f45080b2880 0x7f45080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f451b7fe700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f45080b2880 0x7f45080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f45197fa700 1 -- 172.21.15.46:0/2805252593 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f45040e1c50 con 0x7f451c107df0 2024-05-09T00:18:03.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f45306fdb80 1 Processor -- start 2024-05-09T00:18:03.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f45306fdb80 1 -- start start 2024-05-09T00:18:03.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f45306fdb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e935160 0x562d8e9d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f45306fdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 0x562d8e9d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f45306fdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e9ca8b0 0x562d8e9cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f45306fdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d8e8eaff0 con 0x562d8e84b4a0 2024-05-09T00:18:03.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f45306fdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562d8e8897f0 con 0x562d8e9ca8b0 2024-05-09T00:18:03.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f45306fdb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562d8e8e6fb0 con 0x562d8e935160 2024-05-09T00:18:03.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f451b7fe700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f45080b2880 0x7f45080b4d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f450c005950 tx=0x7f450c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f4512ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 0x562d8e9d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f4512ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 0x562d8e9d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56346/0 (socket says 172.21.15.46:56346) 2024-05-09T00:18:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.093+0000 7f4512ffd700 1 -- 172.21.15.46:0/2086689702 learned_addr learned my addr 172.21.15.46:0/2086689702 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.094+0000 7f451bfff700 1 --2- 172.21.15.46:0/2086689702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e9ca8b0 0x562d8e9cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.094+0000 7f451b7fe700 1 --2- 172.21.15.46:0/2086689702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e935160 0x562d8e9d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.094+0000 7f4512ffd700 1 -- 172.21.15.46:0/2086689702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e935160 msgr2=0x562d8e9d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.094+0000 7f4512ffd700 1 --2- 172.21.15.46:0/2086689702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e935160 0x562d8e9d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.094+0000 7f4512ffd700 1 -- 172.21.15.46:0/2086689702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e9ca8b0 msgr2=0x562d8e9cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.094+0000 7f4512ffd700 1 --2- 172.21.15.46:0/2086689702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e9ca8b0 0x562d8e9cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.094+0000 7f4512ffd700 1 -- 172.21.15.46:0/2086689702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d8e881370 con 0x562d8e84b4a0 2024-05-09T00:18:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.094+0000 7f4512ffd700 1 --2- 172.21.15.46:0/2086689702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 0x562d8e9d0c10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f45040a91c0 tx=0x7f45040a8a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3029d4a1d3e71188 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.094+0000 7f4511ffb700 1 -- 172.21.15.46:0/2086689702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504020040 con 0x562d8e84b4a0 2024-05-09T00:18:03.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.094+0000 7f4511ffb700 1 -- 172.21.15.46:0/2086689702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45040a9930 con 0x562d8e84b4a0 2024-05-09T00:18:03.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.094+0000 7f4511ffb700 1 -- 172.21.15.46:0/2086689702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45040b1e60 con 0x562d8e84b4a0 2024-05-09T00:18:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.095+0000 7f45306fdb80 1 -- 172.21.15.46:0/2086689702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 msgr2=0x562d8e9d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.095+0000 7f45306fdb80 1 --2- 172.21.15.46:0/2086689702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 0x562d8e9d0c10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f45040a91c0 tx=0x7f45040a8a20 comp rx=0 tx=0).stop 2024-05-09T00:18:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.095+0000 7f45306fdb80 1 -- 172.21.15.46:0/2086689702 shutdown_connections 2024-05-09T00:18:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.095+0000 7f45306fdb80 1 --2- 172.21.15.46:0/2086689702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e9ca8b0 0x562d8e9cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.095+0000 7f45306fdb80 1 --2- 172.21.15.46:0/2086689702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e935160 0x562d8e9d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.095+0000 7f45306fdb80 1 --2- 172.21.15.46:0/2086689702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 0x562d8e9d0c10 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.095+0000 7f45306fdb80 1 -- 172.21.15.46:0/2086689702 >> 172.21.15.46:0/2086689702 conn(0x562d8e8ab6a0 msgr2=0x562d8e9c3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.095+0000 7f45306fdb80 1 -- 172.21.15.46:0/2086689702 shutdown_connections 2024-05-09T00:18:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.095+0000 7f45306fdb80 1 -- 172.21.15.46:0/2086689702 wait complete. 2024-05-09T00:18:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.096+0000 7f45306fdb80 1 -- start start 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.096+0000 7f45306fdb80 1 Processor -- start 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.096+0000 7f45306fdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e84b4a0 0x562d8e9d4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.096+0000 7f45306fdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e935160 0x562d8e9d4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.096+0000 7f45306fdb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e9ca8b0 0x562d8e9d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.096+0000 7f45306fdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d8e766ba0 con 0x562d8e935160 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.096+0000 7f45306fdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562d8e8a5150 con 0x562d8e84b4a0 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.096+0000 7f45306fdb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562d8e9cb600 con 0x562d8e9ca8b0 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.096+0000 7f45306fdb80 10 client.0 _ll_register_callbacks cb 0x562d8e75c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.096+0000 7f4512ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e935160 0x562d8e9d4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f4512ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e935160 0x562d8e9d4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56358/0 (socket says 172.21.15.46:56358) 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f4512ffd700 1 -- 172.21.15.46:0/2864639911 learned_addr learned my addr 172.21.15.46:0/2864639911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f451b7fe700 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e84b4a0 0x562d8e9d4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f451bfff700 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e9ca8b0 0x562d8e9d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f4512ffd700 1 -- 172.21.15.46:0/2864639911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e9ca8b0 msgr2=0x562d8e9d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f4512ffd700 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e9ca8b0 0x562d8e9d4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f4512ffd700 1 -- 172.21.15.46:0/2864639911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e84b4a0 msgr2=0x562d8e9d4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f4512ffd700 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e84b4a0 0x562d8e9d4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f4512ffd700 1 -- 172.21.15.46:0/2864639911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d8e766490 con 0x562d8e935160 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f4512ffd700 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e935160 0x562d8e9d4990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4504010960 tx=0x7f45040dff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7bfe2d9aea66b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f44faffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f44faffd700 1 -- 172.21.15.46:0/2864639911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45040c2020 con 0x562d8e935160 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f44faffd700 1 -- 172.21.15.46:0/2864639911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f45040b1e10 con 0x562d8e935160 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f45306fdb80 1 -- 172.21.15.46:0/2864639911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45040a9930 con 0x562d8e935160 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f44faffd700 1 -- 172.21.15.46:0/2864639911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4504020030 con 0x562d8e935160 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f45306fdb80 10 client.16377 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.097+0000 7f45306fdb80 1 -- 172.21.15.46:0/2864639911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d8e9e1ca0 con 0x562d8e935160 2024-05-09T00:18:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.098+0000 7f44faffd700 1 -- 172.21.15.46:0/2864639911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45040a9930 con 0x562d8e935160 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.098+0000 7f44faffd700 1 -- 172.21.15.46:0/2864639911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f45040a8490 con 0x562d8e935160 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.098+0000 7f45306fdb80 10 client.16377 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.098+0000 7f45306fdb80 10 client.16377 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.098+0000 7f45306fdb80 20 client.16377 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.098+0000 7f45306fdb80 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d8e9e4170 0x562d8e9e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.098+0000 7f45306fdb80 4 client.16377 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.098+0000 7f45306fdb80 1 -- 172.21.15.46:0/2864639911 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d8e8f89d0 con 0x562d8e9e4170 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.098+0000 7f451b7fe700 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d8e9e4170 0x562d8e9e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.099+0000 7f451b7fe700 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d8e9e4170 0x562d8e9e6590 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74d494f78daf7620 server_cookie=88705d5af042e4cd in_seq=0 out_seq=0 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.099+0000 7f44faffd700 10 client.16377 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f44faffd700 1 -- 172.21.15.46:0/2864639911 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562d8e8f89d0 con 0x562d8e9e4170 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f44faffd700 10 client.16377 handle_command_reply: tid=0 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f45306fdb80 1 client.16377 shutdown 2024-05-09T00:18:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f45306fdb80 1 -- 172.21.15.46:0/2864639911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e935160 msgr2=0x562d8e9d4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f45306fdb80 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e935160 0x562d8e9d4990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4504010960 tx=0x7f45040dff00 comp rx=0 tx=0).stop 2024-05-09T00:18:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f45306fdb80 1 -- 172.21.15.46:0/2864639911 shutdown_connections 2024-05-09T00:18:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f45306fdb80 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d8e9e4170 0x562d8e9e6590 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f44faffd700 0 client.16377 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:18:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f45306fdb80 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e9ca8b0 0x562d8e9d4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f45306fdb80 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e84b4a0 0x562d8e9d4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f45306fdb80 1 --2- 172.21.15.46:0/2864639911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e935160 0x562d8e9d4990 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f45306fdb80 1 -- 172.21.15.46:0/2864639911 >> 172.21.15.46:0/2864639911 conn(0x562d8e8ab6a0 msgr2=0x562d8e9c31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f45306fdb80 1 -- 172.21.15.46:0/2864639911 shutdown_connections 2024-05-09T00:18:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f45306fdb80 1 -- 172.21.15.46:0/2864639911 wait complete. 2024-05-09T00:18:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.100+0000 7f45306fdb80 20 client.16377 trim_cache size 0 max 16384 2024-05-09T00:18:03.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:02 smithi169 ceph-mon[34777]: pgmap v576: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:18:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.115+0000 7f45306fdb80 1 Processor -- start 2024-05-09T00:18:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f45306fdb80 1 -- start start 2024-05-09T00:18:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f45306fdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e84b4a0 0x562d8e9d68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f45306fdb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e935160 0x562d8e9e95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f45306fdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e9ca8b0 0x562d8e9e9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f45306fdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f451c1af3f0 con 0x562d8e9ca8b0 2024-05-09T00:18:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f45306fdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f451c1acc80 con 0x562d8e84b4a0 2024-05-09T00:18:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f45306fdb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f451c1aaab0 con 0x562d8e935160 2024-05-09T00:18:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f451bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e9ca8b0 0x562d8e9e9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f451bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e9ca8b0 0x562d8e9e9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56368/0 (socket says 172.21.15.46:56368) 2024-05-09T00:18:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f451bfff700 1 -- 172.21.15.46:0/3462275717 learned_addr learned my addr 172.21.15.46:0/3462275717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f4512ffd700 1 --2- 172.21.15.46:0/3462275717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e935160 0x562d8e9e95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f451b7fe700 1 --2- 172.21.15.46:0/3462275717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e84b4a0 0x562d8e9d68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f451bfff700 1 -- 172.21.15.46:0/3462275717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e935160 msgr2=0x562d8e9e95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f451bfff700 1 --2- 172.21.15.46:0/3462275717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e935160 0x562d8e9e95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f451bfff700 1 -- 172.21.15.46:0/3462275717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e84b4a0 msgr2=0x562d8e9d68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f451bfff700 1 --2- 172.21.15.46:0/3462275717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e84b4a0 0x562d8e9d68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.116+0000 7f451bfff700 1 -- 172.21.15.46:0/3462275717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d8e9ea030 con 0x562d8e9ca8b0 2024-05-09T00:18:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.117+0000 7f451bfff700 1 --2- 172.21.15.46:0/3462275717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e9ca8b0 0x562d8e9e9af0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4514015000 tx=0x7f4514009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ba440744ac11d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:03.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.117+0000 7f45117fa700 1 -- 172.21.15.46:0/3462275717 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451402a070 con 0x562d8e9ca8b0 2024-05-09T00:18:03.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.117+0000 7f45117fa700 1 -- 172.21.15.46:0/3462275717 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45140075c0 con 0x562d8e9ca8b0 2024-05-09T00:18:03.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.117+0000 7f45117fa700 1 -- 172.21.15.46:0/3462275717 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514005320 con 0x562d8e9ca8b0 2024-05-09T00:18:03.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.117+0000 7f45306fdb80 1 -- 172.21.15.46:0/3462275717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e9ca8b0 msgr2=0x562d8e9e9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.117+0000 7f45306fdb80 1 --2- 172.21.15.46:0/3462275717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e9ca8b0 0x562d8e9e9af0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4514015000 tx=0x7f4514009350 comp rx=0 tx=0).stop 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.118+0000 7f45306fdb80 1 -- 172.21.15.46:0/3462275717 shutdown_connections 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.118+0000 7f45306fdb80 1 --2- 172.21.15.46:0/3462275717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e9ca8b0 0x562d8e9e9af0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.118+0000 7f45306fdb80 1 --2- 172.21.15.46:0/3462275717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e84b4a0 0x562d8e9d68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.118+0000 7f45306fdb80 1 --2- 172.21.15.46:0/3462275717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e935160 0x562d8e9e95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.118+0000 7f45306fdb80 1 -- 172.21.15.46:0/3462275717 >> 172.21.15.46:0/3462275717 conn(0x562d8e8ab6a0 msgr2=0x562d8e9d4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.118+0000 7f45306fdb80 1 -- 172.21.15.46:0/3462275717 shutdown_connections 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.118+0000 7f45306fdb80 1 -- 172.21.15.46:0/3462275717 wait complete. 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.118+0000 7f45306fdb80 1 -- start start 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.118+0000 7f45306fdb80 1 Processor -- start 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f45306fdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 0x562d8e9e8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f45306fdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e935160 0x562d8e9e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f45306fdb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e9ca8b0 0x562d8ea4c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f45306fdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f451c1ab140 con 0x562d8e84b4a0 2024-05-09T00:18:03.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f45306fdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f451c1ac5b0 con 0x562d8e935160 2024-05-09T00:18:03.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f45306fdb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f451c1ab810 con 0x562d8e9ca8b0 2024-05-09T00:18:03.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f45306fdb80 10 client.0 _ll_register_callbacks cb 0x562d8e75c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:03.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f451b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 0x562d8e9e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f451b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 0x562d8e9e8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56382/0 (socket says 172.21.15.46:56382) 2024-05-09T00:18:03.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f451b7fe700 1 -- 172.21.15.46:0/1346433816 learned_addr learned my addr 172.21.15.46:0/1346433816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:03.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f451bfff700 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e9ca8b0 0x562d8ea4c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f4512ffd700 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e935160 0x562d8e9e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f451b7fe700 1 -- 172.21.15.46:0/1346433816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e9ca8b0 msgr2=0x562d8ea4c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:03.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f451b7fe700 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e9ca8b0 0x562d8ea4c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f451b7fe700 1 -- 172.21.15.46:0/1346433816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e935160 msgr2=0x562d8e9e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:03.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f451b7fe700 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e935160 0x562d8e9e8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f451b7fe700 1 -- 172.21.15.46:0/1346433816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d8ea4cc00 con 0x562d8e84b4a0 2024-05-09T00:18:03.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f451b7fe700 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 0x562d8e9e8690 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f450c004ce0 tx=0x7f450c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af6e547c2b28c89 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:03.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f44faffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:03.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f44faffd700 1 -- 172.21.15.46:0/1346433816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c016070 con 0x562d8e84b4a0 2024-05-09T00:18:03.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f44faffd700 1 -- 172.21.15.46:0/1346433816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f450c028380 con 0x562d8e84b4a0 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.119+0000 7f45306fdb80 1 -- 172.21.15.46:0/1346433816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45140075c0 con 0x562d8e84b4a0 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.120+0000 7f44faffd700 1 -- 172.21.15.46:0/1346433816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f450c028580 con 0x562d8e84b4a0 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.120+0000 7f45306fdb80 10 client.16383 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.120+0000 7f45306fdb80 1 -- 172.21.15.46:0/1346433816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d8ea4dde0 con 0x562d8e84b4a0 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.120+0000 7f44faffd700 1 -- 172.21.15.46:0/1346433816 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45140075c0 con 0x562d8e84b4a0 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.120+0000 7f44faffd700 1 -- 172.21.15.46:0/1346433816 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f450c010a30 con 0x562d8e84b4a0 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.120+0000 7f45306fdb80 10 client.16383 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.120+0000 7f45306fdb80 10 client.16383 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.120+0000 7f45306fdb80 20 client.16383 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.120+0000 7f45306fdb80 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d8e9e4170 0x562d8ea50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.120+0000 7f45306fdb80 4 client.16383 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.120+0000 7f45306fdb80 1 -- 172.21.15.46:0/1346433816 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d8e8f2a70 con 0x562d8e9e4170 2024-05-09T00:18:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.120+0000 7f4512ffd700 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d8e9e4170 0x562d8ea50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f4512ffd700 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d8e9e4170 0x562d8ea50560 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0436d8cb48ff63a server_cookie=fba031fe35e7834f in_seq=0 out_seq=0 2024-05-09T00:18:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f44faffd700 10 client.16383 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:18:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f44faffd700 1 -- 172.21.15.46:0/1346433816 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d8e8f2a70 con 0x562d8e9e4170 2024-05-09T00:18:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f44faffd700 10 client.16383 handle_command_reply: tid=0 2024-05-09T00:18:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f45306fdb80 1 client.16383 shutdown 2024-05-09T00:18:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f45306fdb80 1 -- 172.21.15.46:0/1346433816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 msgr2=0x562d8e9e8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f45306fdb80 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 0x562d8e9e8690 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f450c004ce0 tx=0x7f450c0047c0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f45306fdb80 1 -- 172.21.15.46:0/1346433816 shutdown_connections 2024-05-09T00:18:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f45306fdb80 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d8e9e4170 0x562d8ea50560 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f45306fdb80 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d8e9ca8b0 0x562d8ea4c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f45306fdb80 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d8e84b4a0 0x562d8e9e8690 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f44faffd700 0 client.16383 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:18:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f45306fdb80 1 --2- 172.21.15.46:0/1346433816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d8e935160 0x562d8e9e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f45306fdb80 1 -- 172.21.15.46:0/1346433816 >> 172.21.15.46:0/1346433816 conn(0x562d8e8ab6a0 msgr2=0x562d8e9d4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f45306fdb80 1 -- 172.21.15.46:0/1346433816 shutdown_connections 2024-05-09T00:18:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f45306fdb80 1 -- 172.21.15.46:0/1346433816 wait complete. 2024-05-09T00:18:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.121+0000 7f45306fdb80 20 client.16383 trim_cache size 0 max 16384 2024-05-09T00:18:03.128 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:18:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.122+0000 7f4521ce7700 1 -- 172.21.15.46:0/2805252593 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f45080b2880 msgr2=0x7f45080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.122+0000 7f4521ce7700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f45080b2880 0x7f45080b4d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f450c005950 tx=0x7f450c00a300 comp rx=0 tx=0).stop 2024-05-09T00:18:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.122+0000 7f4521ce7700 1 -- 172.21.15.46:0/2805252593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f451c107df0 msgr2=0x7f451c19a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.122+0000 7f4521ce7700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f451c107df0 0x7f451c19a2b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f450400d980 tx=0x7f450400dc90 comp rx=0 tx=0).stop 2024-05-09T00:18:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.122+0000 7f4521ce7700 1 -- 172.21.15.46:0/2805252593 shutdown_connections 2024-05-09T00:18:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.122+0000 7f4521ce7700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f45080b2880 0x7f45080b4d90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.122+0000 7f4521ce7700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f451c108740 0x7f451c19e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.122+0000 7f4521ce7700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f451c1034c0 0x7f451c199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.122+0000 7f4521ce7700 1 --2- 172.21.15.46:0/2805252593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f451c107df0 0x7f451c19a2b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.122+0000 7f4521ce7700 1 -- 172.21.15.46:0/2805252593 >> 172.21.15.46:0/2805252593 conn(0x7f451c0fedc0 msgr2=0x7f451c107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.122+0000 7f4521ce7700 1 -- 172.21.15.46:0/2805252593 shutdown_connections 2024-05-09T00:18:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:03.122+0000 7f4521ce7700 1 -- 172.21.15.46:0/2805252593 wait complete. 2024-05-09T00:18:03.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:18:04.113 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:18:04.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:04 smithi046 ceph-mon[30151]: pgmap v577: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:05.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:04 smithi169 ceph-mon[34777]: pgmap v577: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:05.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:04 smithi142 ceph-mon[34533]: pgmap v577: 97 pgs: 97 active+clean; 3.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.994+0000 7feb0d5e5700 1 -- 172.21.15.46:0/2895559311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb08109860 msgr2=0x7feb081111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.994+0000 7feb0d5e5700 1 --2- 172.21.15.46:0/2895559311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb08109860 0x7feb081111c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7feb0000b0d0 tx=0x7feb0000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:18:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.994+0000 7feb0d5e5700 1 -- 172.21.15.46:0/2895559311 shutdown_connections 2024-05-09T00:18:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.994+0000 7feb0d5e5700 1 --2- 172.21.15.46:0/2895559311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb08109860 0x7feb081111c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.994+0000 7feb0d5e5700 1 --2- 172.21.15.46:0/2895559311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb081045e0 0x7feb081049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.994+0000 7feb0d5e5700 1 --2- 172.21.15.46:0/2895559311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb08108f10 0x7feb08109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.994+0000 7feb0d5e5700 1 -- 172.21.15.46:0/2895559311 >> 172.21.15.46:0/2895559311 conn(0x7feb080ffed0 msgr2=0x7feb08102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.994+0000 7feb0d5e5700 1 -- 172.21.15.46:0/2895559311 shutdown_connections 2024-05-09T00:18:05.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.994+0000 7feb0d5e5700 1 -- 172.21.15.46:0/2895559311 wait complete. 2024-05-09T00:18:05.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.995+0000 7feb0d5e5700 1 Processor -- start 2024-05-09T00:18:05.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.995+0000 7feb0d5e5700 1 -- start start 2024-05-09T00:18:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb0d5e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb081045e0 0x7feb08076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb0d5e5700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb08108f10 0x7feb08077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb0d5e5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb08109860 0x7feb08077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb0d5e5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb08114510 con 0x7feb081045e0 2024-05-09T00:18:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb0d5e5700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb08114390 con 0x7feb08108f10 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb0d5e5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feb08114690 con 0x7feb08109860 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb06ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb081045e0 0x7feb08076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb06ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb081045e0 0x7feb08076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56414/0 (socket says 172.21.15.46:56414) 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb06ffd700 1 -- 172.21.15.46:0/3316807634 learned_addr learned my addr 172.21.15.46:0/3316807634 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb077fe700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb08109860 0x7feb08077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb067fc700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb08108f10 0x7feb08077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb06ffd700 1 -- 172.21.15.46:0/3316807634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb08109860 msgr2=0x7feb08077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb06ffd700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb08109860 0x7feb08077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb06ffd700 1 -- 172.21.15.46:0/3316807634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb08108f10 msgr2=0x7feb08077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb06ffd700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb08108f10 0x7feb08077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb06ffd700 1 -- 172.21.15.46:0/3316807634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb081a9240 con 0x7feb081045e0 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb067fc700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb08108f10 0x7feb08077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.996+0000 7feb06ffd700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb081045e0 0x7feb08076bd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7feaf800b530 tx=0x7feaf800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.997+0000 7feafffff700 1 -- 172.21.15.46:0/3316807634 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8015020 con 0x7feb081045e0 2024-05-09T00:18:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.997+0000 7feb0d5e5700 1 -- 172.21.15.46:0/3316807634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb081a9530 con 0x7feb081045e0 2024-05-09T00:18:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.997+0000 7feafffff700 1 -- 172.21.15.46:0/3316807634 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf8011ce0 con 0x7feb081045e0 2024-05-09T00:18:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.997+0000 7feafffff700 1 -- 172.21.15.46:0/3316807634 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8010d20 con 0x7feb081045e0 2024-05-09T00:18:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:05.997+0000 7feb0d5e5700 1 -- 172.21.15.46:0/3316807634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb081a9a70 con 0x7feb081045e0 2024-05-09T00:18:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.000+0000 7feafffff700 1 -- 172.21.15.46:0/3316807634 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7feaf8018020 con 0x7feb081045e0 2024-05-09T00:18:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.000+0000 7feafffff700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7feaf40b2880 0x7feaf40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.000+0000 7feafffff700 1 -- 172.21.15.46:0/3316807634 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7feaf80e1a10 con 0x7feb081045e0 2024-05-09T00:18:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.000+0000 7feb067fc700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7feaf40b2880 0x7feaf40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.000+0000 7feb1bffbb80 1 Processor -- start 2024-05-09T00:18:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb1bffbb80 1 -- start start 2024-05-09T00:18:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb067fc700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7feaf40b2880 0x7feaf40b4d90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005fd0 tx=0x7feaf0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb1bffbb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876dfb160 0x564876e944f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb1bffbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 0x564876e94a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb1bffbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876e8e6a0 0x564876e8ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb1bffbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564876db0ff0 con 0x564876d114a0 2024-05-09T00:18:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb1bffbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564876d4f7f0 con 0x564876e8e6a0 2024-05-09T00:18:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb1bffbb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564876dacfb0 con 0x564876dfb160 2024-05-09T00:18:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb067fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 0x564876e94a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb067fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 0x564876e94a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56424/0 (socket says 172.21.15.46:56424) 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb067fc700 1 -- 172.21.15.46:0/3329269168 learned_addr learned my addr 172.21.15.46:0/3329269168 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb06ffd700 1 --2- 172.21.15.46:0/3329269168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876dfb160 0x564876e944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.001+0000 7feb077fe700 1 --2- 172.21.15.46:0/3329269168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876e8e6a0 0x564876e8ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.002+0000 7feb067fc700 1 -- 172.21.15.46:0/3329269168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876dfb160 msgr2=0x564876e944f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.002+0000 7feb067fc700 1 --2- 172.21.15.46:0/3329269168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876dfb160 0x564876e944f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.002+0000 7feb067fc700 1 -- 172.21.15.46:0/3329269168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876e8e6a0 msgr2=0x564876e8ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.002+0000 7feb067fc700 1 --2- 172.21.15.46:0/3329269168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876e8e6a0 0x564876e8ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.002+0000 7feb067fc700 1 -- 172.21.15.46:0/3329269168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564876d47370 con 0x564876d114a0 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.002+0000 7feb06ffd700 1 --2- 172.21.15.46:0/3329269168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876dfb160 0x564876e944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.002+0000 7feb067fc700 1 --2- 172.21.15.46:0/3329269168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 0x564876e94a30 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7feaf000db50 tx=0x7feaf0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc599140e73ba1e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.002+0000 7feafd7fa700 1 -- 172.21.15.46:0/3329269168 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0020070 con 0x564876d114a0 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.002+0000 7feafd7fa700 1 -- 172.21.15.46:0/3329269168 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf000fd60 con 0x564876d114a0 2024-05-09T00:18:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.002+0000 7feafd7fa700 1 -- 172.21.15.46:0/3329269168 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf001b3b0 con 0x564876d114a0 2024-05-09T00:18:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.003+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3329269168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 msgr2=0x564876e94a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.003+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/3329269168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 0x564876e94a30 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7feaf000db50 tx=0x7feaf0010000 comp rx=0 tx=0).stop 2024-05-09T00:18:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.003+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3329269168 shutdown_connections 2024-05-09T00:18:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.003+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/3329269168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876e8e6a0 0x564876e8ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.003+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/3329269168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876dfb160 0x564876e944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.003+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/3329269168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 0x564876e94a30 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.003+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3329269168 >> 172.21.15.46:0/3329269168 conn(0x564876d716a0 msgr2=0x564876e89400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.003+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3329269168 shutdown_connections 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.003+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3329269168 wait complete. 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb1bffbb80 1 -- start start 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb1bffbb80 1 Processor -- start 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb1bffbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876d114a0 0x564876e98230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb1bffbb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876dfb160 0x564876e98770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb1bffbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 0x564876e98cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb1bffbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564876c2cba0 con 0x564876e8e6a0 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb1bffbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564876d6b150 con 0x564876d114a0 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb1bffbb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564876e9b1d0 con 0x564876dfb160 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb1bffbb80 10 client.0 _ll_register_callbacks cb 0x564876c226c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 0x564876e98cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 0x564876e98cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56432/0 (socket says 172.21.15.46:56432) 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb077fe700 1 -- 172.21.15.46:0/3237332541 learned_addr learned my addr 172.21.15.46:0/3237332541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb06ffd700 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876d114a0 0x564876e98230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb067fc700 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876dfb160 0x564876e98770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb077fe700 1 -- 172.21.15.46:0/3237332541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876dfb160 msgr2=0x564876e98770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb077fe700 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876dfb160 0x564876e98770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb077fe700 1 -- 172.21.15.46:0/3237332541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876d114a0 msgr2=0x564876e98230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb077fe700 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876d114a0 0x564876e98230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.004+0000 7feb077fe700 1 -- 172.21.15.46:0/3237332541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564876c2c490 con 0x564876e8e6a0 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.005+0000 7feb067fc700 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876dfb160 0x564876e98770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.005+0000 7feb077fe700 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 0x564876e98cb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7feb00004c80 tx=0x7feb00004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fe0638a09d6434 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.005+0000 7feaee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.005+0000 7feaee7fc700 1 -- 172.21.15.46:0/3237332541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00021070 con 0x564876e8e6a0 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.005+0000 7feaee7fc700 1 -- 172.21.15.46:0/3237332541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7feb000077c0 con 0x564876e8e6a0 2024-05-09T00:18:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.005+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3237332541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf000fd60 con 0x564876e8e6a0 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.005+0000 7feaee7fc700 1 -- 172.21.15.46:0/3237332541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feb000079c0 con 0x564876e8e6a0 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.005+0000 7feb1bffbb80 10 client.16395 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.005+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3237332541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564876ea5c10 con 0x564876e8e6a0 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.005+0000 7feaee7fc700 1 -- 172.21.15.46:0/3237332541 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf000fd60 con 0x564876e8e6a0 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.005+0000 7feaee7fc700 1 -- 172.21.15.46:0/3237332541 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7feb00011070 con 0x564876e8e6a0 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.006+0000 7feb1bffbb80 10 client.16395 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.006+0000 7feb1bffbb80 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.006+0000 7feb1bffbb80 20 client.16395 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.006+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564876ea8250 0x564876eaa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.006+0000 7feb1bffbb80 4 client.16395 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.006+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3237332541 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564876dbe9d0 con 0x564876ea8250 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.006+0000 7feb06ffd700 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564876ea8250 0x564876eaa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.006+0000 7feb06ffd700 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564876ea8250 0x564876eaa670 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d6507ef324646b9 server_cookie=47cd2672bbfcd07 in_seq=0 out_seq=0 2024-05-09T00:18:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.006+0000 7feaee7fc700 10 client.16395 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feaee7fc700 1 -- 172.21.15.46:0/3237332541 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564876dbe9d0 con 0x564876ea8250 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feaee7fc700 10 client.16395 handle_command_reply: tid=0 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feb1bffbb80 1 client.16395 shutdown 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3237332541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 msgr2=0x564876e98cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 0x564876e98cb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7feb00004c80 tx=0x7feb00004cb0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3237332541 shutdown_connections 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564876ea8250 0x564876eaa670 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 0x564876e98cb0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876d114a0 0x564876e98230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/3237332541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876dfb160 0x564876e98770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feaee7fc700 0 client.16395 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3237332541 >> 172.21.15.46:0/3237332541 conn(0x564876d716a0 msgr2=0x564876e891c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.007+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3237332541 shutdown_connections 2024-05-09T00:18:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.008+0000 7feb1bffbb80 1 -- 172.21.15.46:0/3237332541 wait complete. 2024-05-09T00:18:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.008+0000 7feb1bffbb80 20 client.16395 trim_cache size 0 max 16384 2024-05-09T00:18:06.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.023+0000 7feb1bffbb80 1 Processor -- start 2024-05-09T00:18:06.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.023+0000 7feb1bffbb80 1 -- start start 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb1bffbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 0x564876e8dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb1bffbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876dfb160 0x564876ea4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb1bffbb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876e8e6a0 0x564876e92830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb1bffbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb081b36d0 con 0x564876d114a0 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb1bffbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb081b0f60 con 0x564876dfb160 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb1bffbb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feb081aed90 con 0x564876e8e6a0 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb06ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 0x564876e8dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb06ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 0x564876e8dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56448/0 (socket says 172.21.15.46:56448) 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb06ffd700 1 -- 172.21.15.46:0/774577569 learned_addr learned my addr 172.21.15.46:0/774577569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb06ffd700 1 -- 172.21.15.46:0/774577569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876e8e6a0 msgr2=0x564876e92830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb06ffd700 1 --2- 172.21.15.46:0/774577569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876e8e6a0 0x564876e92830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb06ffd700 1 -- 172.21.15.46:0/774577569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876dfb160 msgr2=0x564876ea4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb067fc700 1 --2- 172.21.15.46:0/774577569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876dfb160 0x564876ea4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb06ffd700 1 --2- 172.21.15.46:0/774577569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876dfb160 0x564876ea4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb06ffd700 1 -- 172.21.15.46:0/774577569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564876e9aad0 con 0x564876d114a0 2024-05-09T00:18:06.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feb06ffd700 1 --2- 172.21.15.46:0/774577569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 0x564876e8dd00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7feaf80df510 tx=0x7feaf80107c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4ccc3590b3bbcd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:06.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feafcff9700 1 -- 172.21.15.46:0/774577569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf806d070 con 0x564876d114a0 2024-05-09T00:18:06.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.024+0000 7feafcff9700 1 -- 172.21.15.46:0/774577569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf80ad3e0 con 0x564876d114a0 2024-05-09T00:18:06.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.025+0000 7feafcff9700 1 -- 172.21.15.46:0/774577569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf80b1c40 con 0x564876d114a0 2024-05-09T00:18:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.025+0000 7feb1bffbb80 1 -- 172.21.15.46:0/774577569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 msgr2=0x564876e8dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.025+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/774577569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 0x564876e8dd00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7feaf80df510 tx=0x7feaf80107c0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.025+0000 7feb1bffbb80 1 -- 172.21.15.46:0/774577569 shutdown_connections 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.025+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/774577569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876e8e6a0 0x564876e92830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.025+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/774577569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876d114a0 0x564876e8dd00 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.025+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/774577569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876dfb160 0x564876ea4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.025+0000 7feb1bffbb80 1 -- 172.21.15.46:0/774577569 >> 172.21.15.46:0/774577569 conn(0x564876d716a0 msgr2=0x564876e969f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.025+0000 7feb1bffbb80 1 -- 172.21.15.46:0/774577569 shutdown_connections 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.025+0000 7feb1bffbb80 1 -- 172.21.15.46:0/774577569 wait complete. 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb1bffbb80 1 -- start start 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb1bffbb80 1 Processor -- start 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb1bffbb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876d114a0 0x564876ea0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb1bffbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876dfb160 0x564876ea06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb1bffbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 0x564876e948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb1bffbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb081af420 con 0x564876e8e6a0 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb1bffbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb081b0890 con 0x564876dfb160 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb1bffbb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feb081afaf0 con 0x564876d114a0 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb1bffbb80 10 client.0 _ll_register_callbacks cb 0x564876c226c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 0x564876e948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb067fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876dfb160 0x564876ea06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 0x564876e948f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56456/0 (socket says 172.21.15.46:56456) 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.026+0000 7feb077fe700 1 -- 172.21.15.46:0/1645877422 learned_addr learned my addr 172.21.15.46:0/1645877422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feb06ffd700 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876d114a0 0x564876ea0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feb077fe700 1 -- 172.21.15.46:0/1645877422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876d114a0 msgr2=0x564876ea0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feb077fe700 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876d114a0 0x564876ea0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feb077fe700 1 -- 172.21.15.46:0/1645877422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876dfb160 msgr2=0x564876ea06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feb077fe700 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876dfb160 0x564876ea06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feb077fe700 1 -- 172.21.15.46:0/1645877422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564876e94e30 con 0x564876e8e6a0 2024-05-09T00:18:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feb06ffd700 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876d114a0 0x564876ea0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feb077fe700 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 0x564876e948f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7feb00002230 tx=0x7feb00007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5733827790a83d16 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feaee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feaee7fc700 1 -- 172.21.15.46:0/1645877422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00021070 con 0x564876e8e6a0 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feaee7fc700 1 -- 172.21.15.46:0/1645877422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7feb0000f040 con 0x564876e8e6a0 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feb1bffbb80 1 -- 172.21.15.46:0/1645877422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf80ad3e0 con 0x564876e8e6a0 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feaee7fc700 1 -- 172.21.15.46:0/1645877422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feb00007b60 con 0x564876e8e6a0 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feb1bffbb80 10 client.16401 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.027+0000 7feb1bffbb80 1 -- 172.21.15.46:0/1645877422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564876f12060 con 0x564876e8e6a0 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.028+0000 7feaee7fc700 1 -- 172.21.15.46:0/1645877422 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf80ad3e0 con 0x564876e8e6a0 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.028+0000 7feaee7fc700 1 -- 172.21.15.46:0/1645877422 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7feb00038030 con 0x564876e8e6a0 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.028+0000 7feb1bffbb80 10 client.16401 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.028+0000 7feb1bffbb80 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.028+0000 7feb1bffbb80 20 client.16401 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:06.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.028+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564876f12700 0x564876f14b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.028+0000 7feb1bffbb80 4 client.16401 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:18:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.028+0000 7feb1bffbb80 1 -- 172.21.15.46:0/1645877422 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564876db8a70 con 0x564876f12700 2024-05-09T00:18:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.028+0000 7feb06ffd700 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564876f12700 0x564876f14b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.028+0000 7feaee7fc700 10 client.16401 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.028+0000 7feb06ffd700 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564876f12700 0x564876f14b20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e08baed263a40c server_cookie=a68c720802f5c657 in_seq=0 out_seq=0 2024-05-09T00:18:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feaee7fc700 1 -- 172.21.15.46:0/1645877422 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564876db8a70 con 0x564876f12700 2024-05-09T00:18:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feaee7fc700 10 client.16401 handle_command_reply: tid=0 2024-05-09T00:18:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feb1bffbb80 1 client.16401 shutdown 2024-05-09T00:18:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feb1bffbb80 1 -- 172.21.15.46:0/1645877422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 msgr2=0x564876e948f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 0x564876e948f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7feb00002230 tx=0x7feb00007850 comp rx=0 tx=0).stop 2024-05-09T00:18:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feb1bffbb80 1 -- 172.21.15.46:0/1645877422 shutdown_connections 2024-05-09T00:18:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564876f12700 0x564876f14b20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564876e8e6a0 0x564876e948f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564876d114a0 0x564876ea0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feb1bffbb80 1 --2- 172.21.15.46:0/1645877422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564876dfb160 0x564876ea06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feb1bffbb80 1 -- 172.21.15.46:0/1645877422 >> 172.21.15.46:0/1645877422 conn(0x564876d716a0 msgr2=0x564876e97180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feaee7fc700 0 client.16401 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feb1bffbb80 1 -- 172.21.15.46:0/1645877422 shutdown_connections 2024-05-09T00:18:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feb1bffbb80 1 -- 172.21.15.46:0/1645877422 wait complete. 2024-05-09T00:18:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.029+0000 7feb1bffbb80 20 client.16401 trim_cache size 0 max 16384 2024-05-09T00:18:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:18:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.030+0000 7feb0d5e5700 1 -- 172.21.15.46:0/3316807634 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7feaf40b2880 msgr2=0x7feaf40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.030+0000 7feb0d5e5700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7feaf40b2880 0x7feaf40b4d90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005fd0 tx=0x7feaf0005dc0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.030+0000 7feb0d5e5700 1 -- 172.21.15.46:0/3316807634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb081045e0 msgr2=0x7feb08076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.030+0000 7feb0d5e5700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb081045e0 0x7feb08076bd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7feaf800b530 tx=0x7feaf800b8f0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.030+0000 7feb0d5e5700 1 -- 172.21.15.46:0/3316807634 shutdown_connections 2024-05-09T00:18:06.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.030+0000 7feb0d5e5700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7feaf40b2880 0x7feaf40b4d90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.030+0000 7feb0d5e5700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb08109860 0x7feb08077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.030+0000 7feb0d5e5700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb081045e0 0x7feb08076bd0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.030+0000 7feb0d5e5700 1 --2- 172.21.15.46:0/3316807634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb08108f10 0x7feb08077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:06.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.030+0000 7feb0d5e5700 1 -- 172.21.15.46:0/3316807634 >> 172.21.15.46:0/3316807634 conn(0x7feb080ffed0 msgr2=0x7feb08108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:06.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.030+0000 7feb0d5e5700 1 -- 172.21.15.46:0/3316807634 shutdown_connections 2024-05-09T00:18:06.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:06.030+0000 7feb0d5e5700 1 -- 172.21.15.46:0/3316807634 wait complete. 2024-05-09T00:18:07.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:06 smithi046 ceph-mon[30151]: pgmap v578: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:18:07.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:06 smithi169 ceph-mon[34777]: pgmap v578: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:18:07.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:06 smithi142 ceph-mon[34533]: pgmap v578: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:18:07.881 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:18:07.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:18:08.148 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:18:08.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:08 smithi046 ceph-mon[30151]: pgmap v579: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:09.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:08 smithi169 ceph-mon[34777]: pgmap v579: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:09.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:08 smithi142 ceph-mon[34533]: pgmap v579: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:09.999+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3388860913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7f01045d0 msgr2=0x7fe7f01049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:09.999+0000 7fe7f50b6700 1 --2- 172.21.15.46:0/3388860913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7f01045d0 0x7fe7f01049e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00099c0 tx=0x7fe7e0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:18:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:09.999+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3388860913 shutdown_connections 2024-05-09T00:18:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:09.999+0000 7fe7f50b6700 1 --2- 172.21.15.46:0/3388860913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7f0109850 0x7fe7f01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:09.999+0000 7fe7f50b6700 1 --2- 172.21.15.46:0/3388860913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7f01045d0 0x7fe7f01049e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:09.999+0000 7fe7f50b6700 1 --2- 172.21.15.46:0/3388860913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7f0108f00 0x7fe7f0109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:09.999+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3388860913 >> 172.21.15.46:0/3388860913 conn(0x7fe7f00ffea0 msgr2=0x7fe7f01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:09.999+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3388860913 shutdown_connections 2024-05-09T00:18:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:09.999+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3388860913 wait complete. 2024-05-09T00:18:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.000+0000 7fe7f50b6700 1 Processor -- start 2024-05-09T00:18:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.000+0000 7fe7f50b6700 1 -- start start 2024-05-09T00:18:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.000+0000 7fe7f50b6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7f01045d0 0x7fe7f019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.000+0000 7fe7f50b6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7f0108f00 0x7fe7f019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.000+0000 7fe7f50b6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7f0109850 0x7fe7f019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.000+0000 7fe7f50b6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0114480 con 0x7fe7f01045d0 2024-05-09T00:18:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.000+0000 7fe7f50b6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0114300 con 0x7fe7f0109850 2024-05-09T00:18:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.000+0000 7fe7f50b6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0114600 con 0x7fe7f0108f00 2024-05-09T00:18:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.000+0000 7fe7eed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7f01045d0 0x7fe7f019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.000+0000 7fe7eed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7f01045d0 0x7fe7f019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39106/0 (socket says 172.21.15.46:39106) 2024-05-09T00:18:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.000+0000 7fe7eed9d700 1 -- 172.21.15.46:0/3811146755 learned_addr learned my addr 172.21.15.46:0/3811146755 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7ef59e700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7f0109850 0x7fe7f019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7f0108f00 0x7fe7f019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7eed9d700 1 -- 172.21.15.46:0/3811146755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7f0108f00 msgr2=0x7fe7f019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7eed9d700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7f0108f00 0x7fe7f019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7eed9d700 1 -- 172.21.15.46:0/3811146755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7f0109850 msgr2=0x7fe7f019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7eed9d700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7f0109850 0x7fe7f019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7eed9d700 1 -- 172.21.15.46:0/3811146755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f019fb90 con 0x7fe7f01045d0 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7f0108f00 0x7fe7f019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7eed9d700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7f01045d0 0x7fe7f019ad70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000b580 tx=0x7fe7e00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7d7fff700 1 -- 172.21.15.46:0/3811146755 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0027070 con 0x7fe7f01045d0 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3811146755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f019fe20 con 0x7fe7f01045d0 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7d7fff700 1 -- 172.21.15.46:0/3811146755 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e00040a0 con 0x7fe7f01045d0 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7d7fff700 1 -- 172.21.15.46:0/3811146755 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e000f460 con 0x7fe7f01045d0 2024-05-09T00:18:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.001+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3811146755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f01a0300 con 0x7fe7f01045d0 2024-05-09T00:18:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.004+0000 7fe7d7fff700 1 -- 172.21.15.46:0/3811146755 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe7e002a070 con 0x7fe7f01045d0 2024-05-09T00:18:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.004+0000 7fe7d7fff700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe7d80b2880 0x7fe7d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.005+0000 7fe7d7fff700 1 -- 172.21.15.46:0/3811146755 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe7e00e32a0 con 0x7fe7f01045d0 2024-05-09T00:18:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.005+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe7d80b2880 0x7fe7d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.005+0000 7fe803accb80 1 Processor -- start 2024-05-09T00:18:10.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.005+0000 7fe803accb80 1 -- start start 2024-05-09T00:18:10.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.005+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe7d80b2880 0x7fe7d80b4d90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4005fd0 tx=0x7fe7e4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:10.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.005+0000 7fe803accb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec60921e0 0x564ec612b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.005+0000 7fe803accb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec5fa8520 0x564ec612bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.005+0000 7fe803accb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6125720 0x564ec6125b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.005+0000 7fe803accb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ec6048070 con 0x564ec6125720 2024-05-09T00:18:10.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.005+0000 7fe803accb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564ec5fe6870 con 0x564ec60921e0 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.005+0000 7fe803accb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ec6044030 con 0x564ec5fa8520 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7ef59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6125720 0x564ec6125b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7ef59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6125720 0x564ec6125b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39116/0 (socket says 172.21.15.46:39116) 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7ef59e700 1 -- 172.21.15.46:0/2504414377 learned_addr learned my addr 172.21.15.46:0/2504414377 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/2504414377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec5fa8520 0x564ec612bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7eed9d700 1 --2- 172.21.15.46:0/2504414377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec60921e0 0x564ec612b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7ef59e700 1 -- 172.21.15.46:0/2504414377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec5fa8520 msgr2=0x564ec612bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7ef59e700 1 --2- 172.21.15.46:0/2504414377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec5fa8520 0x564ec612bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7ef59e700 1 -- 172.21.15.46:0/2504414377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec60921e0 msgr2=0x564ec612b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7ef59e700 1 --2- 172.21.15.46:0/2504414377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec60921e0 0x564ec612b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7ef59e700 1 -- 172.21.15.46:0/2504414377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec5fde3f0 con 0x564ec6125720 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7ef59e700 1 --2- 172.21.15.46:0/2504414377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6125720 0x564ec6125b50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800f410 tx=0x7fe7e800f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d852ea997607cb41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7d57fa700 1 -- 172.21.15.46:0/2504414377 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8018030 con 0x564ec6125720 2024-05-09T00:18:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7d57fa700 1 -- 172.21.15.46:0/2504414377 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e8004bb0 con 0x564ec6125720 2024-05-09T00:18:10.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.006+0000 7fe7d57fa700 1 -- 172.21.15.46:0/2504414377 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8005020 con 0x564ec6125720 2024-05-09T00:18:10.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.007+0000 7fe803accb80 1 -- 172.21.15.46:0/2504414377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6125720 msgr2=0x564ec6125b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:10.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.007+0000 7fe803accb80 1 --2- 172.21.15.46:0/2504414377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6125720 0x564ec6125b50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800f410 tx=0x7fe7e800f720 comp rx=0 tx=0).stop 2024-05-09T00:18:10.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.007+0000 7fe803accb80 1 -- 172.21.15.46:0/2504414377 shutdown_connections 2024-05-09T00:18:10.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.007+0000 7fe803accb80 1 --2- 172.21.15.46:0/2504414377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6125720 0x564ec6125b50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.007+0000 7fe803accb80 1 --2- 172.21.15.46:0/2504414377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec60921e0 0x564ec612b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.007+0000 7fe803accb80 1 --2- 172.21.15.46:0/2504414377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec5fa8520 0x564ec612bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.007+0000 7fe803accb80 1 -- 172.21.15.46:0/2504414377 >> 172.21.15.46:0/2504414377 conn(0x564ec6008720 msgr2=0x564ec6120480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:10.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.007+0000 7fe803accb80 1 -- 172.21.15.46:0/2504414377 shutdown_connections 2024-05-09T00:18:10.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.007+0000 7fe803accb80 1 -- 172.21.15.46:0/2504414377 wait complete. 2024-05-09T00:18:10.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe803accb80 1 -- start start 2024-05-09T00:18:10.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe803accb80 1 Processor -- start 2024-05-09T00:18:10.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe803accb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 0x564ec612f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe803accb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec60921e0 0x564ec612f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe803accb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec6125720 0x564ec612fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe803accb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ec5ec3c20 con 0x564ec5fa8520 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe803accb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564ec60021d0 con 0x564ec6125720 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe803accb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ec6132250 con 0x564ec60921e0 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe803accb80 10 client.0 _ll_register_callbacks cb 0x564ec5eb9740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe7eed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 0x564ec612f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe7eed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 0x564ec612f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39124/0 (socket says 172.21.15.46:39124) 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe7eed9d700 1 -- 172.21.15.46:0/2096249949 learned_addr learned my addr 172.21.15.46:0/2096249949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.008+0000 7fe7ef59e700 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec6125720 0x564ec612fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec60921e0 0x564ec612f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7eed9d700 1 -- 172.21.15.46:0/2096249949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec60921e0 msgr2=0x564ec612f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7eed9d700 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec60921e0 0x564ec612f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7eed9d700 1 -- 172.21.15.46:0/2096249949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec6125720 msgr2=0x564ec612fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7eed9d700 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec6125720 0x564ec612fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7eed9d700 1 -- 172.21.15.46:0/2096249949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec5ec3510 con 0x564ec5fa8520 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7ef59e700 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec6125720 0x564ec612fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7eed9d700 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 0x564ec612f270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00e0da0 tx=0x7fe7e0022900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5175e65860f5393d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7d2ff5700 1 -- 172.21.15.46:0/2096249949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00a60e0 con 0x564ec5fa8520 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7d2ff5700 1 -- 172.21.15.46:0/2096249949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe7e00b3290 con 0x564ec5fa8520 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe803accb80 1 -- 172.21.15.46:0/2096249949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e8004bb0 con 0x564ec5fa8520 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7d2ff5700 1 -- 172.21.15.46:0/2096249949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe7e00b3060 con 0x564ec5fa8520 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe803accb80 10 client.16413 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe803accb80 1 -- 172.21.15.46:0/2096249949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ec613cbb0 con 0x564ec5fa8520 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.009+0000 7fe7d2ff5700 1 -- 172.21.15.46:0/2096249949 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e8004bb0 con 0x564ec5fa8520 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.010+0000 7fe7d2ff5700 1 -- 172.21.15.46:0/2096249949 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe7e0071130 con 0x564ec5fa8520 2024-05-09T00:18:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.010+0000 7fe803accb80 10 client.16413 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.010+0000 7fe803accb80 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:18:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.010+0000 7fe803accb80 20 client.16413 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.010+0000 7fe803accb80 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ec613f120 0x564ec6141560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.010+0000 7fe803accb80 4 client.16413 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:18:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.010+0000 7fe803accb80 1 -- 172.21.15.46:0/2096249949 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ec6055a50 con 0x564ec613f120 2024-05-09T00:18:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.010+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ec613f120 0x564ec6141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.010+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ec613f120 0x564ec6141560 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857c9fe4dbda7a04 server_cookie=e190f676620ba88 in_seq=0 out_seq=0 2024-05-09T00:18:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.010+0000 7fe7d2ff5700 10 client.16413 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.011+0000 7fe7d2ff5700 1 -- 172.21.15.46:0/2096249949 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ec6055a50 con 0x564ec613f120 2024-05-09T00:18:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.011+0000 7fe7d2ff5700 10 client.16413 handle_command_reply: tid=0 2024-05-09T00:18:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.011+0000 7fe803accb80 1 client.16413 shutdown 2024-05-09T00:18:10.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.011+0000 7fe803accb80 1 -- 172.21.15.46:0/2096249949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 msgr2=0x564ec612f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:10.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.011+0000 7fe803accb80 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 0x564ec612f270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00e0da0 tx=0x7fe7e0022900 comp rx=0 tx=0).stop 2024-05-09T00:18:10.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.011+0000 7fe803accb80 1 -- 172.21.15.46:0/2096249949 shutdown_connections 2024-05-09T00:18:10.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.011+0000 7fe803accb80 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ec613f120 0x564ec6141560 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.011+0000 7fe803accb80 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec6125720 0x564ec612fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.011+0000 7fe803accb80 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 0x564ec612f270 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.011+0000 7fe803accb80 1 --2- 172.21.15.46:0/2096249949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec60921e0 0x564ec612f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.011+0000 7fe803accb80 1 -- 172.21.15.46:0/2096249949 >> 172.21.15.46:0/2096249949 conn(0x564ec6008720 msgr2=0x564ec6120240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:10.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.011+0000 7fe7d2ff5700 0 client.16413 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:10.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.012+0000 7fe803accb80 1 -- 172.21.15.46:0/2096249949 shutdown_connections 2024-05-09T00:18:10.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.012+0000 7fe803accb80 1 -- 172.21.15.46:0/2096249949 wait complete. 2024-05-09T00:18:10.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.012+0000 7fe803accb80 20 client.16413 trim_cache size 0 max 16384 2024-05-09T00:18:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe803accb80 1 Processor -- start 2024-05-09T00:18:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe803accb80 1 -- start start 2024-05-09T00:18:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe803accb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec6008720 0x564ec61298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe803accb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 0x564ec6131350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe803accb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec60921e0 0x564ec612fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe803accb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01b12c0 con 0x564ec5fa8520 2024-05-09T00:18:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe803accb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01aeb50 con 0x564ec6008720 2024-05-09T00:18:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe803accb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01ac980 con 0x564ec60921e0 2024-05-09T00:18:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe7ee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 0x564ec6131350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe7ee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 0x564ec6131350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39138/0 (socket says 172.21.15.46:39138) 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe7ee59c700 1 -- 172.21.15.46:0/3348815527 learned_addr learned my addr 172.21.15.46:0/3348815527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe7ef59e700 1 --2- 172.21.15.46:0/3348815527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec60921e0 0x564ec612fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe7ee59c700 1 -- 172.21.15.46:0/3348815527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec60921e0 msgr2=0x564ec612fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe7eed9d700 1 --2- 172.21.15.46:0/3348815527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec6008720 0x564ec61298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/3348815527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec60921e0 0x564ec612fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.033+0000 7fe7ee59c700 1 -- 172.21.15.46:0/3348815527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec6008720 msgr2=0x564ec61298e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.034+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/3348815527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec6008720 0x564ec61298e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.034+0000 7fe7ee59c700 1 -- 172.21.15.46:0/3348815527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec61260b0 con 0x564ec5fa8520 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.034+0000 7fe7eed9d700 1 --2- 172.21.15.46:0/3348815527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec6008720 0x564ec61298e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.034+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/3348815527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 0x564ec6131350 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0107860 tx=0x7fe7e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a632e6361686e547 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.034+0000 7fe7d4ff9700 1 -- 172.21.15.46:0/3348815527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e401ee20 con 0x564ec5fa8520 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.034+0000 7fe7d4ff9700 1 -- 172.21.15.46:0/3348815527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e4027770 con 0x564ec5fa8520 2024-05-09T00:18:10.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.034+0000 7fe7d4ff9700 1 -- 172.21.15.46:0/3348815527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4010c10 con 0x564ec5fa8520 2024-05-09T00:18:10.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.035+0000 7fe803accb80 1 -- 172.21.15.46:0/3348815527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 msgr2=0x564ec6131350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:10.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.035+0000 7fe803accb80 1 --2- 172.21.15.46:0/3348815527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 0x564ec6131350 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0107860 tx=0x7fe7e4014000 comp rx=0 tx=0).stop 2024-05-09T00:18:10.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.035+0000 7fe803accb80 1 -- 172.21.15.46:0/3348815527 shutdown_connections 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.035+0000 7fe803accb80 1 --2- 172.21.15.46:0/3348815527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec60921e0 0x564ec612fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.035+0000 7fe803accb80 1 --2- 172.21.15.46:0/3348815527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec6008720 0x564ec61298e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.035+0000 7fe803accb80 1 --2- 172.21.15.46:0/3348815527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec5fa8520 0x564ec6131350 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.035+0000 7fe803accb80 1 -- 172.21.15.46:0/3348815527 >> 172.21.15.46:0/3348815527 conn(0x564ec612bd30 msgr2=0x564ec6124d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.035+0000 7fe803accb80 1 -- 172.21.15.46:0/3348815527 shutdown_connections 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.035+0000 7fe803accb80 1 -- 172.21.15.46:0/3348815527 wait complete. 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.035+0000 7fe803accb80 1 -- start start 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.035+0000 7fe803accb80 1 Processor -- start 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe803accb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec5fa8520 0x564ec6132250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe803accb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6008720 0x564ec6132790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe803accb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec60921e0 0x564ec6132d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe803accb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01ad010 con 0x564ec6008720 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe803accb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01ae480 con 0x564ec60921e0 2024-05-09T00:18:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe803accb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01ad6e0 con 0x564ec5fa8520 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe803accb80 10 client.0 _ll_register_callbacks cb 0x564ec5eb9740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe7ee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6008720 0x564ec6132790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe7ee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6008720 0x564ec6132790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39146/0 (socket says 172.21.15.46:39146) 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe7ee59c700 1 -- 172.21.15.46:0/2112584962 learned_addr learned my addr 172.21.15.46:0/2112584962 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe7eed9d700 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec5fa8520 0x564ec6132250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe7ef59e700 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec60921e0 0x564ec6132d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe7ee59c700 1 -- 172.21.15.46:0/2112584962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec5fa8520 msgr2=0x564ec6132250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec5fa8520 0x564ec6132250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe7ee59c700 1 -- 172.21.15.46:0/2112584962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec60921e0 msgr2=0x564ec6132d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec60921e0 0x564ec6132d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe7ee59c700 1 -- 172.21.15.46:0/2112584962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec61a9b30 con 0x564ec6008720 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe7ee59c700 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6008720 0x564ec6132790 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4005ea0 tx=0x7fe7e40279d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91914a1d6ceee5e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.036+0000 7fe7d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe7d2ff5700 1 -- 172.21.15.46:0/2112584962 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e403c060 con 0x564ec6008720 2024-05-09T00:18:10.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe7d2ff5700 1 -- 172.21.15.46:0/2112584962 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe7e40274c0 con 0x564ec6008720 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe803accb80 1 -- 172.21.15.46:0/2112584962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e4027770 con 0x564ec6008720 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe7d2ff5700 1 -- 172.21.15.46:0/2112584962 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe7e40108b0 con 0x564ec6008720 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe803accb80 10 client.16419 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe803accb80 1 -- 172.21.15.46:0/2112584962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ec61aad40 con 0x564ec6008720 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe7d2ff5700 1 -- 172.21.15.46:0/2112584962 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e4027770 con 0x564ec6008720 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe7d2ff5700 1 -- 172.21.15.46:0/2112584962 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe7e4041070 con 0x564ec6008720 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe803accb80 10 client.16419 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe803accb80 10 client.16419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe803accb80 20 client.16419 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe803accb80 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ec6125720 0x564ec61ad380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe803accb80 4 client.16419 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.037+0000 7fe803accb80 1 -- 172.21.15.46:0/2112584962 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564ec604faf0 con 0x564ec6125720 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.038+0000 7fe7eed9d700 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ec6125720 0x564ec61ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.038+0000 7fe7d2ff5700 10 client.16419 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:10.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.038+0000 7fe7eed9d700 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ec6125720 0x564ec61ad380 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf3e203e12ad343 server_cookie=5796f591114d8f5b in_seq=0 out_seq=0 2024-05-09T00:18:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.039+0000 7fe7d2ff5700 1 -- 172.21.15.46:0/2112584962 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564ec604faf0 con 0x564ec6125720 2024-05-09T00:18:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.039+0000 7fe7d2ff5700 10 client.16419 handle_command_reply: tid=0 2024-05-09T00:18:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.039+0000 7fe803accb80 1 client.16419 shutdown 2024-05-09T00:18:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.039+0000 7fe803accb80 1 -- 172.21.15.46:0/2112584962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6008720 msgr2=0x564ec6132790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.039+0000 7fe803accb80 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6008720 0x564ec6132790 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4005ea0 tx=0x7fe7e40279d0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.039+0000 7fe803accb80 1 -- 172.21.15.46:0/2112584962 shutdown_connections 2024-05-09T00:18:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.039+0000 7fe803accb80 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x564ec6125720 0x564ec61ad380 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe803accb80 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ec60921e0 0x564ec6132d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe803accb80 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ec5fa8520 0x564ec6132250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe7d2ff5700 0 client.16419 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe803accb80 1 --2- 172.21.15.46:0/2112584962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec6008720 0x564ec6132790 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe803accb80 1 -- 172.21.15.46:0/2112584962 >> 172.21.15.46:0/2112584962 conn(0x564ec612bd30 msgr2=0x564ec6128e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:10.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe803accb80 1 -- 172.21.15.46:0/2112584962 shutdown_connections 2024-05-09T00:18:10.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe803accb80 1 -- 172.21.15.46:0/2112584962 wait complete. 2024-05-09T00:18:10.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe803accb80 20 client.16419 trim_cache size 0 max 16384 2024-05-09T00:18:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:18:10.043 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:18:10.043 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "780d3f98-b116-446a-970a-2f72ae18e645", 2024-05-09T00:18:10.043 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:18:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:18:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3811146755 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe7d80b2880 msgr2=0x7fe7d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe7f50b6700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe7d80b2880 0x7fe7d80b4d90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4005fd0 tx=0x7fe7e4005e30 comp rx=0 tx=0).stop 2024-05-09T00:18:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3811146755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7f01045d0 msgr2=0x7fe7f019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe7f50b6700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7f01045d0 0x7fe7f019ad70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000b580 tx=0x7fe7e00047b0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3811146755 shutdown_connections 2024-05-09T00:18:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe7f50b6700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe7d80b2880 0x7fe7d80b4d90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe7f50b6700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7f0109850 0x7fe7f019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe7f50b6700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7f01045d0 0x7fe7f019ad70 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe7f50b6700 1 --2- 172.21.15.46:0/3811146755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7f0108f00 0x7fe7f019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.040+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3811146755 >> 172.21.15.46:0/3811146755 conn(0x7fe7f00ffea0 msgr2=0x7fe7f0108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.041+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3811146755 shutdown_connections 2024-05-09T00:18:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:10.041+0000 7fe7f50b6700 1 -- 172.21.15.46:0/3811146755 wait complete. 2024-05-09T00:18:10.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:18:11.023 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:18:11.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:10 smithi046 ceph-mon[30151]: pgmap v580: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:11.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:10 smithi169 ceph-mon[34777]: pgmap v580: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:11.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:10 smithi142 ceph-mon[34533]: pgmap v580: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:12.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:11 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:18:12.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:11 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:18:12.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:11 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:18:12.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:11 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:18:12.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:11 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:18:12.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:11 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:18:12.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:11 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:18:12.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:11 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:18:12.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:11 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:18:12.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:11 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:18:12.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:11 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:18:12.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:11 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.794+0000 7f5176119700 1 -- 172.21.15.46:0/759506529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170108740 msgr2=0x7f51701100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.794+0000 7f5176119700 1 --2- 172.21.15.46:0/759506529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170108740 0x7f51701100a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f516800b0d0 tx=0x7f516800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.794+0000 7f5176119700 1 -- 172.21.15.46:0/759506529 shutdown_connections 2024-05-09T00:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.794+0000 7f5176119700 1 --2- 172.21.15.46:0/759506529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170108740 0x7f51701100a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.794+0000 7f5176119700 1 --2- 172.21.15.46:0/759506529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51701034c0 0x7f51701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.794+0000 7f5176119700 1 --2- 172.21.15.46:0/759506529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5170107df0 0x7f5170108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.794+0000 7f5176119700 1 -- 172.21.15.46:0/759506529 >> 172.21.15.46:0/759506529 conn(0x7f51700fedc0 msgr2=0x7f51701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.795+0000 7f5176119700 1 -- 172.21.15.46:0/759506529 shutdown_connections 2024-05-09T00:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.795+0000 7f5176119700 1 -- 172.21.15.46:0/759506529 wait complete. 2024-05-09T00:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.795+0000 7f5176119700 1 Processor -- start 2024-05-09T00:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f5176119700 1 -- start start 2024-05-09T00:18:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f5176119700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51701034c0 0x7f51700ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f5176119700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170107df0 0x7f51700ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f5176119700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5170108740 0x7f51700ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f5176119700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5170113370 con 0x7f5170107df0 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f5176119700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f51701131f0 con 0x7f5170108740 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f5176119700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f51701134f0 con 0x7f51701034c0 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f516effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170107df0 0x7f51700ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f516effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170107df0 0x7f51700ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39160/0 (socket says 172.21.15.46:39160) 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f516effd700 1 -- 172.21.15.46:0/1715132967 learned_addr learned my addr 172.21.15.46:0/1715132967 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f516ffff700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5170108740 0x7f51700ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f516f7fe700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51701034c0 0x7f51700ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f516effd700 1 -- 172.21.15.46:0/1715132967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51701034c0 msgr2=0x7f51700ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f516effd700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51701034c0 0x7f51700ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f516effd700 1 -- 172.21.15.46:0/1715132967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5170108740 msgr2=0x7f51700ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f516effd700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5170108740 0x7f51700ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.796+0000 7f516effd700 1 -- 172.21.15.46:0/1715132967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51701a9110 con 0x7f5170107df0 2024-05-09T00:18:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.797+0000 7f516effd700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170107df0 0x7f51700ff7e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f516400d950 tx=0x7f516400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.797+0000 7f516cff9700 1 -- 172.21.15.46:0/1715132967 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164016070 con 0x7f5170107df0 2024-05-09T00:18:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.797+0000 7f5176119700 1 -- 172.21.15.46:0/1715132967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51701a9400 con 0x7f5170107df0 2024-05-09T00:18:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.797+0000 7f516cff9700 1 -- 172.21.15.46:0/1715132967 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5164004830 con 0x7f5170107df0 2024-05-09T00:18:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.797+0000 7f516cff9700 1 -- 172.21.15.46:0/1715132967 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164005520 con 0x7f5170107df0 2024-05-09T00:18:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.797+0000 7f5176119700 1 -- 172.21.15.46:0/1715132967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51701a9940 con 0x7f5170107df0 2024-05-09T00:18:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.800+0000 7f516cff9700 1 -- 172.21.15.46:0/1715132967 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f516400f690 con 0x7f5170107df0 2024-05-09T00:18:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.801+0000 7f516cff9700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f51580b2880 0x7f51580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.801+0000 7f516cff9700 1 -- 172.21.15.46:0/1715132967 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f51640e1f20 con 0x7f5170107df0 2024-05-09T00:18:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.801+0000 7f516f7fe700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f51580b2880 0x7f51580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.801+0000 7f5184b2fb80 1 Processor -- start 2024-05-09T00:18:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.801+0000 7f5184b2fb80 1 -- start start 2024-05-09T00:18:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.801+0000 7f516f7fe700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f51580b2880 0x7f51580b4d90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5160005950 tx=0x7f516000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.801+0000 7f5184b2fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d130ae160 0x559d1314fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.801+0000 7f5184b2fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 0x559d13150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.801+0000 7f5184b2fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d1314a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.801+0000 7f5184b2fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559d13063ff0 con 0x559d12fc44a0 2024-05-09T00:18:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.801+0000 7f5184b2fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559d130027f0 con 0x559d130ae160 2024-05-09T00:18:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.801+0000 7f5184b2fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559d1305ffb0 con 0x559d13149eb0 2024-05-09T00:18:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 0x559d13150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 0x559d13150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39172/0 (socket says 172.21.15.46:39172) 2024-05-09T00:18:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516effd700 1 -- 172.21.15.46:0/3608615897 learned_addr learned my addr 172.21.15.46:0/3608615897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516f7fe700 1 --2- 172.21.15.46:0/3608615897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d130ae160 0x559d1314fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516ffff700 1 --2- 172.21.15.46:0/3608615897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d1314a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516effd700 1 -- 172.21.15.46:0/3608615897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 msgr2=0x559d1314a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516effd700 1 --2- 172.21.15.46:0/3608615897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d1314a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516effd700 1 -- 172.21.15.46:0/3608615897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d130ae160 msgr2=0x559d1314fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516effd700 1 --2- 172.21.15.46:0/3608615897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d130ae160 0x559d1314fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516effd700 1 -- 172.21.15.46:0/3608615897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d12ffa370 con 0x559d12fc44a0 2024-05-09T00:18:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516f7fe700 1 --2- 172.21.15.46:0/3608615897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d130ae160 0x559d1314fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516ffff700 1 --2- 172.21.15.46:0/3608615897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d1314a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f516effd700 1 --2- 172.21.15.46:0/3608615897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 0x559d13150240 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5164005cd0 tx=0x7f5164007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8cd6eec8b28e735 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.802+0000 7f5155ffb700 1 -- 172.21.15.46:0/3608615897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51640ad8f0 con 0x559d12fc44a0 2024-05-09T00:18:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.803+0000 7f5155ffb700 1 -- 172.21.15.46:0/3608615897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51640b23a0 con 0x559d12fc44a0 2024-05-09T00:18:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.803+0000 7f5155ffb700 1 -- 172.21.15.46:0/3608615897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51640a8a00 con 0x559d12fc44a0 2024-05-09T00:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.803+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3608615897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 msgr2=0x559d13150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.803+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3608615897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 0x559d13150240 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5164005cd0 tx=0x7f5164007620 comp rx=0 tx=0).stop 2024-05-09T00:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.804+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3608615897 shutdown_connections 2024-05-09T00:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.804+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3608615897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d1314a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.804+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3608615897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d130ae160 0x559d1314fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.804+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3608615897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 0x559d13150240 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.804+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3608615897 >> 172.21.15.46:0/3608615897 conn(0x559d130246a0 msgr2=0x559d1313c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.804+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3608615897 shutdown_connections 2024-05-09T00:18:12.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.804+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3608615897 wait complete. 2024-05-09T00:18:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.804+0000 7f5184b2fb80 1 -- start start 2024-05-09T00:18:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.804+0000 7f5184b2fb80 1 Processor -- start 2024-05-09T00:18:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.805+0000 7f5184b2fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d12fc44a0 0x559d13153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:12 smithi046 ceph-mon[30151]: pgmap v581: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:18:12.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:12 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:18:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.805+0000 7f5184b2fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d130ae160 0x559d13154010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.805+0000 7f5184b2fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d13149eb0 0x559d13154550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.805+0000 7f5184b2fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559d12edfba0 con 0x559d13149eb0 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.805+0000 7f5184b2fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559d1301e150 con 0x559d12fc44a0 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.805+0000 7f5184b2fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559d131569e0 con 0x559d130ae160 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.805+0000 7f516effd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d130ae160 0x559d13154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.805+0000 7f516f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d12fc44a0 0x559d13153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.805+0000 7f516effd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d130ae160 0x559d13154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:36310/0 (socket says 172.21.15.46:36310) 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.805+0000 7f516f7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d12fc44a0 0x559d13153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:44248/0 (socket says 172.21.15.46:44248) 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.805+0000 7f516effd700 1 -- 172.21.15.46:0/2093261015 learned_addr learned my addr 172.21.15.46:0/2093261015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.805+0000 7f5184b2fb80 10 client.0 _ll_register_callbacks cb 0x559d12ed56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f516effd700 1 -- 172.21.15.46:0/2093261015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d12fc44a0 msgr2=0x559d13153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f516effd700 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d12fc44a0 0x559d13153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f516effd700 1 -- 172.21.15.46:0/2093261015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d13149eb0 msgr2=0x559d13154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:12.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f516ffff700 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d13149eb0 0x559d13154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f516effd700 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d13149eb0 0x559d13154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f516effd700 1 -- 172.21.15.46:0/2093261015 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d12edf490 con 0x559d130ae160 2024-05-09T00:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f516ffff700 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d13149eb0 0x559d13154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f516effd700 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d130ae160 0x559d13154010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f51640a5570 tx=0x7f51640a5be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81bcf9c13472110 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f514effd700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f514effd700 1 -- 172.21.15.46:0/2093261015 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516406e060 con 0x559d130ae160 2024-05-09T00:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f514effd700 1 -- 172.21.15.46:0/2093261015 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f51640b2120 con 0x559d130ae160 2024-05-09T00:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f5184b2fb80 1 -- 172.21.15.46:0/2093261015 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51640b23a0 con 0x559d130ae160 2024-05-09T00:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f514effd700 1 -- 172.21.15.46:0/2093261015 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51640c2030 con 0x559d130ae160 2024-05-09T00:18:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.806+0000 7f5184b2fb80 10 client.26011 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.807+0000 7f5184b2fb80 1 -- 172.21.15.46:0/2093261015 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559d13161460 con 0x559d130ae160 2024-05-09T00:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.807+0000 7f514effd700 1 -- 172.21.15.46:0/2093261015 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51640b23a0 con 0x559d130ae160 2024-05-09T00:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.807+0000 7f514effd700 1 -- 172.21.15.46:0/2093261015 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f51640a5040 con 0x559d130ae160 2024-05-09T00:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.807+0000 7f5184b2fb80 10 client.26011 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.807+0000 7f5184b2fb80 10 client.26011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.807+0000 7f5184b2fb80 20 client.26011 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.807+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559d13163760 0x559d13165b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.807+0000 7f5184b2fb80 4 client.26011 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.807+0000 7f5184b2fb80 1 -- 172.21.15.46:0/2093261015 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d130719d0 con 0x559d13163760 2024-05-09T00:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.807+0000 7f516f7fe700 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559d13163760 0x559d13165b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.808+0000 7f516f7fe700 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559d13163760 0x559d13165b80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b2c9129692bf916 server_cookie=da38d5dd29e19adc in_seq=0 out_seq=0 2024-05-09T00:18:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.808+0000 7f514effd700 10 client.26011 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f514effd700 1 -- 172.21.15.46:0/2093261015 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559d130719d0 con 0x559d13163760 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f514effd700 10 client.26011 handle_command_reply: tid=0 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f5184b2fb80 1 client.26011 shutdown 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f5184b2fb80 1 -- 172.21.15.46:0/2093261015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d130ae160 msgr2=0x559d13154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d130ae160 0x559d13154010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f51640a5570 tx=0x7f51640a5be0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f5184b2fb80 1 -- 172.21.15.46:0/2093261015 shutdown_connections 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559d13163760 0x559d13165b80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f514effd700 0 client.26011 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d13149eb0 0x559d13154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d12fc44a0 0x559d13153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/2093261015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d130ae160 0x559d13154010 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f5184b2fb80 1 -- 172.21.15.46:0/2093261015 >> 172.21.15.46:0/2093261015 conn(0x559d130246a0 msgr2=0x559d1313c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f5184b2fb80 1 -- 172.21.15.46:0/2093261015 shutdown_connections 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f5184b2fb80 1 -- 172.21.15.46:0/2093261015 wait complete. 2024-05-09T00:18:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.809+0000 7f5184b2fb80 20 client.26011 trim_cache size 0 max 16384 2024-05-09T00:18:12.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.830+0000 7f5184b2fb80 1 Processor -- start 2024-05-09T00:18:12.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.830+0000 7f5184b2fb80 1 -- start start 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f5184b2fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d12fc44a0 0x559d1314e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f5184b2fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d130ae160 0x559d13152df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f5184b2fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d13154390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f5184b2fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f51701af360 con 0x559d130ae160 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f5184b2fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f51701acbf0 con 0x559d12fc44a0 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f5184b2fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f517019eb50 con 0x559d13149eb0 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f516effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d130ae160 0x559d13152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f516effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d130ae160 0x559d13152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39178/0 (socket says 172.21.15.46:39178) 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f516effd700 1 -- 172.21.15.46:0/3038201145 learned_addr learned my addr 172.21.15.46:0/3038201145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f516f7fe700 1 --2- 172.21.15.46:0/3038201145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d12fc44a0 0x559d1314e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f516ffff700 1 --2- 172.21.15.46:0/3038201145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d13154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f516effd700 1 -- 172.21.15.46:0/3038201145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 msgr2=0x559d13154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:12.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f516effd700 1 --2- 172.21.15.46:0/3038201145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d13154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f516effd700 1 -- 172.21.15.46:0/3038201145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d12fc44a0 msgr2=0x559d1314e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f516effd700 1 --2- 172.21.15.46:0/3038201145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d12fc44a0 0x559d1314e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f516effd700 1 -- 172.21.15.46:0/3038201145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d13156370 con 0x559d130ae160 2024-05-09T00:18:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.831+0000 7f516ffff700 1 --2- 172.21.15.46:0/3038201145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d13154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.832+0000 7f516effd700 1 --2- 172.21.15.46:0/3038201145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d130ae160 0x559d13152df0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f51640a5d00 tx=0x7f5164007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f6fbd5f4445679 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.832+0000 7f51557fa700 1 -- 172.21.15.46:0/3038201145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164088070 con 0x559d130ae160 2024-05-09T00:18:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.832+0000 7f51557fa700 1 -- 172.21.15.46:0/3038201145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51640b23a0 con 0x559d130ae160 2024-05-09T00:18:12.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.832+0000 7f51557fa700 1 -- 172.21.15.46:0/3038201145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516406e070 con 0x559d130ae160 2024-05-09T00:18:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.833+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3038201145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d130ae160 msgr2=0x559d13152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.833+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3038201145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d130ae160 0x559d13152df0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f51640a5d00 tx=0x7f5164007620 comp rx=0 tx=0).stop 2024-05-09T00:18:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.833+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3038201145 shutdown_connections 2024-05-09T00:18:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.833+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3038201145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d13154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.833+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3038201145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d12fc44a0 0x559d1314e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.833+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3038201145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d130ae160 0x559d13152df0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.833+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3038201145 >> 172.21.15.46:0/3038201145 conn(0x559d130246a0 msgr2=0x559d1315eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.833+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3038201145 shutdown_connections 2024-05-09T00:18:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.833+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3038201145 wait complete. 2024-05-09T00:18:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.833+0000 7f5184b2fb80 1 -- start start 2024-05-09T00:18:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f5184b2fb80 1 Processor -- start 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f5184b2fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 0x559d131c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f5184b2fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d130ae160 0x559d131c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f5184b2fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d13153780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f5184b2fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f51701ab0b0 con 0x559d12fc44a0 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f5184b2fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f51701ac520 con 0x559d130ae160 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f5184b2fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f51701ab780 con 0x559d13149eb0 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f5184b2fb80 10 client.0 _ll_register_callbacks cb 0x559d12ed56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f516f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 0x559d131c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f516effd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d130ae160 0x559d131c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f516f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 0x559d131c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39194/0 (socket says 172.21.15.46:39194) 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f516f7fe700 1 -- 172.21.15.46:0/3280092155 learned_addr learned my addr 172.21.15.46:0/3280092155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.834+0000 7f516ffff700 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d13153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f516f7fe700 1 -- 172.21.15.46:0/3280092155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 msgr2=0x559d13153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f516f7fe700 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d13153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f516f7fe700 1 -- 172.21.15.46:0/3280092155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d130ae160 msgr2=0x559d131c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f516f7fe700 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d130ae160 0x559d131c66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f516f7fe700 1 -- 172.21.15.46:0/3280092155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d13153dc0 con 0x559d12fc44a0 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f516effd700 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d130ae160 0x559d131c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f516f7fe700 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 0x559d131c6160 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x559d13153350 tx=0x7f5160013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adad202dcb8596a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f514effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f514effd700 1 -- 172.21.15.46:0/3280092155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516002c070 con 0x559d12fc44a0 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3280092155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51640b23a0 con 0x559d12fc44a0 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f514effd700 1 -- 172.21.15.46:0/3280092155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f5160028650 con 0x559d12fc44a0 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f514effd700 1 -- 172.21.15.46:0/3280092155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5160038050 con 0x559d12fc44a0 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f5184b2fb80 10 client.16434 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.835+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3280092155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559d131cd720 con 0x559d12fc44a0 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.836+0000 7f514effd700 1 -- 172.21.15.46:0/3280092155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51640b23a0 con 0x559d12fc44a0 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.836+0000 7f514effd700 1 -- 172.21.15.46:0/3280092155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5160028ce0 con 0x559d12fc44a0 2024-05-09T00:18:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.836+0000 7f5184b2fb80 10 client.16434 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.836+0000 7f5184b2fb80 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.836+0000 7f5184b2fb80 20 client.16434 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.836+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559d13163760 0x559d131cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.836+0000 7f5184b2fb80 4 client.16434 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.836+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3280092155 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d1306ba70 con 0x559d13163760 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.836+0000 7f516effd700 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559d13163760 0x559d131cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.837+0000 7f514effd700 10 client.16434 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.837+0000 7f516effd700 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559d13163760 0x559d131cfdf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2e8d2c1bb28a4ba server_cookie=a3aebf54bc6e4103 in_seq=0 out_seq=0 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.837+0000 7f514effd700 1 -- 172.21.15.46:0/3280092155 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559d1306ba70 con 0x559d13163760 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.837+0000 7f514effd700 10 client.16434 handle_command_reply: tid=0 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.837+0000 7f5184b2fb80 1 client.16434 shutdown 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.838+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3280092155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 msgr2=0x559d131c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.838+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 0x559d131c6160 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x559d13153350 tx=0x7f5160013070 comp rx=0 tx=0).stop 2024-05-09T00:18:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.838+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3280092155 shutdown_connections 2024-05-09T00:18:12.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.838+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559d13163760 0x559d131cfdf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.838+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559d13149eb0 0x559d13153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.838+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d12fc44a0 0x559d131c6160 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.838+0000 7f5184b2fb80 1 --2- 172.21.15.46:0/3280092155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d130ae160 0x559d131c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.838+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3280092155 >> 172.21.15.46:0/3280092155 conn(0x559d130246a0 msgr2=0x559d13152b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:12.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.838+0000 7f514effd700 0 client.16434 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:12.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.838+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3280092155 shutdown_connections 2024-05-09T00:18:12.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.838+0000 7f5184b2fb80 1 -- 172.21.15.46:0/3280092155 wait complete. 2024-05-09T00:18:12.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.838+0000 7f5184b2fb80 20 client.16434 trim_cache size 0 max 16384 2024-05-09T00:18:12.842 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:18:12.843 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:18:12.843 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:18:12.843 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:18:12.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.839+0000 7f5176119700 1 -- 172.21.15.46:0/1715132967 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f51580b2880 msgr2=0x7f51580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:12.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.839+0000 7f5176119700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f51580b2880 0x7f51580b4d90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5160005950 tx=0x7f516000b410 comp rx=0 tx=0).stop 2024-05-09T00:18:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.839+0000 7f5176119700 1 -- 172.21.15.46:0/1715132967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170107df0 msgr2=0x7f51700ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.839+0000 7f5176119700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170107df0 0x7f51700ff7e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f516400d950 tx=0x7f516400dc60 comp rx=0 tx=0).stop 2024-05-09T00:18:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.839+0000 7f5176119700 1 -- 172.21.15.46:0/1715132967 shutdown_connections 2024-05-09T00:18:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.839+0000 7f5176119700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f51580b2880 0x7f51580b4d90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.839+0000 7f5176119700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5170108740 0x7f51700ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.839+0000 7f5176119700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51701034c0 0x7f51700ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.839+0000 7f5176119700 1 --2- 172.21.15.46:0/1715132967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5170107df0 0x7f51700ff7e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.839+0000 7f5176119700 1 -- 172.21.15.46:0/1715132967 >> 172.21.15.46:0/1715132967 conn(0x7f51700fedc0 msgr2=0x7f517006b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.839+0000 7f5176119700 1 -- 172.21.15.46:0/1715132967 shutdown_connections 2024-05-09T00:18:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:12.839+0000 7f5176119700 1 -- 172.21.15.46:0/1715132967 wait complete. 2024-05-09T00:18:13.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:12 smithi169 ceph-mon[34777]: pgmap v581: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:18:13.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:12 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:18:13.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:12 smithi142 ceph-mon[34533]: pgmap v581: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:18:13.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:12 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:18:13.553 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:18:13.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:18:13.817 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:18:15.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:14 smithi046 ceph-mon[30151]: pgmap v582: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:18:15.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:14 smithi169 ceph-mon[34777]: pgmap v582: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:18:15.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:14 smithi142 ceph-mon[34533]: pgmap v582: 97 pgs: 97 active+clean; 3.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:18:15.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.626+0000 7f821daad700 1 -- 172.21.15.46:0/2980936457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82181034c0 msgr2=0x7f82181038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:15.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.626+0000 7f821daad700 1 --2- 172.21.15.46:0/2980936457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82181034c0 0x7f82181038d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f820800b0d0 tx=0x7f820800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:18:15.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.626+0000 7f821daad700 1 -- 172.21.15.46:0/2980936457 shutdown_connections 2024-05-09T00:18:15.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.626+0000 7f821daad700 1 --2- 172.21.15.46:0/2980936457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8218108740 0x7f82181100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:15.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.626+0000 7f821daad700 1 --2- 172.21.15.46:0/2980936457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82181034c0 0x7f82181038d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:15.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.626+0000 7f821daad700 1 --2- 172.21.15.46:0/2980936457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8218107df0 0x7f8218108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:15.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.626+0000 7f821daad700 1 -- 172.21.15.46:0/2980936457 >> 172.21.15.46:0/2980936457 conn(0x7f82180fedc0 msgr2=0x7f82181011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:15.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.627+0000 7f821daad700 1 -- 172.21.15.46:0/2980936457 shutdown_connections 2024-05-09T00:18:15.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.627+0000 7f821daad700 1 -- 172.21.15.46:0/2980936457 wait complete. 2024-05-09T00:18:15.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.627+0000 7f821daad700 1 Processor -- start 2024-05-09T00:18:15.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f821daad700 1 -- start start 2024-05-09T00:18:15.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f821daad700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82181034c0 0x7f821819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f821daad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218107df0 0x7f821819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f821daad700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8218108740 0x7f821819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f821daad700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8218113370 con 0x7f8218107df0 2024-05-09T00:18:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f821daad700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82181131f0 con 0x7f82181034c0 2024-05-09T00:18:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f821daad700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82181134f0 con 0x7f8218108740 2024-05-09T00:18:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f8216ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218107df0 0x7f821819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f8216ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218107df0 0x7f821819c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39220/0 (socket says 172.21.15.46:39220) 2024-05-09T00:18:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f8216ffd700 1 -- 172.21.15.46:0/25736568 learned_addr learned my addr 172.21.15.46:0/25736568 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f82177fe700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82181034c0 0x7f821819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f8217fff700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8218108740 0x7f821819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f8216ffd700 1 -- 172.21.15.46:0/25736568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8218108740 msgr2=0x7f821819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f8216ffd700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8218108740 0x7f821819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f8216ffd700 1 -- 172.21.15.46:0/25736568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82181034c0 msgr2=0x7f821819bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f8216ffd700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82181034c0 0x7f821819bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.628+0000 7f8216ffd700 1 -- 172.21.15.46:0/25736568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82181a6f80 con 0x7f8218107df0 2024-05-09T00:18:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.629+0000 7f8216ffd700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218107df0 0x7f821819c170 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f820c00d980 tx=0x7f820c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.629+0000 7f8214ff9700 1 -- 172.21.15.46:0/25736568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c017070 con 0x7f8218107df0 2024-05-09T00:18:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.629+0000 7f821daad700 1 -- 172.21.15.46:0/25736568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82181a7270 con 0x7f8218107df0 2024-05-09T00:18:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.629+0000 7f8214ff9700 1 -- 172.21.15.46:0/25736568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f820c009de0 con 0x7f8218107df0 2024-05-09T00:18:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.629+0000 7f8214ff9700 1 -- 172.21.15.46:0/25736568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c00f3c0 con 0x7f8218107df0 2024-05-09T00:18:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.629+0000 7f821daad700 1 -- 172.21.15.46:0/25736568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82181a7800 con 0x7f8218107df0 2024-05-09T00:18:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.630+0000 7f821daad700 1 -- 172.21.15.46:0/25736568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82180512c0 con 0x7f8218107df0 2024-05-09T00:18:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.632+0000 7f8214ff9700 1 -- 172.21.15.46:0/25736568 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f820c009930 con 0x7f8218107df0 2024-05-09T00:18:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.633+0000 7f8214ff9700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f82000b2880 0x7f82000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.633+0000 7f82177fe700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f82000b2880 0x7f82000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.633+0000 7f8214ff9700 1 -- 172.21.15.46:0/25736568 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f820c0e1da0 con 0x7f8218107df0 2024-05-09T00:18:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.633+0000 7f82177fe700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f82000b2880 0x7f82000b4d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8208009900 tx=0x7f820800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:15.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.638+0000 7f8214ff9700 1 -- 172.21.15.46:0/25736568 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f820c0a9a80 con 0x7f8218107df0 2024-05-09T00:18:15.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.902+0000 7f821daad700 1 -- 172.21.15.46:0/25736568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f821806bc00 con 0x7f8218107df0 2024-05-09T00:18:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.903+0000 7f8214ff9700 1 -- 172.21.15.46:0/25736568 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f820c0a9380 con 0x7f8218107df0 2024-05-09T00:18:15.903 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:18:15.903 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:18:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.905+0000 7f821daad700 1 -- 172.21.15.46:0/25736568 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f82000b2880 msgr2=0x7f82000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.905+0000 7f821daad700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f82000b2880 0x7f82000b4d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8208009900 tx=0x7f820800b040 comp rx=0 tx=0).stop 2024-05-09T00:18:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.905+0000 7f821daad700 1 -- 172.21.15.46:0/25736568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218107df0 msgr2=0x7f821819c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.905+0000 7f821daad700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218107df0 0x7f821819c170 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f820c00d980 tx=0x7f820c00dc90 comp rx=0 tx=0).stop 2024-05-09T00:18:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.905+0000 7f821daad700 1 -- 172.21.15.46:0/25736568 shutdown_connections 2024-05-09T00:18:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.905+0000 7f821daad700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f82000b2880 0x7f82000b4d90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.905+0000 7f821daad700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8218108740 0x7f821819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.905+0000 7f821daad700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82181034c0 0x7f821819bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.905+0000 7f821daad700 1 --2- 172.21.15.46:0/25736568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218107df0 0x7f821819c170 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.905+0000 7f821daad700 1 -- 172.21.15.46:0/25736568 >> 172.21.15.46:0/25736568 conn(0x7f82180fedc0 msgr2=0x7f8218101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.906+0000 7f821daad700 1 -- 172.21.15.46:0/25736568 shutdown_connections 2024-05-09T00:18:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:15.906+0000 7f821daad700 1 -- 172.21.15.46:0/25736568 wait complete. 2024-05-09T00:18:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:18:16.722 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:18:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:18:17.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:16 smithi046 ceph-mon[30151]: pgmap v583: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:17.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:16 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/25736568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:18:17.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:16 smithi169 ceph-mon[34777]: pgmap v583: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:17.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:16 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/25736568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:18:17.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:16 smithi142 ceph-mon[34533]: pgmap v583: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:17.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:16 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/25736568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:18:18.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.753+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/85607740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e28107e10 msgr2=0x7f8e28108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.753+0000 7f8e2ec1b700 1 --2- 172.21.15.46:0/85607740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e28107e10 0x7f8e28108220 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8e180099c0 tx=0x7f8e18009cd0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.753+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/85607740 shutdown_connections 2024-05-09T00:18:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.753+0000 7f8e2ec1b700 1 --2- 172.21.15.46:0/85607740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e28108760 0x7f8e281100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.753+0000 7f8e2ec1b700 1 --2- 172.21.15.46:0/85607740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e281034e0 0x7f8e281038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.753+0000 7f8e2ec1b700 1 --2- 172.21.15.46:0/85607740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e28107e10 0x7f8e28108220 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.753+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/85607740 >> 172.21.15.46:0/85607740 conn(0x7f8e280fef30 msgr2=0x7f8e28101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.754+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/85607740 shutdown_connections 2024-05-09T00:18:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.754+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/85607740 wait complete. 2024-05-09T00:18:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.754+0000 7f8e2ec1b700 1 Processor -- start 2024-05-09T00:18:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e2ec1b700 1 -- start start 2024-05-09T00:18:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e2ec1b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e281034e0 0x7f8e2819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e2ec1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e28107e10 0x7f8e2819c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e2ec1b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e28108760 0x7f8e281a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e2ec1b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28113390 con 0x7f8e28107e10 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e2ec1b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28113210 con 0x7f8e281034e0 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e2ec1b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28113510 con 0x7f8e28108760 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e28107e10 0x7f8e2819c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e28107e10 0x7f8e2819c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46274/0 (socket says 172.21.15.46:46274) 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e27fff700 1 -- 172.21.15.46:0/926546215 learned_addr learned my addr 172.21.15.46:0/926546215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e281034e0 0x7f8e2819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e2d1b8700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e28108760 0x7f8e281a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e27fff700 1 -- 172.21.15.46:0/926546215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e28108760 msgr2=0x7f8e281a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e27fff700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e28108760 0x7f8e281a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e27fff700 1 -- 172.21.15.46:0/926546215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e281034e0 msgr2=0x7f8e2819bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:18.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e27fff700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e281034e0 0x7f8e2819bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.755+0000 7f8e27fff700 1 -- 172.21.15.46:0/926546215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2819ca60 con 0x7f8e28107e10 2024-05-09T00:18:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.756+0000 7f8e27fff700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e28107e10 0x7f8e2819c1b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8e18006010 tx=0x7f8e180047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.756+0000 7f8e25ffb700 1 -- 172.21.15.46:0/926546215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18027070 con 0x7f8e28107e10 2024-05-09T00:18:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.756+0000 7f8e25ffb700 1 -- 172.21.15.46:0/926546215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e18003f20 con 0x7f8e28107e10 2024-05-09T00:18:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.756+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/926546215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e281a72f0 con 0x7f8e28107e10 2024-05-09T00:18:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.756+0000 7f8e25ffb700 1 -- 172.21.15.46:0/926546215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18022cd0 con 0x7f8e28107e10 2024-05-09T00:18:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.756+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/926546215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e281a7740 con 0x7f8e28107e10 2024-05-09T00:18:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.759+0000 7f8e25ffb700 1 -- 172.21.15.46:0/926546215 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8e1802a070 con 0x7f8e28107e10 2024-05-09T00:18:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.759+0000 7f8e25ffb700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8e140b2880 0x7f8e140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.760+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8e140b2880 0x7f8e140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.760+0000 7f8e25ffb700 1 -- 172.21.15.46:0/926546215 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8e180e3080 con 0x7f8e28107e10 2024-05-09T00:18:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.760+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8e140b2880 0x7f8e140b4d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8e10005950 tx=0x7f8e1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.760+0000 7f8e3d631b80 1 Processor -- start 2024-05-09T00:18:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.760+0000 7f8e3d631b80 1 -- start start 2024-05-09T00:18:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e3d631b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba3f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e3d631b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 0x55877ba3fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e3d631b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877ba396a0 0x55877ba39ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e3d631b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55877b95bff0 con 0x55877b8bc4a0 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e3d631b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55877b8fa7f0 con 0x55877ba396a0 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e3d631b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55877b957fb0 con 0x55877b9a6160 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 0x55877ba3fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 0x55877ba3fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46278/0 (socket says 172.21.15.46:46278) 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e27fff700 1 -- 172.21.15.46:0/627025112 learned_addr learned my addr 172.21.15.46:0/627025112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/627025112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba3f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e2d1b8700 1 --2- 172.21.15.46:0/627025112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877ba396a0 0x55877ba39ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e27fff700 1 -- 172.21.15.46:0/627025112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 msgr2=0x55877ba3f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e27fff700 1 --2- 172.21.15.46:0/627025112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba3f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e27fff700 1 -- 172.21.15.46:0/627025112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877ba396a0 msgr2=0x55877ba39ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e27fff700 1 --2- 172.21.15.46:0/627025112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877ba396a0 0x55877ba39ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e27fff700 1 -- 172.21.15.46:0/627025112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55877b8f2370 con 0x55877b8bc4a0 2024-05-09T00:18:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e2d1b8700 1 --2- 172.21.15.46:0/627025112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877ba396a0 0x55877ba39ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/627025112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba3f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.761+0000 7f8e27fff700 1 --2- 172.21.15.46:0/627025112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 0x55877ba3fa30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8e180a67b0 tx=0x7f8e18003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e516363c99c66e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.762+0000 7f8e1effd700 1 -- 172.21.15.46:0/627025112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e180aa730 con 0x55877b8bc4a0 2024-05-09T00:18:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.762+0000 7f8e1effd700 1 -- 172.21.15.46:0/627025112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e180a4e50 con 0x55877b8bc4a0 2024-05-09T00:18:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.762+0000 7f8e1effd700 1 -- 172.21.15.46:0/627025112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e180aa990 con 0x55877b8bc4a0 2024-05-09T00:18:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.762+0000 7f8e3d631b80 1 -- 172.21.15.46:0/627025112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 msgr2=0x55877ba3fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.762+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/627025112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 0x55877ba3fa30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8e180a67b0 tx=0x7f8e18003780 comp rx=0 tx=0).stop 2024-05-09T00:18:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.763+0000 7f8e3d631b80 1 -- 172.21.15.46:0/627025112 shutdown_connections 2024-05-09T00:18:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.763+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/627025112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877ba396a0 0x55877ba39ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.763+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/627025112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba3f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.763+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/627025112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 0x55877ba3fa30 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.763+0000 7f8e3d631b80 1 -- 172.21.15.46:0/627025112 >> 172.21.15.46:0/627025112 conn(0x55877b91c6a0 msgr2=0x55877ba34400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.763+0000 7f8e3d631b80 1 -- 172.21.15.46:0/627025112 shutdown_connections 2024-05-09T00:18:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.763+0000 7f8e3d631b80 1 -- 172.21.15.46:0/627025112 wait complete. 2024-05-09T00:18:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.763+0000 7f8e3d631b80 1 -- start start 2024-05-09T00:18:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.763+0000 7f8e3d631b80 1 Processor -- start 2024-05-09T00:18:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e3d631b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b8bc4a0 0x55877ba48d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e3d631b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba49250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e3d631b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877ba396a0 0x55877ba43110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e3d631b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55877b7d7ba0 con 0x55877ba396a0 2024-05-09T00:18:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e3d631b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55877b916150 con 0x55877b8bc4a0 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e3d631b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55877ba3a360 con 0x55877b9a6160 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e3d631b80 10 client.0 _ll_register_callbacks cb 0x55877b7cd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e2d1b8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877ba396a0 0x55877ba43110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e2d1b8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877ba396a0 0x55877ba43110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46290/0 (socket says 172.21.15.46:46290) 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e2d1b8700 1 -- 172.21.15.46:0/4244907089 learned_addr learned my addr 172.21.15.46:0/4244907089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b8bc4a0 0x55877ba48d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e27fff700 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba49250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e2d1b8700 1 -- 172.21.15.46:0/4244907089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 msgr2=0x55877ba49250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e2d1b8700 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba49250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e2d1b8700 1 -- 172.21.15.46:0/4244907089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b8bc4a0 msgr2=0x55877ba48d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e2d1b8700 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b8bc4a0 0x55877ba48d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.764+0000 7f8e2d1b8700 1 -- 172.21.15.46:0/4244907089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55877b7d7490 con 0x55877ba396a0 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.765+0000 7f8e2d1b8700 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877ba396a0 0x55877ba43110 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8e20007ba0 tx=0x7f8e20007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9783b3c4b1122ce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.765+0000 7f8e07fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.765+0000 7f8e07fff700 1 -- 172.21.15.46:0/4244907089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2001a020 con 0x55877ba396a0 2024-05-09T00:18:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.765+0000 7f8e07fff700 1 -- 172.21.15.46:0/4244907089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8e200118a0 con 0x55877ba396a0 2024-05-09T00:18:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.765+0000 7f8e3d631b80 1 -- 172.21.15.46:0/4244907089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e180a4e50 con 0x55877ba396a0 2024-05-09T00:18:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.768+0000 7f8e07fff700 1 -- 172.21.15.46:0/4244907089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8e20011aa0 con 0x55877ba396a0 2024-05-09T00:18:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.768+0000 7f8e3d631b80 10 client.16452 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.768+0000 7f8e3d631b80 1 -- 172.21.15.46:0/4244907089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55877ba50990 con 0x55877ba396a0 2024-05-09T00:18:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.769+0000 7f8e07fff700 1 -- 172.21.15.46:0/4244907089 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e180a4e50 con 0x55877ba396a0 2024-05-09T00:18:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.769+0000 7f8e07fff700 1 -- 172.21.15.46:0/4244907089 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8e20021430 con 0x55877ba396a0 2024-05-09T00:18:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.769+0000 7f8e3d631b80 10 client.16452 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.769+0000 7f8e3d631b80 10 client.16452 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:18:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.769+0000 7f8e3d631b80 20 client.16452 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.769+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55877ba52fd0 0x55877ba553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.769+0000 7f8e3d631b80 4 client.16452 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:18:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.769+0000 7f8e3d631b80 1 -- 172.21.15.46:0/4244907089 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55877b9699d0 con 0x55877ba52fd0 2024-05-09T00:18:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.769+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55877ba52fd0 0x55877ba553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.770+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55877ba52fd0 0x55877ba553f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33a3960b2311b872 server_cookie=c5fc9e57000fa5b9 in_seq=0 out_seq=0 2024-05-09T00:18:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.770+0000 7f8e07fff700 10 client.16452 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:18:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.770+0000 7f8e07fff700 1 -- 172.21.15.46:0/4244907089 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55877b9699d0 con 0x55877ba52fd0 2024-05-09T00:18:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.770+0000 7f8e07fff700 10 client.16452 handle_command_reply: tid=0 2024-05-09T00:18:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.770+0000 7f8e3d631b80 1 client.16452 shutdown 2024-05-09T00:18:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.771+0000 7f8e3d631b80 1 -- 172.21.15.46:0/4244907089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877ba396a0 msgr2=0x55877ba43110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.771+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877ba396a0 0x55877ba43110 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8e20007ba0 tx=0x7f8e20007eb0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.771+0000 7f8e3d631b80 1 -- 172.21.15.46:0/4244907089 shutdown_connections 2024-05-09T00:18:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.771+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55877ba52fd0 0x55877ba553f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.771+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877ba396a0 0x55877ba43110 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.771+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b8bc4a0 0x55877ba48d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.771+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/4244907089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba49250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.771+0000 7f8e3d631b80 1 -- 172.21.15.46:0/4244907089 >> 172.21.15.46:0/4244907089 conn(0x55877b91c6a0 msgr2=0x55877ba341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.771+0000 7f8e07fff700 0 client.16452 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:18:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.771+0000 7f8e3d631b80 1 -- 172.21.15.46:0/4244907089 shutdown_connections 2024-05-09T00:18:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.771+0000 7f8e3d631b80 1 -- 172.21.15.46:0/4244907089 wait complete. 2024-05-09T00:18:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.771+0000 7f8e3d631b80 20 client.16452 trim_cache size 0 max 16384 2024-05-09T00:18:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.786+0000 7f8e3d631b80 1 Processor -- start 2024-05-09T00:18:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.786+0000 7f8e3d631b80 1 -- start start 2024-05-09T00:18:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.786+0000 7f8e3d631b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b91c6a0 0x55877ba4f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.786+0000 7f8e3d631b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 0x55877ba430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.786+0000 7f8e3d631b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba49170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.786+0000 7f8e3d631b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281b1490 con 0x55877b8bc4a0 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.786+0000 7f8e3d631b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281aed20 con 0x55877b91c6a0 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.786+0000 7f8e3d631b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281acb70 con 0x55877b9a6160 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 0x55877ba430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e27fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 0x55877ba430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46302/0 (socket says 172.21.15.46:46302) 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e27fff700 1 -- 172.21.15.46:0/773452840 learned_addr learned my addr 172.21.15.46:0/773452840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e2d1b8700 1 --2- 172.21.15.46:0/773452840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba49170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/773452840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b91c6a0 0x55877ba4f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e27fff700 1 -- 172.21.15.46:0/773452840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 msgr2=0x55877ba49170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e27fff700 1 --2- 172.21.15.46:0/773452840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba49170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e27fff700 1 -- 172.21.15.46:0/773452840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b91c6a0 msgr2=0x55877ba4f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e27fff700 1 --2- 172.21.15.46:0/773452840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b91c6a0 0x55877ba4f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e27fff700 1 -- 172.21.15.46:0/773452840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55877ba2df60 con 0x55877b8bc4a0 2024-05-09T00:18:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/773452840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b91c6a0 0x55877ba4f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:18:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e27fff700 1 --2- 172.21.15.46:0/773452840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 0x55877ba430d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8e18022750 tx=0x7f8e180b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f014ed4b1b7dc114 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e1e7fc700 1 -- 172.21.15.46:0/773452840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e180aa730 con 0x55877b8bc4a0 2024-05-09T00:18:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e1e7fc700 1 -- 172.21.15.46:0/773452840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1806d070 con 0x55877b8bc4a0 2024-05-09T00:18:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.787+0000 7f8e1e7fc700 1 -- 172.21.15.46:0/773452840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e180aab90 con 0x55877b8bc4a0 2024-05-09T00:18:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.788+0000 7f8e3d631b80 1 -- 172.21.15.46:0/773452840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 msgr2=0x55877ba430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.788+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/773452840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 0x55877ba430d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8e18022750 tx=0x7f8e180b1040 comp rx=0 tx=0).stop 2024-05-09T00:18:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.788+0000 7f8e3d631b80 1 -- 172.21.15.46:0/773452840 shutdown_connections 2024-05-09T00:18:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.788+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/773452840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b9a6160 0x55877ba49170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.788+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/773452840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b91c6a0 0x55877ba4f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.788+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/773452840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b8bc4a0 0x55877ba430d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.788+0000 7f8e3d631b80 1 -- 172.21.15.46:0/773452840 >> 172.21.15.46:0/773452840 conn(0x55877ba39e10 msgr2=0x55877ba429c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.788+0000 7f8e3d631b80 1 -- 172.21.15.46:0/773452840 shutdown_connections 2024-05-09T00:18:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.788+0000 7f8e3d631b80 1 -- 172.21.15.46:0/773452840 wait complete. 2024-05-09T00:18:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.789+0000 7f8e3d631b80 1 -- start start 2024-05-09T00:18:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.789+0000 7f8e3d631b80 1 Processor -- start 2024-05-09T00:18:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.789+0000 7f8e3d631b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b8bc4a0 0x55877ba4efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.789+0000 7f8e3d631b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b91c6a0 0x55877ba4f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.789+0000 7f8e3d631b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b9a6160 0x55877ba4c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.789+0000 7f8e3d631b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281ad1e0 con 0x55877b9a6160 2024-05-09T00:18:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.789+0000 7f8e3d631b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281ae650 con 0x55877b8bc4a0 2024-05-09T00:18:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.789+0000 7f8e3d631b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281ad8b0 con 0x55877b91c6a0 2024-05-09T00:18:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.789+0000 7f8e3d631b80 10 client.0 _ll_register_callbacks cb 0x55877b7cd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e2d1b8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b9a6160 0x55877ba4c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e2d1b8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b9a6160 0x55877ba4c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46318/0 (socket says 172.21.15.46:46318) 2024-05-09T00:18:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e2d1b8700 1 -- 172.21.15.46:0/1797941077 learned_addr learned my addr 172.21.15.46:0/1797941077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e27fff700 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b91c6a0 0x55877ba4f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b8bc4a0 0x55877ba4efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e2d1b8700 1 -- 172.21.15.46:0/1797941077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b91c6a0 msgr2=0x55877ba4f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e2d1b8700 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b91c6a0 0x55877ba4f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e2d1b8700 1 -- 172.21.15.46:0/1797941077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b8bc4a0 msgr2=0x55877ba4efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e2d1b8700 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b8bc4a0 0x55877ba4efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e2d1b8700 1 -- 172.21.15.46:0/1797941077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55877ba4cac0 con 0x55877b9a6160 2024-05-09T00:18:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b8bc4a0 0x55877ba4efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e2d1b8700 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b9a6160 0x55877ba4c480 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8e20000f80 tx=0x7f8e20005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb7b5bc231ac19f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e07fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e07fff700 1 -- 172.21.15.46:0/1797941077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20030020 con 0x55877b9a6160 2024-05-09T00:18:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e07fff700 1 -- 172.21.15.46:0/1797941077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8e20021b10 con 0x55877b9a6160 2024-05-09T00:18:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.790+0000 7f8e3d631b80 1 -- 172.21.15.46:0/1797941077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e1806d070 con 0x55877b9a6160 2024-05-09T00:18:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.791+0000 7f8e07fff700 1 -- 172.21.15.46:0/1797941077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8e2001a040 con 0x55877b9a6160 2024-05-09T00:18:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.791+0000 7f8e3d631b80 10 client.16458 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.791+0000 7f8e3d631b80 1 -- 172.21.15.46:0/1797941077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55877babd0c0 con 0x55877b9a6160 2024-05-09T00:18:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.791+0000 7f8e07fff700 1 -- 172.21.15.46:0/1797941077 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1806d070 con 0x55877b9a6160 2024-05-09T00:18:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.791+0000 7f8e07fff700 1 -- 172.21.15.46:0/1797941077 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8e2002a6f0 con 0x55877b9a6160 2024-05-09T00:18:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.791+0000 7f8e3d631b80 10 client.16458 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.791+0000 7f8e3d631b80 10 client.16458 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:18:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.791+0000 7f8e3d631b80 20 client.16458 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.791+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55877ba396a0 0x55877babf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.791+0000 7f8e3d631b80 4 client.16458 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:18:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.791+0000 7f8e3d631b80 1 -- 172.21.15.46:0/1797941077 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55877b963a70 con 0x55877ba396a0 2024-05-09T00:18:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.791+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55877ba396a0 0x55877babf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e07fff700 10 client.16458 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:18:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e2c9b7700 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55877ba396a0 0x55877babf780 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98130a4c33c88726 server_cookie=96e59d431ddb24b1 in_seq=0 out_seq=0 2024-05-09T00:18:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e07fff700 1 -- 172.21.15.46:0/1797941077 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55877b963a70 con 0x55877ba396a0 2024-05-09T00:18:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e07fff700 10 client.16458 handle_command_reply: tid=0 2024-05-09T00:18:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e3d631b80 1 client.16458 shutdown 2024-05-09T00:18:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e3d631b80 1 -- 172.21.15.46:0/1797941077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b9a6160 msgr2=0x55877ba4c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b9a6160 0x55877ba4c480 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8e20000f80 tx=0x7f8e20005f10 comp rx=0 tx=0).stop 2024-05-09T00:18:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e3d631b80 1 -- 172.21.15.46:0/1797941077 shutdown_connections 2024-05-09T00:18:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55877ba396a0 0x55877babf780 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877b9a6160 0x55877ba4c480 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55877b8bc4a0 0x55877ba4efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e3d631b80 1 --2- 172.21.15.46:0/1797941077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55877b91c6a0 0x55877ba4f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e3d631b80 1 -- 172.21.15.46:0/1797941077 >> 172.21.15.46:0/1797941077 conn(0x55877ba39e10 msgr2=0x55877ba3e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e07fff700 0 client.16458 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:18:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e3d631b80 1 -- 172.21.15.46:0/1797941077 shutdown_connections 2024-05-09T00:18:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.792+0000 7f8e3d631b80 1 -- 172.21.15.46:0/1797941077 wait complete. 2024-05-09T00:18:18.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e3d631b80 20 client.16458 trim_cache size 0 max 16384 2024-05-09T00:18:18.800 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:18:18.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/926546215 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8e140b2880 msgr2=0x7f8e140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:18.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e2ec1b700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8e140b2880 0x7f8e140b4d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8e10005950 tx=0x7f8e1000b410 comp rx=0 tx=0).stop 2024-05-09T00:18:18.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/926546215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e28107e10 msgr2=0x7f8e2819c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:18.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e2ec1b700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e28107e10 0x7f8e2819c1b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8e18006010 tx=0x7f8e180047b0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/926546215 shutdown_connections 2024-05-09T00:18:18.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e2ec1b700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8e140b2880 0x7f8e140b4d90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e2ec1b700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e28108760 0x7f8e281a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e2ec1b700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e281034e0 0x7f8e2819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e2ec1b700 1 --2- 172.21.15.46:0/926546215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e28107e10 0x7f8e2819c1b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:18.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/926546215 >> 172.21.15.46:0/926546215 conn(0x7f8e280fef30 msgr2=0x7f8e281096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:18.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/926546215 shutdown_connections 2024-05-09T00:18:18.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:18.793+0000 7f8e2ec1b700 1 -- 172.21.15.46:0/926546215 wait complete. 2024-05-09T00:18:19.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:18 smithi046 ceph-mon[30151]: pgmap v584: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:19.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:18 smithi169 ceph-mon[34777]: pgmap v584: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:19.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:18 smithi142 ceph-mon[34533]: pgmap v584: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:19.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:18:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:18:21.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:20 smithi046 ceph-mon[30151]: pgmap v585: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:18:21.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:20 smithi169 ceph-mon[34777]: pgmap v585: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:18:21.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:20 smithi142 ceph-mon[34533]: pgmap v585: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-05-09T00:18:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.657+0000 7fedfdb92700 1 -- 172.21.15.46:0/1257671403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedf8075d30 msgr2=0x7fedf8076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.657+0000 7fedfdb92700 1 --2- 172.21.15.46:0/1257671403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedf8075d30 0x7fedf8076140 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fede8009990 tx=0x7fede8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.657+0000 7fedfdb92700 1 -- 172.21.15.46:0/1257671403 shutdown_connections 2024-05-09T00:18:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.657+0000 7fedfdb92700 1 --2- 172.21.15.46:0/1257671403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf8110ba0 0x7fedf8113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.657+0000 7fedfdb92700 1 --2- 172.21.15.46:0/1257671403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedf8075d30 0x7fedf8076140 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.657+0000 7fedfdb92700 1 --2- 172.21.15.46:0/1257671403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf8076680 0x7fedf8110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.657+0000 7fedfdb92700 1 -- 172.21.15.46:0/1257671403 >> 172.21.15.46:0/1257671403 conn(0x7fedf8100040 msgr2=0x7fedf8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.657+0000 7fedfdb92700 1 -- 172.21.15.46:0/1257671403 shutdown_connections 2024-05-09T00:18:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.657+0000 7fedfdb92700 1 -- 172.21.15.46:0/1257671403 wait complete. 2024-05-09T00:18:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.658+0000 7fedfdb92700 1 Processor -- start 2024-05-09T00:18:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.658+0000 7fedfdb92700 1 -- start start 2024-05-09T00:18:21.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.658+0000 7fedfdb92700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedf8075d30 0x7fedf819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.658+0000 7fedfdb92700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf8076680 0x7fedf819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.658+0000 7fedfdb92700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf8110ba0 0x7fedf81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.658+0000 7fedfdb92700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fedf8116690 con 0x7fedf8075d30 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.658+0000 7fedfdb92700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fedf8116510 con 0x7fedf8076680 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.658+0000 7fedfdb92700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fedf8116810 con 0x7fedf8110ba0 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.658+0000 7fedf77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedf8075d30 0x7fedf819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedf8075d30 0x7fedf819d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46334/0 (socket says 172.21.15.46:46334) 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf77fe700 1 -- 172.21.15.46:0/360957292 learned_addr learned my addr 172.21.15.46:0/360957292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf6ffd700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf8076680 0x7fedf819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf7fff700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf8110ba0 0x7fedf81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf77fe700 1 -- 172.21.15.46:0/360957292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf8110ba0 msgr2=0x7fedf81a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf77fe700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf8110ba0 0x7fedf81a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf77fe700 1 -- 172.21.15.46:0/360957292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf8076680 msgr2=0x7fedf819d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf77fe700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf8076680 0x7fedf819d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf77fe700 1 -- 172.21.15.46:0/360957292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf81a1ef0 con 0x7fedf8075d30 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf77fe700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedf8075d30 0x7fedf819d020 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fede80057c0 tx=0x7fede80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf4ff9700 1 -- 172.21.15.46:0/360957292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8027070 con 0x7fedf8075d30 2024-05-09T00:18:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf4ff9700 1 -- 172.21.15.46:0/360957292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede8003f20 con 0x7fedf8075d30 2024-05-09T00:18:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedfdb92700 1 -- 172.21.15.46:0/360957292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf81a2180 con 0x7fedf8075d30 2024-05-09T00:18:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedf4ff9700 1 -- 172.21.15.46:0/360957292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8022c30 con 0x7fedf8075d30 2024-05-09T00:18:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.659+0000 7fedfdb92700 1 -- 172.21.15.46:0/360957292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf81a2660 con 0x7fedf8075d30 2024-05-09T00:18:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.662+0000 7fedf4ff9700 1 -- 172.21.15.46:0/360957292 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fede802a070 con 0x7fedf8075d30 2024-05-09T00:18:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.663+0000 7fedf4ff9700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fede40b2880 0x7fede40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.663+0000 7fedf4ff9700 1 -- 172.21.15.46:0/360957292 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fede80e31f0 con 0x7fedf8075d30 2024-05-09T00:18:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.663+0000 7fedf6ffd700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fede40b2880 0x7fede40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.663+0000 7fee0c5a8b80 1 Processor -- start 2024-05-09T00:18:21.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fee0c5a8b80 1 -- start start 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fedf6ffd700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fede40b2880 0x7fede40b4d90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fedf8103490 tx=0x7fede00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fee0c5a8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 0x564ffce4b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fee0c5a8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce4ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fee0c5a8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce45ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fee0c5a8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ffcd67ff0 con 0x564ffccc84a0 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fee0c5a8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564ffcd067f0 con 0x564ffce456a0 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fee0c5a8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ffcd63fb0 con 0x564ffcdb2160 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fedf6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce4ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fedf6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce4ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46350/0 (socket says 172.21.15.46:46350) 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fedf6ffd700 1 -- 172.21.15.46:0/728551688 learned_addr learned my addr 172.21.15.46:0/728551688 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fedf77fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 0x564ffce4b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fedf7fff700 1 --2- 172.21.15.46:0/728551688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce45ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fedf6ffd700 1 -- 172.21.15.46:0/728551688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 msgr2=0x564ffce4b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fedf6ffd700 1 --2- 172.21.15.46:0/728551688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 0x564ffce4b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fedf6ffd700 1 -- 172.21.15.46:0/728551688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 msgr2=0x564ffce45ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fedf6ffd700 1 --2- 172.21.15.46:0/728551688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce45ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.664+0000 7fedf6ffd700 1 -- 172.21.15.46:0/728551688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ffccfe370 con 0x564ffccc84a0 2024-05-09T00:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.665+0000 7fedf6ffd700 1 --2- 172.21.15.46:0/728551688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce4ba30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fede0010e40 tx=0x7fede0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e4fe5d245c7982 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.665+0000 7fededffb700 1 -- 172.21.15.46:0/728551688 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0024070 con 0x564ffccc84a0 2024-05-09T00:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.665+0000 7fededffb700 1 -- 172.21.15.46:0/728551688 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede0015070 con 0x564ffccc84a0 2024-05-09T00:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.665+0000 7fededffb700 1 -- 172.21.15.46:0/728551688 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0011860 con 0x564ffccc84a0 2024-05-09T00:18:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.666+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/728551688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 msgr2=0x564ffce4ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.666+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/728551688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce4ba30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fede0010e40 tx=0x7fede0013000 comp rx=0 tx=0).stop 2024-05-09T00:18:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.666+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/728551688 shutdown_connections 2024-05-09T00:18:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.666+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/728551688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce45ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.666+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/728551688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 0x564ffce4b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.666+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/728551688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce4ba30 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.666+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/728551688 >> 172.21.15.46:0/728551688 conn(0x564ffcd286a0 msgr2=0x564ffce40400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.666+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/728551688 shutdown_connections 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.666+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/728551688 wait complete. 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.666+0000 7fee0c5a8b80 1 -- start start 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fee0c5a8b80 1 Processor -- start 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fee0c5a8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce4f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fee0c5a8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 0x564ffce4f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fee0c5a8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce4fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fee0c5a8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ffcbe3ba0 con 0x564ffccc84a0 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fee0c5a8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564ffcd22150 con 0x564ffce456a0 2024-05-09T00:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fee0c5a8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ffce521d0 con 0x564ffcdb2160 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fee0c5a8b80 10 client.0 _ll_register_callbacks cb 0x564ffcbd96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fedf77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce4f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fedf77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce4f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46352/0 (socket says 172.21.15.46:46352) 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fedf77fe700 1 -- 172.21.15.46:0/839498821 learned_addr learned my addr 172.21.15.46:0/839498821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fedf6ffd700 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 0x564ffce4f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fedf7fff700 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce4fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fedf77fe700 1 -- 172.21.15.46:0/839498821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 msgr2=0x564ffce4f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fedf77fe700 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 0x564ffce4f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fedf77fe700 1 -- 172.21.15.46:0/839498821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 msgr2=0x564ffce4fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fedf77fe700 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce4fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.667+0000 7fedf77fe700 1 -- 172.21.15.46:0/839498821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ffcbe3490 con 0x564ffccc84a0 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fedf77fe700 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce4f2f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fede8006010 tx=0x7fede80aada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d9e9342b55662e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fedd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fedd6ffd700 1 -- 172.21.15.46:0/839498821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede80e2070 con 0x564ffccc84a0 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fedd6ffd700 1 -- 172.21.15.46:0/839498821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fede80b1070 con 0x564ffccc84a0 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/839498821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fede0015070 con 0x564ffccc84a0 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fedd6ffd700 1 -- 172.21.15.46:0/839498821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fede80095d0 con 0x564ffccc84a0 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fee0c5a8b80 10 client.16470 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/839498821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ffce5ccc0 con 0x564ffccc84a0 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fedd6ffd700 1 -- 172.21.15.46:0/839498821 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede0015070 con 0x564ffccc84a0 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fedd6ffd700 1 -- 172.21.15.46:0/839498821 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fede80a9ac0 con 0x564ffccc84a0 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fee0c5a8b80 10 client.16470 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fee0c5a8b80 10 client.16470 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.668+0000 7fee0c5a8b80 20 client.16470 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.669+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564ffce5f230 0x564ffce61670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.669+0000 7fee0c5a8b80 4 client.16470 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.669+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/839498821 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ffcd759d0 con 0x564ffce5f230 2024-05-09T00:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.669+0000 7fedf6ffd700 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564ffce5f230 0x564ffce61670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.669+0000 7fedf6ffd700 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564ffce5f230 0x564ffce61670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=afe428431890adfa server_cookie=ab624db4583090aa in_seq=0 out_seq=0 2024-05-09T00:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.669+0000 7fedd6ffd700 10 client.16470 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:18:21.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fedd6ffd700 1 -- 172.21.15.46:0/839498821 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ffcd759d0 con 0x564ffce5f230 2024-05-09T00:18:21.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fedd6ffd700 10 client.16470 handle_command_reply: tid=0 2024-05-09T00:18:21.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fee0c5a8b80 1 client.16470 shutdown 2024-05-09T00:18:21.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/839498821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 msgr2=0x564ffce4f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:21.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce4f2f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fede8006010 tx=0x7fede80aada0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/839498821 shutdown_connections 2024-05-09T00:18:21.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564ffce5f230 0x564ffce61670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fedd6ffd700 0 client.16470 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:18:21.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce4fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce4f2f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/839498821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 0x564ffce4f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/839498821 >> 172.21.15.46:0/839498821 conn(0x564ffcd286a0 msgr2=0x564ffce401c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:21.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.670+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/839498821 shutdown_connections 2024-05-09T00:18:21.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.671+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/839498821 wait complete. 2024-05-09T00:18:21.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.671+0000 7fee0c5a8b80 20 client.16470 trim_cache size 0 max 16384 2024-05-09T00:18:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fee0c5a8b80 1 Processor -- start 2024-05-09T00:18:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fee0c5a8b80 1 -- start start 2024-05-09T00:18:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fee0c5a8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce5af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fee0c5a8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 0x564ffce49860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fee0c5a8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce4fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fee0c5a8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fedf81b36f0 con 0x564ffccc84a0 2024-05-09T00:18:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fee0c5a8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fedf81b0f80 con 0x564ffce456a0 2024-05-09T00:18:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fee0c5a8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fedf81aedb0 con 0x564ffcdb2160 2024-05-09T00:18:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fedf77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce5af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fedf77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce5af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46358/0 (socket says 172.21.15.46:46358) 2024-05-09T00:18:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fedf77fe700 1 -- 172.21.15.46:0/3463558013 learned_addr learned my addr 172.21.15.46:0/3463558013 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fedf6ffd700 1 --2- 172.21.15.46:0/3463558013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 0x564ffce49860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.686+0000 7fedf7fff700 1 --2- 172.21.15.46:0/3463558013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce4fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.687+0000 7fedf77fe700 1 -- 172.21.15.46:0/3463558013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 msgr2=0x564ffce49860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.687+0000 7fedf77fe700 1 --2- 172.21.15.46:0/3463558013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 0x564ffce49860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.687+0000 7fedf77fe700 1 -- 172.21.15.46:0/3463558013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 msgr2=0x564ffce4fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.687+0000 7fedf77fe700 1 --2- 172.21.15.46:0/3463558013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce4fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.687+0000 7fedf77fe700 1 -- 172.21.15.46:0/3463558013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ffce39f60 con 0x564ffccc84a0 2024-05-09T00:18:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.687+0000 7fedf7fff700 1 --2- 172.21.15.46:0/3463558013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce4fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.687+0000 7fedf77fe700 1 --2- 172.21.15.46:0/3463558013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce5af30 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fede80aa560 tx=0x7fede8005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c13f0e0f207748d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.687+0000 7feded7fa700 1 -- 172.21.15.46:0/3463558013 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede80c7070 con 0x564ffccc84a0 2024-05-09T00:18:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.687+0000 7feded7fa700 1 -- 172.21.15.46:0/3463558013 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede80aa370 con 0x564ffccc84a0 2024-05-09T00:18:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.687+0000 7feded7fa700 1 -- 172.21.15.46:0/3463558013 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede80b10b0 con 0x564ffccc84a0 2024-05-09T00:18:21.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.688+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/3463558013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 msgr2=0x564ffce5af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.688+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/3463558013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce5af30 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fede80aa560 tx=0x7fede8005960 comp rx=0 tx=0).stop 2024-05-09T00:18:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.688+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/3463558013 shutdown_connections 2024-05-09T00:18:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.688+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/3463558013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffce456a0 0x564ffce4fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.688+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/3463558013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffccc84a0 0x564ffce5af30 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.688+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/3463558013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffcdb2160 0x564ffce49860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.688+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/3463558013 >> 172.21.15.46:0/3463558013 conn(0x564ffcd286a0 msgr2=0x564ffce4dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.688+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/3463558013 shutdown_connections 2024-05-09T00:18:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.688+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/3463558013 wait complete. 2024-05-09T00:18:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.688+0000 7fee0c5a8b80 1 -- start start 2024-05-09T00:18:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.688+0000 7fee0c5a8b80 1 Processor -- start 2024-05-09T00:18:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fee0c5a8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffccc84a0 0x564ffce57730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fee0c5a8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffcdb2160 0x564ffce48770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fee0c5a8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffce456a0 0x564ffce48cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fee0c5a8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fedf81af440 con 0x564ffce456a0 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fee0c5a8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fedf81b08b0 con 0x564ffcdb2160 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fee0c5a8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fedf81afb10 con 0x564ffccc84a0 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fee0c5a8b80 10 client.0 _ll_register_callbacks cb 0x564ffcbd96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fedf7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffce456a0 0x564ffce48cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fedf7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffce456a0 0x564ffce48cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46372/0 (socket says 172.21.15.46:46372) 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fedf7fff700 1 -- 172.21.15.46:0/712176017 learned_addr learned my addr 172.21.15.46:0/712176017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fedf6ffd700 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffcdb2160 0x564ffce48770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fedf77fe700 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffccc84a0 0x564ffce57730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fedf7fff700 1 -- 172.21.15.46:0/712176017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffccc84a0 msgr2=0x564ffce57730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fedf7fff700 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffccc84a0 0x564ffce57730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fedf7fff700 1 -- 172.21.15.46:0/712176017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffcdb2160 msgr2=0x564ffce48770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fedf7fff700 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffcdb2160 0x564ffce48770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fedf7fff700 1 -- 172.21.15.46:0/712176017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ffce492a0 con 0x564ffce456a0 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fedf7fff700 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffce456a0 0x564ffce48cb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fedf0007bc0 tx=0x7fedf0007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5453505782335ce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.689+0000 7fedd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fedd6ffd700 1 -- 172.21.15.46:0/712176017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0004db0 con 0x564ffce456a0 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fedd6ffd700 1 -- 172.21.15.46:0/712176017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fedf00175b0 con 0x564ffce456a0 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/712176017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fede80aa370 con 0x564ffce456a0 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fedd6ffd700 1 -- 172.21.15.46:0/712176017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fedf00177b0 con 0x564ffce456a0 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fee0c5a8b80 10 client.16476 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/712176017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ffce56000 con 0x564ffce456a0 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fedd6ffd700 1 -- 172.21.15.46:0/712176017 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede80aa370 con 0x564ffce456a0 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fedd6ffd700 1 -- 172.21.15.46:0/712176017 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fedf0021a90 con 0x564ffce456a0 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fee0c5a8b80 10 client.16476 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fee0c5a8b80 10 client.16476 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:18:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fee0c5a8b80 20 client.16476 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564ffce5f230 0x564ffcecbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fee0c5a8b80 4 client.16476 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/712176017 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ffcd6fa70 con 0x564ffce5f230 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.690+0000 7fedf77fe700 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564ffce5f230 0x564ffcecbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fedd6ffd700 10 client.16476 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fedf77fe700 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564ffce5f230 0x564ffcecbf10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91a75d82bf3d71d1 server_cookie=b19a5d155447019 in_seq=0 out_seq=0 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fedd6ffd700 1 -- 172.21.15.46:0/712176017 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ffcd6fa70 con 0x564ffce5f230 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fedd6ffd700 10 client.16476 handle_command_reply: tid=0 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fee0c5a8b80 1 client.16476 shutdown 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/712176017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffce456a0 msgr2=0x564ffce48cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffce456a0 0x564ffce48cb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fedf0007bc0 tx=0x7fedf0007bf0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/712176017 shutdown_connections 2024-05-09T00:18:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564ffce5f230 0x564ffcecbf10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ffce456a0 0x564ffce48cb0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ffccc84a0 0x564ffce57730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fedd6ffd700 0 client.16476 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:18:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fee0c5a8b80 1 --2- 172.21.15.46:0/712176017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ffcdb2160 0x564ffce48770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/712176017 >> 172.21.15.46:0/712176017 conn(0x564ffcd286a0 msgr2=0x564ffce4fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.691+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/712176017 shutdown_connections 2024-05-09T00:18:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fee0c5a8b80 1 -- 172.21.15.46:0/712176017 wait complete. 2024-05-09T00:18:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fee0c5a8b80 20 client.16476 trim_cache size 0 max 16384 2024-05-09T00:18:21.699 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:18:21.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fedfdb92700 1 -- 172.21.15.46:0/360957292 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fede40b2880 msgr2=0x7fede40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:21.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fedfdb92700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fede40b2880 0x7fede40b4d90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fedf8103490 tx=0x7fede00073d0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fedfdb92700 1 -- 172.21.15.46:0/360957292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedf8075d30 msgr2=0x7fedf819d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:21.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fedfdb92700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedf8075d30 0x7fedf819d020 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fede80057c0 tx=0x7fede80057f0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fedfdb92700 1 -- 172.21.15.46:0/360957292 shutdown_connections 2024-05-09T00:18:21.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fedfdb92700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fede40b2880 0x7fede40b4d90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fedfdb92700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf8110ba0 0x7fedf81a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fedfdb92700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedf8075d30 0x7fedf819d020 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fedfdb92700 1 --2- 172.21.15.46:0/360957292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf8076680 0x7fedf819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:21.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fedfdb92700 1 -- 172.21.15.46:0/360957292 >> 172.21.15.46:0/360957292 conn(0x7fedf8100040 msgr2=0x7fedf81006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:21.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fedfdb92700 1 -- 172.21.15.46:0/360957292 shutdown_connections 2024-05-09T00:18:21.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:21.692+0000 7fedfdb92700 1 -- 172.21.15.46:0/360957292 wait complete. 2024-05-09T00:18:22.585 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:18:22.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:22 smithi046 ceph-mon[30151]: pgmap v586: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:22.843 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:18:23.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:22 smithi169 ceph-mon[34777]: pgmap v586: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:23.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:22 smithi142 ceph-mon[34533]: pgmap v586: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:24.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.676+0000 7ffaf406e700 1 -- 172.21.15.46:0/95206341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec107e10 msgr2=0x7ffaec108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:24.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.676+0000 7ffaf406e700 1 --2- 172.21.15.46:0/95206341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec107e10 0x7ffaec108220 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0099c0 tx=0x7ffadc009cd0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.676+0000 7ffaf406e700 1 -- 172.21.15.46:0/95206341 shutdown_connections 2024-05-09T00:18:24.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.676+0000 7ffaf406e700 1 --2- 172.21.15.46:0/95206341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaec108760 0x7ffaec1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.676+0000 7ffaf406e700 1 --2- 172.21.15.46:0/95206341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaec1034e0 0x7ffaec1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.676+0000 7ffaf406e700 1 --2- 172.21.15.46:0/95206341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec107e10 0x7ffaec108220 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.676+0000 7ffaf406e700 1 -- 172.21.15.46:0/95206341 >> 172.21.15.46:0/95206341 conn(0x7ffaec0fef30 msgr2=0x7ffaec101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.676+0000 7ffaf406e700 1 -- 172.21.15.46:0/95206341 shutdown_connections 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.676+0000 7ffaf406e700 1 -- 172.21.15.46:0/95206341 wait complete. 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf406e700 1 Processor -- start 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf406e700 1 -- start start 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf406e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec1034e0 0x7ffaec0ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf406e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaec107e10 0x7ffaec0ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf406e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaec108760 0x7ffaec0ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf406e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec113370 con 0x7ffaec1034e0 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf406e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1131f0 con 0x7ffaec108760 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf406e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1134f0 con 0x7ffaec107e10 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf1e0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec1034e0 0x7ffaec0ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf1609700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaec107e10 0x7ffaec0ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf1e0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec1034e0 0x7ffaec0ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46384/0 (socket says 172.21.15.46:46384) 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf1e0a700 1 -- 172.21.15.46:0/452496022 learned_addr learned my addr 172.21.15.46:0/452496022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf1609700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaec107e10 0x7ffaec0ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:45868/0 (socket says 172.21.15.46:45868) 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.677+0000 7ffaf260b700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaec108760 0x7ffaec0ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.678+0000 7ffaf1e0a700 1 -- 172.21.15.46:0/452496022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaec107e10 msgr2=0x7ffaec0ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.678+0000 7ffaf1e0a700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaec107e10 0x7ffaec0ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.678+0000 7ffaf1e0a700 1 -- 172.21.15.46:0/452496022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaec108760 msgr2=0x7ffaec0ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.678+0000 7ffaf1e0a700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaec108760 0x7ffaec0ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.678+0000 7ffaf1e0a700 1 -- 172.21.15.46:0/452496022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec1a91d0 con 0x7ffaec1034e0 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.678+0000 7ffaf1e0a700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec1034e0 0x7ffaec0ff410 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ffae800c920 tx=0x7ffae800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.678+0000 7ffae2ffd700 1 -- 172.21.15.46:0/452496022 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8004a30 con 0x7ffaec1034e0 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.678+0000 7ffaf406e700 1 -- 172.21.15.46:0/452496022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaec1a94c0 con 0x7ffaec1034e0 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.678+0000 7ffae2ffd700 1 -- 172.21.15.46:0/452496022 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae80054d0 con 0x7ffaec1034e0 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.678+0000 7ffae2ffd700 1 -- 172.21.15.46:0/452496022 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8007bd0 con 0x7ffaec1034e0 2024-05-09T00:18:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.678+0000 7ffaf406e700 1 -- 172.21.15.46:0/452496022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaec1a9a20 con 0x7ffaec1034e0 2024-05-09T00:18:24.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.681+0000 7ffae2ffd700 1 -- 172.21.15.46:0/452496022 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffae800f450 con 0x7ffaec1034e0 2024-05-09T00:18:24.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.682+0000 7ffae2ffd700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ffad80b2880 0x7ffad80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.682+0000 7ffaf1609700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ffad80b2880 0x7ffad80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.682+0000 7ffae2ffd700 1 -- 172.21.15.46:0/452496022 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ffae80e1d70 con 0x7ffaec1034e0 2024-05-09T00:18:24.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.682+0000 7ffb02a84b80 1 Processor -- start 2024-05-09T00:18:24.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.682+0000 7ffaf1609700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ffad80b2880 0x7ffad80b4d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffadc006010 tx=0x7ffadc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:24.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.682+0000 7ffb02a84b80 1 -- start start 2024-05-09T00:18:24.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffb02a84b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3db46d00 0x55ca3dc4f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffb02a84b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc4f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffb02a84b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 0x55ca3dc49810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffb02a84b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ca3db6c0a0 con 0x55ca3dc49400 2024-05-09T00:18:24.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffb02a84b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ca3db119a0 con 0x55ca3db46d00 2024-05-09T00:18:24.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffb02a84b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ca3db68060 con 0x55ca3dacc4a0 2024-05-09T00:18:24.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffaf260b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 0x55ca3dc49810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffaf1e0a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3db46d00 0x55ca3dc4f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffaf1609700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc4f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffaf260b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 0x55ca3dc49810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46390/0 (socket says 172.21.15.46:46390) 2024-05-09T00:18:24.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffaf260b700 1 -- 172.21.15.46:0/422657426 learned_addr learned my addr 172.21.15.46:0/422657426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:24.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffaf1e0a700 1 --2- 172.21.15.46:0/422657426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3db46d00 0x55ca3dc4f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:42974/0 (socket says 172.21.15.46:42974) 2024-05-09T00:18:24.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffaf260b700 1 -- 172.21.15.46:0/422657426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dacc4a0 msgr2=0x55ca3dc4f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:24.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffaf260b700 1 --2- 172.21.15.46:0/422657426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc4f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffaf260b700 1 -- 172.21.15.46:0/422657426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3db46d00 msgr2=0x55ca3dc4f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:24.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffaf260b700 1 --2- 172.21.15.46:0/422657426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3db46d00 0x55ca3dc4f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.683+0000 7ffaf260b700 1 -- 172.21.15.46:0/422657426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca3db02500 con 0x55ca3dc49400 2024-05-09T00:18:24.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.684+0000 7ffaf260b700 1 --2- 172.21.15.46:0/422657426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 0x55ca3dc49810 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ffae400a730 tx=0x7ffae40078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0b00adc68d60e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:24.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.684+0000 7ffad7fff700 1 -- 172.21.15.46:0/422657426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4003a00 con 0x55ca3dc49400 2024-05-09T00:18:24.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.684+0000 7ffad7fff700 1 -- 172.21.15.46:0/422657426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae4003ba0 con 0x55ca3dc49400 2024-05-09T00:18:24.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.684+0000 7ffad7fff700 1 -- 172.21.15.46:0/422657426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4014600 con 0x55ca3dc49400 2024-05-09T00:18:24.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.685+0000 7ffb02a84b80 1 -- 172.21.15.46:0/422657426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 msgr2=0x55ca3dc49810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:24.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.685+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/422657426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 0x55ca3dc49810 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ffae400a730 tx=0x7ffae40078d0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.685+0000 7ffb02a84b80 1 -- 172.21.15.46:0/422657426 shutdown_connections 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.685+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/422657426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 0x55ca3dc49810 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.685+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/422657426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3db46d00 0x55ca3dc4f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.685+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/422657426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc4f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.685+0000 7ffb02a84b80 1 -- 172.21.15.46:0/422657426 >> 172.21.15.46:0/422657426 conn(0x55ca3dacff70 msgr2=0x55ca3dc44220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.685+0000 7ffb02a84b80 1 -- 172.21.15.46:0/422657426 shutdown_connections 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.685+0000 7ffb02a84b80 1 -- 172.21.15.46:0/422657426 wait complete. 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.685+0000 7ffb02a84b80 1 -- start start 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.685+0000 7ffb02a84b80 1 Processor -- start 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.686+0000 7ffb02a84b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc53080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.686+0000 7ffb02a84b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3db46d00 0x55ca3dc535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.686+0000 7ffb02a84b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dc49400 0x55ca3dc53b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.686+0000 7ffb02a84b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ca3dc4a100 con 0x55ca3db46d00 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.686+0000 7ffb02a84b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ca3db06b20 con 0x55ca3dacc4a0 2024-05-09T00:18:24.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.686+0000 7ffb02a84b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ca3dc5f1d0 con 0x55ca3dc49400 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.686+0000 7ffb02a84b80 10 client.0 _ll_register_callbacks cb 0x55ca3d9dd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.686+0000 7ffaf1e0a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc53080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.686+0000 7ffaf1609700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3db46d00 0x55ca3dc535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.686+0000 7ffaf260b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dc49400 0x55ca3dc53b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.686+0000 7ffaf1e0a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc53080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:42986/0 (socket says 172.21.15.46:42986) 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.686+0000 7ffaf1e0a700 1 -- 172.21.15.46:0/2184466497 learned_addr learned my addr 172.21.15.46:0/2184466497 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffaf260b700 1 -- 172.21.15.46:0/2184466497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3dacc4a0 msgr2=0x55ca3dc53080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffaf260b700 1 --2- 172.21.15.46:0/2184466497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc53080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffaf260b700 1 -- 172.21.15.46:0/2184466497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3db46d00 msgr2=0x55ca3dc535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffaf260b700 1 --2- 172.21.15.46:0/2184466497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3db46d00 0x55ca3dc535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffaf260b700 1 -- 172.21.15.46:0/2184466497 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca3d9e7490 con 0x55ca3dc49400 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffaf260b700 1 --2- 172.21.15.46:0/2184466497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dc49400 0x55ca3dc53b00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffae4000f80 tx=0x7ffae4004450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=694515c4c81d8c86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffad57fa700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffad57fa700 1 -- 172.21.15.46:0/2184466497 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4022070 con 0x55ca3dc49400 2024-05-09T00:18:24.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2184466497 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffae4003ba0 con 0x55ca3dc49400 2024-05-09T00:18:24.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffad57fa700 1 -- 172.21.15.46:0/2184466497 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ffae400a2e0 con 0x55ca3dc49400 2024-05-09T00:18:24.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffad57fa700 1 -- 172.21.15.46:0/2184466497 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffae4003a00 con 0x55ca3dc49400 2024-05-09T00:18:24.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffb02a84b80 10 client.26068 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:24.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.687+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2184466497 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca3dc60b70 con 0x55ca3dc49400 2024-05-09T00:18:24.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.688+0000 7ffad57fa700 1 -- 172.21.15.46:0/2184466497 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae4003ba0 con 0x55ca3dc49400 2024-05-09T00:18:24.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.688+0000 7ffad57fa700 1 -- 172.21.15.46:0/2184466497 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ffae401ea70 con 0x55ca3dc49400 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.688+0000 7ffb02a84b80 10 client.26068 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.688+0000 7ffb02a84b80 10 client.26068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.688+0000 7ffb02a84b80 20 client.26068 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.688+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/2184466497 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ca3dc630a0 0x55ca3dc654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.688+0000 7ffb02a84b80 4 client.26068 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.688+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2184466497 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca3db79a80 con 0x55ca3dc630a0 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.688+0000 7ffaf1e0a700 1 --2- 172.21.15.46:0/2184466497 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ca3dc630a0 0x55ca3dc654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.689+0000 7ffaf1e0a700 1 --2- 172.21.15.46:0/2184466497 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ca3dc630a0 0x55ca3dc654c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca63b0e18457e5c6 server_cookie=1ba11864ec0d1a30 in_seq=0 out_seq=0 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.689+0000 7ffad57fa700 10 client.26068 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.689+0000 7ffad57fa700 1 -- 172.21.15.46:0/2184466497 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca3db79a80 con 0x55ca3dc630a0 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.689+0000 7ffad57fa700 10 client.26068 handle_command_reply: tid=0 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffb02a84b80 1 client.26068 shutdown 2024-05-09T00:18:24.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2184466497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dc49400 msgr2=0x55ca3dc53b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:24.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/2184466497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dc49400 0x55ca3dc53b00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffae4000f80 tx=0x7ffae4004450 comp rx=0 tx=0).stop 2024-05-09T00:18:24.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2184466497 shutdown_connections 2024-05-09T00:18:24.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/2184466497 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ca3dc630a0 0x55ca3dc654c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/2184466497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dc49400 0x55ca3dc53b00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/2184466497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc53080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffad57fa700 0 client.26068 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:24.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/2184466497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3db46d00 0x55ca3dc535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2184466497 >> 172.21.15.46:0/2184466497 conn(0x55ca3dacff70 msgr2=0x55ca3dc3dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:24.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2184466497 shutdown_connections 2024-05-09T00:18:24.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2184466497 wait complete. 2024-05-09T00:18:24.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.690+0000 7ffb02a84b80 20 client.26068 trim_cache size 0 max 16384 2024-05-09T00:18:24.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.705+0000 7ffb02a84b80 1 Processor -- start 2024-05-09T00:18:24.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.705+0000 7ffb02a84b80 1 -- start start 2024-05-09T00:18:24.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.705+0000 7ffb02a84b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc3e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.705+0000 7ffb02a84b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3db46d00 0x55ca3dc684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.705+0000 7ffb02a84b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 0x55ca3dc68a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.705+0000 7ffb02a84b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1b3780 con 0x55ca3dc49400 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.705+0000 7ffb02a84b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1b1010 con 0x55ca3dacc4a0 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.705+0000 7ffb02a84b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1aee40 con 0x55ca3db46d00 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.705+0000 7ffaf260b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 0x55ca3dc68a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffaf260b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 0x55ca3dc68a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46402/0 (socket says 172.21.15.46:46402) 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffaf260b700 1 -- 172.21.15.46:0/3819793474 learned_addr learned my addr 172.21.15.46:0/3819793474 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffaf1609700 1 --2- 172.21.15.46:0/3819793474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3db46d00 0x55ca3dc684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffaf1e0a700 1 --2- 172.21.15.46:0/3819793474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc3e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffaf260b700 1 -- 172.21.15.46:0/3819793474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3db46d00 msgr2=0x55ca3dc684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffaf260b700 1 --2- 172.21.15.46:0/3819793474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3db46d00 0x55ca3dc684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffaf260b700 1 -- 172.21.15.46:0/3819793474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3dacc4a0 msgr2=0x55ca3dc3e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffaf260b700 1 --2- 172.21.15.46:0/3819793474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc3e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffaf260b700 1 -- 172.21.15.46:0/3819793474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca3dc55920 con 0x55ca3dc49400 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffaf1e0a700 1 --2- 172.21.15.46:0/3819793474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc3e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffaf260b700 1 --2- 172.21.15.46:0/3819793474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 0x55ca3dc68a20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ffae400afd0 tx=0x7ffae4007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703b39633f7e3ca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:24.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffad77fe700 1 -- 172.21.15.46:0/3819793474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4022040 con 0x55ca3dc49400 2024-05-09T00:18:24.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffad77fe700 1 -- 172.21.15.46:0/3819793474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae4014da0 con 0x55ca3dc49400 2024-05-09T00:18:24.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.706+0000 7ffad77fe700 1 -- 172.21.15.46:0/3819793474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4005c60 con 0x55ca3dc49400 2024-05-09T00:18:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.707+0000 7ffb02a84b80 1 -- 172.21.15.46:0/3819793474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 msgr2=0x55ca3dc68a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.707+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/3819793474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 0x55ca3dc68a20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ffae400afd0 tx=0x7ffae4007ad0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.707+0000 7ffb02a84b80 1 -- 172.21.15.46:0/3819793474 shutdown_connections 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.707+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/3819793474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dc49400 0x55ca3dc68a20 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.707+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/3819793474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3dacc4a0 0x55ca3dc3e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.707+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/3819793474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3db46d00 0x55ca3dc684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.707+0000 7ffb02a84b80 1 -- 172.21.15.46:0/3819793474 >> 172.21.15.46:0/3819793474 conn(0x55ca3dacff70 msgr2=0x55ca3dc3dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.707+0000 7ffb02a84b80 1 -- 172.21.15.46:0/3819793474 shutdown_connections 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.707+0000 7ffb02a84b80 1 -- 172.21.15.46:0/3819793474 wait complete. 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.707+0000 7ffb02a84b80 1 -- start start 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.708+0000 7ffb02a84b80 1 Processor -- start 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.708+0000 7ffb02a84b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dacc4a0 0x55ca3dcc49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.708+0000 7ffb02a84b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3db46d00 0x55ca3dcc4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.708+0000 7ffb02a84b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dc49400 0x55ca3dc67780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.708+0000 7ffb02a84b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1af4d0 con 0x55ca3dacc4a0 2024-05-09T00:18:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.708+0000 7ffb02a84b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1b0940 con 0x55ca3db46d00 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.708+0000 7ffb02a84b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1afba0 con 0x55ca3dc49400 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.708+0000 7ffb02a84b80 10 client.0 _ll_register_callbacks cb 0x55ca3d9dd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.708+0000 7ffaf1e0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dacc4a0 0x55ca3dcc49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.708+0000 7ffaf1e0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dacc4a0 0x55ca3dcc49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46414/0 (socket says 172.21.15.46:46414) 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.708+0000 7ffaf1e0a700 1 -- 172.21.15.46:0/2804990411 learned_addr learned my addr 172.21.15.46:0/2804990411 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.708+0000 7ffaf1609700 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3db46d00 0x55ca3dcc4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffaf260b700 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dc49400 0x55ca3dc67780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffaf1e0a700 1 -- 172.21.15.46:0/2804990411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dc49400 msgr2=0x55ca3dc67780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffaf1e0a700 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dc49400 0x55ca3dc67780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffaf1e0a700 1 -- 172.21.15.46:0/2804990411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3db46d00 msgr2=0x55ca3dcc4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffaf1e0a700 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3db46d00 0x55ca3dcc4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffaf1e0a700 1 -- 172.21.15.46:0/2804990411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca3dc67ed0 con 0x55ca3dacc4a0 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffaf260b700 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dc49400 0x55ca3dc67780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffaf1e0a700 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dacc4a0 0x55ca3dcc49f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ffae80aa260 tx=0x7ffae80a9eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e2260c18d9d8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffad57fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffad57fa700 1 -- 172.21.15.46:0/2804990411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae801d040 con 0x55ca3dacc4a0 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffad57fa700 1 -- 172.21.15.46:0/2804990411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ffae8005220 con 0x55ca3dacc4a0 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2804990411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffae4014da0 con 0x55ca3dacc4a0 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffad57fa700 1 -- 172.21.15.46:0/2804990411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffae80aa2d0 con 0x55ca3dacc4a0 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffb02a84b80 10 client.16494 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2804990411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca3dccd090 con 0x55ca3dacc4a0 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.709+0000 7ffad57fa700 1 -- 172.21.15.46:0/2804990411 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae4014da0 con 0x55ca3dacc4a0 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.710+0000 7ffad57fa700 1 -- 172.21.15.46:0/2804990411 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ffae80c7050 con 0x55ca3dacc4a0 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.710+0000 7ffb02a84b80 10 client.16494 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.710+0000 7ffb02a84b80 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.710+0000 7ffb02a84b80 20 client.16494 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.710+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ca3dccd750 0x55ca3dccfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.710+0000 7ffb02a84b80 4 client.16494 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:18:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.710+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2804990411 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca3db73b20 con 0x55ca3dccd750 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.710+0000 7ffaf1609700 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ca3dccd750 0x55ca3dccfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.710+0000 7ffad57fa700 10 client.16494 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.710+0000 7ffaf1609700 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ca3dccd750 0x55ca3dccfb70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8feee534da91e1f server_cookie=3d60d7f44c814c6f in_seq=0 out_seq=0 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffad57fa700 1 -- 172.21.15.46:0/2804990411 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca3db73b20 con 0x55ca3dccd750 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffad57fa700 10 client.16494 handle_command_reply: tid=0 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffb02a84b80 1 client.16494 shutdown 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2804990411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dacc4a0 msgr2=0x55ca3dcc49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dacc4a0 0x55ca3dcc49f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ffae80aa260 tx=0x7ffae80a9eb0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2804990411 shutdown_connections 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ca3dccd750 0x55ca3dccfb70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca3dc49400 0x55ca3dc67780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca3dacc4a0 0x55ca3dcc49f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffad57fa700 0 client.16494 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffb02a84b80 1 --2- 172.21.15.46:0/2804990411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca3db46d00 0x55ca3dcc4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2804990411 >> 172.21.15.46:0/2804990411 conn(0x55ca3dacff70 msgr2=0x55ca3dc4cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2804990411 shutdown_connections 2024-05-09T00:18:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffb02a84b80 1 -- 172.21.15.46:0/2804990411 wait complete. 2024-05-09T00:18:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.711+0000 7ffb02a84b80 20 client.16494 trim_cache size 0 max 16384 2024-05-09T00:18:24.716 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:18:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.712+0000 7ffaf406e700 1 -- 172.21.15.46:0/452496022 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ffad80b2880 msgr2=0x7ffad80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.712+0000 7ffaf406e700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ffad80b2880 0x7ffad80b4d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffadc006010 tx=0x7ffadc000bc0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.712+0000 7ffaf406e700 1 -- 172.21.15.46:0/452496022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec1034e0 msgr2=0x7ffaec0ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.712+0000 7ffaf406e700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec1034e0 0x7ffaec0ff410 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ffae800c920 tx=0x7ffae800cc30 comp rx=0 tx=0).stop 2024-05-09T00:18:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.712+0000 7ffaf406e700 1 -- 172.21.15.46:0/452496022 shutdown_connections 2024-05-09T00:18:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.712+0000 7ffaf406e700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ffad80b2880 0x7ffad80b4d90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.712+0000 7ffaf406e700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffaec108760 0x7ffaec0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.712+0000 7ffaf406e700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec1034e0 0x7ffaec0ff410 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.712+0000 7ffaf406e700 1 --2- 172.21.15.46:0/452496022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaec107e10 0x7ffaec0ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.712+0000 7ffaf406e700 1 -- 172.21.15.46:0/452496022 >> 172.21.15.46:0/452496022 conn(0x7ffaec0fef30 msgr2=0x7ffaec06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.712+0000 7ffaf406e700 1 -- 172.21.15.46:0/452496022 shutdown_connections 2024-05-09T00:18:24.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:24.712+0000 7ffaf406e700 1 -- 172.21.15.46:0/452496022 wait complete. 2024-05-09T00:18:25.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:24 smithi046 ceph-mon[30151]: pgmap v587: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:18:25.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:24 smithi169 ceph-mon[34777]: pgmap v587: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:18:25.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:24 smithi142 ceph-mon[34533]: pgmap v587: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:18:26.542 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:18:26.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:18:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:18:26.825 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:26 smithi046 ceph-mon[30151]: pgmap v588: 97 pgs: 97 active+clean; 3.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:27.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:26 smithi169 ceph-mon[34777]: pgmap v588: 97 pgs: 97 active+clean; 3.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:27.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:26 smithi142 ceph-mon[34533]: pgmap v588: 97 pgs: 97 active+clean; 3.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.745+0000 7faee5909700 1 -- 172.21.15.46:0/4065866623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee01092d0 msgr2=0x7faee01096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.745+0000 7faee5909700 1 --2- 172.21.15.46:0/4065866623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee01092d0 0x7faee01096e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7faed0009990 tx=0x7faed0009ca0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.745+0000 7faee5909700 1 -- 172.21.15.46:0/4065866623 shutdown_connections 2024-05-09T00:18:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.745+0000 7faee5909700 1 --2- 172.21.15.46:0/4065866623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee0104330 0x7faee0110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.745+0000 7faee5909700 1 --2- 172.21.15.46:0/4065866623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee01092d0 0x7faee01096e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.745+0000 7faee5909700 1 --2- 172.21.15.46:0/4065866623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee01039e0 0x7faee0103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.745+0000 7faee5909700 1 -- 172.21.15.46:0/4065866623 >> 172.21.15.46:0/4065866623 conn(0x7faee00fe4e0 msgr2=0x7faee0100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.745+0000 7faee5909700 1 -- 172.21.15.46:0/4065866623 shutdown_connections 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.745+0000 7faee5909700 1 -- 172.21.15.46:0/4065866623 wait complete. 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.746+0000 7faee5909700 1 Processor -- start 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.746+0000 7faee5909700 1 -- start start 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.746+0000 7faee5909700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee01039e0 0x7faee019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.746+0000 7faee5909700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee0104330 0x7faee019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.746+0000 7faee5909700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee01092d0 0x7faee019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.746+0000 7faee5909700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faee0113370 con 0x7faee01039e0 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.746+0000 7faee5909700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faee01131f0 con 0x7faee0104330 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.746+0000 7faee5909700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faee01134f0 con 0x7faee01092d0 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.746+0000 7faedeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee01039e0 0x7faee019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.746+0000 7faedeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee01039e0 0x7faee019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51764/0 (socket says 172.21.15.46:51764) 2024-05-09T00:18:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.746+0000 7faedeffd700 1 -- 172.21.15.46:0/3635013501 learned_addr learned my addr 172.21.15.46:0/3635013501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.746+0000 7faede7fc700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee0104330 0x7faee019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.747+0000 7faedf7fe700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee01092d0 0x7faee019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.747+0000 7faedeffd700 1 -- 172.21.15.46:0/3635013501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee01092d0 msgr2=0x7faee019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.747+0000 7faedeffd700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee01092d0 0x7faee019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.747+0000 7faedeffd700 1 -- 172.21.15.46:0/3635013501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee0104330 msgr2=0x7faee019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.747+0000 7faedeffd700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee0104330 0x7faee019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.747+0000 7faedeffd700 1 -- 172.21.15.46:0/3635013501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee019fd20 con 0x7faee01039e0 2024-05-09T00:18:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.747+0000 7faedeffd700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee01039e0 0x7faee019ae50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7faed0000c00 tx=0x7faed00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.747+0000 7faee4907700 1 -- 172.21.15.46:0/3635013501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0027070 con 0x7faee01039e0 2024-05-09T00:18:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.747+0000 7faee4907700 1 -- 172.21.15.46:0/3635013501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed00041d0 con 0x7faee01039e0 2024-05-09T00:18:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.747+0000 7faee4907700 1 -- 172.21.15.46:0/3635013501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed000f460 con 0x7faee01039e0 2024-05-09T00:18:28.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.747+0000 7faee5909700 1 -- 172.21.15.46:0/3635013501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee019ffb0 con 0x7faee01039e0 2024-05-09T00:18:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.747+0000 7faee5909700 1 -- 172.21.15.46:0/3635013501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee01a0440 con 0x7faee01039e0 2024-05-09T00:18:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.750+0000 7faee4907700 1 -- 172.21.15.46:0/3635013501 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7faed002a070 con 0x7faee01039e0 2024-05-09T00:18:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.751+0000 7faee4907700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7faec80b2880 0x7faec80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.751+0000 7faef431fb80 1 Processor -- start 2024-05-09T00:18:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.751+0000 7faef431fb80 1 -- start start 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.751+0000 7faede7fc700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7faec80b2880 0x7faec80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.751+0000 7faee4907700 1 -- 172.21.15.46:0/3635013501 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7faed00e3260 con 0x7faee01039e0 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.751+0000 7faef431fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 0x559c9561bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.751+0000 7faef431fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c9561c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.751+0000 7faef431fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c95620890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.751+0000 7faef431fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559c9553a070 con 0x559c95620480 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.751+0000 7faef431fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559c954d8870 con 0x559c9549a520 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.751+0000 7faef431fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559c95536030 con 0x559c955841e0 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faedf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c95620890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faedf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c95620890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51776/0 (socket says 172.21.15.46:51776) 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faedf7fe700 1 -- 172.21.15.46:0/3188393550 learned_addr learned my addr 172.21.15.46:0/3188393550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faede7fc700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7faec80b2880 0x7faec80b4d90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faed4005fd0 tx=0x7faed400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faede7fc700 1 --2- 172.21.15.46:0/3188393550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c9561c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faedeffd700 1 --2- 172.21.15.46:0/3188393550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 0x559c9561bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faedf7fe700 1 -- 172.21.15.46:0/3188393550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 msgr2=0x559c9561bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faedf7fe700 1 --2- 172.21.15.46:0/3188393550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 0x559c9561bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faedf7fe700 1 -- 172.21.15.46:0/3188393550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 msgr2=0x559c9561c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faedf7fe700 1 --2- 172.21.15.46:0/3188393550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c9561c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faedf7fe700 1 -- 172.21.15.46:0/3188393550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c954d03f0 con 0x559c95620480 2024-05-09T00:18:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faede7fc700 1 --2- 172.21.15.46:0/3188393550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c9561c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:28.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faedf7fe700 1 --2- 172.21.15.46:0/3188393550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c95620890 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7faed80084f0 tx=0x7faed8008800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2dd6bfc22468337 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faec5ffb700 1 -- 172.21.15.46:0/3188393550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8003c00 con 0x559c95620480 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faec5ffb700 1 -- 172.21.15.46:0/3188393550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed8003da0 con 0x559c95620480 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.752+0000 7faec5ffb700 1 -- 172.21.15.46:0/3188393550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8012890 con 0x559c95620480 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.753+0000 7faef431fb80 1 -- 172.21.15.46:0/3188393550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 msgr2=0x559c95620890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.753+0000 7faef431fb80 1 --2- 172.21.15.46:0/3188393550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c95620890 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7faed80084f0 tx=0x7faed8008800 comp rx=0 tx=0).stop 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.753+0000 7faef431fb80 1 -- 172.21.15.46:0/3188393550 shutdown_connections 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.753+0000 7faef431fb80 1 --2- 172.21.15.46:0/3188393550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c95620890 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.753+0000 7faef431fb80 1 --2- 172.21.15.46:0/3188393550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 0x559c9561bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.753+0000 7faef431fb80 1 --2- 172.21.15.46:0/3188393550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c9561c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.753+0000 7faef431fb80 1 -- 172.21.15.46:0/3188393550 >> 172.21.15.46:0/3188393550 conn(0x559c954fa720 msgr2=0x559c95612480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.753+0000 7faef431fb80 1 -- 172.21.15.46:0/3188393550 shutdown_connections 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.753+0000 7faef431fb80 1 -- 172.21.15.46:0/3188393550 wait complete. 2024-05-09T00:18:28.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faef431fb80 1 -- start start 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faef431fb80 1 Processor -- start 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faef431fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c9549a520 0x559c95625610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faef431fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c955841e0 0x559c95625b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faef431fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c95620480 0x559c95626090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faef431fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559c953b5c20 con 0x559c955841e0 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faef431fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559c954f41d0 con 0x559c95620480 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faef431fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559c95621230 con 0x559c9549a520 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faef431fb80 10 client.0 _ll_register_callbacks cb 0x559c953ab740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faede7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c955841e0 0x559c95625b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faede7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c955841e0 0x559c95625b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51788/0 (socket says 172.21.15.46:51788) 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faede7fc700 1 -- 172.21.15.46:0/688606179 learned_addr learned my addr 172.21.15.46:0/688606179 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faedeffd700 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c9549a520 0x559c95625610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.754+0000 7faedf7fe700 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c95620480 0x559c95626090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faede7fc700 1 -- 172.21.15.46:0/688606179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c9549a520 msgr2=0x559c95625610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faede7fc700 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c9549a520 0x559c95625610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faede7fc700 1 -- 172.21.15.46:0/688606179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c95620480 msgr2=0x559c95626090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faede7fc700 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c95620480 0x559c95626090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faede7fc700 1 -- 172.21.15.46:0/688606179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c953b5510 con 0x559c955841e0 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faedeffd700 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c9549a520 0x559c95625610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faede7fc700 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c955841e0 0x559c95625b50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7faee006ca20 tx=0x7faed4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c5606a9f6727fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faebeffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faebeffd700 1 -- 172.21.15.46:0/688606179 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4004a30 con 0x559c955841e0 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faebeffd700 1 -- 172.21.15.46:0/688606179 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7faed4004bd0 con 0x559c955841e0 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faef431fb80 1 -- 172.21.15.46:0/688606179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faed8003da0 con 0x559c955841e0 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faebeffd700 1 -- 172.21.15.46:0/688606179 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faed4004dd0 con 0x559c955841e0 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.755+0000 7faef431fb80 10 client.16506 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.756+0000 7faef431fb80 1 -- 172.21.15.46:0/688606179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559c95632f90 con 0x559c955841e0 2024-05-09T00:18:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.756+0000 7faebeffd700 1 -- 172.21.15.46:0/688606179 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed8003da0 con 0x559c955841e0 2024-05-09T00:18:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.756+0000 7faebeffd700 1 -- 172.21.15.46:0/688606179 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7faed401ea50 con 0x559c955841e0 2024-05-09T00:18:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.756+0000 7faef431fb80 10 client.16506 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.756+0000 7faef431fb80 10 client.16506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:18:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.756+0000 7faef431fb80 20 client.16506 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.756+0000 7faef431fb80 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559c95635620 0x559c95637a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.756+0000 7faef431fb80 4 client.16506 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:18:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.756+0000 7faef431fb80 1 -- 172.21.15.46:0/688606179 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c95547a50 con 0x559c95635620 2024-05-09T00:18:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.756+0000 7faedeffd700 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559c95635620 0x559c95637a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.757+0000 7faebeffd700 10 client.16506 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.757+0000 7faedeffd700 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559c95635620 0x559c95637a80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9e1fb725f0d1ff server_cookie=b70f3663333d6c3a in_seq=0 out_seq=0 2024-05-09T00:18:28.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faebeffd700 1 -- 172.21.15.46:0/688606179 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559c95547a50 con 0x559c95635620 2024-05-09T00:18:28.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faebeffd700 10 client.16506 handle_command_reply: tid=0 2024-05-09T00:18:28.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faef431fb80 1 client.16506 shutdown 2024-05-09T00:18:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faef431fb80 1 -- 172.21.15.46:0/688606179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c955841e0 msgr2=0x559c95625b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faef431fb80 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c955841e0 0x559c95625b50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7faee006ca20 tx=0x7faed4004750 comp rx=0 tx=0).stop 2024-05-09T00:18:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faef431fb80 1 -- 172.21.15.46:0/688606179 shutdown_connections 2024-05-09T00:18:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faef431fb80 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559c95635620 0x559c95637a80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faef431fb80 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c95620480 0x559c95626090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faef431fb80 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c9549a520 0x559c95625610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faef431fb80 1 --2- 172.21.15.46:0/688606179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c955841e0 0x559c95625b50 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faebeffd700 0 client.16506 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faef431fb80 1 -- 172.21.15.46:0/688606179 >> 172.21.15.46:0/688606179 conn(0x559c954fa720 msgr2=0x559c95612240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faef431fb80 1 -- 172.21.15.46:0/688606179 shutdown_connections 2024-05-09T00:18:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faef431fb80 1 -- 172.21.15.46:0/688606179 wait complete. 2024-05-09T00:18:28.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.758+0000 7faef431fb80 20 client.16506 trim_cache size 0 max 16384 2024-05-09T00:18:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.779+0000 7faef431fb80 1 Processor -- start 2024-05-09T00:18:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.779+0000 7faef431fb80 1 -- start start 2024-05-09T00:18:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.779+0000 7faef431fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c95617140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.779+0000 7faef431fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 0x559c95625ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.779+0000 7faef431fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c95631480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.779+0000 7faef431fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faee01af210 con 0x559c95620480 2024-05-09T00:18:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.779+0000 7faef431fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faee01acaa0 con 0x559c9549a520 2024-05-09T00:18:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.779+0000 7faef431fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faee01aa8f0 con 0x559c955841e0 2024-05-09T00:18:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.779+0000 7faedf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c95631480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.779+0000 7faedeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c95617140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.779+0000 7faedf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c95631480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51792/0 (socket says 172.21.15.46:51792) 2024-05-09T00:18:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.779+0000 7faedf7fe700 1 -- 172.21.15.46:0/4280322332 learned_addr learned my addr 172.21.15.46:0/4280322332 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.780+0000 7faede7fc700 1 --2- 172.21.15.46:0/4280322332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 0x559c95625ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.780+0000 7faedf7fe700 1 -- 172.21.15.46:0/4280322332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 msgr2=0x559c95625ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.780+0000 7faedf7fe700 1 --2- 172.21.15.46:0/4280322332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 0x559c95625ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.780+0000 7faedf7fe700 1 -- 172.21.15.46:0/4280322332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 msgr2=0x559c95617140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.780+0000 7faedf7fe700 1 --2- 172.21.15.46:0/4280322332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c95617140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.780+0000 7faedf7fe700 1 -- 172.21.15.46:0/4280322332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c95627eb0 con 0x559c95620480 2024-05-09T00:18:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.780+0000 7faedeffd700 1 --2- 172.21.15.46:0/4280322332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c95617140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:18:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.780+0000 7faedf7fe700 1 --2- 172.21.15.46:0/4280322332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c95631480 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7faed8015000 tx=0x7faed8004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46197020c7bc5965 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.783+0000 7faec57fa700 1 -- 172.21.15.46:0/4280322332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8023030 con 0x559c95620480 2024-05-09T00:18:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.783+0000 7faec57fa700 1 -- 172.21.15.46:0/4280322332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed8004020 con 0x559c95620480 2024-05-09T00:18:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.783+0000 7faec57fa700 1 -- 172.21.15.46:0/4280322332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8012500 con 0x559c95620480 2024-05-09T00:18:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.784+0000 7faef431fb80 1 -- 172.21.15.46:0/4280322332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 msgr2=0x559c95631480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.784+0000 7faef431fb80 1 --2- 172.21.15.46:0/4280322332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c95631480 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7faed8015000 tx=0x7faed8004cd0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.784+0000 7faef431fb80 1 -- 172.21.15.46:0/4280322332 shutdown_connections 2024-05-09T00:18:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.784+0000 7faef431fb80 1 --2- 172.21.15.46:0/4280322332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c95631480 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.784+0000 7faef431fb80 1 --2- 172.21.15.46:0/4280322332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c95617140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.784+0000 7faef431fb80 1 --2- 172.21.15.46:0/4280322332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 0x559c95625ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.784+0000 7faef431fb80 1 -- 172.21.15.46:0/4280322332 >> 172.21.15.46:0/4280322332 conn(0x559c954fa720 msgr2=0x559c956226c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:28.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.784+0000 7faef431fb80 1 -- 172.21.15.46:0/4280322332 shutdown_connections 2024-05-09T00:18:28.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.784+0000 7faef431fb80 1 -- 172.21.15.46:0/4280322332 wait complete. 2024-05-09T00:18:28.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faef431fb80 1 -- start start 2024-05-09T00:18:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faef431fb80 1 Processor -- start 2024-05-09T00:18:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faef431fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c9562d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faef431fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 0x559c9562dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faef431fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c956287c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faef431fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faee01aaf60 con 0x559c95620480 2024-05-09T00:18:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faef431fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faee01ac3d0 con 0x559c9549a520 2024-05-09T00:18:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faef431fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faee01ab630 con 0x559c955841e0 2024-05-09T00:18:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faef431fb80 10 client.0 _ll_register_callbacks cb 0x559c953ab740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faedf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c956287c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faedf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c956287c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51794/0 (socket says 172.21.15.46:51794) 2024-05-09T00:18:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faedf7fe700 1 -- 172.21.15.46:0/2280080551 learned_addr learned my addr 172.21.15.46:0/2280080551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faedeffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c9562d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.785+0000 7faede7fc700 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 0x559c9562dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faedf7fe700 1 -- 172.21.15.46:0/2280080551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 msgr2=0x559c9562dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faedf7fe700 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 0x559c9562dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faedf7fe700 1 -- 172.21.15.46:0/2280080551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 msgr2=0x559c9562d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faedf7fe700 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c9562d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faedf7fe700 1 -- 172.21.15.46:0/2280080551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c95628ea0 con 0x559c95620480 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faedeffd700 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c9562d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faedf7fe700 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c956287c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7faed8008840 tx=0x7faed8004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63854d7a7b2fc33d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faebeffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faebeffd700 1 -- 172.21.15.46:0/2280080551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8012500 con 0x559c95620480 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faebeffd700 1 -- 172.21.15.46:0/2280080551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7faed80126a0 con 0x559c95620480 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faef431fb80 1 -- 172.21.15.46:0/2280080551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faed8004020 con 0x559c95620480 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faebeffd700 1 -- 172.21.15.46:0/2280080551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faed80128a0 con 0x559c95620480 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faef431fb80 10 client.16512 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.786+0000 7faef431fb80 1 -- 172.21.15.46:0/2280080551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559c956a07b0 con 0x559c95620480 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.787+0000 7faebeffd700 1 -- 172.21.15.46:0/2280080551 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed8004020 con 0x559c95620480 2024-05-09T00:18:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.787+0000 7faebeffd700 1 -- 172.21.15.46:0/2280080551 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7faed8035030 con 0x559c95620480 2024-05-09T00:18:28.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.787+0000 7faef431fb80 10 client.16512 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:28.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.787+0000 7faef431fb80 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:18:28.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.787+0000 7faef431fb80 20 client.16512 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:28.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.787+0000 7faef431fb80 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559c956a0c00 0x559c956a3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:28.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.787+0000 7faef431fb80 4 client.16512 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:18:28.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.787+0000 7faef431fb80 1 -- 172.21.15.46:0/2280080551 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559c95541af0 con 0x559c956a0c00 2024-05-09T00:18:28.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.787+0000 7faedeffd700 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559c956a0c00 0x559c956a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:28.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.788+0000 7faedeffd700 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559c956a0c00 0x559c956a3020 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5451dab042c7fae server_cookie=62c13f79ef593329 in_seq=0 out_seq=0 2024-05-09T00:18:28.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.788+0000 7faebeffd700 10 client.16512 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:28.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.789+0000 7faebeffd700 1 -- 172.21.15.46:0/2280080551 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559c95541af0 con 0x559c956a0c00 2024-05-09T00:18:28.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.789+0000 7faebeffd700 10 client.16512 handle_command_reply: tid=0 2024-05-09T00:18:28.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.789+0000 7faef431fb80 1 client.16512 shutdown 2024-05-09T00:18:28.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.789+0000 7faef431fb80 1 -- 172.21.15.46:0/2280080551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 msgr2=0x559c956287c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.789+0000 7faef431fb80 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c956287c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7faed8008840 tx=0x7faed8004400 comp rx=0 tx=0).stop 2024-05-09T00:18:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faef431fb80 1 -- 172.21.15.46:0/2280080551 shutdown_connections 2024-05-09T00:18:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faef431fb80 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x559c956a0c00 0x559c956a3020 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faef431fb80 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c95620480 0x559c956287c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faef431fb80 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c9549a520 0x559c9562d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faef431fb80 1 --2- 172.21.15.46:0/2280080551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c955841e0 0x559c9562dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faef431fb80 1 -- 172.21.15.46:0/2280080551 >> 172.21.15.46:0/2280080551 conn(0x559c954fa720 msgr2=0x559c95612390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faebeffd700 0 client.16512 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faef431fb80 1 -- 172.21.15.46:0/2280080551 shutdown_connections 2024-05-09T00:18:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faef431fb80 1 -- 172.21.15.46:0/2280080551 wait complete. 2024-05-09T00:18:28.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faef431fb80 20 client.16512 trim_cache size 0 max 16384 2024-05-09T00:18:28.793 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:18:28.793 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:18:28.793 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "21669b2c-47dc-450f-8fa5-7c791b3efc15", 2024-05-09T00:18:28.793 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:18:28.793 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:18:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faee5909700 1 -- 172.21.15.46:0/3635013501 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7faec80b2880 msgr2=0x7faec80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faee5909700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7faec80b2880 0x7faec80b4d90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faed4005fd0 tx=0x7faed400a560 comp rx=0 tx=0).stop 2024-05-09T00:18:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faee5909700 1 -- 172.21.15.46:0/3635013501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee01039e0 msgr2=0x7faee019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.790+0000 7faee5909700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee01039e0 0x7faee019ae50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7faed0000c00 tx=0x7faed00047b0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.791+0000 7faee5909700 1 -- 172.21.15.46:0/3635013501 shutdown_connections 2024-05-09T00:18:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.791+0000 7faee5909700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7faec80b2880 0x7faec80b4d90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.791+0000 7faee5909700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee01092d0 0x7faee019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.791+0000 7faee5909700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faee01039e0 0x7faee019ae50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.791+0000 7faee5909700 1 --2- 172.21.15.46:0/3635013501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee0104330 0x7faee019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.791+0000 7faee5909700 1 -- 172.21.15.46:0/3635013501 >> 172.21.15.46:0/3635013501 conn(0x7faee00fe4e0 msgr2=0x7faee0101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.791+0000 7faee5909700 1 -- 172.21.15.46:0/3635013501 shutdown_connections 2024-05-09T00:18:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:28.791+0000 7faee5909700 1 -- 172.21.15.46:0/3635013501 wait complete. 2024-05-09T00:18:29.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:28 smithi046 ceph-mon[30151]: pgmap v589: 97 pgs: 97 active+clean; 3.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:29.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:28 smithi169 ceph-mon[34777]: pgmap v589: 97 pgs: 97 active+clean; 3.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:29.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:28 smithi142 ceph-mon[34533]: pgmap v589: 97 pgs: 97 active+clean; 3.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:29.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:18:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:18:30.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:29 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:18:30.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:29 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:18:30.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:29 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:18:30.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:29 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:18:30.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:29 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:18:30.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:29 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:18:30.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:29 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:18:30.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:29 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:18:30.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:29 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:18:30.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:29 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:18:30.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:29 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:18:30.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:29 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:18:31.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:30 smithi046 ceph-mon[30151]: pgmap v590: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:31.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:30 smithi169 ceph-mon[34777]: pgmap v590: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:31.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:30 smithi142 ceph-mon[34533]: pgmap v590: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.710+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1567643709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0108f00 msgr2=0x7fd2b0109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.710+0000 7fd2b5b14700 1 --2- 172.21.15.46:0/1567643709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0108f00 0x7fd2b0109310 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd2980099c0 tx=0x7fd298009cd0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.710+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1567643709 shutdown_connections 2024-05-09T00:18:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.710+0000 7fd2b5b14700 1 --2- 172.21.15.46:0/1567643709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2b0109850 0x7fd2b01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.710+0000 7fd2b5b14700 1 --2- 172.21.15.46:0/1567643709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2b01045d0 0x7fd2b01049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.710+0000 7fd2b5b14700 1 --2- 172.21.15.46:0/1567643709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0108f00 0x7fd2b0109310 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.711+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1567643709 >> 172.21.15.46:0/1567643709 conn(0x7fd2b00ffea0 msgr2=0x7fd2b01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.711+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1567643709 shutdown_connections 2024-05-09T00:18:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.711+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1567643709 wait complete. 2024-05-09T00:18:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.711+0000 7fd2b5b14700 1 Processor -- start 2024-05-09T00:18:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.712+0000 7fd2b5b14700 1 -- start start 2024-05-09T00:18:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.712+0000 7fd2b5b14700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2b01045d0 0x7fd2b0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.712+0000 7fd2b5b14700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2b0108f00 0x7fd2b0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.712+0000 7fd2b5b14700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0109850 0x7fd2b0077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.712+0000 7fd2b5b14700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b01144b0 con 0x7fd2b0109850 2024-05-09T00:18:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.712+0000 7fd2b5b14700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b0114330 con 0x7fd2b01045d0 2024-05-09T00:18:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.712+0000 7fd2b5b14700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b0114630 con 0x7fd2b0108f00 2024-05-09T00:18:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.712+0000 7fd2affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0109850 0x7fd2b0077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.712+0000 7fd2affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0109850 0x7fd2b0077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51820/0 (socket says 172.21.15.46:51820) 2024-05-09T00:18:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.712+0000 7fd2affff700 1 -- 172.21.15.46:0/1152607428 learned_addr learned my addr 172.21.15.46:0/1152607428 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.712+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2b01045d0 0x7fd2b0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.712+0000 7fd2a6ffd700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2b0108f00 0x7fd2b0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.713+0000 7fd2affff700 1 -- 172.21.15.46:0/1152607428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2b0108f00 msgr2=0x7fd2b0077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.713+0000 7fd2affff700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2b0108f00 0x7fd2b0077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.713+0000 7fd2affff700 1 -- 172.21.15.46:0/1152607428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2b01045d0 msgr2=0x7fd2b0076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.713+0000 7fd2affff700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2b01045d0 0x7fd2b0076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.713+0000 7fd2affff700 1 -- 172.21.15.46:0/1152607428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b01a91c0 con 0x7fd2b0109850 2024-05-09T00:18:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.713+0000 7fd2affff700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0109850 0x7fd2b0077650 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8007d70 tx=0x7fd2a800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.713+0000 7fd2ad7fa700 1 -- 172.21.15.46:0/1152607428 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8016020 con 0x7fd2b0109850 2024-05-09T00:18:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.713+0000 7fd2ad7fa700 1 -- 172.21.15.46:0/1152607428 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2a800ac90 con 0x7fd2b0109850 2024-05-09T00:18:31.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.713+0000 7fd2ad7fa700 1 -- 172.21.15.46:0/1152607428 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a80115e0 con 0x7fd2b0109850 2024-05-09T00:18:31.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.713+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1152607428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b01a94b0 con 0x7fd2b0109850 2024-05-09T00:18:31.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.713+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1152607428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b01a9970 con 0x7fd2b0109850 2024-05-09T00:18:31.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.716+0000 7fd2ad7fa700 1 -- 172.21.15.46:0/1152607428 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd2a8019020 con 0x7fd2b0109850 2024-05-09T00:18:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.717+0000 7fd2ad7fa700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd29c0b2880 0x7fd29c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.717+0000 7fd2ad7fa700 1 -- 172.21.15.46:0/1152607428 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd2a80e2120 con 0x7fd2b0109850 2024-05-09T00:18:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.717+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd29c0b2880 0x7fd29c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.717+0000 7fd2c452ab80 1 Processor -- start 2024-05-09T00:18:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.717+0000 7fd2c452ab80 1 -- start start 2024-05-09T00:18:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.717+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd29c0b2880 0x7fd29c0b4d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0005950 tx=0x7fd2a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2c452ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103cb4d00 0x557103dbd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2c452ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103c3a4a0 0x557103dbd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2c452ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103db7400 0x557103db7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2c452ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557103cda0a0 con 0x557103db7400 2024-05-09T00:18:31.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2c452ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557103c7f9a0 con 0x557103c3a4a0 2024-05-09T00:18:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2c452ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557103cd6060 con 0x557103cb4d00 2024-05-09T00:18:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103db7400 0x557103db7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2a6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103c3a4a0 0x557103dbd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103db7400 0x557103db7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51822/0 (socket says 172.21.15.46:51822) 2024-05-09T00:18:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2affff700 1 -- 172.21.15.46:0/4290594941 learned_addr learned my addr 172.21.15.46:0/4290594941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/4290594941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103cb4d00 0x557103dbd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2affff700 1 -- 172.21.15.46:0/4290594941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103cb4d00 msgr2=0x557103dbd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2affff700 1 --2- 172.21.15.46:0/4290594941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103cb4d00 0x557103dbd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2affff700 1 -- 172.21.15.46:0/4290594941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103c3a4a0 msgr2=0x557103dbd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2affff700 1 --2- 172.21.15.46:0/4290594941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103c3a4a0 0x557103dbd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2affff700 1 -- 172.21.15.46:0/4290594941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557103c70500 con 0x557103db7400 2024-05-09T00:18:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.718+0000 7fd2affff700 1 --2- 172.21.15.46:0/4290594941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103db7400 0x557103db7810 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80a5780 tx=0x7fd2a80129f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c0bb63339c7b71 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.719+0000 7fd2a5ffb700 1 -- 172.21.15.46:0/4290594941 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a80a9210 con 0x557103db7400 2024-05-09T00:18:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.719+0000 7fd2a5ffb700 1 -- 172.21.15.46:0/4290594941 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2a80a9e00 con 0x557103db7400 2024-05-09T00:18:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.719+0000 7fd2a5ffb700 1 -- 172.21.15.46:0/4290594941 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a80b2330 con 0x557103db7400 2024-05-09T00:18:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.719+0000 7fd2c452ab80 1 -- 172.21.15.46:0/4290594941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103db7400 msgr2=0x557103db7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.719+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/4290594941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103db7400 0x557103db7810 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80a5780 tx=0x7fd2a80129f0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.720+0000 7fd2c452ab80 1 -- 172.21.15.46:0/4290594941 shutdown_connections 2024-05-09T00:18:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.720+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/4290594941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103db7400 0x557103db7810 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.720+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/4290594941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103cb4d00 0x557103dbd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.720+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/4290594941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103c3a4a0 0x557103dbd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.720+0000 7fd2c452ab80 1 -- 172.21.15.46:0/4290594941 >> 172.21.15.46:0/4290594941 conn(0x557103c3df70 msgr2=0x557103db2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.720+0000 7fd2c452ab80 1 -- 172.21.15.46:0/4290594941 shutdown_connections 2024-05-09T00:18:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.720+0000 7fd2c452ab80 1 -- 172.21.15.46:0/4290594941 wait complete. 2024-05-09T00:18:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.720+0000 7fd2c452ab80 1 -- start start 2024-05-09T00:18:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.720+0000 7fd2c452ab80 1 Processor -- start 2024-05-09T00:18:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2c452ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103c3a4a0 0x557103dc1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2c452ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103cb4d00 0x557103dc1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2c452ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 0x557103dc1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2c452ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557103db8100 con 0x557103cb4d00 2024-05-09T00:18:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2c452ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557103c74b20 con 0x557103c3a4a0 2024-05-09T00:18:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2c452ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557103dcd1d0 con 0x557103db7400 2024-05-09T00:18:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2c452ab80 10 client.0 _ll_register_callbacks cb 0x557103b4b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2a6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103cb4d00 0x557103dc1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2a6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103cb4d00 0x557103dc1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51838/0 (socket says 172.21.15.46:51838) 2024-05-09T00:18:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2a6ffd700 1 -- 172.21.15.46:0/1102619743 learned_addr learned my addr 172.21.15.46:0/1102619743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2affff700 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 0x557103dc1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103c3a4a0 0x557103dc1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2a6ffd700 1 -- 172.21.15.46:0/1102619743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 msgr2=0x557103dc1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2a6ffd700 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 0x557103dc1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2a6ffd700 1 -- 172.21.15.46:0/1102619743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103c3a4a0 msgr2=0x557103dc1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2a6ffd700 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103c3a4a0 0x557103dc1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2a6ffd700 1 -- 172.21.15.46:0/1102619743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557103b55490 con 0x557103cb4d00 2024-05-09T00:18:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103c3a4a0 0x557103dc1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:18:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd2a6ffd700 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103cb4d00 0x557103dc1560 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd298005b00 tx=0x7fd298009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee096d83a1c7359 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.721+0000 7fd28effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd28effd700 1 -- 172.21.15.46:0/1102619743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298005600 con 0x557103cb4d00 2024-05-09T00:18:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd28effd700 1 -- 172.21.15.46:0/1102619743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd29800b790 con 0x557103cb4d00 2024-05-09T00:18:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd2c452ab80 1 -- 172.21.15.46:0/1102619743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2a80a9e00 con 0x557103cb4d00 2024-05-09T00:18:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd28effd700 1 -- 172.21.15.46:0/1102619743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd29800b990 con 0x557103cb4d00 2024-05-09T00:18:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd2c452ab80 10 client.16524 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd2c452ab80 1 -- 172.21.15.46:0/1102619743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557103dceb00 con 0x557103cb4d00 2024-05-09T00:18:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd28effd700 1 -- 172.21.15.46:0/1102619743 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2a80a9e00 con 0x557103cb4d00 2024-05-09T00:18:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd28effd700 1 -- 172.21.15.46:0/1102619743 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd298023bd0 con 0x557103cb4d00 2024-05-09T00:18:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd2c452ab80 10 client.16524 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd2c452ab80 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:18:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd2c452ab80 20 client.16524 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557103dd1140 0x557103dd3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd2c452ab80 4 client.16524 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.722+0000 7fd2c452ab80 1 -- 172.21.15.46:0/1102619743 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557103ce7a80 con 0x557103dd1140 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.723+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557103dd1140 0x557103dd3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.723+0000 7fd28effd700 10 client.16524 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.723+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557103dd1140 0x557103dd3560 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ecdfcb7afedadf server_cookie=b8b8c557f267de1b in_seq=0 out_seq=0 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.724+0000 7fd28effd700 1 -- 172.21.15.46:0/1102619743 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557103ce7a80 con 0x557103dd1140 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.724+0000 7fd28effd700 10 client.16524 handle_command_reply: tid=0 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.724+0000 7fd2c452ab80 1 client.16524 shutdown 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.724+0000 7fd2c452ab80 1 -- 172.21.15.46:0/1102619743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103cb4d00 msgr2=0x557103dc1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.724+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103cb4d00 0x557103dc1560 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd298005b00 tx=0x7fd298009590 comp rx=0 tx=0).stop 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.724+0000 7fd2c452ab80 1 -- 172.21.15.46:0/1102619743 shutdown_connections 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.724+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557103dd1140 0x557103dd3560 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.724+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 0x557103dc1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.724+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103c3a4a0 0x557103dc1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.724+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/1102619743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103cb4d00 0x557103dc1560 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.724+0000 7fd2c452ab80 1 -- 172.21.15.46:0/1102619743 >> 172.21.15.46:0/1102619743 conn(0x557103c3df70 msgr2=0x557103dabff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:31.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.724+0000 7fd28effd700 0 client.16524 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:31.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.725+0000 7fd2c452ab80 1 -- 172.21.15.46:0/1102619743 shutdown_connections 2024-05-09T00:18:31.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.725+0000 7fd2c452ab80 1 -- 172.21.15.46:0/1102619743 wait complete. 2024-05-09T00:18:31.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.725+0000 7fd2c452ab80 20 client.16524 trim_cache size 0 max 16384 2024-05-09T00:18:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.745+0000 7fd2c452ab80 1 Processor -- start 2024-05-09T00:18:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.745+0000 7fd2c452ab80 1 -- start start 2024-05-09T00:18:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.745+0000 7fd2c452ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 0x557103dd8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.745+0000 7fd2c452ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103cb4d00 0x557103dd9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.745+0000 7fd2c452ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 0x557103dabff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.745+0000 7fd2c452ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b01b35c0 con 0x557103c3a4a0 2024-05-09T00:18:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.745+0000 7fd2c452ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b01b0e50 con 0x557103cb4d00 2024-05-09T00:18:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.745+0000 7fd2c452ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b01aec80 con 0x557103db7400 2024-05-09T00:18:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2af7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 0x557103dd8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2af7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 0x557103dd8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51846/0 (socket says 172.21.15.46:51846) 2024-05-09T00:18:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2af7fe700 1 -- 172.21.15.46:0/364007432 learned_addr learned my addr 172.21.15.46:0/364007432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2a6ffd700 1 --2- 172.21.15.46:0/364007432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103cb4d00 0x557103dd9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2affff700 1 --2- 172.21.15.46:0/364007432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 0x557103dabff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2af7fe700 1 -- 172.21.15.46:0/364007432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 msgr2=0x557103dabff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/364007432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 0x557103dabff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2af7fe700 1 -- 172.21.15.46:0/364007432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103cb4d00 msgr2=0x557103dd9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/364007432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103cb4d00 0x557103dd9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2af7fe700 1 -- 172.21.15.46:0/364007432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557103dae9d0 con 0x557103c3a4a0 2024-05-09T00:18:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/364007432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 0x557103dd8d10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0005d90 tx=0x7fd2a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5895987c2b14f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2a57fa700 1 -- 172.21.15.46:0/364007432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0021c80 con 0x557103c3a4a0 2024-05-09T00:18:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2a57fa700 1 -- 172.21.15.46:0/364007432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2a0021e20 con 0x557103c3a4a0 2024-05-09T00:18:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.746+0000 7fd2a57fa700 1 -- 172.21.15.46:0/364007432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0005460 con 0x557103c3a4a0 2024-05-09T00:18:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.747+0000 7fd2c452ab80 1 -- 172.21.15.46:0/364007432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 msgr2=0x557103dd8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.747+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/364007432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 0x557103dd8d10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0005d90 tx=0x7fd2a0013000 comp rx=0 tx=0).stop 2024-05-09T00:18:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.747+0000 7fd2c452ab80 1 -- 172.21.15.46:0/364007432 shutdown_connections 2024-05-09T00:18:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.747+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/364007432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 0x557103dabff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.747+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/364007432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 0x557103dd8d10 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.747+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/364007432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103cb4d00 0x557103dd9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.747+0000 7fd2c452ab80 1 -- 172.21.15.46:0/364007432 >> 172.21.15.46:0/364007432 conn(0x557103c3df70 msgr2=0x557103dbf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.747+0000 7fd2c452ab80 1 -- 172.21.15.46:0/364007432 shutdown_connections 2024-05-09T00:18:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.747+0000 7fd2c452ab80 1 -- 172.21.15.46:0/364007432 wait complete. 2024-05-09T00:18:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2c452ab80 1 -- start start 2024-05-09T00:18:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2c452ab80 1 Processor -- start 2024-05-09T00:18:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2c452ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 0x557103dc9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2c452ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103cb4d00 0x557103dbdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2c452ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 0x557103dbe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2c452ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b01af310 con 0x557103c3a4a0 2024-05-09T00:18:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2c452ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b01b0780 con 0x557103cb4d00 2024-05-09T00:18:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2c452ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b01af9e0 con 0x557103db7400 2024-05-09T00:18:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2c452ab80 10 client.0 _ll_register_callbacks cb 0x557103db7dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:18:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2af7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 0x557103dc9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2af7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 0x557103dc9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51848/0 (socket says 172.21.15.46:51848) 2024-05-09T00:18:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2af7fe700 1 -- 172.21.15.46:0/2919014694 learned_addr learned my addr 172.21.15.46:0/2919014694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:18:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2a6ffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103cb4d00 0x557103dbdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.748+0000 7fd2affff700 1 --2- 172.21.15.46:0/2919014694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 0x557103dbe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd2af7fe700 1 -- 172.21.15.46:0/2919014694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 msgr2=0x557103dbe320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/2919014694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 0x557103dbe320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd2af7fe700 1 -- 172.21.15.46:0/2919014694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103cb4d00 msgr2=0x557103dbdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/2919014694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103cb4d00 0x557103dbdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd2af7fe700 1 -- 172.21.15.46:0/2919014694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557103dbe860 con 0x557103c3a4a0 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd2af7fe700 1 --2- 172.21.15.46:0/2919014694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 0x557103dc9410 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000ba80 tx=0x7fd2a0021d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554c905a560167c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd28effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd28effd700 1 -- 172.21.15.46:0/2919014694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0020260 con 0x557103c3a4a0 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd28effd700 1 -- 172.21.15.46:0/2919014694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd2a0027040 con 0x557103c3a4a0 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd2c452ab80 1 -- 172.21.15.46:0/2919014694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2a0021e20 con 0x557103c3a4a0 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd28effd700 1 -- 172.21.15.46:0/2919014694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd2a0021140 con 0x557103c3a4a0 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd2c452ab80 10 client.16530 fetch_fsmap learned FSMap version 26 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.749+0000 7fd2c452ab80 1 -- 172.21.15.46:0/2919014694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557103e3b5a0 con 0x557103c3a4a0 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.750+0000 7fd28effd700 1 -- 172.21.15.46:0/2919014694 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2a0021e20 con 0x557103c3a4a0 2024-05-09T00:18:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.750+0000 7fd28effd700 1 -- 172.21.15.46:0/2919014694 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd2a0028ce0 con 0x557103c3a4a0 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.750+0000 7fd2c452ab80 10 client.16530 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.750+0000 7fd2c452ab80 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.750+0000 7fd2c452ab80 20 client.16530 populate_metadata read hostname 'smithi046' 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.750+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/2919014694 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557103e3bc40 0x557103e3e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.750+0000 7fd2c452ab80 4 client.16530 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.750+0000 7fd2c452ab80 1 -- 172.21.15.46:0/2919014694 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557103ce1b20 con 0x557103e3bc40 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.750+0000 7fd2a6ffd700 1 --2- 172.21.15.46:0/2919014694 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557103e3bc40 0x557103e3e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.750+0000 7fd2a6ffd700 1 --2- 172.21.15.46:0/2919014694 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557103e3bc40 0x557103e3e060 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9515e35acbadadc6 server_cookie=5990c09af6686046 in_seq=0 out_seq=0 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.750+0000 7fd28effd700 10 client.16530 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd28effd700 1 -- 172.21.15.46:0/2919014694 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557103ce1b20 con 0x557103e3bc40 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd28effd700 10 client.16530 handle_command_reply: tid=0 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd2c452ab80 1 client.16530 shutdown 2024-05-09T00:18:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd2c452ab80 1 -- 172.21.15.46:0/2919014694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 msgr2=0x557103dc9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/2919014694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 0x557103dc9410 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000ba80 tx=0x7fd2a0021d30 comp rx=0 tx=0).stop 2024-05-09T00:18:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd2c452ab80 1 -- 172.21.15.46:0/2919014694 shutdown_connections 2024-05-09T00:18:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/2919014694 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557103e3bc40 0x557103e3e060 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/2919014694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557103db7400 0x557103dbe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/2919014694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557103c3a4a0 0x557103dc9410 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd2c452ab80 1 --2- 172.21.15.46:0/2919014694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557103cb4d00 0x557103dbdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd2c452ab80 1 -- 172.21.15.46:0/2919014694 >> 172.21.15.46:0/2919014694 conn(0x557103c3df70 msgr2=0x557103dabff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd28effd700 0 client.16530 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:18:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd2c452ab80 1 -- 172.21.15.46:0/2919014694 shutdown_connections 2024-05-09T00:18:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd2c452ab80 1 -- 172.21.15.46:0/2919014694 wait complete. 2024-05-09T00:18:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.751+0000 7fd2c452ab80 20 client.16530 trim_cache size 0 max 16384 2024-05-09T00:18:31.758 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:18:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:18:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T00:18:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "21669b2c-47dc-450f-8fa5-7c791b3efc15": { 2024-05-09T00:18:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T00:18:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "21669b2c-47dc-450f-8fa5-7c791b3efc15", 2024-05-09T00:18:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T00:18:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:18:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:18:31.759 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:18:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.752+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1152607428 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd29c0b2880 msgr2=0x7fd29c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.752+0000 7fd2b5b14700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd29c0b2880 0x7fd29c0b4d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0005950 tx=0x7fd2a000b410 comp rx=0 tx=0).stop 2024-05-09T00:18:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.752+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1152607428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0109850 msgr2=0x7fd2b0077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:18:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.752+0000 7fd2b5b14700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0109850 0x7fd2b0077650 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8007d70 tx=0x7fd2a800a340 comp rx=0 tx=0).stop 2024-05-09T00:18:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.752+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1152607428 shutdown_connections 2024-05-09T00:18:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.752+0000 7fd2b5b14700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd29c0b2880 0x7fd29c0b4d90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.752+0000 7fd2b5b14700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0109850 0x7fd2b0077650 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.752+0000 7fd2b5b14700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2b01045d0 0x7fd2b0076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.752+0000 7fd2b5b14700 1 --2- 172.21.15.46:0/1152607428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2b0108f00 0x7fd2b0077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:18:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.752+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1152607428 >> 172.21.15.46:0/1152607428 conn(0x7fd2b00ffea0 msgr2=0x7fd2b010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:18:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.752+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1152607428 shutdown_connections 2024-05-09T00:18:31.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:18:31.752+0000 7fd2b5b14700 1 -- 172.21.15.46:0/1152607428 wait complete. 2024-05-09T00:18:32.535 INFO:tasks.cephfs.filesystem:scrub status for tag:21669b2c-47dc-450f-8fa5-7c791b3efc15 - {'path': '/', 'tag': '21669b2c-47dc-450f-8fa5-7c791b3efc15', 'options': 'recursive,force'} 2024-05-09T00:18:33.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:32 smithi046 ceph-mon[30151]: pgmap v591: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:18:33.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:32 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:18:33.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:32 smithi169 ceph-mon[34777]: pgmap v591: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:18:33.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:32 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:18:33.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:32 smithi142 ceph-mon[34533]: pgmap v591: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:18:33.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:32 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:18:35.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:34 smithi046 ceph-mon[30151]: pgmap v592: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:18:35.109 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:34 smithi169 ceph-mon[34777]: pgmap v592: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:18:35.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:34 smithi142 ceph-mon[34533]: pgmap v592: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:18:37.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:36 smithi046 ceph-mon[30151]: pgmap v593: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:18:37.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:36 smithi169 ceph-mon[34777]: pgmap v593: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:18:37.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:36 smithi142 ceph-mon[34533]: pgmap v593: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:18:39.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:38 smithi046 ceph-mon[30151]: pgmap v594: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:18:39.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:38 smithi169 ceph-mon[34777]: pgmap v594: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:18:39.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:38 smithi142 ceph-mon[34533]: pgmap v594: 97 pgs: 97 active+clean; 4.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:18:41.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:40 smithi046 ceph-mon[30151]: pgmap v595: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:18:41.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:40 smithi169 ceph-mon[34777]: pgmap v595: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:18:41.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:40 smithi142 ceph-mon[34533]: pgmap v595: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:18:43.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:42 smithi046 ceph-mon[30151]: pgmap v596: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:18:43.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:42 smithi169 ceph-mon[34777]: pgmap v596: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:18:43.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:42 smithi142 ceph-mon[34533]: pgmap v596: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:18:45.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:44 smithi046 ceph-mon[30151]: pgmap v597: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:18:45.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:44 smithi169 ceph-mon[34777]: pgmap v597: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:18:45.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:44 smithi142 ceph-mon[34533]: pgmap v597: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:18:47.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:46 smithi046 ceph-mon[30151]: pgmap v598: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-05-09T00:18:47.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:46 smithi169 ceph-mon[34777]: pgmap v598: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-05-09T00:18:47.194 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:46 smithi142 ceph-mon[34533]: pgmap v598: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-05-09T00:18:49.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:48 smithi169 ceph-mon[34777]: pgmap v599: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:49.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:48 smithi142 ceph-mon[34533]: pgmap v599: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:49.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:48 smithi046 ceph-mon[30151]: pgmap v599: 97 pgs: 97 active+clean; 4.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:50 smithi169 ceph-mon[34777]: pgmap v600: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:18:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:18:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:18:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:50 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:18:51.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:50 smithi142 ceph-mon[34533]: pgmap v600: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:51.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:18:51.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:18:51.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:18:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:50 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:18:51.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:50 smithi046 ceph-mon[30151]: pgmap v600: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:51.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:18:51.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:18:51.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:18:51.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:50 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:18:51.920 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.920 INFO:tasks.workunit.client.0.smithi046.stdout:251/600: open fd = 3 2024-05-09T00:18:51.920 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.920 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.920 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.920 INFO:tasks.workunit.client.0.smithi046.stdout:252/600: open fd = 3 2024-05-09T00:18:51.920 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.920 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.921 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.921 INFO:tasks.workunit.client.0.smithi046.stdout:253/600: open fd = 3 2024-05-09T00:18:51.921 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.921 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.921 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.921 INFO:tasks.workunit.client.0.smithi046.stdout:254/600: open fd = 3 2024-05-09T00:18:51.921 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.921 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.921 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:255/600: open fd = 3 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:256/600: open fd = 3 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:257/600: open fd = 3 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:258/600: open fd = 3 2024-05-09T00:18:51.922 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:259/600: open fd = 3 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:260/600: open fd = 3 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:261/600: open fd = 3 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.923 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:262/600: open fd = 3 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:263/600: open fd = 3 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:264/600: open fd = 3 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.924 INFO:tasks.workunit.client.0.smithi046.stdout:265/600: open fd = 3 2024-05-09T00:18:51.925 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.925 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.925 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.925 INFO:tasks.workunit.client.0.smithi046.stdout:266/600: open fd = 3 2024-05-09T00:18:51.925 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.925 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.925 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.925 INFO:tasks.workunit.client.0.smithi046.stdout:267/600: open fd = 3 2024-05-09T00:18:51.925 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.925 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.925 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.925 INFO:tasks.workunit.client.0.smithi046.stdout:268/600: open fd = 3 2024-05-09T00:18:51.926 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.926 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.926 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.926 INFO:tasks.workunit.client.0.smithi046.stdout:269/600: open fd = 3 2024-05-09T00:18:51.926 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.926 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.926 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.926 INFO:tasks.workunit.client.0.smithi046.stdout:270/600: open fd = 3 2024-05-09T00:18:51.926 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.926 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:271/600: open fd = 3 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:272/600: open fd = 3 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:273/600: open fd = 3 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.927 INFO:tasks.workunit.client.0.smithi046.stdout:274/600: open fd = 3 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:275/600: open fd = 3 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:276/600: open fd = 3 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:277/600: open fd = 3 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.928 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:278/600: open fd = 3 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:279/600: open fd = 3 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:280/600: open fd = 3 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.929 INFO:tasks.workunit.client.0.smithi046.stdout:281/600: open fd = 3 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:282/600: open fd = 3 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:283/600: open fd = 3 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:284/600: open fd = 3 2024-05-09T00:18:51.930 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:285/600: open fd = 3 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:286/600: open fd = 3 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:287/600: open fd = 3 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.931 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:288/600: open fd = 3 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:289/600: open fd = 3 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:290/600: open fd = 3 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:291/600: open fd = 3 2024-05-09T00:18:51.932 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.933 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.933 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.933 INFO:tasks.workunit.client.0.smithi046.stdout:292/600: open fd = 3 2024-05-09T00:18:51.933 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.933 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.933 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.933 INFO:tasks.workunit.client.0.smithi046.stdout:293/600: open fd = 3 2024-05-09T00:18:51.933 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.933 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.933 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:294/600: open fd = 3 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:295/600: open fd = 3 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:296/600: open fd = 3 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:297/600: open fd = 3 2024-05-09T00:18:51.934 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:298/600: open fd = 3 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:299/600: open fd = 3 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:300/600: open fd = 3 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.935 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:301/600: open fd = 3 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:302/600: open fd = 3 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:303/600: open fd = 3 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:304/600: open fd = 3 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.936 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:305/600: open fd = 3 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:306/600: open fd = 3 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:307/600: open fd = 3 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.937 INFO:tasks.workunit.client.0.smithi046.stdout:308/600: open fd = 3 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:309/600: open fd = 3 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:310/600: open fd = 3 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:311/600: open fd = 3 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.939 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:18:51.939 INFO:tasks.workunit.client.0.smithi046.stdout:312/600: open fd = 3 2024-05-09T00:18:51.939 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:51.939 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:18:53.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:52 smithi046 ceph-mon[30151]: pgmap v601: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:18:53.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:52 smithi169 ceph-mon[34777]: pgmap v601: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:18:53.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:52 smithi142 ceph-mon[34533]: pgmap v601: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:18:55.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:54 smithi169 ceph-mon[34777]: pgmap v602: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:18:55.195 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:54 smithi142 ceph-mon[34533]: pgmap v602: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:18:55.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:54 smithi046 ceph-mon[30151]: pgmap v602: 97 pgs: 97 active+clean; 4.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:18:57.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:56 smithi169 ceph-mon[34777]: pgmap v603: 97 pgs: 97 active+clean; 4.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:57.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:56 smithi142 ceph-mon[34533]: pgmap v603: 97 pgs: 97 active+clean; 4.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:57.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:56 smithi046 ceph-mon[30151]: pgmap v603: 97 pgs: 97 active+clean; 4.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:18:59.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:18:58 smithi169 ceph-mon[34777]: pgmap v604: 97 pgs: 97 active+clean; 4.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:59.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:18:58 smithi142 ceph-mon[34533]: pgmap v604: 97 pgs: 97 active+clean; 4.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:18:59.311 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:18:58 smithi046 ceph-mon[30151]: pgmap v604: 97 pgs: 97 active+clean; 4.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:01.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:00 smithi169 ceph-mon[34777]: pgmap v605: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:19:01.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:00 smithi142 ceph-mon[34533]: pgmap v605: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:19:01.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:00 smithi046 ceph-mon[30151]: pgmap v605: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:19:02.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:19:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:03.089 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:02 smithi046 ceph-mon[30151]: pgmap v606: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:19:03.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:02 smithi169 ceph-mon[34777]: pgmap v606: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:19:03.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:02 smithi142 ceph-mon[34533]: pgmap v606: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:19:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.658+0000 7f3ae2027700 1 -- 172.21.15.46:0/1764958155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc108f00 msgr2=0x7f3adc109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.658+0000 7f3ae2027700 1 --2- 172.21.15.46:0/1764958155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc108f00 0x7f3adc109310 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00099c0 tx=0x7f3ad0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.658+0000 7f3ae2027700 1 -- 172.21.15.46:0/1764958155 shutdown_connections 2024-05-09T00:19:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.658+0000 7f3ae2027700 1 --2- 172.21.15.46:0/1764958155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3adc109850 0x7f3adc1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.658+0000 7f3ae2027700 1 --2- 172.21.15.46:0/1764958155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3adc1045d0 0x7f3adc1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.658+0000 7f3ae2027700 1 --2- 172.21.15.46:0/1764958155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc108f00 0x7f3adc109310 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.658+0000 7f3ae2027700 1 -- 172.21.15.46:0/1764958155 >> 172.21.15.46:0/1764958155 conn(0x7f3adc0ffea0 msgr2=0x7f3adc1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.658+0000 7f3ae2027700 1 -- 172.21.15.46:0/1764958155 shutdown_connections 2024-05-09T00:19:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.658+0000 7f3ae2027700 1 -- 172.21.15.46:0/1764958155 wait complete. 2024-05-09T00:19:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.659+0000 7f3ae2027700 1 Processor -- start 2024-05-09T00:19:04.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.659+0000 7f3ae2027700 1 -- start start 2024-05-09T00:19:04.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.659+0000 7f3ae2027700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc1045d0 0x7f3adc19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.659+0000 7f3ae2027700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3adc108f00 0x7f3adc19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.659+0000 7f3ae2027700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3adc109850 0x7f3adc19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.659+0000 7f3ae2027700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1144b0 con 0x7f3adc1045d0 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.659+0000 7f3ae2027700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc114330 con 0x7f3adc109850 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.659+0000 7f3ae2027700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc114630 con 0x7f3adc108f00 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.659+0000 7f3adb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc1045d0 0x7f3adc19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.659+0000 7f3adb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc1045d0 0x7f3adc19af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33830/0 (socket says 172.21.15.46:33830) 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.659+0000 7f3adb7fe700 1 -- 172.21.15.46:0/1994341955 learned_addr learned my addr 172.21.15.46:0/1994341955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3adbfff700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3adc109850 0x7f3adc19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3adaffd700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3adc108f00 0x7f3adc19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3adb7fe700 1 -- 172.21.15.46:0/1994341955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3adc108f00 msgr2=0x7f3adc19b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3adc108f00 0x7f3adc19b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3adb7fe700 1 -- 172.21.15.46:0/1994341955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3adc109850 msgr2=0x7f3adc19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3adc109850 0x7f3adc19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3adb7fe700 1 -- 172.21.15.46:0/1994341955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc19fdd0 con 0x7f3adc1045d0 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc1045d0 0x7f3adc19af00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00d950 tx=0x7f3acc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:04.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3ad8ff9700 1 -- 172.21.15.46:0/1994341955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc016070 con 0x7f3adc1045d0 2024-05-09T00:19:04.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3ae2027700 1 -- 172.21.15.46:0/1994341955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3adc1a00c0 con 0x7f3adc1045d0 2024-05-09T00:19:04.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3ad8ff9700 1 -- 172.21.15.46:0/1994341955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3acc00fd40 con 0x7f3adc1045d0 2024-05-09T00:19:04.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3ad8ff9700 1 -- 172.21.15.46:0/1994341955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc00e3b0 con 0x7f3adc1045d0 2024-05-09T00:19:04.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.660+0000 7f3ae2027700 1 -- 172.21.15.46:0/1994341955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3adc076b00 con 0x7f3adc1045d0 2024-05-09T00:19:04.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.663+0000 7f3ad8ff9700 1 -- 172.21.15.46:0/1994341955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3acc019070 con 0x7f3adc1045d0 2024-05-09T00:19:04.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.664+0000 7f3ad8ff9700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3ac40b2880 0x7f3ac40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.664+0000 7f3af0a3db80 1 Processor -- start 2024-05-09T00:19:04.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.664+0000 7f3af0a3db80 1 -- start start 2024-05-09T00:19:04.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.664+0000 7f3ad8ff9700 1 -- 172.21.15.46:0/1994341955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f3acc0e1d00 con 0x7f3adc1045d0 2024-05-09T00:19:04.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.664+0000 7f3adaffd700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3ac40b2880 0x7f3ac40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3af0a3db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd018160 0x5646cd0b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3af0a3db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646ccf2e4a0 0x5646cd0b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3af0a3db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646cd0ab6a0 0x5646cd0abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3af0a3db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5646ccfcdff0 con 0x5646cd018160 2024-05-09T00:19:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3af0a3db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5646ccf6c7f0 con 0x5646ccf2e4a0 2024-05-09T00:19:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3af0a3db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5646ccfc9fb0 con 0x5646cd0ab6a0 2024-05-09T00:19:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3adb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd018160 0x5646cd0b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3adaffd700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3ac40b2880 0x7f3ac40b4d90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0000c00 tx=0x7f3ad001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3adb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd018160 0x5646cd0b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33832/0 (socket says 172.21.15.46:33832) 2024-05-09T00:19:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3adb7fe700 1 -- 172.21.15.46:0/2000609192 learned_addr learned my addr 172.21.15.46:0/2000609192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3adbfff700 1 --2- 172.21.15.46:0/2000609192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646cd0ab6a0 0x5646cd0abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3adaffd700 1 --2- 172.21.15.46:0/2000609192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646ccf2e4a0 0x5646cd0b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3adb7fe700 1 -- 172.21.15.46:0/2000609192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646cd0ab6a0 msgr2=0x5646cd0abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/2000609192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646cd0ab6a0 0x5646cd0abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3adb7fe700 1 -- 172.21.15.46:0/2000609192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646ccf2e4a0 msgr2=0x5646cd0b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/2000609192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646ccf2e4a0 0x5646cd0b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3adb7fe700 1 -- 172.21.15.46:0/2000609192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ccf64370 con 0x5646cd018160 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.665+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/2000609192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd018160 0x5646cd0b14f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0a9260 tx=0x7f3acc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291a3f8a501aa959 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.666+0000 7f3ac1ffb700 1 -- 172.21.15.46:0/2000609192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc020070 con 0x5646cd018160 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.666+0000 7f3ac1ffb700 1 -- 172.21.15.46:0/2000609192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3acc0ad6d0 con 0x5646cd018160 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.666+0000 7f3ac1ffb700 1 -- 172.21.15.46:0/2000609192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0a8580 con 0x5646cd018160 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.666+0000 7f3af0a3db80 1 -- 172.21.15.46:0/2000609192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd018160 msgr2=0x5646cd0b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.666+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/2000609192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd018160 0x5646cd0b14f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0a9260 tx=0x7f3acc010000 comp rx=0 tx=0).stop 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.667+0000 7f3af0a3db80 1 -- 172.21.15.46:0/2000609192 shutdown_connections 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.667+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/2000609192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646cd0ab6a0 0x5646cd0abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.667+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/2000609192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd018160 0x5646cd0b14f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.667+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/2000609192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646ccf2e4a0 0x5646cd0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.667+0000 7f3af0a3db80 1 -- 172.21.15.46:0/2000609192 >> 172.21.15.46:0/2000609192 conn(0x5646ccf8e6a0 msgr2=0x5646cd0a6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:04.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.667+0000 7f3af0a3db80 1 -- 172.21.15.46:0/2000609192 shutdown_connections 2024-05-09T00:19:04.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.667+0000 7f3af0a3db80 1 -- 172.21.15.46:0/2000609192 wait complete. 2024-05-09T00:19:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.667+0000 7f3af0a3db80 1 -- start start 2024-05-09T00:19:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.667+0000 7f3af0a3db80 1 Processor -- start 2024-05-09T00:19:04.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3af0a3db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646ccf2e4a0 0x5646cd0b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3af0a3db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0b5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3af0a3db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 0x5646cd0b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3af0a3db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5646cce49ba0 con 0x5646cd0ab6a0 2024-05-09T00:19:04.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3af0a3db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5646ccf88150 con 0x5646cd018160 2024-05-09T00:19:04.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3af0a3db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5646cd0ac4e0 con 0x5646ccf2e4a0 2024-05-09T00:19:04.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3af0a3db80 10 client.0 _ll_register_callbacks cb 0x5646cce3f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:04.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adaffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 0x5646cd0b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adaffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:35708/0 (socket says 172.21.15.46:35708) 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adaffd700 1 -- 172.21.15.46:0/1660918101 learned_addr learned my addr 172.21.15.46:0/1660918101 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 0x5646cd0b5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33846/0 (socket says 172.21.15.46:33846) 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646ccf2e4a0 0x5646cd0b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adbfff700 1 -- 172.21.15.46:0/1660918101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646ccf2e4a0 msgr2=0x5646cd0b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adbfff700 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646ccf2e4a0 0x5646cd0b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adbfff700 1 -- 172.21.15.46:0/1660918101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 msgr2=0x5646cd0b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adbfff700 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adbfff700 1 -- 172.21.15.46:0/1660918101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646cce49490 con 0x5646cd0ab6a0 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adaffd700 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.668+0000 7f3adbfff700 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 0x5646cd0b5c50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4007af0 tx=0x7f3ad4007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d123e95fd75193 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3abaffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3abaffd700 1 -- 172.21.15.46:0/1660918101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad401a070 con 0x5646cd0ab6a0 2024-05-09T00:19:04.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3abaffd700 1 -- 172.21.15.46:0/1660918101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f3ad4004db0 con 0x5646cd0ab6a0 2024-05-09T00:19:04.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1660918101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3acc0ad6d0 con 0x5646cd0ab6a0 2024-05-09T00:19:04.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3abaffd700 1 -- 172.21.15.46:0/1660918101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ad4025050 con 0x5646cd0ab6a0 2024-05-09T00:19:04.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3af0a3db80 10 client.16542 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:04.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1660918101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646cd0c2b90 con 0x5646cd0ab6a0 2024-05-09T00:19:04.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3abaffd700 1 -- 172.21.15.46:0/1660918101 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3acc0ad6d0 con 0x5646cd0ab6a0 2024-05-09T00:19:04.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3abaffd700 1 -- 172.21.15.46:0/1660918101 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3ad40167b0 con 0x5646cd0ab6a0 2024-05-09T00:19:04.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3af0a3db80 10 client.16542 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:04.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3af0a3db80 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:04.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3af0a3db80 20 client.16542 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646cd0c51c0 0x5646cd0c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3af0a3db80 4 client.16542 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.669+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1660918101 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646ccfdb9d0 con 0x5646cd0c51c0 2024-05-09T00:19:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.670+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646cd0c51c0 0x5646cd0c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.670+0000 7f3abaffd700 10 client.16542 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.670+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646cd0c51c0 0x5646cd0c7600 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe5a5addc2a1b536 server_cookie=b6f6bad9a4f259e2 in_seq=0 out_seq=0 2024-05-09T00:19:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3abaffd700 1 -- 172.21.15.46:0/1660918101 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646ccfdb9d0 con 0x5646cd0c51c0 2024-05-09T00:19:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3abaffd700 10 client.16542 handle_command_reply: tid=0 2024-05-09T00:19:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3af0a3db80 1 client.16542 shutdown 2024-05-09T00:19:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1660918101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 msgr2=0x5646cd0b5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 0x5646cd0b5c50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4007af0 tx=0x7f3ad4007b20 comp rx=0 tx=0).stop 2024-05-09T00:19:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1660918101 shutdown_connections 2024-05-09T00:19:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646cd0c51c0 0x5646cd0c7600 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 0x5646cd0b5c50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646ccf2e4a0 0x5646cd0b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/1660918101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1660918101 >> 172.21.15.46:0/1660918101 conn(0x5646ccf8e6a0 msgr2=0x5646cd0a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3abaffd700 0 client.16542 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1660918101 shutdown_connections 2024-05-09T00:19:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.671+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1660918101 wait complete. 2024-05-09T00:19:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.672+0000 7f3af0a3db80 20 client.16542 trim_cache size 0 max 16384 2024-05-09T00:19:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3af0a3db80 1 Processor -- start 2024-05-09T00:19:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3af0a3db80 1 -- start start 2024-05-09T00:19:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3af0a3db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646ccf2e4a0 0x5646cd0af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3af0a3db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3af0a3db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646cd0ab6a0 0x5646cd0c0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3af0a3db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1b38d0 con 0x5646ccf2e4a0 2024-05-09T00:19:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3af0a3db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1b1160 con 0x5646cd018160 2024-05-09T00:19:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3af0a3db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1aefb0 con 0x5646cd0ab6a0 2024-05-09T00:19:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3adb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646ccf2e4a0 0x5646cd0af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3adb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646ccf2e4a0 0x5646cd0af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33856/0 (socket says 172.21.15.46:33856) 2024-05-09T00:19:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3adb7fe700 1 -- 172.21.15.46:0/1194925370 learned_addr learned my addr 172.21.15.46:0/1194925370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3adbfff700 1 --2- 172.21.15.46:0/1194925370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646cd0ab6a0 0x5646cd0c0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.692+0000 7f3adaffd700 1 --2- 172.21.15.46:0/1194925370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.693+0000 7f3adb7fe700 1 -- 172.21.15.46:0/1194925370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646cd0ab6a0 msgr2=0x5646cd0c0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.693+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/1194925370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646cd0ab6a0 0x5646cd0c0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.693+0000 7f3adb7fe700 1 -- 172.21.15.46:0/1194925370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 msgr2=0x5646cd0b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.693+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/1194925370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.693+0000 7f3adb7fe700 1 -- 172.21.15.46:0/1194925370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646cd0ac620 con 0x5646ccf2e4a0 2024-05-09T00:19:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.693+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/1194925370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646ccf2e4a0 0x5646cd0af8a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00b2f0 tx=0x7f3acc091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1294d13e8617cc47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.693+0000 7f3ac17fa700 1 -- 172.21.15.46:0/1194925370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc020040 con 0x5646ccf2e4a0 2024-05-09T00:19:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.693+0000 7f3ac17fa700 1 -- 172.21.15.46:0/1194925370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3acc0ad6d0 con 0x5646ccf2e4a0 2024-05-09T00:19:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.693+0000 7f3ac17fa700 1 -- 172.21.15.46:0/1194925370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0a8580 con 0x5646ccf2e4a0 2024-05-09T00:19:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.694+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1194925370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646ccf2e4a0 msgr2=0x5646cd0af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.694+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/1194925370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646ccf2e4a0 0x5646cd0af8a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00b2f0 tx=0x7f3acc091040 comp rx=0 tx=0).stop 2024-05-09T00:19:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.694+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1194925370 shutdown_connections 2024-05-09T00:19:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.694+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/1194925370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646cd0ab6a0 0x5646cd0c0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.694+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/1194925370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646ccf2e4a0 0x5646cd0af8a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.694+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/1194925370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.694+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1194925370 >> 172.21.15.46:0/1194925370 conn(0x5646ccf8e6a0 msgr2=0x5646cd0b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.694+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1194925370 shutdown_connections 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.694+0000 7f3af0a3db80 1 -- 172.21.15.46:0/1194925370 wait complete. 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.694+0000 7f3af0a3db80 1 -- start start 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3af0a3db80 1 Processor -- start 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3af0a3db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646ccf2e4a0 0x5646cd0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3af0a3db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3af0a3db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 0x5646cd0cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3af0a3db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1af620 con 0x5646cd0ab6a0 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3af0a3db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1b0a90 con 0x5646cd018160 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3af0a3db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1afcf0 con 0x5646ccf2e4a0 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3af0a3db80 10 client.0 _ll_register_callbacks cb 0x5646cce3f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3adbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 0x5646cd0cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3adbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 0x5646cd0cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33868/0 (socket says 172.21.15.46:33868) 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3adbfff700 1 -- 172.21.15.46:0/3049088941 learned_addr learned my addr 172.21.15.46:0/3049088941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646ccf2e4a0 0x5646cd0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3adaffd700 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3adbfff700 1 -- 172.21.15.46:0/3049088941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646ccf2e4a0 msgr2=0x5646cd0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3adbfff700 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646ccf2e4a0 0x5646cd0cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3adbfff700 1 -- 172.21.15.46:0/3049088941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 msgr2=0x5646cd0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3adbfff700 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.695+0000 7f3adbfff700 1 -- 172.21.15.46:0/3049088941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646cd0bb830 con 0x5646cd0ab6a0 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3adbfff700 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 0x5646cd0cd1f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4016d70 tx=0x7f3ad4005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c93a9ae72850f8f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3abaffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3abaffd700 1 -- 172.21.15.46:0/3049088941 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad401a040 con 0x5646cd0ab6a0 2024-05-09T00:19:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3abaffd700 1 -- 172.21.15.46:0/3049088941 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f3ad4003c00 con 0x5646cd0ab6a0 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3af0a3db80 1 -- 172.21.15.46:0/3049088941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3acc0ad6d0 con 0x5646cd0ab6a0 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3abaffd700 1 -- 172.21.15.46:0/3049088941 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ad4003e00 con 0x5646cd0ab6a0 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3af0a3db80 10 client.16548 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3af0a3db80 1 -- 172.21.15.46:0/3049088941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646cd12fca0 con 0x5646cd0ab6a0 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3abaffd700 1 -- 172.21.15.46:0/3049088941 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3acc0ad6d0 con 0x5646cd0ab6a0 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3abaffd700 1 -- 172.21.15.46:0/3049088941 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3ad4015c50 con 0x5646cd0ab6a0 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3af0a3db80 10 client.16548 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3af0a3db80 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3af0a3db80 20 client.16548 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646cd1302c0 0x5646cd1326e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3af0a3db80 4 client.16548 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.696+0000 7f3af0a3db80 1 -- 172.21.15.46:0/3049088941 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646ccfd5a70 con 0x5646cd1302c0 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.697+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646cd1302c0 0x5646cd1326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.697+0000 7f3abaffd700 10 client.16548 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.697+0000 7f3adb7fe700 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646cd1302c0 0x5646cd1326e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dfeabb55d7de029 server_cookie=7ba32622b9aeb62f in_seq=0 out_seq=0 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.697+0000 7f3abaffd700 1 -- 172.21.15.46:0/3049088941 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5646ccfd5a70 con 0x5646cd1302c0 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.697+0000 7f3abaffd700 10 client.16548 handle_command_reply: tid=0 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.697+0000 7f3af0a3db80 1 client.16548 shutdown 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3af0a3db80 1 -- 172.21.15.46:0/3049088941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 msgr2=0x5646cd0cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 0x5646cd0cd1f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4016d70 tx=0x7f3ad4005cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3af0a3db80 1 -- 172.21.15.46:0/3049088941 shutdown_connections 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5646cd1302c0 0x5646cd1326e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646cd0ab6a0 0x5646cd0cd1f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646ccf2e4a0 0x5646cd0cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3af0a3db80 1 --2- 172.21.15.46:0/3049088941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646cd018160 0x5646cd0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3af0a3db80 1 -- 172.21.15.46:0/3049088941 >> 172.21.15.46:0/3049088941 conn(0x5646ccf8e6a0 msgr2=0x5646cd0aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3abaffd700 0 client.16548 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3af0a3db80 1 -- 172.21.15.46:0/3049088941 shutdown_connections 2024-05-09T00:19:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3af0a3db80 1 -- 172.21.15.46:0/3049088941 wait complete. 2024-05-09T00:19:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3af0a3db80 20 client.16548 trim_cache size 0 max 16384 2024-05-09T00:19:04.702 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:19:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:19:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:19:04.702 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:19:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3ae2027700 1 -- 172.21.15.46:0/1994341955 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3ac40b2880 msgr2=0x7f3ac40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3ae2027700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3ac40b2880 0x7f3ac40b4d90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0000c00 tx=0x7f3ad001a040 comp rx=0 tx=0).stop 2024-05-09T00:19:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3ae2027700 1 -- 172.21.15.46:0/1994341955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc1045d0 msgr2=0x7f3adc19af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3ae2027700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc1045d0 0x7f3adc19af00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00d950 tx=0x7f3acc00dc60 comp rx=0 tx=0).stop 2024-05-09T00:19:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3ae2027700 1 -- 172.21.15.46:0/1994341955 shutdown_connections 2024-05-09T00:19:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3ae2027700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f3ac40b2880 0x7f3ac40b4d90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3ae2027700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3adc109850 0x7f3adc19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.698+0000 7f3ae2027700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc1045d0 0x7f3adc19af00 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.699+0000 7f3ae2027700 1 --2- 172.21.15.46:0/1994341955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3adc108f00 0x7f3adc19b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.699+0000 7f3ae2027700 1 -- 172.21.15.46:0/1994341955 >> 172.21.15.46:0/1994341955 conn(0x7f3adc0ffea0 msgr2=0x7f3adc10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.699+0000 7f3ae2027700 1 -- 172.21.15.46:0/1994341955 shutdown_connections 2024-05-09T00:19:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:04.699+0000 7f3ae2027700 1 -- 172.21.15.46:0/1994341955 wait complete. 2024-05-09T00:19:05.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:04 smithi169 ceph-mon[34777]: pgmap v607: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:05.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:04 smithi142 ceph-mon[34533]: pgmap v607: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:05.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:04 smithi046 ceph-mon[30151]: pgmap v607: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:05.508 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:19:05.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:19:05.784 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:06.951 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:06 smithi046 ceph-mon[30151]: pgmap v608: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:19:07.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:06 smithi169 ceph-mon[34777]: pgmap v608: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:19:07.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:06 smithi142 ceph-mon[34533]: pgmap v608: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:19:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.522+0000 7fc02afbb700 1 -- 172.21.15.46:0/3711708472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0241034e0 msgr2=0x7fc0241038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.522+0000 7fc02afbb700 1 --2- 172.21.15.46:0/3711708472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0241034e0 0x7fc0241038f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0099c0 tx=0x7fc00c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.522+0000 7fc02afbb700 1 -- 172.21.15.46:0/3711708472 shutdown_connections 2024-05-09T00:19:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.522+0000 7fc02afbb700 1 --2- 172.21.15.46:0/3711708472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc024108760 0x7fc0241100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.522+0000 7fc02afbb700 1 --2- 172.21.15.46:0/3711708472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0241034e0 0x7fc0241038f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.522+0000 7fc02afbb700 1 --2- 172.21.15.46:0/3711708472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc024107e10 0x7fc024108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.522+0000 7fc02afbb700 1 -- 172.21.15.46:0/3711708472 >> 172.21.15.46:0/3711708472 conn(0x7fc0240fef30 msgr2=0x7fc024101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.522+0000 7fc02afbb700 1 -- 172.21.15.46:0/3711708472 shutdown_connections 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.522+0000 7fc02afbb700 1 -- 172.21.15.46:0/3711708472 wait complete. 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc02afbb700 1 Processor -- start 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc02afbb700 1 -- start start 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc02afbb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0241034e0 0x7fc024199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc02afbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc024107e10 0x7fc02419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc02afbb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc024108760 0x7fc02419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc02afbb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241133b0 con 0x7fc024107e10 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc02afbb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc024113230 con 0x7fc0241034e0 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc02afbb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc024113530 con 0x7fc024108760 2024-05-09T00:19:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc023fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc024107e10 0x7fc02419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc023fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc024107e10 0x7fc02419a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48626/0 (socket says 172.21.15.46:48626) 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc023fff700 1 -- 172.21.15.46:0/3463415463 learned_addr learned my addr 172.21.15.46:0/3463415463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc028d57700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0241034e0 0x7fc024199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.523+0000 7fc029558700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc024108760 0x7fc02419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.524+0000 7fc023fff700 1 -- 172.21.15.46:0/3463415463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc024108760 msgr2=0x7fc02419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.524+0000 7fc023fff700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc024108760 0x7fc02419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.524+0000 7fc023fff700 1 -- 172.21.15.46:0/3463415463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0241034e0 msgr2=0x7fc024199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.524+0000 7fc023fff700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0241034e0 0x7fc024199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.524+0000 7fc023fff700 1 -- 172.21.15.46:0/3463415463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0241a9160 con 0x7fc024107e10 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.524+0000 7fc023fff700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc024107e10 0x7fc02419a020 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc014007f10 tx=0x7fc01400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.524+0000 7fc021ffb700 1 -- 172.21.15.46:0/3463415463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01400de90 con 0x7fc024107e10 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.524+0000 7fc02afbb700 1 -- 172.21.15.46:0/3463415463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0241a9450 con 0x7fc024107e10 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.524+0000 7fc021ffb700 1 -- 172.21.15.46:0/3463415463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc014004bb0 con 0x7fc024107e10 2024-05-09T00:19:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.524+0000 7fc021ffb700 1 -- 172.21.15.46:0/3463415463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014015660 con 0x7fc024107e10 2024-05-09T00:19:07.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.525+0000 7fc02afbb700 1 -- 172.21.15.46:0/3463415463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0241a9940 con 0x7fc024107e10 2024-05-09T00:19:07.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.526+0000 7fc02afbb700 1 -- 172.21.15.46:0/3463415463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc024104990 con 0x7fc024107e10 2024-05-09T00:19:07.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.527+0000 7fc021ffb700 1 -- 172.21.15.46:0/3463415463 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc014005020 con 0x7fc024107e10 2024-05-09T00:19:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.528+0000 7fc021ffb700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0100b2a20 0x7fc0100b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.528+0000 7fc028d57700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0100b2a20 0x7fc0100b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.528+0000 7fc021ffb700 1 -- 172.21.15.46:0/3463415463 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc0140e1e90 con 0x7fc024107e10 2024-05-09T00:19:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.529+0000 7fc028d57700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0100b2a20 0x7fc0100b4f30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc00c003800 tx=0x7fc00c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:07.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.533+0000 7fc021ffb700 1 -- 172.21.15.46:0/3463415463 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc014010070 con 0x7fc024107e10 2024-05-09T00:19:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.793+0000 7fc02afbb700 1 -- 172.21.15.46:0/3463415463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc02402e5f0 con 0x7fc024107e10 2024-05-09T00:19:07.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.794+0000 7fc021ffb700 1 -- 172.21.15.46:0/3463415463 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7fc0140a9b70 con 0x7fc024107e10 2024-05-09T00:19:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:19:07.794 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:19:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.796+0000 7fc02afbb700 1 -- 172.21.15.46:0/3463415463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0100b2a20 msgr2=0x7fc0100b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.796+0000 7fc02afbb700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0100b2a20 0x7fc0100b4f30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc00c003800 tx=0x7fc00c01a040 comp rx=0 tx=0).stop 2024-05-09T00:19:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.796+0000 7fc02afbb700 1 -- 172.21.15.46:0/3463415463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc024107e10 msgr2=0x7fc02419a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.796+0000 7fc02afbb700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc024107e10 0x7fc02419a020 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc014007f10 tx=0x7fc01400d3b0 comp rx=0 tx=0).stop 2024-05-09T00:19:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.796+0000 7fc02afbb700 1 -- 172.21.15.46:0/3463415463 shutdown_connections 2024-05-09T00:19:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.796+0000 7fc02afbb700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0100b2a20 0x7fc0100b4f30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.796+0000 7fc02afbb700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc024108760 0x7fc02419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.796+0000 7fc02afbb700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0241034e0 0x7fc024199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.796+0000 7fc02afbb700 1 --2- 172.21.15.46:0/3463415463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc024107e10 0x7fc02419a020 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.796+0000 7fc02afbb700 1 -- 172.21.15.46:0/3463415463 >> 172.21.15.46:0/3463415463 conn(0x7fc0240fef30 msgr2=0x7fc0241096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.797+0000 7fc02afbb700 1 -- 172.21.15.46:0/3463415463 shutdown_connections 2024-05-09T00:19:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:07.797+0000 7fc02afbb700 1 -- 172.21.15.46:0/3463415463 wait complete. 2024-05-09T00:19:07.797 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:19:08.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:07 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3463415463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:19:08.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:07 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3463415463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:19:08.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:07 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3463415463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:19:08.614 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:19:08.877 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:09.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:08 smithi169 ceph-mon[34777]: pgmap v609: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:09.155 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:08 smithi046 ceph-mon[30151]: pgmap v609: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:09.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:08 smithi142 ceph-mon[34533]: pgmap v609: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:10.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:09 smithi046 ceph-mon[30151]: pgmap v610: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:19:10.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:09 smithi142 ceph-mon[34533]: pgmap v610: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:19:10.355 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:09 smithi169 ceph-mon[34777]: pgmap v610: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:19:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.755+0000 7f28f8a2c700 1 -- 172.21.15.46:0/1674486313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f4108740 msgr2=0x7f28f41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.755+0000 7f28f8a2c700 1 --2- 172.21.15.46:0/1674486313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f4108740 0x7f28f41100a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0099c0 tx=0x7f28ec009cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.756+0000 7f28f8a2c700 1 -- 172.21.15.46:0/1674486313 shutdown_connections 2024-05-09T00:19:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.756+0000 7f28f8a2c700 1 --2- 172.21.15.46:0/1674486313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f4108740 0x7f28f41100a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.756+0000 7f28f8a2c700 1 --2- 172.21.15.46:0/1674486313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f41034c0 0x7f28f41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.756+0000 7f28f8a2c700 1 --2- 172.21.15.46:0/1674486313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f4107df0 0x7f28f4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.756+0000 7f28f8a2c700 1 -- 172.21.15.46:0/1674486313 >> 172.21.15.46:0/1674486313 conn(0x7f28f40fedc0 msgr2=0x7f28f41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.756+0000 7f28f8a2c700 1 -- 172.21.15.46:0/1674486313 shutdown_connections 2024-05-09T00:19:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.756+0000 7f28f8a2c700 1 -- 172.21.15.46:0/1674486313 wait complete. 2024-05-09T00:19:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f8a2c700 1 Processor -- start 2024-05-09T00:19:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f8a2c700 1 -- start start 2024-05-09T00:19:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f8a2c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f41034c0 0x7f28f410c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f8a2c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f4107df0 0x7f28f410c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f8a2c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f4108740 0x7f28f410ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f8a2c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4113370 con 0x7f28f4107df0 2024-05-09T00:19:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f8a2c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41131f0 con 0x7f28f4108740 2024-05-09T00:19:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f8a2c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41134f0 con 0x7f28f41034c0 2024-05-09T00:19:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f4107df0 0x7f28f410c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f4107df0 0x7f28f410c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48656/0 (socket says 172.21.15.46:48656) 2024-05-09T00:19:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f1d9b700 1 -- 172.21.15.46:0/3198644768 learned_addr learned my addr 172.21.15.46:0/3198644768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f259c700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f41034c0 0x7f28f410c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f2d9d700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f4108740 0x7f28f410ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f1d9b700 1 -- 172.21.15.46:0/3198644768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f41034c0 msgr2=0x7f28f410c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f1d9b700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f41034c0 0x7f28f410c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f1d9b700 1 -- 172.21.15.46:0/3198644768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f4108740 msgr2=0x7f28f410ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f1d9b700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f4108740 0x7f28f410ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.757+0000 7f28f1d9b700 1 -- 172.21.15.46:0/3198644768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f41ad530 con 0x7f28f4107df0 2024-05-09T00:19:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.758+0000 7f28f1d9b700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f4107df0 0x7f28f410c950 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f28e800c9e0 tx=0x7f28e800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.758+0000 7f28e37fe700 1 -- 172.21.15.46:0/3198644768 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8007aa0 con 0x7f28f4107df0 2024-05-09T00:19:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.758+0000 7f28f8a2c700 1 -- 172.21.15.46:0/3198644768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f41ad820 con 0x7f28f4107df0 2024-05-09T00:19:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.758+0000 7f28e37fe700 1 -- 172.21.15.46:0/3198644768 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e8007c40 con 0x7f28f4107df0 2024-05-09T00:19:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.758+0000 7f28f8a2c700 1 -- 172.21.15.46:0/3198644768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f41addb0 con 0x7f28f4107df0 2024-05-09T00:19:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.758+0000 7f28e37fe700 1 -- 172.21.15.46:0/3198644768 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8018660 con 0x7f28f4107df0 2024-05-09T00:19:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.761+0000 7f28e37fe700 1 -- 172.21.15.46:0/3198644768 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f28e8004830 con 0x7f28f4107df0 2024-05-09T00:19:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.761+0000 7f28e37fe700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28dc0b2900 0x7f28dc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.762+0000 7f28f259c700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28dc0b2900 0x7f28dc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.762+0000 7f28e37fe700 1 -- 172.21.15.46:0/3198644768 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f28e80e20b0 con 0x7f28f4107df0 2024-05-09T00:19:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.762+0000 7f2907442b80 1 Processor -- start 2024-05-09T00:19:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.762+0000 7f2907442b80 1 -- start start 2024-05-09T00:19:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.762+0000 7f28f259c700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28dc0b2900 0x7f28dc0b4e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009550 tx=0x7f28e40091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.762+0000 7f2907442b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda530cd00 0x55eda5415240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.762+0000 7f2907442b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda52924a0 0x55eda5415780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.762+0000 7f2907442b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 0x55eda540f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.762+0000 7f2907442b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eda53320a0 con 0x55eda540f400 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.762+0000 7f2907442b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55eda52d79a0 con 0x55eda530cd00 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.762+0000 7f2907442b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eda532e060 con 0x55eda52924a0 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28f2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 0x55eda540f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28f2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 0x55eda540f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48662/0 (socket says 172.21.15.46:48662) 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28f2d9d700 1 -- 172.21.15.46:0/1923099704 learned_addr learned my addr 172.21.15.46:0/1923099704 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28f1d9b700 1 --2- 172.21.15.46:0/1923099704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda52924a0 0x55eda5415780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28f259c700 1 --2- 172.21.15.46:0/1923099704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda530cd00 0x55eda5415240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28f2d9d700 1 -- 172.21.15.46:0/1923099704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda52924a0 msgr2=0x55eda5415780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28f2d9d700 1 --2- 172.21.15.46:0/1923099704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda52924a0 0x55eda5415780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28f2d9d700 1 -- 172.21.15.46:0/1923099704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda530cd00 msgr2=0x55eda5415240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28f2d9d700 1 --2- 172.21.15.46:0/1923099704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda530cd00 0x55eda5415240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28f2d9d700 1 -- 172.21.15.46:0/1923099704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eda52c8500 con 0x55eda540f400 2024-05-09T00:19:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28f2d9d700 1 --2- 172.21.15.46:0/1923099704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 0x55eda540f810 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0099c0 tx=0x7f28ec009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bfeecaa8b05787 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28e0ff9700 1 -- 172.21.15.46:0/1923099704 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec026070 con 0x55eda540f400 2024-05-09T00:19:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28e0ff9700 1 -- 172.21.15.46:0/1923099704 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28ec005070 con 0x55eda540f400 2024-05-09T00:19:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.763+0000 7f28e0ff9700 1 -- 172.21.15.46:0/1923099704 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec00ba80 con 0x55eda540f400 2024-05-09T00:19:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.764+0000 7f2907442b80 1 -- 172.21.15.46:0/1923099704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 msgr2=0x55eda540f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.764+0000 7f2907442b80 1 --2- 172.21.15.46:0/1923099704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 0x55eda540f810 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0099c0 tx=0x7f28ec009510 comp rx=0 tx=0).stop 2024-05-09T00:19:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.764+0000 7f2907442b80 1 -- 172.21.15.46:0/1923099704 shutdown_connections 2024-05-09T00:19:10.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.764+0000 7f2907442b80 1 --2- 172.21.15.46:0/1923099704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 0x55eda540f810 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.764+0000 7f2907442b80 1 --2- 172.21.15.46:0/1923099704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda530cd00 0x55eda5415240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.764+0000 7f2907442b80 1 --2- 172.21.15.46:0/1923099704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda52924a0 0x55eda5415780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.764+0000 7f2907442b80 1 -- 172.21.15.46:0/1923099704 >> 172.21.15.46:0/1923099704 conn(0x55eda5295f70 msgr2=0x55eda540a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.764+0000 7f2907442b80 1 -- 172.21.15.46:0/1923099704 shutdown_connections 2024-05-09T00:19:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.764+0000 7f2907442b80 1 -- 172.21.15.46:0/1923099704 wait complete. 2024-05-09T00:19:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.765+0000 7f2907442b80 1 -- start start 2024-05-09T00:19:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.765+0000 7f2907442b80 1 Processor -- start 2024-05-09T00:19:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.765+0000 7f2907442b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda52924a0 0x55eda5419140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.765+0000 7f2907442b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 0x55eda5419680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.765+0000 7f2907442b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 0x55eda5419bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.765+0000 7f2907442b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eda5410100 con 0x55eda540f400 2024-05-09T00:19:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.765+0000 7f2907442b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55eda52ccb20 con 0x55eda52924a0 2024-05-09T00:19:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.765+0000 7f2907442b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eda5425280 con 0x55eda530cd00 2024-05-09T00:19:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.765+0000 7f2907442b80 10 client.0 _ll_register_callbacks cb 0x55eda51a36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.765+0000 7f28f2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 0x55eda5419bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.765+0000 7f28f2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 0x55eda5419bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48666/0 (socket says 172.21.15.46:48666) 2024-05-09T00:19:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28f2d9d700 1 -- 172.21.15.46:0/3551175890 learned_addr learned my addr 172.21.15.46:0/3551175890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28f259c700 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda52924a0 0x55eda5419140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28f1d9b700 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 0x55eda5419680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28f2d9d700 1 -- 172.21.15.46:0/3551175890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 msgr2=0x55eda5419680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28f2d9d700 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 0x55eda5419680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28f2d9d700 1 -- 172.21.15.46:0/3551175890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda52924a0 msgr2=0x55eda5419140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28f2d9d700 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda52924a0 0x55eda5419140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28f2d9d700 1 -- 172.21.15.46:0/3551175890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eda51ad490 con 0x55eda540f400 2024-05-09T00:19:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28f2d9d700 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 0x55eda5419bc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009990 tx=0x7f28ec005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb6d934bc014a81 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28d9ffb700 1 -- 172.21.15.46:0/3551175890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec026070 con 0x55eda540f400 2024-05-09T00:19:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f2907442b80 1 -- 172.21.15.46:0/3551175890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28ec005070 con 0x55eda540f400 2024-05-09T00:19:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28d9ffb700 1 -- 172.21.15.46:0/3551175890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f28ec00b790 con 0x55eda540f400 2024-05-09T00:19:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f28d9ffb700 1 -- 172.21.15.46:0/3551175890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28ec003e30 con 0x55eda540f400 2024-05-09T00:19:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f2907442b80 10 client.16566 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.766+0000 7f2907442b80 1 -- 172.21.15.46:0/3551175890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eda5426c20 con 0x55eda540f400 2024-05-09T00:19:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.767+0000 7f28d9ffb700 1 -- 172.21.15.46:0/3551175890 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28ec005070 con 0x55eda540f400 2024-05-09T00:19:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.767+0000 7f28d9ffb700 1 -- 172.21.15.46:0/3551175890 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f28ec021b70 con 0x55eda540f400 2024-05-09T00:19:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.767+0000 7f2907442b80 10 client.16566 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.767+0000 7f2907442b80 10 client.16566 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:19:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.767+0000 7f2907442b80 20 client.16566 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.767+0000 7f2907442b80 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55eda54290f0 0x55eda542b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.767+0000 7f2907442b80 4 client.16566 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.767+0000 7f2907442b80 1 -- 172.21.15.46:0/3551175890 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eda533fa80 con 0x55eda54290f0 2024-05-09T00:19:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.767+0000 7f28f259c700 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55eda54290f0 0x55eda542b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.768+0000 7f28f259c700 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55eda54290f0 0x55eda542b510 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=371dc02da7f1d322 server_cookie=3db7ecc1f55278af in_seq=0 out_seq=0 2024-05-09T00:19:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.768+0000 7f28d9ffb700 10 client.16566 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:19:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f28d9ffb700 1 -- 172.21.15.46:0/3551175890 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eda533fa80 con 0x55eda54290f0 2024-05-09T00:19:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f28d9ffb700 10 client.16566 handle_command_reply: tid=0 2024-05-09T00:19:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f2907442b80 1 client.16566 shutdown 2024-05-09T00:19:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f2907442b80 1 -- 172.21.15.46:0/3551175890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 msgr2=0x55eda5419bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f2907442b80 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 0x55eda5419bc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009990 tx=0x7f28ec005810 comp rx=0 tx=0).stop 2024-05-09T00:19:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f2907442b80 1 -- 172.21.15.46:0/3551175890 shutdown_connections 2024-05-09T00:19:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f2907442b80 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55eda54290f0 0x55eda542b510 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f2907442b80 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda540f400 0x55eda5419bc0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f2907442b80 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda52924a0 0x55eda5419140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f28d9ffb700 0 client.16566 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:19:10.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f2907442b80 1 --2- 172.21.15.46:0/3551175890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 0x55eda5419680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f2907442b80 1 -- 172.21.15.46:0/3551175890 >> 172.21.15.46:0/3551175890 conn(0x55eda5295f70 msgr2=0x55eda5403ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:10.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f2907442b80 1 -- 172.21.15.46:0/3551175890 shutdown_connections 2024-05-09T00:19:10.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f2907442b80 1 -- 172.21.15.46:0/3551175890 wait complete. 2024-05-09T00:19:10.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.769+0000 7f2907442b80 20 client.16566 trim_cache size 0 max 16384 2024-05-09T00:19:10.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.784+0000 7f2907442b80 1 Processor -- start 2024-05-09T00:19:10.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.784+0000 7f2907442b80 1 -- start start 2024-05-09T00:19:10.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.784+0000 7f2907442b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda5295f70 0x55eda542e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.784+0000 7f2907442b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda52924a0 0x55eda542ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.784+0000 7f2907442b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 0x55eda5403ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.784+0000 7f2907442b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41b79d0 con 0x55eda52924a0 2024-05-09T00:19:10.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.784+0000 7f2907442b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41b5260 con 0x55eda5295f70 2024-05-09T00:19:10.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.784+0000 7f2907442b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41b3090 con 0x55eda530cd00 2024-05-09T00:19:10.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28f1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda52924a0 0x55eda542ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28f1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda52924a0 0x55eda542ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48672/0 (socket says 172.21.15.46:48672) 2024-05-09T00:19:10.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28f1d9b700 1 -- 172.21.15.46:0/3103111121 learned_addr learned my addr 172.21.15.46:0/3103111121 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:10.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28f2d9d700 1 --2- 172.21.15.46:0/3103111121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 0x55eda5403ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28f259c700 1 --2- 172.21.15.46:0/3103111121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda5295f70 0x55eda542e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28f1d9b700 1 -- 172.21.15.46:0/3103111121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 msgr2=0x55eda5403ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:10.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28f1d9b700 1 --2- 172.21.15.46:0/3103111121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 0x55eda5403ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28f1d9b700 1 -- 172.21.15.46:0/3103111121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda5295f70 msgr2=0x55eda542e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:10.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28f1d9b700 1 --2- 172.21.15.46:0/3103111121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda5295f70 0x55eda542e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28f1d9b700 1 -- 172.21.15.46:0/3103111121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eda54069d0 con 0x55eda52924a0 2024-05-09T00:19:10.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28f1d9b700 1 --2- 172.21.15.46:0/3103111121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda52924a0 0x55eda542ea70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f28e8004800 tx=0x7f28e8091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca67414458424b68 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:10.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28dbfff700 1 -- 172.21.15.46:0/3103111121 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e80a9dd0 con 0x55eda52924a0 2024-05-09T00:19:10.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28dbfff700 1 -- 172.21.15.46:0/3103111121 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e80a96d0 con 0x55eda52924a0 2024-05-09T00:19:10.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.785+0000 7f28dbfff700 1 -- 172.21.15.46:0/3103111121 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e80a9930 con 0x55eda52924a0 2024-05-09T00:19:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.786+0000 7f2907442b80 1 -- 172.21.15.46:0/3103111121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda52924a0 msgr2=0x55eda542ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.786+0000 7f2907442b80 1 --2- 172.21.15.46:0/3103111121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda52924a0 0x55eda542ea70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f28e8004800 tx=0x7f28e8091040 comp rx=0 tx=0).stop 2024-05-09T00:19:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.786+0000 7f2907442b80 1 -- 172.21.15.46:0/3103111121 shutdown_connections 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.786+0000 7f2907442b80 1 --2- 172.21.15.46:0/3103111121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 0x55eda5403ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.786+0000 7f2907442b80 1 --2- 172.21.15.46:0/3103111121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda5295f70 0x55eda542e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.786+0000 7f2907442b80 1 --2- 172.21.15.46:0/3103111121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda52924a0 0x55eda542ea70 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.786+0000 7f2907442b80 1 -- 172.21.15.46:0/3103111121 >> 172.21.15.46:0/3103111121 conn(0x55eda5419d10 msgr2=0x55eda5414230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.786+0000 7f2907442b80 1 -- 172.21.15.46:0/3103111121 shutdown_connections 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.786+0000 7f2907442b80 1 -- 172.21.15.46:0/3103111121 wait complete. 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f2907442b80 1 -- start start 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f2907442b80 1 Processor -- start 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f2907442b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda52924a0 0x55eda548aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f2907442b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda5295f70 0x55eda548afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f2907442b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 0x55eda542db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f2907442b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41b3720 con 0x55eda5295f70 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f2907442b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41b4b90 con 0x55eda52924a0 2024-05-09T00:19:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f2907442b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41b3df0 con 0x55eda530cd00 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f2907442b80 10 client.0 _ll_register_callbacks cb 0x55eda51a36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f28f1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda5295f70 0x55eda548afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f28f1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda5295f70 0x55eda548afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48684/0 (socket says 172.21.15.46:48684) 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f28f1d9b700 1 -- 172.21.15.46:0/1887606121 learned_addr learned my addr 172.21.15.46:0/1887606121 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f28f259c700 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda52924a0 0x55eda548aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.787+0000 7f28f2d9d700 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 0x55eda542db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f28f1d9b700 1 -- 172.21.15.46:0/1887606121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 msgr2=0x55eda542db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f28f1d9b700 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 0x55eda542db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f28f1d9b700 1 -- 172.21.15.46:0/1887606121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda52924a0 msgr2=0x55eda548aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f28f1d9b700 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda52924a0 0x55eda548aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f28f1d9b700 1 -- 172.21.15.46:0/1887606121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eda5415c70 con 0x55eda5295f70 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f28f1d9b700 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda5295f70 0x55eda548afd0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f28e80a9380 tx=0x7f28e80ada80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d922e28ba0c7c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f28d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f28d9ffb700 1 -- 172.21.15.46:0/1887606121 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e80a9dd0 con 0x55eda5295f70 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f28d9ffb700 1 -- 172.21.15.46:0/1887606121 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f28e80b0e00 con 0x55eda5295f70 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f2907442b80 1 -- 172.21.15.46:0/1887606121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e80a96d0 con 0x55eda5295f70 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f28d9ffb700 1 -- 172.21.15.46:0/1887606121 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28e80a5cc0 con 0x55eda5295f70 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f2907442b80 10 client.16572 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f2907442b80 1 -- 172.21.15.46:0/1887606121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eda5493010 con 0x55eda5295f70 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f28d9ffb700 1 -- 172.21.15.46:0/1887606121 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e80a96d0 con 0x55eda5295f70 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f28d9ffb700 1 -- 172.21.15.46:0/1887606121 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f28e808d030 con 0x55eda5295f70 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f2907442b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.788+0000 7f2907442b80 10 client.16572 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.789+0000 7f2907442b80 20 client.16572 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.789+0000 7f2907442b80 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55eda540f400 0x55eda5495620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.789+0000 7f2907442b80 4 client.16572 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.789+0000 7f2907442b80 1 -- 172.21.15.46:0/1887606121 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eda5339b20 con 0x55eda540f400 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.789+0000 7f28f259c700 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55eda540f400 0x55eda5495620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:10.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.789+0000 7f28d9ffb700 10 client.16572 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.789+0000 7f28f259c700 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55eda540f400 0x55eda5495620 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70a81e72182edec1 server_cookie=f202372296de61d9 in_seq=0 out_seq=0 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.789+0000 7f28d9ffb700 1 -- 172.21.15.46:0/1887606121 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eda5339b20 con 0x55eda540f400 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.789+0000 7f28d9ffb700 10 client.16572 handle_command_reply: tid=0 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.789+0000 7f2907442b80 1 client.16572 shutdown 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f2907442b80 1 -- 172.21.15.46:0/1887606121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda5295f70 msgr2=0x55eda548afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f2907442b80 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda5295f70 0x55eda548afd0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f28e80a9380 tx=0x7f28e80ada80 comp rx=0 tx=0).stop 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f2907442b80 1 -- 172.21.15.46:0/1887606121 shutdown_connections 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f2907442b80 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55eda540f400 0x55eda5495620 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f2907442b80 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eda530cd00 0x55eda542db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f2907442b80 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eda52924a0 0x55eda548aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f28d9ffb700 0 client.16572 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f2907442b80 1 --2- 172.21.15.46:0/1887606121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eda5295f70 0x55eda548afd0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f2907442b80 1 -- 172.21.15.46:0/1887606121 >> 172.21.15.46:0/1887606121 conn(0x55eda5419d10 msgr2=0x55eda5403ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:10.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f2907442b80 1 -- 172.21.15.46:0/1887606121 shutdown_connections 2024-05-09T00:19:10.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f2907442b80 1 -- 172.21.15.46:0/1887606121 wait complete. 2024-05-09T00:19:10.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f2907442b80 20 client.16572 trim_cache size 0 max 16384 2024-05-09T00:19:10.795 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:19:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f28f8a2c700 1 -- 172.21.15.46:0/3198644768 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28dc0b2900 msgr2=0x7f28dc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f28f8a2c700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28dc0b2900 0x7f28dc0b4e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009550 tx=0x7f28e40091f0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f28f8a2c700 1 -- 172.21.15.46:0/3198644768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f4107df0 msgr2=0x7f28f410c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f28f8a2c700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f4107df0 0x7f28f410c950 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f28e800c9e0 tx=0x7f28e800ccf0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f28f8a2c700 1 -- 172.21.15.46:0/3198644768 shutdown_connections 2024-05-09T00:19:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f28f8a2c700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f28dc0b2900 0x7f28dc0b4e10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f28f8a2c700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f4108740 0x7f28f410ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f28f8a2c700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f41034c0 0x7f28f410c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f28f8a2c700 1 --2- 172.21.15.46:0/3198644768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f4107df0 0x7f28f410c950 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.790+0000 7f28f8a2c700 1 -- 172.21.15.46:0/3198644768 >> 172.21.15.46:0/3198644768 conn(0x7f28f40fedc0 msgr2=0x7f28f41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.791+0000 7f28f8a2c700 1 -- 172.21.15.46:0/3198644768 shutdown_connections 2024-05-09T00:19:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:10.791+0000 7f28f8a2c700 1 -- 172.21.15.46:0/3198644768 wait complete. 2024-05-09T00:19:11.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:19:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:12.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:12 smithi046 ceph-mon[30151]: pgmap v611: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:19:12.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:12 smithi169 ceph-mon[34777]: pgmap v611: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:19:12.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:12 smithi142 ceph-mon[34533]: pgmap v611: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:19:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.690+0000 7f6dd9d57700 1 -- 172.21.15.46:0/3274169839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd4108740 msgr2=0x7f6dd41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.690+0000 7f6dd9d57700 1 --2- 172.21.15.46:0/3274169839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd4108740 0x7f6dd41100a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0099c0 tx=0x7f6dcc009cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.690+0000 7f6dd9d57700 1 -- 172.21.15.46:0/3274169839 shutdown_connections 2024-05-09T00:19:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.690+0000 7f6dd9d57700 1 --2- 172.21.15.46:0/3274169839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd4108740 0x7f6dd41100a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.690+0000 7f6dd9d57700 1 --2- 172.21.15.46:0/3274169839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dd41034c0 0x7f6dd41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.690+0000 7f6dd9d57700 1 --2- 172.21.15.46:0/3274169839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd4107df0 0x7f6dd4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.690+0000 7f6dd9d57700 1 -- 172.21.15.46:0/3274169839 >> 172.21.15.46:0/3274169839 conn(0x7f6dd40fedc0 msgr2=0x7f6dd41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:13.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.690+0000 7f6dd9d57700 1 -- 172.21.15.46:0/3274169839 shutdown_connections 2024-05-09T00:19:13.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.690+0000 7f6dd9d57700 1 -- 172.21.15.46:0/3274169839 wait complete. 2024-05-09T00:19:13.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.690+0000 7f6dd9d57700 1 Processor -- start 2024-05-09T00:19:13.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.691+0000 7f6dd9d57700 1 -- start start 2024-05-09T00:19:13.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.691+0000 7f6dd9d57700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dd41034c0 0x7f6dd419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.691+0000 7f6dd9d57700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd4107df0 0x7f6dd419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.691+0000 7f6dd9d57700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd4108740 0x7f6dd41a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.691+0000 7f6dd9d57700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd4113370 con 0x7f6dd4108740 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.691+0000 7f6dd9d57700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41131f0 con 0x7f6dd41034c0 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.691+0000 7f6dd9d57700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41134f0 con 0x7f6dd4107df0 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.691+0000 7f6dd3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd4108740 0x7f6dd41a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.691+0000 7f6dd3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd4108740 0x7f6dd41a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48700/0 (socket says 172.21.15.46:48700) 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.691+0000 7f6dd3fff700 1 -- 172.21.15.46:0/4189243065 learned_addr learned my addr 172.21.15.46:0/4189243065 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.691+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dd41034c0 0x7f6dd419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.691+0000 7f6dd2ffd700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd4107df0 0x7f6dd419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.692+0000 7f6dd3fff700 1 -- 172.21.15.46:0/4189243065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd4107df0 msgr2=0x7f6dd419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.692+0000 7f6dd3fff700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd4107df0 0x7f6dd419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.692+0000 7f6dd3fff700 1 -- 172.21.15.46:0/4189243065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dd41034c0 msgr2=0x7f6dd419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.692+0000 7f6dd3fff700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dd41034c0 0x7f6dd419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.692+0000 7f6dd3fff700 1 -- 172.21.15.46:0/4189243065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd41a1e30 con 0x7f6dd4108740 2024-05-09T00:19:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.692+0000 7f6dd3fff700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd4108740 0x7f6dd41a1840 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc005910 tx=0x7f6dcc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.692+0000 7f6dd0ff9700 1 -- 172.21.15.46:0/4189243065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc027070 con 0x7f6dd4108740 2024-05-09T00:19:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.692+0000 7f6dd9d57700 1 -- 172.21.15.46:0/4189243065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd41a20c0 con 0x7f6dd4108740 2024-05-09T00:19:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.692+0000 7f6dd0ff9700 1 -- 172.21.15.46:0/4189243065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dcc003f20 con 0x7f6dd4108740 2024-05-09T00:19:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.692+0000 7f6dd0ff9700 1 -- 172.21.15.46:0/4189243065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc022ba0 con 0x7f6dd4108740 2024-05-09T00:19:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.692+0000 7f6dd9d57700 1 -- 172.21.15.46:0/4189243065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd41a25a0 con 0x7f6dd4108740 2024-05-09T00:19:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.695+0000 7f6dd0ff9700 1 -- 172.21.15.46:0/4189243065 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6dcc02a070 con 0x7f6dd4108740 2024-05-09T00:19:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.695+0000 7f6dd0ff9700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6dbc0b2880 0x7f6dbc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.695+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6dbc0b2880 0x7f6dbc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6dd0ff9700 1 -- 172.21.15.46:0/4189243065 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6dcc0e3170 con 0x7f6dd4108740 2024-05-09T00:19:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6de876db80 1 Processor -- start 2024-05-09T00:19:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6de876db80 1 -- start start 2024-05-09T00:19:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6dbc0b2880 0x7f6dbc0b4d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400f440 tx=0x7f6dc40079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6de876db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1be9d00 0x55b3f1cf2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6de876db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6de876db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1cec400 0x55b3f1cec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6de876db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b3f1c0f0a0 con 0x55b3f1b6f4a0 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6de876db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b3f1bb49a0 con 0x55b3f1be9d00 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6de876db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b3f1c0b060 con 0x55b3f1cec400 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6dd2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6dd2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48716/0 (socket says 172.21.15.46:48716) 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6dd2ffd700 1 -- 172.21.15.46:0/724193571 learned_addr learned my addr 172.21.15.46:0/724193571 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.696+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/724193571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1be9d00 0x55b3f1cf2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.697+0000 7f6dd3fff700 1 --2- 172.21.15.46:0/724193571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1cec400 0x55b3f1cec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.697+0000 7f6dd2ffd700 1 -- 172.21.15.46:0/724193571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1cec400 msgr2=0x55b3f1cec810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.697+0000 7f6dd2ffd700 1 --2- 172.21.15.46:0/724193571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1cec400 0x55b3f1cec810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.697+0000 7f6dd2ffd700 1 -- 172.21.15.46:0/724193571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1be9d00 msgr2=0x55b3f1cf2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.697+0000 7f6dd2ffd700 1 --2- 172.21.15.46:0/724193571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1be9d00 0x55b3f1cf2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.697+0000 7f6dd2ffd700 1 -- 172.21.15.46:0/724193571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f1ba5500 con 0x55b3f1b6f4a0 2024-05-09T00:19:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.697+0000 7f6dd3fff700 1 --2- 172.21.15.46:0/724193571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1cec400 0x55b3f1cec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:19:13.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.697+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/724193571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1be9d00 0x55b3f1cf2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:13.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.697+0000 7f6dd2ffd700 1 --2- 172.21.15.46:0/724193571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf2780 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800bc70 tx=0x7f6dc800bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97242c9c334b6151 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:13.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.697+0000 7f6db9ffb700 1 -- 172.21.15.46:0/724193571 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8010c90 con 0x55b3f1b6f4a0 2024-05-09T00:19:13.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.697+0000 7f6db9ffb700 1 -- 172.21.15.46:0/724193571 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc8010e30 con 0x55b3f1b6f4a0 2024-05-09T00:19:13.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.697+0000 7f6db9ffb700 1 -- 172.21.15.46:0/724193571 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8005220 con 0x55b3f1b6f4a0 2024-05-09T00:19:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.698+0000 7f6de876db80 1 -- 172.21.15.46:0/724193571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1b6f4a0 msgr2=0x55b3f1cf2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.698+0000 7f6de876db80 1 --2- 172.21.15.46:0/724193571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf2780 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800bc70 tx=0x7f6dc800bf80 comp rx=0 tx=0).stop 2024-05-09T00:19:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.698+0000 7f6de876db80 1 -- 172.21.15.46:0/724193571 shutdown_connections 2024-05-09T00:19:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.698+0000 7f6de876db80 1 --2- 172.21.15.46:0/724193571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1cec400 0x55b3f1cec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.698+0000 7f6de876db80 1 --2- 172.21.15.46:0/724193571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1be9d00 0x55b3f1cf2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.698+0000 7f6de876db80 1 --2- 172.21.15.46:0/724193571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf2780 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.698+0000 7f6de876db80 1 -- 172.21.15.46:0/724193571 >> 172.21.15.46:0/724193571 conn(0x55b3f1b72f70 msgr2=0x55b3f1ce7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.698+0000 7f6de876db80 1 -- 172.21.15.46:0/724193571 shutdown_connections 2024-05-09T00:19:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.698+0000 7f6de876db80 1 -- 172.21.15.46:0/724193571 wait complete. 2024-05-09T00:19:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6de876db80 1 -- start start 2024-05-09T00:19:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6de876db80 1 Processor -- start 2024-05-09T00:19:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6de876db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6de876db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1be9d00 0x55b3f1cf64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6de876db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 0x55b3f1cf6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6de876db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b3f1ced160 con 0x55b3f1cec400 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6de876db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b3f1ba9b20 con 0x55b3f1be9d00 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6de876db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b3f1cf9020 con 0x55b3f1b6f4a0 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6de876db80 10 client.0 _ll_register_callbacks cb 0x55b3f1a806c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6dd3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 0x55b3f1cf6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6dd3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 0x55b3f1cf6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48718/0 (socket says 172.21.15.46:48718) 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6dd3fff700 1 -- 172.21.15.46:0/231475450 learned_addr learned my addr 172.21.15.46:0/231475450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.699+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6dd2ffd700 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1be9d00 0x55b3f1cf64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6dd3fff700 1 -- 172.21.15.46:0/231475450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1b6f4a0 msgr2=0x55b3f1cf5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6dd3fff700 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6dd3fff700 1 -- 172.21.15.46:0/231475450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1be9d00 msgr2=0x55b3f1cf64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6dd3fff700 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1be9d00 0x55b3f1cf64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6dd3fff700 1 -- 172.21.15.46:0/231475450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f1a8a490 con 0x55b3f1cec400 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6dd3fff700 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 0x55b3f1cf6a10 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0a7a90 tx=0x7f6dcc0aa010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5aa59bf2f41b80 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6db2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6db2ffd700 1 -- 172.21.15.46:0/231475450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0cd0f0 con 0x55b3f1cec400 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6db2ffd700 1 -- 172.21.15.46:0/231475450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6dcc0b3370 con 0x55b3f1cec400 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6de876db80 1 -- 172.21.15.46:0/231475450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc8010e30 con 0x55b3f1cec400 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6db2ffd700 1 -- 172.21.15.46:0/231475450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dcc0aaf20 con 0x55b3f1cec400 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6de876db80 10 client.16584 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6de876db80 1 -- 172.21.15.46:0/231475450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b3f1d038d0 con 0x55b3f1cec400 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.700+0000 7f6db2ffd700 1 -- 172.21.15.46:0/231475450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dcc0aa820 con 0x55b3f1cec400 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.701+0000 7f6db2ffd700 1 -- 172.21.15.46:0/231475450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6dcc0a9a80 con 0x55b3f1cec400 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.701+0000 7f6de876db80 10 client.16584 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.701+0000 7f6de876db80 10 client.16584 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.701+0000 7f6de876db80 20 client.16584 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.701+0000 7f6de876db80 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b3f1d05e90 0x55b3f1d082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.701+0000 7f6de876db80 4 client.16584 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.701+0000 7f6de876db80 1 -- 172.21.15.46:0/231475450 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6dc8010e30 con 0x55b3f1d05e90 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.701+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b3f1d05e90 0x55b3f1d082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.702+0000 7f6db2ffd700 10 client.16584 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.702+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b3f1d05e90 0x55b3f1d082e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8acaaeafd6317ac server_cookie=35a7d106627979b7 in_seq=0 out_seq=0 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.702+0000 7f6db2ffd700 1 -- 172.21.15.46:0/231475450 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f6dc8010e30 con 0x55b3f1d05e90 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.702+0000 7f6db2ffd700 10 client.16584 handle_command_reply: tid=0 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.702+0000 7f6de876db80 1 client.16584 shutdown 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.702+0000 7f6de876db80 1 -- 172.21.15.46:0/231475450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 msgr2=0x55b3f1cf6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.702+0000 7f6de876db80 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 0x55b3f1cf6a10 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0a7a90 tx=0x7f6dcc0aa010 comp rx=0 tx=0).stop 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.703+0000 7f6de876db80 1 -- 172.21.15.46:0/231475450 shutdown_connections 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.703+0000 7f6de876db80 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b3f1d05e90 0x55b3f1d082e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.703+0000 7f6de876db80 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 0x55b3f1cf6a10 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.703+0000 7f6db2ffd700 0 client.16584 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:19:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.703+0000 7f6de876db80 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.703+0000 7f6de876db80 1 --2- 172.21.15.46:0/231475450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1be9d00 0x55b3f1cf64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.703+0000 7f6de876db80 1 -- 172.21.15.46:0/231475450 >> 172.21.15.46:0/231475450 conn(0x55b3f1b72f70 msgr2=0x55b3f1ce0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.703+0000 7f6de876db80 1 -- 172.21.15.46:0/231475450 shutdown_connections 2024-05-09T00:19:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.703+0000 7f6de876db80 1 -- 172.21.15.46:0/231475450 wait complete. 2024-05-09T00:19:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.703+0000 7f6de876db80 20 client.16584 trim_cache size 0 max 16384 2024-05-09T00:19:13.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6de876db80 1 Processor -- start 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6de876db80 1 -- start start 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6de876db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6de876db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1be9d00 0x55b3f1ce1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6de876db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1cec400 0x55b3f1cf4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6de876db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41b34b0 con 0x55b3f1be9d00 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6de876db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41b0d40 con 0x55b3f1cec400 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6de876db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41aeb90 con 0x55b3f1b6f4a0 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6dd2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1be9d00 0x55b3f1ce1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6dd2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1be9d00 0x55b3f1ce1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48720/0 (socket says 172.21.15.46:48720) 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6dd2ffd700 1 -- 172.21.15.46:0/2642121236 learned_addr learned my addr 172.21.15.46:0/2642121236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/2642121236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6dd2ffd700 1 -- 172.21.15.46:0/2642121236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1b6f4a0 msgr2=0x55b3f1cf1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6dd2ffd700 1 --2- 172.21.15.46:0/2642121236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6dd2ffd700 1 -- 172.21.15.46:0/2642121236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1cec400 msgr2=0x55b3f1cf4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6dd2ffd700 1 --2- 172.21.15.46:0/2642121236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1cec400 0x55b3f1cf4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6dd2ffd700 1 -- 172.21.15.46:0/2642121236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f1cf4500 con 0x55b3f1be9d00 2024-05-09T00:19:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.718+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/2642121236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:19:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.719+0000 7f6dd2ffd700 1 --2- 172.21.15.46:0/2642121236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1be9d00 0x55b3f1ce1a20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x55b3f1ce8c90 tx=0x7f6dc8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d579aa64f7605ba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.719+0000 7f6db97fa700 1 -- 172.21.15.46:0/2642121236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc802a070 con 0x55b3f1be9d00 2024-05-09T00:19:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.719+0000 7f6db97fa700 1 -- 172.21.15.46:0/2642121236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc8003d30 con 0x55b3f1be9d00 2024-05-09T00:19:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.719+0000 7f6db97fa700 1 -- 172.21.15.46:0/2642121236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8005530 con 0x55b3f1be9d00 2024-05-09T00:19:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.720+0000 7f6de876db80 1 -- 172.21.15.46:0/2642121236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1be9d00 msgr2=0x55b3f1ce1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.720+0000 7f6de876db80 1 --2- 172.21.15.46:0/2642121236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1be9d00 0x55b3f1ce1a20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x55b3f1ce8c90 tx=0x7f6dc8004d10 comp rx=0 tx=0).stop 2024-05-09T00:19:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.720+0000 7f6de876db80 1 -- 172.21.15.46:0/2642121236 shutdown_connections 2024-05-09T00:19:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.720+0000 7f6de876db80 1 --2- 172.21.15.46:0/2642121236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1cec400 0x55b3f1cf4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.720+0000 7f6de876db80 1 --2- 172.21.15.46:0/2642121236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1cf1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.720+0000 7f6de876db80 1 --2- 172.21.15.46:0/2642121236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1be9d00 0x55b3f1ce1a20 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.720+0000 7f6de876db80 1 -- 172.21.15.46:0/2642121236 >> 172.21.15.46:0/2642121236 conn(0x55b3f1b72f70 msgr2=0x55b3f1ceb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:13.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.720+0000 7f6de876db80 1 -- 172.21.15.46:0/2642121236 shutdown_connections 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.720+0000 7f6de876db80 1 -- 172.21.15.46:0/2642121236 wait complete. 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.720+0000 7f6de876db80 1 -- start start 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.720+0000 7f6de876db80 1 Processor -- start 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6de876db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1d0d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6de876db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1be9d00 0x55b3f1d0da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6de876db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 0x55b3f1d0dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6de876db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41af200 con 0x55b3f1cec400 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6de876db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41b0670 con 0x55b3f1b6f4a0 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6de876db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41af8d0 con 0x55b3f1be9d00 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6de876db80 10 client.0 _ll_register_callbacks cb 0x55b3f1a806c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6dd3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 0x55b3f1d0dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6dd3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 0x55b3f1d0dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48732/0 (socket says 172.21.15.46:48732) 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6dd3fff700 1 -- 172.21.15.46:0/1538530882 learned_addr learned my addr 172.21.15.46:0/1538530882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1d0d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6dd2ffd700 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1be9d00 0x55b3f1d0da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6dd3fff700 1 -- 172.21.15.46:0/1538530882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1be9d00 msgr2=0x55b3f1d0da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6dd3fff700 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1be9d00 0x55b3f1d0da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6dd3fff700 1 -- 172.21.15.46:0/1538530882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1b6f4a0 msgr2=0x55b3f1d0d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6dd3fff700 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1d0d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6dd3fff700 1 -- 172.21.15.46:0/1538530882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f1d6f7f0 con 0x55b3f1cec400 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6dd2ffd700 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1be9d00 0x55b3f1d0da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6dd3fff700 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 0x55b3f1d0dfd0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0aa7b0 tx=0x7f6dcc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6edee2a81c0e7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.721+0000 7f6db2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6db2ffd700 1 -- 172.21.15.46:0/1538530882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0cd0f0 con 0x55b3f1cec400 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6db2ffd700 1 -- 172.21.15.46:0/1538530882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6dcc003a70 con 0x55b3f1cec400 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6de876db80 1 -- 172.21.15.46:0/1538530882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc8003d30 con 0x55b3f1cec400 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6db2ffd700 1 -- 172.21.15.46:0/1538530882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dcc0c8050 con 0x55b3f1cec400 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6de876db80 10 client.16590 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:13.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6de876db80 1 -- 172.21.15.46:0/1538530882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b3f1d709a0 con 0x55b3f1cec400 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6db2ffd700 1 -- 172.21.15.46:0/1538530882 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc8003d30 con 0x55b3f1cec400 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6db2ffd700 1 -- 172.21.15.46:0/1538530882 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6dcc0a6190 con 0x55b3f1cec400 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6de876db80 10 client.16590 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6de876db80 10 client.16590 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6de876db80 20 client.16590 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6de876db80 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b3f1d70f10 0x55b3f1d73330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6de876db80 4 client.16590 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.722+0000 7f6de876db80 1 -- 172.21.15.46:0/1538530882 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3f1c1ca80 con 0x55b3f1d70f10 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b3f1d70f10 0x55b3f1d73330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6db2ffd700 10 client.16590 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6dd37fe700 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b3f1d70f10 0x55b3f1d73330 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81874b512ca888d2 server_cookie=209894818019d6ab in_seq=0 out_seq=0 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6db2ffd700 1 -- 172.21.15.46:0/1538530882 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3f1c1ca80 con 0x55b3f1d70f10 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6db2ffd700 10 client.16590 handle_command_reply: tid=0 2024-05-09T00:19:13.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6de876db80 1 client.16590 shutdown 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6de876db80 1 -- 172.21.15.46:0/1538530882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 msgr2=0x55b3f1d0dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6de876db80 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 0x55b3f1d0dfd0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0aa7b0 tx=0x7f6dcc009310 comp rx=0 tx=0).stop 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6de876db80 1 -- 172.21.15.46:0/1538530882 shutdown_connections 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6de876db80 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b3f1d70f10 0x55b3f1d73330 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6de876db80 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3f1cec400 0x55b3f1d0dfd0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6de876db80 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3f1b6f4a0 0x55b3f1d0d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6de876db80 1 --2- 172.21.15.46:0/1538530882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b3f1be9d00 0x55b3f1d0da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6de876db80 1 -- 172.21.15.46:0/1538530882 >> 172.21.15.46:0/1538530882 conn(0x55b3f1b72f70 msgr2=0x55b3f1ce1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.723+0000 7f6db2ffd700 0 client.16590 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6de876db80 1 -- 172.21.15.46:0/1538530882 shutdown_connections 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6de876db80 1 -- 172.21.15.46:0/1538530882 wait complete. 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6de876db80 20 client.16590 trim_cache size 0 max 16384 2024-05-09T00:19:13.727 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:19:13.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6dd9d57700 1 -- 172.21.15.46:0/4189243065 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6dbc0b2880 msgr2=0x7f6dbc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:13.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6dd9d57700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6dbc0b2880 0x7f6dbc0b4d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400f440 tx=0x7f6dc40079e0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6dd9d57700 1 -- 172.21.15.46:0/4189243065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd4108740 msgr2=0x7f6dd41a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:13.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6dd9d57700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd4108740 0x7f6dd41a1840 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc005910 tx=0x7f6dcc009580 comp rx=0 tx=0).stop 2024-05-09T00:19:13.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6dd9d57700 1 -- 172.21.15.46:0/4189243065 shutdown_connections 2024-05-09T00:19:13.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6dd9d57700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6dbc0b2880 0x7f6dbc0b4d90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6dd9d57700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd4108740 0x7f6dd41a1840 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6dd9d57700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dd41034c0 0x7f6dd419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6dd9d57700 1 --2- 172.21.15.46:0/4189243065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd4107df0 0x7f6dd419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:13.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.724+0000 7f6dd9d57700 1 -- 172.21.15.46:0/4189243065 >> 172.21.15.46:0/4189243065 conn(0x7f6dd40fedc0 msgr2=0x7f6dd41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:13.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.725+0000 7f6dd9d57700 1 -- 172.21.15.46:0/4189243065 shutdown_connections 2024-05-09T00:19:13.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:13.725+0000 7f6dd9d57700 1 -- 172.21.15.46:0/4189243065 wait complete. 2024-05-09T00:19:14.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:19:14.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:14 smithi046 ceph-mon[30151]: pgmap v612: 97 pgs: 97 active+clean; 4.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:14.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:14 smithi169 ceph-mon[34777]: pgmap v612: 97 pgs: 97 active+clean; 4.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:14.897 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:14.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:14 smithi142 ceph-mon[34533]: pgmap v612: 97 pgs: 97 active+clean; 4.4 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:16.656 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:16 smithi046 ceph-mon[30151]: pgmap v613: 97 pgs: 97 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:19:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.777+0000 7f04ca20d700 1 -- 172.21.15.46:0/1958049272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04c4107e10 msgr2=0x7f04c4108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.777+0000 7f04ca20d700 1 --2- 172.21.15.46:0/1958049272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04c4107e10 0x7f04c4108220 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0099c0 tx=0x7f04ac009cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.777+0000 7f04ca20d700 1 -- 172.21.15.46:0/1958049272 shutdown_connections 2024-05-09T00:19:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.777+0000 7f04ca20d700 1 --2- 172.21.15.46:0/1958049272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c4108760 0x7f04c41100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.777+0000 7f04ca20d700 1 --2- 172.21.15.46:0/1958049272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04c41034e0 0x7f04c41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.777+0000 7f04ca20d700 1 --2- 172.21.15.46:0/1958049272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04c4107e10 0x7f04c4108220 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.777+0000 7f04ca20d700 1 -- 172.21.15.46:0/1958049272 >> 172.21.15.46:0/1958049272 conn(0x7f04c40fef30 msgr2=0x7f04c4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.777+0000 7f04ca20d700 1 -- 172.21.15.46:0/1958049272 shutdown_connections 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.777+0000 7f04ca20d700 1 -- 172.21.15.46:0/1958049272 wait complete. 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04ca20d700 1 Processor -- start 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04ca20d700 1 -- start start 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04ca20d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c41034e0 0x7f04c40ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04ca20d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04c4107e10 0x7f04c40ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04ca20d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04c4108760 0x7f04c40ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04ca20d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04c4113370 con 0x7f04c4108760 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04ca20d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41131f0 con 0x7f04c41034e0 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04ca20d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41134f0 con 0x7f04c4107e10 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04c3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04c4108760 0x7f04c40ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04c3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04c4108760 0x7f04c40ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52970/0 (socket says 172.21.15.46:52970) 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04c3fff700 1 -- 172.21.15.46:0/3290158869 learned_addr learned my addr 172.21.15.46:0/3290158869 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04c37fe700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c41034e0 0x7f04c40ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.778+0000 7f04bbfff700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04c4107e10 0x7f04c40ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.779+0000 7f04c3fff700 1 -- 172.21.15.46:0/3290158869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04c4107e10 msgr2=0x7f04c40ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.779+0000 7f04c3fff700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04c4107e10 0x7f04c40ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.779+0000 7f04c3fff700 1 -- 172.21.15.46:0/3290158869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c41034e0 msgr2=0x7f04c40ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.779+0000 7f04c3fff700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c41034e0 0x7f04c40ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.779+0000 7f04c3fff700 1 -- 172.21.15.46:0/3290158869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c41a9250 con 0x7f04c4108760 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.779+0000 7f04c3fff700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04c4108760 0x7f04c40ffe90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f04bc007ee0 tx=0x7f04bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.779+0000 7f04c17fa700 1 -- 172.21.15.46:0/3290158869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc004bb0 con 0x7f04c4108760 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.779+0000 7f04ca20d700 1 -- 172.21.15.46:0/3290158869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c41a9540 con 0x7f04c4108760 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.779+0000 7f04c17fa700 1 -- 172.21.15.46:0/3290158869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04bc004d50 con 0x7f04c4108760 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.779+0000 7f04c17fa700 1 -- 172.21.15.46:0/3290158869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc005920 con 0x7f04c4108760 2024-05-09T00:19:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.779+0000 7f04ca20d700 1 -- 172.21.15.46:0/3290158869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c41a9a50 con 0x7f04c4108760 2024-05-09T00:19:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.782+0000 7f04c17fa700 1 -- 172.21.15.46:0/3290158869 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f04bc01a020 con 0x7f04c4108760 2024-05-09T00:19:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.782+0000 7f04c17fa700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f04b00b2880 0x7f04b00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04c37fe700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f04b00b2880 0x7f04b00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04c17fa700 1 -- 172.21.15.46:0/3290158869 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f04bc0e1830 con 0x7f04c4108760 2024-05-09T00:19:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04d8c23b80 1 Processor -- start 2024-05-09T00:19:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04d8c23b80 1 -- start start 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04c37fe700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f04b00b2880 0x7f04b00b4d90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f04b4005950 tx=0x7f04b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04d8c23b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b22cd00 0x55cb4b335240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04d8c23b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b1b24a0 0x55cb4b335780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04d8c23b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b32f400 0x55cb4b32f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04d8c23b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cb4b2520a0 con 0x55cb4b22cd00 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04d8c23b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cb4b1f79a0 con 0x55cb4b1b24a0 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04d8c23b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cb4b24e060 con 0x55cb4b32f400 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04c37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b22cd00 0x55cb4b335240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04c37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b22cd00 0x55cb4b335240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52978/0 (socket says 172.21.15.46:52978) 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04c37fe700 1 -- 172.21.15.46:0/684868503 learned_addr learned my addr 172.21.15.46:0/684868503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.783+0000 7f04bbfff700 1 --2- 172.21.15.46:0/684868503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b1b24a0 0x55cb4b335780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.784+0000 7f04c3fff700 1 --2- 172.21.15.46:0/684868503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b32f400 0x55cb4b32f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.784+0000 7f04c37fe700 1 -- 172.21.15.46:0/684868503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b32f400 msgr2=0x55cb4b32f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.784+0000 7f04c37fe700 1 --2- 172.21.15.46:0/684868503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b32f400 0x55cb4b32f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.784+0000 7f04c37fe700 1 -- 172.21.15.46:0/684868503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b1b24a0 msgr2=0x55cb4b335780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.784+0000 7f04c37fe700 1 --2- 172.21.15.46:0/684868503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b1b24a0 0x55cb4b335780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.784+0000 7f04c37fe700 1 -- 172.21.15.46:0/684868503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb4b1e8500 con 0x55cb4b22cd00 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.784+0000 7f04c37fe700 1 --2- 172.21.15.46:0/684868503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b22cd00 0x55cb4b335240 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f04b4010fa0 tx=0x7f04b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8acc50f8acb53b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.784+0000 7f04b9ffb700 1 -- 172.21.15.46:0/684868503 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b400be60 con 0x55cb4b22cd00 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.784+0000 7f04b9ffb700 1 -- 172.21.15.46:0/684868503 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b4011db0 con 0x55cb4b22cd00 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.784+0000 7f04b9ffb700 1 -- 172.21.15.46:0/684868503 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b401c700 con 0x55cb4b22cd00 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.785+0000 7f04d8c23b80 1 -- 172.21.15.46:0/684868503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b22cd00 msgr2=0x55cb4b335240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.785+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/684868503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b22cd00 0x55cb4b335240 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f04b4010fa0 tx=0x7f04b4013000 comp rx=0 tx=0).stop 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.785+0000 7f04d8c23b80 1 -- 172.21.15.46:0/684868503 shutdown_connections 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.785+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/684868503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b32f400 0x55cb4b32f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.785+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/684868503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b22cd00 0x55cb4b335240 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.785+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/684868503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b1b24a0 0x55cb4b335780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.785+0000 7f04d8c23b80 1 -- 172.21.15.46:0/684868503 >> 172.21.15.46:0/684868503 conn(0x55cb4b1b5f70 msgr2=0x55cb4b32a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.785+0000 7f04d8c23b80 1 -- 172.21.15.46:0/684868503 shutdown_connections 2024-05-09T00:19:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.785+0000 7f04d8c23b80 1 -- 172.21.15.46:0/684868503 wait complete. 2024-05-09T00:19:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.786+0000 7f04d8c23b80 1 -- start start 2024-05-09T00:19:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.786+0000 7f04d8c23b80 1 Processor -- start 2024-05-09T00:19:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.786+0000 7f04d8c23b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b339040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.786+0000 7f04d8c23b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b339580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.786+0000 7f04d8c23b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b32f400 0x55cb4b339ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.786+0000 7f04d8c23b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cb4b330160 con 0x55cb4b32f400 2024-05-09T00:19:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.786+0000 7f04d8c23b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cb4b1ecb20 con 0x55cb4b22cd00 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.786+0000 7f04d8c23b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cb4b33c020 con 0x55cb4b1b24a0 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.786+0000 7f04d8c23b80 10 client.0 _ll_register_callbacks cb 0x55cb4b0c36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.786+0000 7f04c3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b32f400 0x55cb4b339ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.786+0000 7f04c37fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b339040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.786+0000 7f04c3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b32f400 0x55cb4b339ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52992/0 (socket says 172.21.15.46:52992) 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04c3fff700 1 -- 172.21.15.46:0/3347683351 learned_addr learned my addr 172.21.15.46:0/3347683351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04bbfff700 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b339580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04c3fff700 1 -- 172.21.15.46:0/3347683351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 msgr2=0x55cb4b339040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04c3fff700 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b339040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04c3fff700 1 -- 172.21.15.46:0/3347683351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 msgr2=0x55cb4b339580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04c3fff700 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b339580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04c3fff700 1 -- 172.21.15.46:0/3347683351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4b0cd490 con 0x55cb4b32f400 2024-05-09T00:19:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04bbfff700 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b339580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04c3fff700 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b32f400 0x55cb4b339ac0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0df330 tx=0x7f04bc0a9330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86183d2caa51c974 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04a2ffd700 1 -- 172.21.15.46:0/3347683351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0a8960 con 0x55cb4b32f400 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04a2ffd700 1 -- 172.21.15.46:0/3347683351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f04bc005110 con 0x55cb4b32f400 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04d8c23b80 1 -- 172.21.15.46:0/3347683351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04b4011db0 con 0x55cb4b32f400 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04a2ffd700 1 -- 172.21.15.46:0/3347683351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04bc0a9550 con 0x55cb4b32f400 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04d8c23b80 10 client.16602 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.787+0000 7f04d8c23b80 1 -- 172.21.15.46:0/3347683351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cb4b346ac0 con 0x55cb4b32f400 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.788+0000 7f04a2ffd700 1 -- 172.21.15.46:0/3347683351 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b4011db0 con 0x55cb4b32f400 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.788+0000 7f04a2ffd700 1 -- 172.21.15.46:0/3347683351 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f04bc0a8240 con 0x55cb4b32f400 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.788+0000 7f04d8c23b80 10 client.16602 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.788+0000 7f04d8c23b80 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.788+0000 7f04d8c23b80 20 client.16602 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.788+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55cb4b349050 0x55cb4b34b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.788+0000 7f04d8c23b80 4 client.16602 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.788+0000 7f04d8c23b80 1 -- 172.21.15.46:0/3347683351 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb4b25fa80 con 0x55cb4b349050 2024-05-09T00:19:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.788+0000 7f04c37fe700 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55cb4b349050 0x55cb4b34b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.789+0000 7f04c37fe700 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55cb4b349050 0x55cb4b34b470 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b9d28b407809f0 server_cookie=dd26ce94c8f16968 in_seq=0 out_seq=0 2024-05-09T00:19:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.789+0000 7f04a2ffd700 10 client.16602 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04a2ffd700 1 -- 172.21.15.46:0/3347683351 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cb4b25fa80 con 0x55cb4b349050 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04a2ffd700 10 client.16602 handle_command_reply: tid=0 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04d8c23b80 1 client.16602 shutdown 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04d8c23b80 1 -- 172.21.15.46:0/3347683351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b32f400 msgr2=0x55cb4b339ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b32f400 0x55cb4b339ac0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0df330 tx=0x7f04bc0a9330 comp rx=0 tx=0).stop 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04d8c23b80 1 -- 172.21.15.46:0/3347683351 shutdown_connections 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55cb4b349050 0x55cb4b34b470 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b32f400 0x55cb4b339ac0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b339040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/3347683351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b339580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04d8c23b80 1 -- 172.21.15.46:0/3347683351 >> 172.21.15.46:0/3347683351 conn(0x55cb4b1b5f70 msgr2=0x55cb4b323ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04a2ffd700 0 client.16602 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04d8c23b80 1 -- 172.21.15.46:0/3347683351 shutdown_connections 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04d8c23b80 1 -- 172.21.15.46:0/3347683351 wait complete. 2024-05-09T00:19:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.790+0000 7f04d8c23b80 20 client.16602 trim_cache size 0 max 16384 2024-05-09T00:19:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.805+0000 7f04d8c23b80 1 Processor -- start 2024-05-09T00:19:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.805+0000 7f04d8c23b80 1 -- start start 2024-05-09T00:19:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04d8c23b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 0x55cb4b338360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04d8c23b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b339900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04d8c23b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b344e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04d8c23b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41b3740 con 0x55cb4b1b5f70 2024-05-09T00:19:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04d8c23b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41b0fd0 con 0x55cb4b22cd00 2024-05-09T00:19:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04d8c23b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41aee20 con 0x55cb4b1b24a0 2024-05-09T00:19:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04c37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 0x55cb4b338360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04c37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 0x55cb4b338360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52998/0 (socket says 172.21.15.46:52998) 2024-05-09T00:19:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04c37fe700 1 -- 172.21.15.46:0/1961968044 learned_addr learned my addr 172.21.15.46:0/1961968044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:16.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04c37fe700 1 -- 172.21.15.46:0/1961968044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 msgr2=0x55cb4b339900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:16.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04bbfff700 1 --2- 172.21.15.46:0/1961968044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b339900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04c37fe700 1 --2- 172.21.15.46:0/1961968044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b339900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04c37fe700 1 -- 172.21.15.46:0/1961968044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 msgr2=0x55cb4b344e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:16.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04c37fe700 1 --2- 172.21.15.46:0/1961968044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b344e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04c37fe700 1 -- 172.21.15.46:0/1961968044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb4b34b9d0 con 0x55cb4b1b5f70 2024-05-09T00:19:16.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04bbfff700 1 --2- 172.21.15.46:0/1961968044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b339900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:19:16.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.806+0000 7f04c37fe700 1 --2- 172.21.15.46:0/1961968044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 0x55cb4b338360 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f04b4010f70 tx=0x7f04b400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df07a0e6df8e2d09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:16.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.807+0000 7f04b97fa700 1 -- 172.21.15.46:0/1961968044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4005040 con 0x55cb4b1b5f70 2024-05-09T00:19:16.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.807+0000 7f04b97fa700 1 -- 172.21.15.46:0/1961968044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b403a030 con 0x55cb4b1b5f70 2024-05-09T00:19:16.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.807+0000 7f04b97fa700 1 -- 172.21.15.46:0/1961968044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b401e420 con 0x55cb4b1b5f70 2024-05-09T00:19:16.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.807+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1961968044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 msgr2=0x55cb4b338360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:16.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.807+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/1961968044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 0x55cb4b338360 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f04b4010f70 tx=0x7f04b400bf90 comp rx=0 tx=0).stop 2024-05-09T00:19:16.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.807+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1961968044 shutdown_connections 2024-05-09T00:19:16.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.807+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/1961968044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b344e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.807+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/1961968044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 0x55cb4b338360 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.807+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/1961968044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b339900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.807+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1961968044 >> 172.21.15.46:0/1961968044 conn(0x55cb4b32a520 msgr2=0x55cb4b334130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.807+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1961968044 shutdown_connections 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.807+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1961968044 wait complete. 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.808+0000 7f04d8c23b80 1 -- start start 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.808+0000 7f04d8c23b80 1 Processor -- start 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.808+0000 7f04d8c23b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b3510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.808+0000 7f04d8c23b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 0x55cb4b341180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.808+0000 7f04d8c23b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b3416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.808+0000 7f04d8c23b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41af490 con 0x55cb4b1b5f70 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.808+0000 7f04d8c23b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41b0900 con 0x55cb4b22cd00 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.808+0000 7f04d8c23b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41afb60 con 0x55cb4b1b24a0 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.808+0000 7f04d8c23b80 10 client.0 _ll_register_callbacks cb 0x55cb4b0c36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 0x55cb4b341180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 0x55cb4b341180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53000/0 (socket says 172.21.15.46:53000) 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04bbfff700 1 -- 172.21.15.46:0/1050865106 learned_addr learned my addr 172.21.15.46:0/1050865106 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04c37fe700 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b3510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04c3fff700 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b3416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04bbfff700 1 -- 172.21.15.46:0/1050865106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 msgr2=0x55cb4b3510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04bbfff700 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b3510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04bbfff700 1 -- 172.21.15.46:0/1050865106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 msgr2=0x55cb4b3416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04bbfff700 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b3416c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04bbfff700 1 -- 172.21.15.46:0/1050865106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4b34fc70 con 0x55cb4b1b5f70 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04c3fff700 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b3416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04bbfff700 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 0x55cb4b341180 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009990 tx=0x7f04ac0052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13450960e5c1ae75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04a2ffd700 1 -- 172.21.15.46:0/1050865106 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00b990 con 0x55cb4b1b5f70 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04a2ffd700 1 -- 172.21.15.46:0/1050865106 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f04ac00bb30 con 0x55cb4b1b5f70 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.809+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1050865106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04b403a030 con 0x55cb4b1b5f70 2024-05-09T00:19:16.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.810+0000 7f04a2ffd700 1 -- 172.21.15.46:0/1050865106 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04ac033050 con 0x55cb4b1b5f70 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.810+0000 7f04d8c23b80 10 client.16608 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.810+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1050865106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cb4b3b2890 con 0x55cb4b1b5f70 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.810+0000 7f04a2ffd700 1 -- 172.21.15.46:0/1050865106 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b403a030 con 0x55cb4b1b5f70 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.810+0000 7f04a2ffd700 1 -- 172.21.15.46:0/1050865106 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f04ac024420 con 0x55cb4b1b5f70 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.810+0000 7f04d8c23b80 10 client.16608 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.810+0000 7f04d8c23b80 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.810+0000 7f04d8c23b80 20 client.16608 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.810+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55cb4b32f400 0x55cb4b3b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.810+0000 7f04d8c23b80 4 client.16608 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.810+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1050865106 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb4b259b20 con 0x55cb4b32f400 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.810+0000 7f04c37fe700 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55cb4b32f400 0x55cb4b3b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04c37fe700 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55cb4b32f400 0x55cb4b3b4fa0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c35c4d1eb4283f32 server_cookie=dfe50b966d26d327 in_seq=0 out_seq=0 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04a2ffd700 10 client.16608 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:16.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04a2ffd700 1 -- 172.21.15.46:0/1050865106 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb4b259b20 con 0x55cb4b32f400 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04a2ffd700 10 client.16608 handle_command_reply: tid=0 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04d8c23b80 1 client.16608 shutdown 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1050865106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 msgr2=0x55cb4b341180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 0x55cb4b341180 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009990 tx=0x7f04ac0052e0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1050865106 shutdown_connections 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55cb4b32f400 0x55cb4b3b4fa0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cb4b22cd00 0x55cb4b3416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cb4b1b24a0 0x55cb4b3510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04a2ffd700 0 client.16608 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04d8c23b80 1 --2- 172.21.15.46:0/1050865106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb4b1b5f70 0x55cb4b341180 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.811+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1050865106 >> 172.21.15.46:0/1050865106 conn(0x55cb4b32a520 msgr2=0x55cb4b323ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1050865106 shutdown_connections 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04d8c23b80 1 -- 172.21.15.46:0/1050865106 wait complete. 2024-05-09T00:19:16.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04d8c23b80 20 client.16608 trim_cache size 0 max 16384 2024-05-09T00:19:16.816 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:19:16.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04ca20d700 1 -- 172.21.15.46:0/3290158869 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f04b00b2880 msgr2=0x7f04b00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:16.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04ca20d700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f04b00b2880 0x7f04b00b4d90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f04b4005950 tx=0x7f04b400b410 comp rx=0 tx=0).stop 2024-05-09T00:19:16.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04ca20d700 1 -- 172.21.15.46:0/3290158869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04c4108760 msgr2=0x7f04c40ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:16.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04ca20d700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04c4108760 0x7f04c40ffe90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f04bc007ee0 tx=0x7f04bc00d3b0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04ca20d700 1 -- 172.21.15.46:0/3290158869 shutdown_connections 2024-05-09T00:19:16.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04ca20d700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f04b00b2880 0x7f04b00b4d90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04ca20d700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04c4108760 0x7f04c40ffe90 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04ca20d700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c41034e0 0x7f04c40ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04ca20d700 1 --2- 172.21.15.46:0/3290158869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04c4107e10 0x7f04c40ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:16.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.812+0000 7f04ca20d700 1 -- 172.21.15.46:0/3290158869 >> 172.21.15.46:0/3290158869 conn(0x7f04c40fef30 msgr2=0x7f04c406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:16.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.813+0000 7f04ca20d700 1 -- 172.21.15.46:0/3290158869 shutdown_connections 2024-05-09T00:19:16.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:16.813+0000 7f04ca20d700 1 -- 172.21.15.46:0/3290158869 wait complete. 2024-05-09T00:19:16.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:16 smithi169 ceph-mon[34777]: pgmap v613: 97 pgs: 97 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:19:16.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:16 smithi142 ceph-mon[34533]: pgmap v613: 97 pgs: 97 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:19:18.558 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:19:18.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:19:18.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:18 smithi046 ceph-mon[30151]: pgmap v614: 97 pgs: 97 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:18.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:18 smithi169 ceph-mon[34777]: pgmap v614: 97 pgs: 97 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:18.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:18 smithi142 ceph-mon[34533]: pgmap v614: 97 pgs: 97 active+clean; 4.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.641+0000 7fc86d62c700 1 -- 172.21.15.46:0/521289567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc868108740 msgr2=0x7fc8681100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.641+0000 7fc86d62c700 1 --2- 172.21.15.46:0/521289567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc868108740 0x7fc8681100a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc8600099c0 tx=0x7fc860009cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.642+0000 7fc86d62c700 1 -- 172.21.15.46:0/521289567 shutdown_connections 2024-05-09T00:19:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.642+0000 7fc86d62c700 1 --2- 172.21.15.46:0/521289567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc868108740 0x7fc8681100a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.642+0000 7fc86d62c700 1 --2- 172.21.15.46:0/521289567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8681034c0 0x7fc8681038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.642+0000 7fc86d62c700 1 --2- 172.21.15.46:0/521289567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc868107df0 0x7fc868108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.642+0000 7fc86d62c700 1 -- 172.21.15.46:0/521289567 >> 172.21.15.46:0/521289567 conn(0x7fc8680fedf0 msgr2=0x7fc868101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.642+0000 7fc86d62c700 1 -- 172.21.15.46:0/521289567 shutdown_connections 2024-05-09T00:19:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.642+0000 7fc86d62c700 1 -- 172.21.15.46:0/521289567 wait complete. 2024-05-09T00:19:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.642+0000 7fc86d62c700 1 Processor -- start 2024-05-09T00:19:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc86d62c700 1 -- start start 2024-05-09T00:19:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc86d62c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8681034c0 0x7fc868110880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc86d62c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc868107df0 0x7fc868110dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc86d62c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc868108740 0x7fc868111300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc86d62c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8681144c0 con 0x7fc868107df0 2024-05-09T00:19:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc86d62c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc868114340 con 0x7fc868108740 2024-05-09T00:19:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc86d62c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc868114640 con 0x7fc8681034c0 2024-05-09T00:19:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc8667fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc868107df0 0x7fc868110dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc8667fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc868107df0 0x7fc868110dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53024/0 (socket says 172.21.15.46:53024) 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc8667fc700 1 -- 172.21.15.46:0/970485983 learned_addr learned my addr 172.21.15.46:0/970485983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc866ffd700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8681034c0 0x7fc868110880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc8677fe700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc868108740 0x7fc868111300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc8667fc700 1 -- 172.21.15.46:0/970485983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8681034c0 msgr2=0x7fc868110880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc8667fc700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8681034c0 0x7fc868110880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc8667fc700 1 -- 172.21.15.46:0/970485983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc868108740 msgr2=0x7fc868111300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc8667fc700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc868108740 0x7fc868111300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.643+0000 7fc8667fc700 1 -- 172.21.15.46:0/970485983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8681a9170 con 0x7fc868107df0 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.644+0000 7fc8667fc700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc868107df0 0x7fc868110dc0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00d980 tx=0x7fc85c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.644+0000 7fc84ffff700 1 -- 172.21.15.46:0/970485983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c017070 con 0x7fc868107df0 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.644+0000 7fc86d62c700 1 -- 172.21.15.46:0/970485983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8681a9460 con 0x7fc868107df0 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.644+0000 7fc84ffff700 1 -- 172.21.15.46:0/970485983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc85c00fd40 con 0x7fc868107df0 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.644+0000 7fc84ffff700 1 -- 172.21.15.46:0/970485983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00e3b0 con 0x7fc868107df0 2024-05-09T00:19:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.644+0000 7fc86d62c700 1 -- 172.21.15.46:0/970485983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8681a99f0 con 0x7fc868107df0 2024-05-09T00:19:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.647+0000 7fc84ffff700 1 -- 172.21.15.46:0/970485983 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc85c010610 con 0x7fc868107df0 2024-05-09T00:19:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.647+0000 7fc84ffff700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc8500b2880 0x7fc8500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc84ffff700 1 -- 172.21.15.46:0/970485983 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc85c0e1c10 con 0x7fc868107df0 2024-05-09T00:19:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc866ffd700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc8500b2880 0x7fc8500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc87c042b80 1 Processor -- start 2024-05-09T00:19:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc87c042b80 1 -- start start 2024-05-09T00:19:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc866ffd700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc8500b2880 0x7fc8500b4d90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc858009550 tx=0x7fc8580091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc87c042b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f8f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc87c042b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 0x561e0f8f7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc87c042b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f8f1480 0x561e0f8f1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc87c042b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561e0f814120 con 0x561e0f7eed80 2024-05-09T00:19:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc87c042b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561e0f7b9a20 con 0x561e0f8f1480 2024-05-09T00:19:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc87c042b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561e0f8100e0 con 0x561e0f774520 2024-05-09T00:19:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc866ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f8f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc866ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f8f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53036/0 (socket says 172.21.15.46:53036) 2024-05-09T00:19:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc866ffd700 1 -- 172.21.15.46:0/2893596667 learned_addr learned my addr 172.21.15.46:0/2893596667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.648+0000 7fc8677fe700 1 --2- 172.21.15.46:0/2893596667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f8f1480 0x561e0f8f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.649+0000 7fc866ffd700 1 -- 172.21.15.46:0/2893596667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 msgr2=0x561e0f8f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.649+0000 7fc866ffd700 1 --2- 172.21.15.46:0/2893596667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 0x561e0f8f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.649+0000 7fc866ffd700 1 -- 172.21.15.46:0/2893596667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f8f1480 msgr2=0x561e0f8f1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.649+0000 7fc866ffd700 1 --2- 172.21.15.46:0/2893596667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f8f1480 0x561e0f8f1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.649+0000 7fc866ffd700 1 -- 172.21.15.46:0/2893596667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0f7aa580 con 0x561e0f7eed80 2024-05-09T00:19:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.649+0000 7fc866ffd700 1 --2- 172.21.15.46:0/2893596667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f8f72c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc8580046e0 tx=0x7fc858013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497239f52eae71c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.649+0000 7fc84d7fa700 1 -- 172.21.15.46:0/2893596667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85801c050 con 0x561e0f7eed80 2024-05-09T00:19:20.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.649+0000 7fc84d7fa700 1 -- 172.21.15.46:0/2893596667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc858026070 con 0x561e0f7eed80 2024-05-09T00:19:20.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.649+0000 7fc84d7fa700 1 -- 172.21.15.46:0/2893596667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858015680 con 0x561e0f7eed80 2024-05-09T00:19:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.650+0000 7fc87c042b80 1 -- 172.21.15.46:0/2893596667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 msgr2=0x561e0f8f72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.650+0000 7fc87c042b80 1 --2- 172.21.15.46:0/2893596667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f8f72c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc8580046e0 tx=0x7fc858013000 comp rx=0 tx=0).stop 2024-05-09T00:19:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.650+0000 7fc87c042b80 1 -- 172.21.15.46:0/2893596667 shutdown_connections 2024-05-09T00:19:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.650+0000 7fc87c042b80 1 --2- 172.21.15.46:0/2893596667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f8f1480 0x561e0f8f1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.650+0000 7fc87c042b80 1 --2- 172.21.15.46:0/2893596667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f8f72c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.650+0000 7fc87c042b80 1 --2- 172.21.15.46:0/2893596667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 0x561e0f8f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.650+0000 7fc87c042b80 1 -- 172.21.15.46:0/2893596667 >> 172.21.15.46:0/2893596667 conn(0x561e0f777ff0 msgr2=0x561e0f8ec2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.650+0000 7fc87c042b80 1 -- 172.21.15.46:0/2893596667 shutdown_connections 2024-05-09T00:19:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.650+0000 7fc87c042b80 1 -- 172.21.15.46:0/2893596667 wait complete. 2024-05-09T00:19:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.650+0000 7fc87c042b80 1 -- start start 2024-05-09T00:19:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc87c042b80 1 Processor -- start 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc87c042b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f774520 0x561e0f8fb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc87c042b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f8fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc87c042b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f8f1480 0x561e0f8fbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc87c042b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561e0f8f21e0 con 0x561e0f7eed80 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc87c042b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561e0f7aeba0 con 0x561e0f774520 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc87c042b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561e0f8fe0a0 con 0x561e0f8f1480 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc87c042b80 10 client.0 _ll_register_callbacks cb 0x561e0f685740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc8667fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f8fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc8667fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f8fb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53042/0 (socket says 172.21.15.46:53042) 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc8667fc700 1 -- 172.21.15.46:0/3396829542 learned_addr learned my addr 172.21.15.46:0/3396829542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc8677fe700 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f8f1480 0x561e0f8fbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc866ffd700 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f774520 0x561e0f8fb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc8667fc700 1 -- 172.21.15.46:0/3396829542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f8f1480 msgr2=0x561e0f8fbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc8667fc700 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f8f1480 0x561e0f8fbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc8667fc700 1 -- 172.21.15.46:0/3396829542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f774520 msgr2=0x561e0f8fb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc8667fc700 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f774520 0x561e0f8fb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.651+0000 7fc8667fc700 1 -- 172.21.15.46:0/3396829542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0f68f510 con 0x561e0f7eed80 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.652+0000 7fc8677fe700 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f8f1480 0x561e0f8fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.652+0000 7fc8667fc700 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f8fb6b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0a91c0 tx=0x7fc85c0a9770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fba326ff3a34d2a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.652+0000 7fc84aff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.652+0000 7fc84aff5700 1 -- 172.21.15.46:0/3396829542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c020040 con 0x561e0f7eed80 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.652+0000 7fc84aff5700 1 -- 172.21.15.46:0/3396829542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc85c0dfde0 con 0x561e0f7eed80 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.652+0000 7fc87c042b80 1 -- 172.21.15.46:0/3396829542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc858026070 con 0x561e0f7eed80 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.652+0000 7fc84aff5700 1 -- 172.21.15.46:0/3396829542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc85c0a8d40 con 0x561e0f7eed80 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.652+0000 7fc87c042b80 10 client.16620 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.652+0000 7fc87c042b80 1 -- 172.21.15.46:0/3396829542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561e0f908b90 con 0x561e0f7eed80 2024-05-09T00:19:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.652+0000 7fc84aff5700 1 -- 172.21.15.46:0/3396829542 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc858026070 con 0x561e0f7eed80 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.652+0000 7fc84aff5700 1 -- 172.21.15.46:0/3396829542 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc85c0a8490 con 0x561e0f7eed80 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.652+0000 7fc87c042b80 10 client.16620 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.653+0000 7fc87c042b80 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.653+0000 7fc87c042b80 20 client.16620 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.653+0000 7fc87c042b80 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561e0f90b1d0 0x561e0f90d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.653+0000 7fc87c042b80 4 client.16620 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.653+0000 7fc87c042b80 1 -- 172.21.15.46:0/3396829542 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e0f821b00 con 0x561e0f90b1d0 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.653+0000 7fc866ffd700 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561e0f90b1d0 0x561e0f90d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.653+0000 7fc84aff5700 10 client.16620 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.653+0000 7fc866ffd700 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561e0f90b1d0 0x561e0f90d5f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43cf758a431ee40b server_cookie=6ba8324ab05c5fb5 in_seq=0 out_seq=0 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.654+0000 7fc84aff5700 1 -- 172.21.15.46:0/3396829542 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561e0f821b00 con 0x561e0f90b1d0 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.654+0000 7fc84aff5700 10 client.16620 handle_command_reply: tid=0 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.654+0000 7fc87c042b80 1 client.16620 shutdown 2024-05-09T00:19:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.654+0000 7fc87c042b80 1 -- 172.21.15.46:0/3396829542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 msgr2=0x561e0f8fb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.654+0000 7fc87c042b80 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f8fb6b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0a91c0 tx=0x7fc85c0a9770 comp rx=0 tx=0).stop 2024-05-09T00:19:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.654+0000 7fc87c042b80 1 -- 172.21.15.46:0/3396829542 shutdown_connections 2024-05-09T00:19:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.654+0000 7fc87c042b80 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561e0f90b1d0 0x561e0f90d5f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.654+0000 7fc87c042b80 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f8f1480 0x561e0f8fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.654+0000 7fc87c042b80 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f774520 0x561e0f8fb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.654+0000 7fc87c042b80 1 --2- 172.21.15.46:0/3396829542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f8fb6b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.654+0000 7fc84aff5700 0 client.16620 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.654+0000 7fc87c042b80 1 -- 172.21.15.46:0/3396829542 >> 172.21.15.46:0/3396829542 conn(0x561e0f777ff0 msgr2=0x561e0f8e6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.655+0000 7fc87c042b80 1 -- 172.21.15.46:0/3396829542 shutdown_connections 2024-05-09T00:19:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.655+0000 7fc87c042b80 1 -- 172.21.15.46:0/3396829542 wait complete. 2024-05-09T00:19:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.655+0000 7fc87c042b80 20 client.16620 trim_cache size 0 max 16384 2024-05-09T00:19:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.675+0000 7fc87c042b80 1 Processor -- start 2024-05-09T00:19:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc87c042b80 1 -- start start 2024-05-09T00:19:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc87c042b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 0x561e0f8f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc87c042b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f7eed80 0x561e0f8f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc87c042b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f8f1480 0x561e0f8fba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc87c042b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8681b37b0 con 0x561e0f8f1480 2024-05-09T00:19:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc87c042b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8681b1040 con 0x561e0f7eed80 2024-05-09T00:19:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc87c042b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc8681aee70 con 0x561e0f774520 2024-05-09T00:19:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc8677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f8f1480 0x561e0f8fba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc8677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f8f1480 0x561e0f8fba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53044/0 (socket says 172.21.15.46:53044) 2024-05-09T00:19:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc8677fe700 1 -- 172.21.15.46:0/691518466 learned_addr learned my addr 172.21.15.46:0/691518466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:20.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc866ffd700 1 --2- 172.21.15.46:0/691518466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 0x561e0f8f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc8667fc700 1 --2- 172.21.15.46:0/691518466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f7eed80 0x561e0f8f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc8677fe700 1 -- 172.21.15.46:0/691518466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 msgr2=0x561e0f8f8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc8677fe700 1 --2- 172.21.15.46:0/691518466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 0x561e0f8f8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc8677fe700 1 -- 172.21.15.46:0/691518466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f7eed80 msgr2=0x561e0f8f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc8677fe700 1 --2- 172.21.15.46:0/691518466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f7eed80 0x561e0f8f0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.676+0000 7fc8677fe700 1 -- 172.21.15.46:0/691518466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0f8e8a50 con 0x561e0f8f1480 2024-05-09T00:19:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.677+0000 7fc8677fe700 1 --2- 172.21.15.46:0/691518466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f8f1480 0x561e0f8fba30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc860015000 tx=0x7fc860005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ef37f1b4ca1de1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.677+0000 7fc84cff9700 1 -- 172.21.15.46:0/691518466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860003a60 con 0x561e0f8f1480 2024-05-09T00:19:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.677+0000 7fc84cff9700 1 -- 172.21.15.46:0/691518466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc860028070 con 0x561e0f8f1480 2024-05-09T00:19:20.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.677+0000 7fc84cff9700 1 -- 172.21.15.46:0/691518466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86000bac0 con 0x561e0f8f1480 2024-05-09T00:19:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.677+0000 7fc87c042b80 1 -- 172.21.15.46:0/691518466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f8f1480 msgr2=0x561e0f8fba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.677+0000 7fc87c042b80 1 --2- 172.21.15.46:0/691518466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f8f1480 0x561e0f8fba30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc860015000 tx=0x7fc860005fb0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 -- 172.21.15.46:0/691518466 shutdown_connections 2024-05-09T00:19:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 --2- 172.21.15.46:0/691518466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f8f1480 0x561e0f8fba30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 --2- 172.21.15.46:0/691518466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 0x561e0f8f8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 --2- 172.21.15.46:0/691518466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f7eed80 0x561e0f8f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 -- 172.21.15.46:0/691518466 >> 172.21.15.46:0/691518466 conn(0x561e0f777ff0 msgr2=0x561e0f8e66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 -- 172.21.15.46:0/691518466 shutdown_connections 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 -- 172.21.15.46:0/691518466 wait complete. 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 -- start start 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 Processor -- start 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 0x561e0f903460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f900150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f8f1480 0x561e0f900690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8681af500 con 0x561e0f7eed80 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8681b0970 con 0x561e0f8f1480 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.678+0000 7fc87c042b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc8681afbd0 con 0x561e0f774520 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc87c042b80 10 client.0 _ll_register_callbacks cb 0x561e0f685740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc8667fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f900150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc8667fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f900150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53052/0 (socket says 172.21.15.46:53052) 2024-05-09T00:19:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc8667fc700 1 -- 172.21.15.46:0/1133266896 learned_addr learned my addr 172.21.15.46:0/1133266896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc866ffd700 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 0x561e0f903460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc8677fe700 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f8f1480 0x561e0f900690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc8667fc700 1 -- 172.21.15.46:0/1133266896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 msgr2=0x561e0f903460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc8667fc700 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 0x561e0f903460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc8667fc700 1 -- 172.21.15.46:0/1133266896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f8f1480 msgr2=0x561e0f900690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc8667fc700 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f8f1480 0x561e0f900690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc8667fc700 1 -- 172.21.15.46:0/1133266896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0f900bd0 con 0x561e0f7eed80 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc8667fc700 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f900150 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0a52b0 tx=0x7fc85c0ad3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540e586fa367fb8c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc84aff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc84aff5700 1 -- 172.21.15.46:0/1133266896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c020070 con 0x561e0f7eed80 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc84aff5700 1 -- 172.21.15.46:0/1133266896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc85c0a4c90 con 0x561e0f7eed80 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.679+0000 7fc87c042b80 1 -- 172.21.15.46:0/1133266896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc860028070 con 0x561e0f7eed80 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.680+0000 7fc84aff5700 1 -- 172.21.15.46:0/1133266896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc85c0a9930 con 0x561e0f7eed80 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.680+0000 7fc87c042b80 10 client.16626 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.680+0000 7fc87c042b80 1 -- 172.21.15.46:0/1133266896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561e0f976460 con 0x561e0f7eed80 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.680+0000 7fc84aff5700 1 -- 172.21.15.46:0/1133266896 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc860028070 con 0x561e0f7eed80 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.680+0000 7fc84aff5700 1 -- 172.21.15.46:0/1133266896 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc85c0c3cb0 con 0x561e0f7eed80 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.680+0000 7fc87c042b80 10 client.16626 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.680+0000 7fc87c042b80 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.680+0000 7fc87c042b80 20 client.16626 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.680+0000 7fc87c042b80 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561e0f976b70 0x561e0f978f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.680+0000 7fc87c042b80 4 client.16626 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.680+0000 7fc87c042b80 1 -- 172.21.15.46:0/1133266896 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561e0f81bba0 con 0x561e0f976b70 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.680+0000 7fc866ffd700 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561e0f976b70 0x561e0f978f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.681+0000 7fc84aff5700 10 client.16626 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.681+0000 7fc866ffd700 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561e0f976b70 0x561e0f978f90 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfd4af7a0949deff server_cookie=a9c200d2d882afaa in_seq=0 out_seq=0 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc84aff5700 1 -- 172.21.15.46:0/1133266896 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561e0f81bba0 con 0x561e0f976b70 2024-05-09T00:19:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc84aff5700 10 client.16626 handle_command_reply: tid=0 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc87c042b80 1 client.16626 shutdown 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc87c042b80 1 -- 172.21.15.46:0/1133266896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 msgr2=0x561e0f900150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc87c042b80 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f900150 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0a52b0 tx=0x7fc85c0ad3c0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc87c042b80 1 -- 172.21.15.46:0/1133266896 shutdown_connections 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc87c042b80 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x561e0f976b70 0x561e0f978f90 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc87c042b80 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e0f8f1480 0x561e0f900690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc87c042b80 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e0f774520 0x561e0f903460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc87c042b80 1 --2- 172.21.15.46:0/1133266896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e0f7eed80 0x561e0f900150 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc87c042b80 1 -- 172.21.15.46:0/1133266896 >> 172.21.15.46:0/1133266896 conn(0x561e0f777ff0 msgr2=0x561e0f8f4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc84aff5700 0 client.16626 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc87c042b80 1 -- 172.21.15.46:0/1133266896 shutdown_connections 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc87c042b80 1 -- 172.21.15.46:0/1133266896 wait complete. 2024-05-09T00:19:20.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.682+0000 7fc87c042b80 20 client.16626 trim_cache size 0 max 16384 2024-05-09T00:19:20.687 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:19:20.687 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:19:20.687 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "68a3f6b1-2cfc-4dc9-b7c5-3b3bfa0eeaa4", 2024-05-09T00:19:20.687 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:19:20.687 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:19:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.683+0000 7fc86d62c700 1 -- 172.21.15.46:0/970485983 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc8500b2880 msgr2=0x7fc8500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.683+0000 7fc86d62c700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc8500b2880 0x7fc8500b4d90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc858009550 tx=0x7fc8580091b0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.683+0000 7fc86d62c700 1 -- 172.21.15.46:0/970485983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc868107df0 msgr2=0x7fc868110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.683+0000 7fc86d62c700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc868107df0 0x7fc868110dc0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00d980 tx=0x7fc85c00dc90 comp rx=0 tx=0).stop 2024-05-09T00:19:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.683+0000 7fc86d62c700 1 -- 172.21.15.46:0/970485983 shutdown_connections 2024-05-09T00:19:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.683+0000 7fc86d62c700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc8500b2880 0x7fc8500b4d90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.683+0000 7fc86d62c700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc868108740 0x7fc868111300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.683+0000 7fc86d62c700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8681034c0 0x7fc868110880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.683+0000 7fc86d62c700 1 --2- 172.21.15.46:0/970485983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc868107df0 0x7fc868110dc0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.683+0000 7fc86d62c700 1 -- 172.21.15.46:0/970485983 >> 172.21.15.46:0/970485983 conn(0x7fc8680fedf0 msgr2=0x7fc868107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.683+0000 7fc86d62c700 1 -- 172.21.15.46:0/970485983 shutdown_connections 2024-05-09T00:19:20.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:20.683+0000 7fc86d62c700 1 -- 172.21.15.46:0/970485983 wait complete. 2024-05-09T00:19:20.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:20 smithi046 ceph-mon[30151]: pgmap v615: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:19:20.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:20 smithi169 ceph-mon[34777]: pgmap v615: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:19:20.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:20 smithi142 ceph-mon[34533]: pgmap v615: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:19:21.590 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:19:21.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:21 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:19:21.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:21 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:21.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:21 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:19:21.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:21 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:21.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:21 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:19:21.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:21 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:21.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:21 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:19:21.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:21 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:21.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:21 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:19:21.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:21 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:21.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:21 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:19:21.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:21 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:22.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:22 smithi046 ceph-mon[30151]: pgmap v616: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:19:22.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:22 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:19:22.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:22 smithi169 ceph-mon[34777]: pgmap v616: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:19:22.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:22 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:19:22.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:22 smithi142 ceph-mon[34533]: pgmap v616: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:19:22.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:22 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:19:23.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.689+0000 7fa52fc6a700 1 -- 172.21.15.46:0/3822414129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa528107df0 msgr2=0x7fa528108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:23.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.689+0000 7fa52fc6a700 1 --2- 172.21.15.46:0/3822414129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa528107df0 0x7fa528108200 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa5180099c0 tx=0x7fa518009cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.689+0000 7fa52fc6a700 1 -- 172.21.15.46:0/3822414129 shutdown_connections 2024-05-09T00:19:23.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.689+0000 7fa52fc6a700 1 --2- 172.21.15.46:0/3822414129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa528108740 0x7fa5281100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.689+0000 7fa52fc6a700 1 --2- 172.21.15.46:0/3822414129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5281034c0 0x7fa5281038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.689+0000 7fa52fc6a700 1 --2- 172.21.15.46:0/3822414129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa528107df0 0x7fa528108200 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.689+0000 7fa52fc6a700 1 -- 172.21.15.46:0/3822414129 >> 172.21.15.46:0/3822414129 conn(0x7fa5280fedf0 msgr2=0x7fa528101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:23.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.689+0000 7fa52fc6a700 1 -- 172.21.15.46:0/3822414129 shutdown_connections 2024-05-09T00:19:23.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.689+0000 7fa52fc6a700 1 -- 172.21.15.46:0/3822414129 wait complete. 2024-05-09T00:19:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.690+0000 7fa52fc6a700 1 Processor -- start 2024-05-09T00:19:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.690+0000 7fa52fc6a700 1 -- start start 2024-05-09T00:19:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.690+0000 7fa52fc6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5281034c0 0x7fa528199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.690+0000 7fa52fc6a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa528107df0 0x7fa52819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.690+0000 7fa52fc6a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa528108740 0x7fa52819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.690+0000 7fa52fc6a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281133c0 con 0x7fa5281034c0 2024-05-09T00:19:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.690+0000 7fa52fc6a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa528113240 con 0x7fa528107df0 2024-05-09T00:19:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.690+0000 7fa52fc6a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa528113540 con 0x7fa528108740 2024-05-09T00:19:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.690+0000 7fa52da06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5281034c0 0x7fa528199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.690+0000 7fa52da06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5281034c0 0x7fa528199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53058/0 (socket says 172.21.15.46:53058) 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa52da06700 1 -- 172.21.15.46:0/2660896169 learned_addr learned my addr 172.21.15.46:0/2660896169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa52d205700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa528107df0 0x7fa52819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa52e207700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa528108740 0x7fa52819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa52da06700 1 -- 172.21.15.46:0/2660896169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa528108740 msgr2=0x7fa52819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa52da06700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa528108740 0x7fa52819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa52da06700 1 -- 172.21.15.46:0/2660896169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa528107df0 msgr2=0x7fa52819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa52da06700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa528107df0 0x7fa52819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa52da06700 1 -- 172.21.15.46:0/2660896169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52819ebd0 con 0x7fa5281034c0 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa52e207700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa528108740 0x7fa52819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa52da06700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5281034c0 0x7fa528199db0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa52400c970 tx=0x7fa52400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa51effd700 1 -- 172.21.15.46:0/2660896169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524007aa0 con 0x7fa5281034c0 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa51effd700 1 -- 172.21.15.46:0/2660896169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa524004ce0 con 0x7fa5281034c0 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa52fc6a700 1 -- 172.21.15.46:0/2660896169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52819eec0 con 0x7fa5281034c0 2024-05-09T00:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa51effd700 1 -- 172.21.15.46:0/2660896169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240055c0 con 0x7fa5281034c0 2024-05-09T00:19:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.691+0000 7fa52fc6a700 1 -- 172.21.15.46:0/2660896169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52819f3d0 con 0x7fa5281034c0 2024-05-09T00:19:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.694+0000 7fa51effd700 1 -- 172.21.15.46:0/2660896169 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa524007c40 con 0x7fa5281034c0 2024-05-09T00:19:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.695+0000 7fa51effd700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa5140b2900 0x7fa5140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.695+0000 7fa52d205700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa5140b2900 0x7fa5140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.695+0000 7fa51effd700 1 -- 172.21.15.46:0/2660896169 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fa5240e1fe0 con 0x7fa5281034c0 2024-05-09T00:19:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.695+0000 7fa53e680b80 1 Processor -- start 2024-05-09T00:19:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.695+0000 7fa53e680b80 1 -- start start 2024-05-09T00:19:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.695+0000 7fa52d205700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa5140b2900 0x7fa5140b4e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa518000c00 tx=0x7fa51801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa53e680b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d048d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa53e680b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 0x55a2d048dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa53e680b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d048e250 0x55a2d049cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa53e680b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a2d03aeff0 con 0x55a2d030f4a0 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa53e680b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a2d034d7f0 con 0x55a2d048e250 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa53e680b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a2d03aafb0 con 0x55a2d03f9160 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa52d205700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 0x55a2d048dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa52d205700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 0x55a2d048dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53068/0 (socket says 172.21.15.46:53068) 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa52d205700 1 -- 172.21.15.46:0/2065153807 learned_addr learned my addr 172.21.15.46:0/2065153807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa52da06700 1 --2- 172.21.15.46:0/2065153807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d048d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa52e207700 1 --2- 172.21.15.46:0/2065153807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d048e250 0x55a2d049cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa52d205700 1 -- 172.21.15.46:0/2065153807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 msgr2=0x55a2d048d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa52d205700 1 --2- 172.21.15.46:0/2065153807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d048d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa52d205700 1 -- 172.21.15.46:0/2065153807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d048e250 msgr2=0x55a2d049cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa52d205700 1 --2- 172.21.15.46:0/2065153807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d048e250 0x55a2d049cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa52d205700 1 -- 172.21.15.46:0/2065153807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d0345370 con 0x55a2d030f4a0 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.696+0000 7fa52d205700 1 --2- 172.21.15.46:0/2065153807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 0x55a2d048dd10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa5180099c0 tx=0x7fa51801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc2a205332c0645 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.697+0000 7fa513fff700 1 -- 172.21.15.46:0/2065153807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51800fa40 con 0x55a2d030f4a0 2024-05-09T00:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.697+0000 7fa513fff700 1 -- 172.21.15.46:0/2065153807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa518031070 con 0x55a2d030f4a0 2024-05-09T00:19:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.697+0000 7fa513fff700 1 -- 172.21.15.46:0/2065153807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518018400 con 0x55a2d030f4a0 2024-05-09T00:19:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.697+0000 7fa53e680b80 1 -- 172.21.15.46:0/2065153807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 msgr2=0x55a2d048dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.697+0000 7fa53e680b80 1 --2- 172.21.15.46:0/2065153807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 0x55a2d048dd10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa5180099c0 tx=0x7fa51801d000 comp rx=0 tx=0).stop 2024-05-09T00:19:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.698+0000 7fa53e680b80 1 -- 172.21.15.46:0/2065153807 shutdown_connections 2024-05-09T00:19:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.698+0000 7fa53e680b80 1 --2- 172.21.15.46:0/2065153807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d048e250 0x55a2d049cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.698+0000 7fa53e680b80 1 --2- 172.21.15.46:0/2065153807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d048d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.698+0000 7fa53e680b80 1 --2- 172.21.15.46:0/2065153807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 0x55a2d048dd10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.698+0000 7fa53e680b80 1 -- 172.21.15.46:0/2065153807 >> 172.21.15.46:0/2065153807 conn(0x55a2d036f6a0 msgr2=0x55a2d0489d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.698+0000 7fa53e680b80 1 -- 172.21.15.46:0/2065153807 shutdown_connections 2024-05-09T00:19:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.698+0000 7fa53e680b80 1 -- 172.21.15.46:0/2065153807 wait complete. 2024-05-09T00:19:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.698+0000 7fa53e680b80 1 -- start start 2024-05-09T00:19:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.698+0000 7fa53e680b80 1 Processor -- start 2024-05-09T00:19:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa53e680b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d030f4a0 0x55a2d049a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa53e680b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d03f9160 0x55a2d049a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa53e680b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 0x55a2d049abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa53e680b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a2d022aba0 con 0x55a2d048e250 2024-05-09T00:19:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa53e680b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a2d0369150 con 0x55a2d03f9160 2024-05-09T00:19:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa53e680b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a2d049d410 con 0x55a2d030f4a0 2024-05-09T00:19:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa53e680b80 10 client.0 _ll_register_callbacks cb 0x55a2d02206c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa52e207700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 0x55a2d049abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa52e207700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 0x55a2d049abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53078/0 (socket says 172.21.15.46:53078) 2024-05-09T00:19:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa52e207700 1 -- 172.21.15.46:0/2500704431 learned_addr learned my addr 172.21.15.46:0/2500704431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa52da06700 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d030f4a0 0x55a2d049a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa52d205700 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d03f9160 0x55a2d049a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa52e207700 1 -- 172.21.15.46:0/2500704431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d030f4a0 msgr2=0x55a2d049a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa52e207700 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d030f4a0 0x55a2d049a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa52e207700 1 -- 172.21.15.46:0/2500704431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d03f9160 msgr2=0x55a2d049a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa52e207700 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d03f9160 0x55a2d049a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa52e207700 1 -- 172.21.15.46:0/2500704431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d022a490 con 0x55a2d048e250 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa52e207700 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 0x55a2d049abb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa52000fd30 tx=0x7fa52000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8c3ede7ab6c832 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.699+0000 7fa5117fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa5117fa700 1 -- 172.21.15.46:0/2500704431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520019030 con 0x55a2d048e250 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa5117fa700 1 -- 172.21.15.46:0/2500704431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fa520010040 con 0x55a2d048e250 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa53e680b80 1 -- 172.21.15.46:0/2500704431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa518031070 con 0x55a2d048e250 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa5117fa700 1 -- 172.21.15.46:0/2500704431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa520004bb0 con 0x55a2d048e250 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa53e680b80 10 client.16638 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa53e680b80 1 -- 172.21.15.46:0/2500704431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2d04a8010 con 0x55a2d048e250 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa5117fa700 1 -- 172.21.15.46:0/2500704431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa518031070 con 0x55a2d048e250 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa5117fa700 1 -- 172.21.15.46:0/2500704431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa52001e700 con 0x55a2d048e250 2024-05-09T00:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa53e680b80 10 client.16638 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa53e680b80 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa53e680b80 20 client.16638 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa53e680b80 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a2d04aa6f0 0x55a2d04acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa53e680b80 4 client.16638 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.700+0000 7fa53e680b80 1 -- 172.21.15.46:0/2500704431 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2d03bc9d0 con 0x55a2d04aa6f0 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.701+0000 7fa52da06700 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a2d04aa6f0 0x55a2d04acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.701+0000 7fa5117fa700 10 client.16638 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.701+0000 7fa52da06700 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a2d04aa6f0 0x55a2d04acb10 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f8bd8a5ce4b4ca6 server_cookie=e09d427ce67ea605 in_seq=0 out_seq=0 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa5117fa700 1 -- 172.21.15.46:0/2500704431 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2d03bc9d0 con 0x55a2d04aa6f0 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa5117fa700 10 client.16638 handle_command_reply: tid=0 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa53e680b80 1 client.16638 shutdown 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa53e680b80 1 -- 172.21.15.46:0/2500704431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 msgr2=0x55a2d049abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa53e680b80 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 0x55a2d049abb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa52000fd30 tx=0x7fa52000d430 comp rx=0 tx=0).stop 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa53e680b80 1 -- 172.21.15.46:0/2500704431 shutdown_connections 2024-05-09T00:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa53e680b80 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a2d04aa6f0 0x55a2d04acb10 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa53e680b80 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 0x55a2d049abb0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa53e680b80 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d030f4a0 0x55a2d049a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa53e680b80 1 --2- 172.21.15.46:0/2500704431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d03f9160 0x55a2d049a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa5117fa700 0 client.16638 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa53e680b80 1 -- 172.21.15.46:0/2500704431 >> 172.21.15.46:0/2500704431 conn(0x55a2d036f6a0 msgr2=0x55a2d04898a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa53e680b80 1 -- 172.21.15.46:0/2500704431 shutdown_connections 2024-05-09T00:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa53e680b80 1 -- 172.21.15.46:0/2500704431 wait complete. 2024-05-09T00:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.702+0000 7fa53e680b80 20 client.16638 trim_cache size 0 max 16384 2024-05-09T00:19:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa53e680b80 1 Processor -- start 2024-05-09T00:19:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa53e680b80 1 -- start start 2024-05-09T00:19:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa53e680b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 0x55a2d049a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa53e680b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d04a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa53e680b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d048e250 0x55a2d04a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa53e680b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281af150 con 0x55a2d030f4a0 2024-05-09T00:19:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa53e680b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281ac9e0 con 0x55a2d048e250 2024-05-09T00:19:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa53e680b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281aa810 con 0x55a2d03f9160 2024-05-09T00:19:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa52da06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 0x55a2d049a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa52da06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 0x55a2d049a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53080/0 (socket says 172.21.15.46:53080) 2024-05-09T00:19:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa52da06700 1 -- 172.21.15.46:0/174216873 learned_addr learned my addr 172.21.15.46:0/174216873 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa52d205700 1 --2- 172.21.15.46:0/174216873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d04a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa52e207700 1 --2- 172.21.15.46:0/174216873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d048e250 0x55a2d04a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa52da06700 1 -- 172.21.15.46:0/174216873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 msgr2=0x55a2d04a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa52da06700 1 --2- 172.21.15.46:0/174216873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d04a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa52da06700 1 -- 172.21.15.46:0/174216873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d048e250 msgr2=0x55a2d04a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa52da06700 1 --2- 172.21.15.46:0/174216873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d048e250 0x55a2d04a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa52da06700 1 -- 172.21.15.46:0/174216873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d0499100 con 0x55a2d030f4a0 2024-05-09T00:19:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa52d205700 1 --2- 172.21.15.46:0/174216873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d04a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:19:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.723+0000 7fa52e207700 1 --2- 172.21.15.46:0/174216873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d048e250 0x55a2d04a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:19:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.724+0000 7fa52da06700 1 --2- 172.21.15.46:0/174216873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 0x55a2d049a9f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa52400c940 tx=0x7fa5240df9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5620513ceeeb1193 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.724+0000 7fa5137fe700 1 -- 172.21.15.46:0/174216873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240a9690 con 0x55a2d030f4a0 2024-05-09T00:19:23.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.724+0000 7fa5137fe700 1 -- 172.21.15.46:0/174216873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5240ada40 con 0x55a2d030f4a0 2024-05-09T00:19:23.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.724+0000 7fa5137fe700 1 -- 172.21.15.46:0/174216873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240a9b50 con 0x55a2d030f4a0 2024-05-09T00:19:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.724+0000 7fa53e680b80 1 -- 172.21.15.46:0/174216873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 msgr2=0x55a2d049a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.724+0000 7fa53e680b80 1 --2- 172.21.15.46:0/174216873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 0x55a2d049a9f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa52400c940 tx=0x7fa5240df9b0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.725+0000 7fa53e680b80 1 -- 172.21.15.46:0/174216873 shutdown_connections 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.725+0000 7fa53e680b80 1 --2- 172.21.15.46:0/174216873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d048e250 0x55a2d04a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.725+0000 7fa53e680b80 1 --2- 172.21.15.46:0/174216873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d030f4a0 0x55a2d049a9f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.725+0000 7fa53e680b80 1 --2- 172.21.15.46:0/174216873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d04a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.725+0000 7fa53e680b80 1 -- 172.21.15.46:0/174216873 >> 172.21.15.46:0/174216873 conn(0x55a2d036f6a0 msgr2=0x55a2d048fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.725+0000 7fa53e680b80 1 -- 172.21.15.46:0/174216873 shutdown_connections 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.725+0000 7fa53e680b80 1 -- 172.21.15.46:0/174216873 wait complete. 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.725+0000 7fa53e680b80 1 -- start start 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.725+0000 7fa53e680b80 1 Processor -- start 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa53e680b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d030f4a0 0x55a2d050e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa53e680b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d050e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa53e680b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 0x55a2d049ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa53e680b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281aaea0 con 0x55a2d048e250 2024-05-09T00:19:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa53e680b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281ac310 con 0x55a2d030f4a0 2024-05-09T00:19:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa53e680b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281ab570 con 0x55a2d03f9160 2024-05-09T00:19:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa53e680b80 10 client.0 _ll_register_callbacks cb 0x55a2d02206c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52e207700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 0x55a2d049ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52e207700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 0x55a2d049ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53082/0 (socket says 172.21.15.46:53082) 2024-05-09T00:19:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52e207700 1 -- 172.21.15.46:0/3649371146 learned_addr learned my addr 172.21.15.46:0/3649371146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52da06700 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d030f4a0 0x55a2d050e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52d205700 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d050e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52e207700 1 -- 172.21.15.46:0/3649371146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 msgr2=0x55a2d050e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52e207700 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d050e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52e207700 1 -- 172.21.15.46:0/3649371146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d030f4a0 msgr2=0x55a2d050e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52e207700 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d030f4a0 0x55a2d050e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52e207700 1 -- 172.21.15.46:0/3649371146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d049f0b0 con 0x55a2d048e250 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52d205700 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d050e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52da06700 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d030f4a0 0x55a2d050e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa52e207700 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 0x55a2d049ea40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa52001ebf0 tx=0x7fa52000d8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b624fb659361e0e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa5117fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa5117fa700 1 -- 172.21.15.46:0/3649371146 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52002e060 con 0x55a2d048e250 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa5117fa700 1 -- 172.21.15.46:0/3649371146 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fa52000d990 con 0x55a2d048e250 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.726+0000 7fa53e680b80 1 -- 172.21.15.46:0/3649371146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5240ada40 con 0x55a2d048e250 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.727+0000 7fa5117fa700 1 -- 172.21.15.46:0/3649371146 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa520019070 con 0x55a2d048e250 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.727+0000 7fa53e680b80 10 client.16644 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.727+0000 7fa53e680b80 1 -- 172.21.15.46:0/3649371146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2d0515740 con 0x55a2d048e250 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.727+0000 7fa5117fa700 1 -- 172.21.15.46:0/3649371146 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5240ada40 con 0x55a2d048e250 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.727+0000 7fa5117fa700 1 -- 172.21.15.46:0/3649371146 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fa520033070 con 0x55a2d048e250 2024-05-09T00:19:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.727+0000 7fa53e680b80 10 client.16644 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.727+0000 7fa53e680b80 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.727+0000 7fa53e680b80 20 client.16644 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.727+0000 7fa53e680b80 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a2d0515df0 0x55a2d0518210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.727+0000 7fa53e680b80 4 client.16644 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.727+0000 7fa53e680b80 1 -- 172.21.15.46:0/3649371146 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2d03b6a70 con 0x55a2d0515df0 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.727+0000 7fa52da06700 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a2d0515df0 0x55a2d0518210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa52da06700 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a2d0515df0 0x55a2d0518210 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1805d4d3ef66b8 server_cookie=c2a2fa33a9280586 in_seq=0 out_seq=0 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa5117fa700 10 client.16644 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa5117fa700 1 -- 172.21.15.46:0/3649371146 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a2d03b6a70 con 0x55a2d0515df0 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa5117fa700 10 client.16644 handle_command_reply: tid=0 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa53e680b80 1 client.16644 shutdown 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa53e680b80 1 -- 172.21.15.46:0/3649371146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 msgr2=0x55a2d049ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa53e680b80 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 0x55a2d049ea40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa52001ebf0 tx=0x7fa52000d8b0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa53e680b80 1 -- 172.21.15.46:0/3649371146 shutdown_connections 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa53e680b80 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a2d0515df0 0x55a2d0518210 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa53e680b80 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2d048e250 0x55a2d049ea40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa53e680b80 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a2d030f4a0 0x55a2d050e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa5117fa700 0 client.16644 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa53e680b80 1 --2- 172.21.15.46:0/3649371146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2d03f9160 0x55a2d050e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa53e680b80 1 -- 172.21.15.46:0/3649371146 >> 172.21.15.46:0/3649371146 conn(0x55a2d036f6a0 msgr2=0x55a2d0494b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.728+0000 7fa53e680b80 1 -- 172.21.15.46:0/3649371146 shutdown_connections 2024-05-09T00:19:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa53e680b80 1 -- 172.21.15.46:0/3649371146 wait complete. 2024-05-09T00:19:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa53e680b80 20 client.16644 trim_cache size 0 max 16384 2024-05-09T00:19:23.733 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:19:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:19:23.733 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:19:23.734 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:19:23.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa52fc6a700 1 -- 172.21.15.46:0/2660896169 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa5140b2900 msgr2=0x7fa5140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:23.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa52fc6a700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa5140b2900 0x7fa5140b4e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa518000c00 tx=0x7fa51801a040 comp rx=0 tx=0).stop 2024-05-09T00:19:23.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa52fc6a700 1 -- 172.21.15.46:0/2660896169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5281034c0 msgr2=0x7fa528199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:23.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa52fc6a700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5281034c0 0x7fa528199db0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa52400c970 tx=0x7fa52400cd30 comp rx=0 tx=0).stop 2024-05-09T00:19:23.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa52fc6a700 1 -- 172.21.15.46:0/2660896169 shutdown_connections 2024-05-09T00:19:23.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa52fc6a700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa5140b2900 0x7fa5140b4e10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa52fc6a700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa528108740 0x7fa52819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa52fc6a700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5281034c0 0x7fa528199db0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa52fc6a700 1 --2- 172.21.15.46:0/2660896169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa528107df0 0x7fa52819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:23.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa52fc6a700 1 -- 172.21.15.46:0/2660896169 >> 172.21.15.46:0/2660896169 conn(0x7fa5280fedf0 msgr2=0x7fa528101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:23.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa52fc6a700 1 -- 172.21.15.46:0/2660896169 shutdown_connections 2024-05-09T00:19:23.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:23.729+0000 7fa52fc6a700 1 -- 172.21.15.46:0/2660896169 wait complete. 2024-05-09T00:19:24.623 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:19:24.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:19:24.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:24 smithi046 ceph-mon[30151]: pgmap v617: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:24.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:24 smithi169 ceph-mon[34777]: pgmap v617: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:24.878 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:24.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:24 smithi142 ceph-mon[34533]: pgmap v617: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:26.561 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:26 smithi046 ceph-mon[30151]: pgmap v618: 97 pgs: 97 active+clean; 4.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:19:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.723+0000 7f1fb707f700 1 -- 172.21.15.46:0/2035454354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb0107df0 msgr2=0x7f1fb0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.723+0000 7f1fb707f700 1 --2- 172.21.15.46:0/2035454354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb0107df0 0x7f1fb0108200 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40099c0 tx=0x7f1fa4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.723+0000 7f1fb707f700 1 -- 172.21.15.46:0/2035454354 shutdown_connections 2024-05-09T00:19:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.723+0000 7f1fb707f700 1 --2- 172.21.15.46:0/2035454354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fb0108740 0x7f1fb01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.723+0000 7f1fb707f700 1 --2- 172.21.15.46:0/2035454354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fb01034c0 0x7f1fb01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.723+0000 7f1fb707f700 1 --2- 172.21.15.46:0/2035454354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb0107df0 0x7f1fb0108200 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.723+0000 7f1fb707f700 1 -- 172.21.15.46:0/2035454354 >> 172.21.15.46:0/2035454354 conn(0x7f1fb00fedf0 msgr2=0x7f1fb0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.723+0000 7f1fb707f700 1 -- 172.21.15.46:0/2035454354 shutdown_connections 2024-05-09T00:19:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.723+0000 7f1fb707f700 1 -- 172.21.15.46:0/2035454354 wait complete. 2024-05-09T00:19:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.723+0000 7f1fb707f700 1 Processor -- start 2024-05-09T00:19:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.723+0000 7f1fb707f700 1 -- start start 2024-05-09T00:19:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb707f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb01034c0 0x7f1fb019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb707f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fb0107df0 0x7f1fb019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb707f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fb0108740 0x7f1fb019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb707f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb0114480 con 0x7f1fb01034c0 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb707f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb0114300 con 0x7f1fb0108740 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb707f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb0114600 con 0x7f1fb0107df0 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb4e1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb01034c0 0x7f1fb019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb4e1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb01034c0 0x7f1fb019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58414/0 (socket says 172.21.15.46:58414) 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb4e1b700 1 -- 172.21.15.46:0/3718323194 learned_addr learned my addr 172.21.15.46:0/3718323194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb561c700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fb0108740 0x7f1fb019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1faffff700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fb0107df0 0x7f1fb019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb4e1b700 1 -- 172.21.15.46:0/3718323194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fb0107df0 msgr2=0x7f1fb019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb4e1b700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fb0107df0 0x7f1fb019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb4e1b700 1 -- 172.21.15.46:0/3718323194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fb0108740 msgr2=0x7f1fb019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb4e1b700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fb0108740 0x7f1fb019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb4e1b700 1 -- 172.21.15.46:0/3718323194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb019fce0 con 0x7f1fb01034c0 2024-05-09T00:19:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.724+0000 7f1fb4e1b700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb01034c0 0x7f1fb019ae10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000d980 tx=0x7f1fa000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.725+0000 7f1fadffb700 1 -- 172.21.15.46:0/3718323194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0016070 con 0x7f1fb01034c0 2024-05-09T00:19:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.725+0000 7f1fb707f700 1 -- 172.21.15.46:0/3718323194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb019ffd0 con 0x7f1fb01034c0 2024-05-09T00:19:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.725+0000 7f1fadffb700 1 -- 172.21.15.46:0/3718323194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fa0004830 con 0x7f1fb01034c0 2024-05-09T00:19:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.725+0000 7f1fadffb700 1 -- 172.21.15.46:0/3718323194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0005540 con 0x7f1fb01034c0 2024-05-09T00:19:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.725+0000 7f1fb707f700 1 -- 172.21.15.46:0/3718323194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb01a78f0 con 0x7f1fb01034c0 2024-05-09T00:19:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.726+0000 7f1fb707f700 1 -- 172.21.15.46:0/3718323194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb00512c0 con 0x7f1fb01034c0 2024-05-09T00:19:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.727+0000 7f1fadffb700 1 -- 172.21.15.46:0/3718323194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1fa0004220 con 0x7f1fb01034c0 2024-05-09T00:19:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.728+0000 7f1fadffb700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1f980b2880 0x7f1f980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.728+0000 7f1faffff700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1f980b2880 0x7f1f980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.728+0000 7f1fadffb700 1 -- 172.21.15.46:0/3718323194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1fa00e1fd0 con 0x7f1fb01034c0 2024-05-09T00:19:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.729+0000 7f1faffff700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1f980b2880 0x7f1f980b4d90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4006010 tx=0x7f1fa4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.733+0000 7f1fadffb700 1 -- 172.21.15.46:0/3718323194 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1fa001f020 con 0x7f1fb01034c0 2024-05-09T00:19:26.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:26 smithi169 ceph-mon[34777]: pgmap v618: 97 pgs: 97 active+clean; 4.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:19:26.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:26 smithi142 ceph-mon[34533]: pgmap v618: 97 pgs: 97 active+clean; 4.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:19:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.995+0000 7f1fb707f700 1 -- 172.21.15.46:0/3718323194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fb002e5f0 con 0x7f1fb01034c0 2024-05-09T00:19:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.996+0000 7f1fadffb700 1 -- 172.21.15.46:0/3718323194 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f1fa00a9cf0 con 0x7f1fb01034c0 2024-05-09T00:19:26.995 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:19:26.995 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:19:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.998+0000 7f1fb707f700 1 -- 172.21.15.46:0/3718323194 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1f980b2880 msgr2=0x7f1f980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.998+0000 7f1fb707f700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1f980b2880 0x7f1f980b4d90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4006010 tx=0x7f1fa4000bc0 comp rx=0 tx=0).stop 2024-05-09T00:19:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.998+0000 7f1fb707f700 1 -- 172.21.15.46:0/3718323194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb01034c0 msgr2=0x7f1fb019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.998+0000 7f1fb707f700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb01034c0 0x7f1fb019ae10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000d980 tx=0x7f1fa000dc90 comp rx=0 tx=0).stop 2024-05-09T00:19:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.998+0000 7f1fb707f700 1 -- 172.21.15.46:0/3718323194 shutdown_connections 2024-05-09T00:19:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.998+0000 7f1fb707f700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1f980b2880 0x7f1f980b4d90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.998+0000 7f1fb707f700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fb0108740 0x7f1fb019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.998+0000 7f1fb707f700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb01034c0 0x7f1fb019ae10 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.998+0000 7f1fb707f700 1 --2- 172.21.15.46:0/3718323194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fb0107df0 0x7f1fb019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.998+0000 7f1fb707f700 1 -- 172.21.15.46:0/3718323194 >> 172.21.15.46:0/3718323194 conn(0x7f1fb00fedf0 msgr2=0x7f1fb01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.998+0000 7f1fb707f700 1 -- 172.21.15.46:0/3718323194 shutdown_connections 2024-05-09T00:19:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:26.998+0000 7f1fb707f700 1 -- 172.21.15.46:0/3718323194 wait complete. 2024-05-09T00:19:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:19:27.769 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:27 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3718323194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:19:27.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:19:27.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:27 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3718323194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:19:27.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:27 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3718323194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:19:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:28.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:28 smithi046 ceph-mon[30151]: pgmap v619: 97 pgs: 97 active+clean; 4.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:19:28.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:28 smithi169 ceph-mon[34777]: pgmap v619: 97 pgs: 97 active+clean; 4.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:19:28.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:28 smithi142 ceph-mon[34533]: pgmap v619: 97 pgs: 97 active+clean; 4.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:19:29.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.952+0000 7f4e1256a700 1 -- 172.21.15.46:0/2242937280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107df0 msgr2=0x7f4e0c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:29.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.952+0000 7f4e1256a700 1 --2- 172.21.15.46:0/2242937280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107df0 0x7f4e0c108200 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4e000099c0 tx=0x7f4e00009cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.952+0000 7f4e1256a700 1 -- 172.21.15.46:0/2242937280 shutdown_connections 2024-05-09T00:19:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.952+0000 7f4e1256a700 1 --2- 172.21.15.46:0/2242937280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c108740 0x7f4e0c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.952+0000 7f4e1256a700 1 --2- 172.21.15.46:0/2242937280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e0c1034c0 0x7f4e0c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.952+0000 7f4e1256a700 1 --2- 172.21.15.46:0/2242937280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107df0 0x7f4e0c108200 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.952+0000 7f4e1256a700 1 -- 172.21.15.46:0/2242937280 >> 172.21.15.46:0/2242937280 conn(0x7f4e0c0fedf0 msgr2=0x7f4e0c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.952+0000 7f4e1256a700 1 -- 172.21.15.46:0/2242937280 shutdown_connections 2024-05-09T00:19:29.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.952+0000 7f4e1256a700 1 -- 172.21.15.46:0/2242937280 wait complete. 2024-05-09T00:19:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.953+0000 7f4e1256a700 1 Processor -- start 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.953+0000 7f4e1256a700 1 -- start start 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.953+0000 7f4e1256a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c1034c0 0x7f4e0c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e1256a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e0c107df0 0x7f4e0c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e1256a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c108740 0x7f4e0c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e1256a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c113350 con 0x7f4e0c1034c0 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e1256a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c1131d0 con 0x7f4e0c107df0 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e1256a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c1134d0 con 0x7f4e0c108740 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e0bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c1034c0 0x7f4e0c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e0bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c1034c0 0x7f4e0c19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58436/0 (socket says 172.21.15.46:58436) 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e0bfff700 1 -- 172.21.15.46:0/3353506677 learned_addr learned my addr 172.21.15.46:0/3353506677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e10b07700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c108740 0x7f4e0c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e0b7fe700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e0c107df0 0x7f4e0c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e0bfff700 1 -- 172.21.15.46:0/3353506677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c108740 msgr2=0x7f4e0c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e0bfff700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c108740 0x7f4e0c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e0bfff700 1 -- 172.21.15.46:0/3353506677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e0c107df0 msgr2=0x7f4e0c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e0bfff700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e0c107df0 0x7f4e0c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e0bfff700 1 -- 172.21.15.46:0/3353506677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0c19fce0 con 0x7f4e0c1034c0 2024-05-09T00:19:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e0bfff700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c1034c0 0x7f4e0c19ae10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00e9a0 tx=0x7f4dfc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e097fa700 1 -- 172.21.15.46:0/3353506677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00cbd0 con 0x7f4e0c1034c0 2024-05-09T00:19:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.954+0000 7f4e1256a700 1 -- 172.21.15.46:0/3353506677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0c19ffd0 con 0x7f4e0c1034c0 2024-05-09T00:19:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.955+0000 7f4e097fa700 1 -- 172.21.15.46:0/3353506677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dfc00cd70 con 0x7f4e0c1034c0 2024-05-09T00:19:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.955+0000 7f4e097fa700 1 -- 172.21.15.46:0/3353506677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc018680 con 0x7f4e0c1034c0 2024-05-09T00:19:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.955+0000 7f4e1256a700 1 -- 172.21.15.46:0/3353506677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0c1a78f0 con 0x7f4e0c1034c0 2024-05-09T00:19:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.957+0000 7f4e097fa700 1 -- 172.21.15.46:0/3353506677 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4dfc018820 con 0x7f4e0c1034c0 2024-05-09T00:19:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.958+0000 7f4e097fa700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4df40b2880 0x7f4df40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.958+0000 7f4e0b7fe700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4df40b2880 0x7f4df40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.958+0000 7f4e097fa700 1 -- 172.21.15.46:0/3353506677 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4dfc0e24f0 con 0x7f4e0c1034c0 2024-05-09T00:19:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.958+0000 7f4e20f80b80 1 Processor -- start 2024-05-09T00:19:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.958+0000 7f4e20f80b80 1 -- start start 2024-05-09T00:19:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e0b7fe700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4df40b2880 0x7f4df40b4d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4e00000c00 tx=0x7f4e0001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e20f80b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8ead00 0x5608dd9f3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e20f80b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9f3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e20f80b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 0x5608dd9ed810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e20f80b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5608dd9100a0 con 0x5608dd9ed400 2024-05-09T00:19:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e20f80b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5608dd8b59a0 con 0x5608dd8ead00 2024-05-09T00:19:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e20f80b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5608dd90c060 con 0x5608dd8704a0 2024-05-09T00:19:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e10b07700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 0x5608dd9ed810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e10b07700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 0x5608dd9ed810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58450/0 (socket says 172.21.15.46:58450) 2024-05-09T00:19:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e0b7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e10b07700 1 -- 172.21.15.46:0/2587277954 learned_addr learned my addr 172.21.15.46:0/2587277954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e0bfff700 1 --2- 172.21.15.46:0/2587277954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8ead00 0x5608dd9f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e10b07700 1 -- 172.21.15.46:0/2587277954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 msgr2=0x5608dd9f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e10b07700 1 --2- 172.21.15.46:0/2587277954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e10b07700 1 -- 172.21.15.46:0/2587277954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8ead00 msgr2=0x5608dd9f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e10b07700 1 --2- 172.21.15.46:0/2587277954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8ead00 0x5608dd9f3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e10b07700 1 -- 172.21.15.46:0/2587277954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608dd8a6500 con 0x5608dd9ed400 2024-05-09T00:19:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.959+0000 7f4e10b07700 1 --2- 172.21.15.46:0/2587277954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 0x5608dd9ed810 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4e04003fa0 tx=0x7f4e04009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44f3751e8385df8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.960+0000 7f4df27fc700 1 -- 172.21.15.46:0/2587277954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04004bb0 con 0x5608dd9ed400 2024-05-09T00:19:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.960+0000 7f4df27fc700 1 -- 172.21.15.46:0/2587277954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e04004d50 con 0x5608dd9ed400 2024-05-09T00:19:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.960+0000 7f4df27fc700 1 -- 172.21.15.46:0/2587277954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04007d20 con 0x5608dd9ed400 2024-05-09T00:19:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.961+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2587277954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 msgr2=0x5608dd9ed810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.961+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/2587277954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 0x5608dd9ed810 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4e04003fa0 tx=0x7f4e04009ba0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.961+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2587277954 shutdown_connections 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.961+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/2587277954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 0x5608dd9ed810 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.961+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/2587277954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8ead00 0x5608dd9f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.961+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/2587277954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.961+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2587277954 >> 172.21.15.46:0/2587277954 conn(0x5608dd873f70 msgr2=0x5608dd9e8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.961+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2587277954 shutdown_connections 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.961+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2587277954 wait complete. 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.961+0000 7f4e20f80b80 1 -- start start 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.961+0000 7f4e20f80b80 1 Processor -- start 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e20f80b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e20f80b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8ead00 0x5608dd9f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e20f80b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 0x5608dd9f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e20f80b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5608dd9ee160 con 0x5608dd9ed400 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e20f80b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5608dd8aab20 con 0x5608dd8ead00 2024-05-09T00:19:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e20f80b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5608dd9fa020 con 0x5608dd8704a0 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e20f80b80 10 client.0 _ll_register_callbacks cb 0x5608dd7816c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e10b07700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 0x5608dd9f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e10b07700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 0x5608dd9f7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58452/0 (socket says 172.21.15.46:58452) 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e10b07700 1 -- 172.21.15.46:0/1660993542 learned_addr learned my addr 172.21.15.46:0/1660993542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e0bfff700 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e0b7fe700 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8ead00 0x5608dd9f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e10b07700 1 -- 172.21.15.46:0/1660993542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 msgr2=0x5608dd9f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e10b07700 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e10b07700 1 -- 172.21.15.46:0/1660993542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8ead00 msgr2=0x5608dd9f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e10b07700 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8ead00 0x5608dd9f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e10b07700 1 -- 172.21.15.46:0/1660993542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608dd78b490 con 0x5608dd9ed400 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4e10b07700 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 0x5608dd9f7b10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004730 tx=0x7f4e04004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518a04aa3249b160 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.962+0000 7f4deb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4deb7fe700 1 -- 172.21.15.46:0/1660993542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04022070 con 0x5608dd9ed400 2024-05-09T00:19:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4deb7fe700 1 -- 172.21.15.46:0/1660993542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4e04007bb0 con 0x5608dd9ed400 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4e20f80b80 1 -- 172.21.15.46:0/1660993542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e04004d50 con 0x5608dd9ed400 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4deb7fe700 1 -- 172.21.15.46:0/1660993542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4e0402e050 con 0x5608dd9ed400 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4e20f80b80 10 client.16662 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4e20f80b80 1 -- 172.21.15.46:0/1660993542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608dda04a60 con 0x5608dd9ed400 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4deb7fe700 1 -- 172.21.15.46:0/1660993542 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e04004d50 con 0x5608dd9ed400 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4deb7fe700 1 -- 172.21.15.46:0/1660993542 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4e0401e410 con 0x5608dd9ed400 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4e20f80b80 10 client.16662 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4e20f80b80 10 client.16662 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4e20f80b80 20 client.16662 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5608dda06f90 0x5608dda093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4e20f80b80 4 client.16662 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.963+0000 7f4e20f80b80 1 -- 172.21.15.46:0/1660993542 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608dd91da80 con 0x5608dda06f90 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.964+0000 7f4e0bfff700 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5608dda06f90 0x5608dda093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.964+0000 7f4e0bfff700 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5608dda06f90 0x5608dda093b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4883af2758b2e740 server_cookie=4564903d04e23669 in_seq=0 out_seq=0 2024-05-09T00:19:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.964+0000 7f4deb7fe700 10 client.16662 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4deb7fe700 1 -- 172.21.15.46:0/1660993542 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5608dd91da80 con 0x5608dda06f90 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4deb7fe700 10 client.16662 handle_command_reply: tid=0 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4e20f80b80 1 client.16662 shutdown 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4e20f80b80 1 -- 172.21.15.46:0/1660993542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 msgr2=0x5608dd9f7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 0x5608dd9f7b10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004730 tx=0x7f4e04004760 comp rx=0 tx=0).stop 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4e20f80b80 1 -- 172.21.15.46:0/1660993542 shutdown_connections 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5608dda06f90 0x5608dda093b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd9ed400 0x5608dd9f7b10 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4deb7fe700 0 client.16662 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/1660993542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8ead00 0x5608dd9f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4e20f80b80 1 -- 172.21.15.46:0/1660993542 >> 172.21.15.46:0/1660993542 conn(0x5608dd873f70 msgr2=0x5608dd9e1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4e20f80b80 1 -- 172.21.15.46:0/1660993542 shutdown_connections 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.965+0000 7f4e20f80b80 1 -- 172.21.15.46:0/1660993542 wait complete. 2024-05-09T00:19:29.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.966+0000 7f4e20f80b80 20 client.16662 trim_cache size 0 max 16384 2024-05-09T00:19:29.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e20f80b80 1 Processor -- start 2024-05-09T00:19:29.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e20f80b80 1 -- start start 2024-05-09T00:19:29.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e20f80b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8704a0 0x5608dd9f2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e20f80b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 0x5608dd9f7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e20f80b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd9ed400 0x5608dd9e1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e20f80b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c1b15e0 con 0x5608dd8ead00 2024-05-09T00:19:29.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e20f80b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c1aee70 con 0x5608dd8704a0 2024-05-09T00:19:29.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e20f80b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c1acca0 con 0x5608dd9ed400 2024-05-09T00:19:29.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e0b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 0x5608dd9f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e0b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 0x5608dd9f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58462/0 (socket says 172.21.15.46:58462) 2024-05-09T00:19:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e0b7fe700 1 -- 172.21.15.46:0/3576117322 learned_addr learned my addr 172.21.15.46:0/3576117322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e10b07700 1 --2- 172.21.15.46:0/3576117322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd9ed400 0x5608dd9e1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e0bfff700 1 --2- 172.21.15.46:0/3576117322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8704a0 0x5608dd9f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e0b7fe700 1 -- 172.21.15.46:0/3576117322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd9ed400 msgr2=0x5608dd9e1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e0b7fe700 1 --2- 172.21.15.46:0/3576117322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd9ed400 0x5608dd9e1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e0b7fe700 1 -- 172.21.15.46:0/3576117322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8704a0 msgr2=0x5608dd9f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e0b7fe700 1 --2- 172.21.15.46:0/3576117322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8704a0 0x5608dd9f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.981+0000 7f4e0b7fe700 1 -- 172.21.15.46:0/3576117322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608dd9f5620 con 0x5608dd8ead00 2024-05-09T00:19:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.982+0000 7f4e0b7fe700 1 --2- 172.21.15.46:0/3576117322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 0x5608dd9f7620 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4e000099c0 tx=0x7f4e0001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865f5eab250533ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.982+0000 7f4df1ffb700 1 -- 172.21.15.46:0/3576117322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e000056c0 con 0x5608dd8ead00 2024-05-09T00:19:29.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.982+0000 7f4df1ffb700 1 -- 172.21.15.46:0/3576117322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e00035070 con 0x5608dd8ead00 2024-05-09T00:19:29.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.982+0000 7f4df1ffb700 1 -- 172.21.15.46:0/3576117322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00030c50 con 0x5608dd8ead00 2024-05-09T00:19:29.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.983+0000 7f4e20f80b80 1 -- 172.21.15.46:0/3576117322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 msgr2=0x5608dd9f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:29.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.983+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/3576117322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 0x5608dd9f7620 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4e000099c0 tx=0x7f4e0001e000 comp rx=0 tx=0).stop 2024-05-09T00:19:29.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.983+0000 7f4e20f80b80 1 -- 172.21.15.46:0/3576117322 shutdown_connections 2024-05-09T00:19:29.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.983+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/3576117322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd9ed400 0x5608dd9e1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.983+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/3576117322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd8704a0 0x5608dd9f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.983+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/3576117322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 0x5608dd9f7620 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.983+0000 7f4e20f80b80 1 -- 172.21.15.46:0/3576117322 >> 172.21.15.46:0/3576117322 conn(0x5608dd873f70 msgr2=0x5608dd9f2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:29.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.983+0000 7f4e20f80b80 1 -- 172.21.15.46:0/3576117322 shutdown_connections 2024-05-09T00:19:29.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.983+0000 7f4e20f80b80 1 -- 172.21.15.46:0/3576117322 wait complete. 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.983+0000 7f4e20f80b80 1 -- start start 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.983+0000 7f4e20f80b80 1 Processor -- start 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e20f80b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e20f80b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 0x5608dda0df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e20f80b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd9ed400 0x5608dda0e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e20f80b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c1ad330 con 0x5608dd8ead00 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e20f80b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c1ae7a0 con 0x5608dd9ed400 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e20f80b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c1ada00 con 0x5608dd8704a0 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e20f80b80 10 client.0 _ll_register_callbacks cb 0x5608dd9edea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e0b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 0x5608dda0df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e0b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 0x5608dda0df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58474/0 (socket says 172.21.15.46:58474) 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e0b7fe700 1 -- 172.21.15.46:0/2689093647 learned_addr learned my addr 172.21.15.46:0/2689093647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e0bfff700 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e0b7fe700 1 -- 172.21.15.46:0/2689093647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 msgr2=0x5608dd9ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e0b7fe700 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9ff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e0b7fe700 1 -- 172.21.15.46:0/2689093647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd9ed400 msgr2=0x5608dda0e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e10b07700 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd9ed400 0x5608dda0e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e0b7fe700 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd9ed400 0x5608dda0e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e0b7fe700 1 -- 172.21.15.46:0/2689093647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608dda0ebe0 con 0x5608dd8ead00 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e0bfff700 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4e0b7fe700 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 0x5608dda0df20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4e00030f00 tx=0x7f4e00030470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0d0d379f89e800 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.984+0000 7f4deb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4deb7fe700 1 -- 172.21.15.46:0/2689093647 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0003d070 con 0x5608dd8ead00 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4deb7fe700 1 -- 172.21.15.46:0/2689093647 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4e0000bba0 con 0x5608dd8ead00 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2689093647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e00035070 con 0x5608dd8ead00 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4deb7fe700 1 -- 172.21.15.46:0/2689093647 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4e00049050 con 0x5608dd8ead00 2024-05-09T00:19:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4e20f80b80 10 client.16668 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2689093647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608dda75820 con 0x5608dd8ead00 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4deb7fe700 1 -- 172.21.15.46:0/2689093647 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e00035070 con 0x5608dd8ead00 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4deb7fe700 1 -- 172.21.15.46:0/2689093647 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4e0000fce0 con 0x5608dd8ead00 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4e20f80b80 10 client.16668 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4e20f80b80 10 client.16668 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4e20f80b80 20 client.16668 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5608dda75e10 0x5608dda78230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4e20f80b80 4 client.16668 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.985+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2689093647 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608dd917b20 con 0x5608dda75e10 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4e0bfff700 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5608dda75e10 0x5608dda78230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4deb7fe700 10 client.16668 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4e0bfff700 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5608dda75e10 0x5608dda78230 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be68d74ab07c6d2f server_cookie=d4a0990b24c048c5 in_seq=0 out_seq=0 2024-05-09T00:19:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4deb7fe700 1 -- 172.21.15.46:0/2689093647 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608dd917b20 con 0x5608dda75e10 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4deb7fe700 10 client.16668 handle_command_reply: tid=0 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4e20f80b80 1 client.16668 shutdown 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2689093647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 msgr2=0x5608dda0df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 0x5608dda0df20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4e00030f00 tx=0x7f4e00030470 comp rx=0 tx=0).stop 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2689093647 shutdown_connections 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x5608dda75e10 0x5608dda78230 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608dd9ed400 0x5608dda0e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608dd8704a0 0x5608dd9ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4deb7fe700 0 client.16668 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4e20f80b80 1 --2- 172.21.15.46:0/2689093647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608dd8ead00 0x5608dda0df20 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.986+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2689093647 >> 172.21.15.46:0/2689093647 conn(0x5608dd873f70 msgr2=0x5608dd9e1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2689093647 shutdown_connections 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e20f80b80 1 -- 172.21.15.46:0/2689093647 wait complete. 2024-05-09T00:19:29.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e20f80b80 20 client.16668 trim_cache size 0 max 16384 2024-05-09T00:19:29.992 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:19:29.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e1256a700 1 -- 172.21.15.46:0/3353506677 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4df40b2880 msgr2=0x7f4df40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:29.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e1256a700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4df40b2880 0x7f4df40b4d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4e00000c00 tx=0x7f4e0001a040 comp rx=0 tx=0).stop 2024-05-09T00:19:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e1256a700 1 -- 172.21.15.46:0/3353506677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c1034c0 msgr2=0x7f4e0c19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e1256a700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c1034c0 0x7f4e0c19ae10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00e9a0 tx=0x7f4dfc00ecb0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e1256a700 1 -- 172.21.15.46:0/3353506677 shutdown_connections 2024-05-09T00:19:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e1256a700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4df40b2880 0x7f4df40b4d90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e1256a700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e0c108740 0x7f4e0c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e1256a700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c1034c0 0x7f4e0c19ae10 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e1256a700 1 --2- 172.21.15.46:0/3353506677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e0c107df0 0x7f4e0c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.987+0000 7f4e1256a700 1 -- 172.21.15.46:0/3353506677 >> 172.21.15.46:0/3353506677 conn(0x7f4e0c0fedf0 msgr2=0x7f4e0c075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.988+0000 7f4e1256a700 1 -- 172.21.15.46:0/3353506677 shutdown_connections 2024-05-09T00:19:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:29.988+0000 7f4e1256a700 1 -- 172.21.15.46:0/3353506677 wait complete. 2024-05-09T00:19:30.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:30 smithi046 ceph-mon[30151]: pgmap v620: 97 pgs: 97 active+clean; 4.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-05-09T00:19:30.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:19:30.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:30 smithi169 ceph-mon[34777]: pgmap v620: 97 pgs: 97 active+clean; 4.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-05-09T00:19:30.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:30 smithi142 ceph-mon[34533]: pgmap v620: 97 pgs: 97 active+clean; 4.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-05-09T00:19:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:32.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:32 smithi046 ceph-mon[30151]: pgmap v621: 97 pgs: 97 active+clean; 4.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-05-09T00:19:32.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.815+0000 7f756f59e700 1 -- 172.21.15.46:0/1509473698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7570109150 msgr2=0x7f7570109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:32.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.815+0000 7f756f59e700 1 --2- 172.21.15.46:0/1509473698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7570109150 0x7f7570109560 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f756000b0a0 tx=0x7f756000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.815+0000 7f756f59e700 1 -- 172.21.15.46:0/1509473698 shutdown_connections 2024-05-09T00:19:32.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.815+0000 7f756f59e700 1 --2- 172.21.15.46:0/1509473698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7570102f30 0x7f7570110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.815+0000 7f756f59e700 1 --2- 172.21.15.46:0/1509473698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7570109150 0x7f7570109560 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.815+0000 7f756f59e700 1 --2- 172.21.15.46:0/1509473698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75701025e0 0x7f75701029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.815+0000 7f756f59e700 1 -- 172.21.15.46:0/1509473698 >> 172.21.15.46:0/1509473698 conn(0x7f7570078370 msgr2=0x7f75700787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:32.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.815+0000 7f756f59e700 1 -- 172.21.15.46:0/1509473698 shutdown_connections 2024-05-09T00:19:32.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.815+0000 7f756f59e700 1 -- 172.21.15.46:0/1509473698 wait complete. 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.816+0000 7f756f59e700 1 Processor -- start 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.816+0000 7f756f59e700 1 -- start start 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.816+0000 7f756f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75701025e0 0x7f757019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.816+0000 7f756f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7570102f30 0x7f757019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.816+0000 7f756f59e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7570109150 0x7f757019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.816+0000 7f756f59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7570114450 con 0x7f7570102f30 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.816+0000 7f756f59e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75701142d0 con 0x7f7570109150 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.816+0000 7f756f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f75701145d0 con 0x7f75701025e0 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.816+0000 7f756dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7570102f30 0x7f757019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.816+0000 7f756dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7570102f30 0x7f757019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58482/0 (socket says 172.21.15.46:58482) 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.816+0000 7f756dd9b700 1 -- 172.21.15.46:0/3030115839 learned_addr learned my addr 172.21.15.46:0/3030115839 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f756ed9d700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7570109150 0x7f757019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f756e59c700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75701025e0 0x7f757019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f756dd9b700 1 -- 172.21.15.46:0/3030115839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75701025e0 msgr2=0x7f757019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f756dd9b700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75701025e0 0x7f757019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f756dd9b700 1 -- 172.21.15.46:0/3030115839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7570109150 msgr2=0x7f757019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f756dd9b700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7570109150 0x7f757019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f756dd9b700 1 -- 172.21.15.46:0/3030115839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757019fc60 con 0x7f7570102f30 2024-05-09T00:19:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f756e59c700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75701025e0 0x7f757019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f756dd9b700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7570102f30 0x7f757019b380 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f755800b500 tx=0x7f755800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f75677fe700 1 -- 172.21.15.46:0/3030115839 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558015020 con 0x7f7570102f30 2024-05-09T00:19:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f756f59e700 1 -- 172.21.15.46:0/3030115839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757019ff50 con 0x7f7570102f30 2024-05-09T00:19:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f75677fe700 1 -- 172.21.15.46:0/3030115839 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7558011ce0 con 0x7f7570102f30 2024-05-09T00:19:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f75677fe700 1 -- 172.21.15.46:0/3030115839 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558010d40 con 0x7f7570102f30 2024-05-09T00:19:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.817+0000 7f756f59e700 1 -- 172.21.15.46:0/3030115839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701a7870 con 0x7f7570102f30 2024-05-09T00:19:32.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.821+0000 7f75677fe700 1 -- 172.21.15.46:0/3030115839 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7558018020 con 0x7f7570102f30 2024-05-09T00:19:32.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.822+0000 7f75677fe700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f755c0b2880 0x7f755c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.822+0000 7f7583817b80 1 Processor -- start 2024-05-09T00:19:32.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.822+0000 7f756e59c700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f755c0b2880 0x7f755c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.822+0000 7f75677fe700 1 -- 172.21.15.46:0/3030115839 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f75580e19f0 con 0x7f7570102f30 2024-05-09T00:19:32.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.822+0000 7f7583817b80 1 -- start start 2024-05-09T00:19:32.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.822+0000 7f7583817b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74c07160 0x564e74ca4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.822+0000 7f7583817b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74b1d4a0 0x564e74ca4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f7583817b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 0x564e74c9f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f7583817b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564e74bbcff0 con 0x564e74c9ec10 2024-05-09T00:19:32.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f7583817b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564e74b5b7f0 con 0x564e74b1d4a0 2024-05-09T00:19:32.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f7583817b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564e74bb8fb0 con 0x564e74c07160 2024-05-09T00:19:32.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756e59c700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f755c0b2880 0x7f755c0b4d90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7560009fd0 tx=0x7f75600098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:32.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 0x564e74c9f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 0x564e74c9f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58494/0 (socket says 172.21.15.46:58494) 2024-05-09T00:19:32.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756ed9d700 1 -- 172.21.15.46:0/748076406 learned_addr learned my addr 172.21.15.46:0/748076406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:32.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756dd9b700 1 --2- 172.21.15.46:0/748076406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74b1d4a0 0x564e74ca4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756e59c700 1 --2- 172.21.15.46:0/748076406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74c07160 0x564e74ca4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756ed9d700 1 -- 172.21.15.46:0/748076406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74c07160 msgr2=0x564e74ca4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:32.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756ed9d700 1 --2- 172.21.15.46:0/748076406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74c07160 0x564e74ca4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756ed9d700 1 -- 172.21.15.46:0/748076406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74b1d4a0 msgr2=0x564e74ca4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:32.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756ed9d700 1 --2- 172.21.15.46:0/748076406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74b1d4a0 0x564e74ca4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756ed9d700 1 -- 172.21.15.46:0/748076406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e74b53370 con 0x564e74c9ec10 2024-05-09T00:19:32.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756dd9b700 1 --2- 172.21.15.46:0/748076406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74b1d4a0 0x564e74ca4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:19:32.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.823+0000 7f756ed9d700 1 --2- 172.21.15.46:0/748076406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 0x564e74c9f020 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f756800ac50 tx=0x7f756800af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1022aaf6e782229a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:32.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.824+0000 7f7564ff9700 1 -- 172.21.15.46:0/748076406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756800fdb0 con 0x564e74c9ec10 2024-05-09T00:19:32.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.824+0000 7f7564ff9700 1 -- 172.21.15.46:0/748076406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75680044d0 con 0x564e74c9ec10 2024-05-09T00:19:32.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.824+0000 7f7564ff9700 1 -- 172.21.15.46:0/748076406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568018640 con 0x564e74c9ec10 2024-05-09T00:19:32.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.825+0000 7f7583817b80 1 -- 172.21.15.46:0/748076406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 msgr2=0x564e74c9f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:32.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.825+0000 7f7583817b80 1 --2- 172.21.15.46:0/748076406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 0x564e74c9f020 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f756800ac50 tx=0x7f756800af60 comp rx=0 tx=0).stop 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.825+0000 7f7583817b80 1 -- 172.21.15.46:0/748076406 shutdown_connections 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.825+0000 7f7583817b80 1 --2- 172.21.15.46:0/748076406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 0x564e74c9f020 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.825+0000 7f7583817b80 1 --2- 172.21.15.46:0/748076406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74c07160 0x564e74ca4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.825+0000 7f7583817b80 1 --2- 172.21.15.46:0/748076406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74b1d4a0 0x564e74ca4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.825+0000 7f7583817b80 1 -- 172.21.15.46:0/748076406 >> 172.21.15.46:0/748076406 conn(0x564e74b7d6a0 msgr2=0x564e74c95620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.825+0000 7f7583817b80 1 -- 172.21.15.46:0/748076406 shutdown_connections 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.825+0000 7f7583817b80 1 -- 172.21.15.46:0/748076406 wait complete. 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.825+0000 7f7583817b80 1 -- start start 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.825+0000 7f7583817b80 1 Processor -- start 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f7583817b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74b1d4a0 0x564e74ca8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f7583817b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 0x564e74ca8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f7583817b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74c9ec10 0x564e74ca91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f7583817b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564e74a38ba0 con 0x564e74c07160 2024-05-09T00:19:32.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f7583817b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564e74b77150 con 0x564e74c9ec10 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f7583817b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564e74c9fa00 con 0x564e74b1d4a0 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f7583817b80 10 client.0 _ll_register_callbacks cb 0x564e74a2e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f756dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 0x564e74ca8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f756dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 0x564e74ca8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58504/0 (socket says 172.21.15.46:58504) 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f756dd9b700 1 -- 172.21.15.46:0/1010335107 learned_addr learned my addr 172.21.15.46:0/1010335107 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f756e59c700 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74b1d4a0 0x564e74ca8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f756ed9d700 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74c9ec10 0x564e74ca91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f756dd9b700 1 -- 172.21.15.46:0/1010335107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74b1d4a0 msgr2=0x564e74ca8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f756dd9b700 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74b1d4a0 0x564e74ca8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f756dd9b700 1 -- 172.21.15.46:0/1010335107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74c9ec10 msgr2=0x564e74ca91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f756dd9b700 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74c9ec10 0x564e74ca91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f756dd9b700 1 -- 172.21.15.46:0/1010335107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e74a38490 con 0x564e74c07160 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f756dd9b700 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 0x564e74ca8ca0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f755800bc10 tx=0x7f7558010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef42488d144ce750 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.826+0000 7f7555ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7555ffb700 1 -- 172.21.15.46:0/1010335107 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580c2020 con 0x564e74c07160 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7555ffb700 1 -- 172.21.15.46:0/1010335107 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f75580b1bb0 con 0x564e74c07160 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7583817b80 1 -- 172.21.15.46:0/1010335107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75680044d0 con 0x564e74c07160 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7555ffb700 1 -- 172.21.15.46:0/1010335107 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75580a8fd0 con 0x564e74c07160 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7583817b80 10 client.16680 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7583817b80 1 -- 172.21.15.46:0/1010335107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e74cb60a0 con 0x564e74c07160 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7555ffb700 1 -- 172.21.15.46:0/1010335107 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75680044d0 con 0x564e74c07160 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7555ffb700 1 -- 172.21.15.46:0/1010335107 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f75580a4940 con 0x564e74c07160 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7583817b80 10 client.16680 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7583817b80 10 client.16680 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7583817b80 20 client.16680 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7583817b80 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564e74cb86e0 0x564e74cbab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7583817b80 4 client.16680 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.827+0000 7f7583817b80 1 -- 172.21.15.46:0/1010335107 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e74bca9d0 con 0x564e74cb86e0 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.828+0000 7f756e59c700 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564e74cb86e0 0x564e74cbab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.828+0000 7f7555ffb700 10 client.16680 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.828+0000 7f756e59c700 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564e74cb86e0 0x564e74cbab50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6cc16176e4c7c933 server_cookie=57084626644b5a80 in_seq=0 out_seq=0 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7555ffb700 1 -- 172.21.15.46:0/1010335107 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564e74bca9d0 con 0x564e74cb86e0 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7555ffb700 10 client.16680 handle_command_reply: tid=0 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7583817b80 1 client.16680 shutdown 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7583817b80 1 -- 172.21.15.46:0/1010335107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 msgr2=0x564e74ca8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7583817b80 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 0x564e74ca8ca0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f755800bc10 tx=0x7f7558010640 comp rx=0 tx=0).stop 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7583817b80 1 -- 172.21.15.46:0/1010335107 shutdown_connections 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7583817b80 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564e74cb86e0 0x564e74cbab50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7555ffb700 0 client.16680 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7583817b80 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74c9ec10 0x564e74ca91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7583817b80 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74b1d4a0 0x564e74ca8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7583817b80 1 --2- 172.21.15.46:0/1010335107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 0x564e74ca8ca0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7583817b80 1 -- 172.21.15.46:0/1010335107 >> 172.21.15.46:0/1010335107 conn(0x564e74b7d6a0 msgr2=0x564e74c95380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:32.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7583817b80 1 -- 172.21.15.46:0/1010335107 shutdown_connections 2024-05-09T00:19:32.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7583817b80 1 -- 172.21.15.46:0/1010335107 wait complete. 2024-05-09T00:19:32.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.829+0000 7f7583817b80 20 client.16680 trim_cache size 0 max 16384 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f7583817b80 1 Processor -- start 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f7583817b80 1 -- start start 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f7583817b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74b1d4a0 0x564e74c9a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f7583817b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74c07160 0x564e74ca9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f7583817b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 0x564e74ca2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f7583817b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75701af290 con 0x564e74c9ec10 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f7583817b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75701acb20 con 0x564e74b1d4a0 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f7583817b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f75701aa950 con 0x564e74c07160 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f756ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 0x564e74ca2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f756ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 0x564e74ca2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58510/0 (socket says 172.21.15.46:58510) 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f756ed9d700 1 -- 172.21.15.46:0/3780539992 learned_addr learned my addr 172.21.15.46:0/3780539992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f756dd9b700 1 --2- 172.21.15.46:0/3780539992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74c07160 0x564e74ca9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f756e59c700 1 --2- 172.21.15.46:0/3780539992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74b1d4a0 0x564e74c9a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f756ed9d700 1 -- 172.21.15.46:0/3780539992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74c07160 msgr2=0x564e74ca9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f756ed9d700 1 --2- 172.21.15.46:0/3780539992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74c07160 0x564e74ca9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f756ed9d700 1 -- 172.21.15.46:0/3780539992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74b1d4a0 msgr2=0x564e74c9a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f756ed9d700 1 --2- 172.21.15.46:0/3780539992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74b1d4a0 0x564e74c9a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f756ed9d700 1 -- 172.21.15.46:0/3780539992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e74c91b60 con 0x564e74c9ec10 2024-05-09T00:19:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f756e59c700 1 --2- 172.21.15.46:0/3780539992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74b1d4a0 0x564e74c9a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:19:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.845+0000 7f756dd9b700 1 --2- 172.21.15.46:0/3780539992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74c07160 0x564e74ca9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.846+0000 7f756ed9d700 1 --2- 172.21.15.46:0/3780539992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 0x564e74ca2b90 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f756800a500 tx=0x7f756800a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473c3256e52a61d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.846+0000 7f7557fff700 1 -- 172.21.15.46:0/3780539992 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568027070 con 0x564e74c9ec10 2024-05-09T00:19:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.846+0000 7f7557fff700 1 -- 172.21.15.46:0/3780539992 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7568004710 con 0x564e74c9ec10 2024-05-09T00:19:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.846+0000 7f7557fff700 1 -- 172.21.15.46:0/3780539992 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75680052a0 con 0x564e74c9ec10 2024-05-09T00:19:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.846+0000 7f7583817b80 1 -- 172.21.15.46:0/3780539992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 msgr2=0x564e74ca2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.846+0000 7f7583817b80 1 --2- 172.21.15.46:0/3780539992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 0x564e74ca2b90 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f756800a500 tx=0x7f756800a840 comp rx=0 tx=0).stop 2024-05-09T00:19:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 -- 172.21.15.46:0/3780539992 shutdown_connections 2024-05-09T00:19:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 --2- 172.21.15.46:0/3780539992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c9ec10 0x564e74ca2b90 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 --2- 172.21.15.46:0/3780539992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74b1d4a0 0x564e74c9a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 --2- 172.21.15.46:0/3780539992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74c07160 0x564e74ca9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 -- 172.21.15.46:0/3780539992 >> 172.21.15.46:0/3780539992 conn(0x564e74b7d6a0 msgr2=0x564e74ca2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 -- 172.21.15.46:0/3780539992 shutdown_connections 2024-05-09T00:19:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 -- 172.21.15.46:0/3780539992 wait complete. 2024-05-09T00:19:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 -- start start 2024-05-09T00:19:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 Processor -- start 2024-05-09T00:19:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74b1d4a0 0x564e74cb08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 0x564e74cb0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74c9ec10 0x564e74cbf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75701aafe0 con 0x564e74c07160 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75701ac450 con 0x564e74c9ec10 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f75701ab6b0 con 0x564e74b1d4a0 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.847+0000 7f7583817b80 10 client.0 _ll_register_callbacks cb 0x564e74a2e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f756dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 0x564e74cb0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f756dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 0x564e74cb0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58518/0 (socket says 172.21.15.46:58518) 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f756dd9b700 1 -- 172.21.15.46:0/3038066152 learned_addr learned my addr 172.21.15.46:0/3038066152 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f756ed9d700 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74c9ec10 0x564e74cbf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f756e59c700 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74b1d4a0 0x564e74cb08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f756dd9b700 1 -- 172.21.15.46:0/3038066152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74b1d4a0 msgr2=0x564e74cb08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f756dd9b700 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74b1d4a0 0x564e74cb08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f756dd9b700 1 -- 172.21.15.46:0/3038066152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74c9ec10 msgr2=0x564e74cbf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f756dd9b700 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74c9ec10 0x564e74cbf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f756dd9b700 1 -- 172.21.15.46:0/3038066152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e74cbf990 con 0x564e74c07160 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f756dd9b700 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 0x564e74cb0e30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f75580a4d90 tx=0x7f75580a8a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cca4fe0195bba0d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f7555ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f7555ffb700 1 -- 172.21.15.46:0/3038066152 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75580c2070 con 0x564e74c07160 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f7555ffb700 1 -- 172.21.15.46:0/3038066152 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f75580a8230 con 0x564e74c07160 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f7583817b80 1 -- 172.21.15.46:0/3038066152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7568004710 con 0x564e74c07160 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f7555ffb700 1 -- 172.21.15.46:0/3038066152 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75580a96d0 con 0x564e74c07160 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.848+0000 7f7583817b80 10 client.16686 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.849+0000 7f7583817b80 1 -- 172.21.15.46:0/3038066152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e74d22720 con 0x564e74c07160 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.849+0000 7f7555ffb700 1 -- 172.21.15.46:0/3038066152 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7568004710 con 0x564e74c07160 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.849+0000 7f7555ffb700 1 -- 172.21.15.46:0/3038066152 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f75580ae510 con 0x564e74c07160 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.849+0000 7f7583817b80 10 client.16686 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.849+0000 7f7583817b80 10 client.16686 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.849+0000 7f7583817b80 20 client.16686 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.849+0000 7f7583817b80 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564e74cb86e0 0x564e74d24c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.849+0000 7f7583817b80 4 client.16686 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.849+0000 7f7583817b80 1 -- 172.21.15.46:0/3038066152 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e74bc4a70 con 0x564e74cb86e0 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.849+0000 7f756e59c700 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564e74cb86e0 0x564e74d24c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7555ffb700 10 client.16686 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f756e59c700 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564e74cb86e0 0x564e74d24c50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ebebe2dde76e143 server_cookie=2ddd464cfcf06b9a in_seq=0 out_seq=0 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7555ffb700 1 -- 172.21.15.46:0/3038066152 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e74bc4a70 con 0x564e74cb86e0 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7555ffb700 10 client.16686 handle_command_reply: tid=0 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7583817b80 1 client.16686 shutdown 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7583817b80 1 -- 172.21.15.46:0/3038066152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 msgr2=0x564e74cb0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7583817b80 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 0x564e74cb0e30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f75580a4d90 tx=0x7f75580a8a50 comp rx=0 tx=0).stop 2024-05-09T00:19:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7583817b80 1 -- 172.21.15.46:0/3038066152 shutdown_connections 2024-05-09T00:19:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7583817b80 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x564e74cb86e0 0x564e74d24c50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7583817b80 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e74c9ec10 0x564e74cbf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7583817b80 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564e74b1d4a0 0x564e74cb08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7583817b80 1 --2- 172.21.15.46:0/3038066152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e74c07160 0x564e74cb0e30 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7583817b80 1 -- 172.21.15.46:0/3038066152 >> 172.21.15.46:0/3038066152 conn(0x564e74b7d6a0 msgr2=0x564e74c99f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7555ffb700 0 client.16686 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:19:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7583817b80 1 -- 172.21.15.46:0/3038066152 shutdown_connections 2024-05-09T00:19:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7583817b80 1 -- 172.21.15.46:0/3038066152 wait complete. 2024-05-09T00:19:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.850+0000 7f7583817b80 20 client.16686 trim_cache size 0 max 16384 2024-05-09T00:19:32.854 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:19:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.851+0000 7f756f59e700 1 -- 172.21.15.46:0/3030115839 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f755c0b2880 msgr2=0x7f755c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.851+0000 7f756f59e700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f755c0b2880 0x7f755c0b4d90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7560009fd0 tx=0x7f75600098a0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.851+0000 7f756f59e700 1 -- 172.21.15.46:0/3030115839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7570102f30 msgr2=0x7f757019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.851+0000 7f756f59e700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7570102f30 0x7f757019b380 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f755800b500 tx=0x7f755800b810 comp rx=0 tx=0).stop 2024-05-09T00:19:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.851+0000 7f756f59e700 1 -- 172.21.15.46:0/3030115839 shutdown_connections 2024-05-09T00:19:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.851+0000 7f756f59e700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f755c0b2880 0x7f755c0b4d90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.851+0000 7f756f59e700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7570109150 0x7f757019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.851+0000 7f756f59e700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75701025e0 0x7f757019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.851+0000 7f756f59e700 1 --2- 172.21.15.46:0/3030115839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7570102f30 0x7f757019b380 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.851+0000 7f756f59e700 1 -- 172.21.15.46:0/3030115839 >> 172.21.15.46:0/3030115839 conn(0x7f7570078370 msgr2=0x7f75700ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.851+0000 7f756f59e700 1 -- 172.21.15.46:0/3030115839 shutdown_connections 2024-05-09T00:19:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:32.851+0000 7f756f59e700 1 -- 172.21.15.46:0/3030115839 wait complete. 2024-05-09T00:19:32.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:32 smithi169 ceph-mon[34777]: pgmap v621: 97 pgs: 97 active+clean; 4.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-05-09T00:19:32.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:32 smithi142 ceph-mon[34533]: pgmap v621: 97 pgs: 97 active+clean; 4.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-05-09T00:19:33.816 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:19:34.073 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:34.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:34 smithi046 ceph-mon[30151]: pgmap v622: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-05-09T00:19:34.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:34 smithi169 ceph-mon[34777]: pgmap v622: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-05-09T00:19:34.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:34 smithi142 ceph-mon[34533]: pgmap v622: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-05-09T00:19:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:35.999+0000 7ff992aca700 1 -- 172.21.15.46:0/357831266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98c1034c0 msgr2=0x7ff98c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:35.999+0000 7ff992aca700 1 --2- 172.21.15.46:0/357831266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98c1034c0 0x7ff98c1038d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff9800099c0 tx=0x7ff980009cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:35.999+0000 7ff992aca700 1 -- 172.21.15.46:0/357831266 shutdown_connections 2024-05-09T00:19:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:35.999+0000 7ff992aca700 1 --2- 172.21.15.46:0/357831266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff98c108740 0x7ff98c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:35.999+0000 7ff992aca700 1 --2- 172.21.15.46:0/357831266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98c1034c0 0x7ff98c1038d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:35.999+0000 7ff992aca700 1 --2- 172.21.15.46:0/357831266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff98c107df0 0x7ff98c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:35.999+0000 7ff992aca700 1 -- 172.21.15.46:0/357831266 >> 172.21.15.46:0/357831266 conn(0x7ff98c0fedc0 msgr2=0x7ff98c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:35.999+0000 7ff992aca700 1 -- 172.21.15.46:0/357831266 shutdown_connections 2024-05-09T00:19:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:35.999+0000 7ff992aca700 1 -- 172.21.15.46:0/357831266 wait complete. 2024-05-09T00:19:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:35.999+0000 7ff992aca700 1 Processor -- start 2024-05-09T00:19:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:35.999+0000 7ff992aca700 1 -- start start 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff992aca700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff98c1034c0 0x7ff98c19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff992aca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98c107df0 0x7ff98c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff992aca700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff98c108740 0x7ff98c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff992aca700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c113460 con 0x7ff98c107df0 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff992aca700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c1132e0 con 0x7ff98c1034c0 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff992aca700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c1135e0 con 0x7ff98c108740 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff98bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98c107df0 0x7ff98c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff98bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98c107df0 0x7ff98c19c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58546/0 (socket says 172.21.15.46:58546) 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff98bfff700 1 -- 172.21.15.46:0/2783667092 learned_addr learned my addr 172.21.15.46:0/2783667092 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff991067700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff98c108740 0x7ff98c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff990866700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff98c1034c0 0x7ff98c19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff98bfff700 1 -- 172.21.15.46:0/2783667092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff98c108740 msgr2=0x7ff98c19c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:36.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff98bfff700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff98c108740 0x7ff98c19c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff98bfff700 1 -- 172.21.15.46:0/2783667092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff98c1034c0 msgr2=0x7ff98c19bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:36.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff98bfff700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff98c1034c0 0x7ff98c19bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff98bfff700 1 -- 172.21.15.46:0/2783667092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98c1a70a0 con 0x7ff98c107df0 2024-05-09T00:19:36.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff990866700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff98c1034c0 0x7ff98c19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:19:36.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff98bfff700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98c107df0 0x7ff98c19c2c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff97800d950 tx=0x7ff97800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:36.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.000+0000 7ff989ffb700 1 -- 172.21.15.46:0/2783667092 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978017070 con 0x7ff98c107df0 2024-05-09T00:19:36.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.001+0000 7ff989ffb700 1 -- 172.21.15.46:0/2783667092 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff97800de30 con 0x7ff98c107df0 2024-05-09T00:19:36.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.001+0000 7ff992aca700 1 -- 172.21.15.46:0/2783667092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff98c1a7390 con 0x7ff98c107df0 2024-05-09T00:19:36.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.001+0000 7ff989ffb700 1 -- 172.21.15.46:0/2783667092 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978005440 con 0x7ff98c107df0 2024-05-09T00:19:36.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.001+0000 7ff992aca700 1 -- 172.21.15.46:0/2783667092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff98c1a7880 con 0x7ff98c107df0 2024-05-09T00:19:36.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.003+0000 7ff989ffb700 1 -- 172.21.15.46:0/2783667092 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff978004750 con 0x7ff98c107df0 2024-05-09T00:19:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.004+0000 7ff989ffb700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff97c0b2880 0x7ff97c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.004+0000 7ff990866700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff97c0b2880 0x7ff97c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.004+0000 7ff989ffb700 1 -- 172.21.15.46:0/2783667092 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff9780e1b20 con 0x7ff98c107df0 2024-05-09T00:19:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff9a14e0b80 1 Processor -- start 2024-05-09T00:19:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff9a14e0b80 1 -- start start 2024-05-09T00:19:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff990866700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff97c0b2880 0x7ff97c0b4d90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff980003800 tx=0x7ff980000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:36.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff9a14e0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f17391f160 0x55f1739b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff9a14e0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1738354a0 0x55f1739b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff9a14e0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 0x55f1739bb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff9a14e0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f1738d4ff0 con 0x55f1739bb400 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff9a14e0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f1738737f0 con 0x55f17391f160 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff9a14e0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f1738d0fb0 con 0x55f1738354a0 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff991067700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 0x55f1739bb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff991067700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 0x55f1739bb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58556/0 (socket says 172.21.15.46:58556) 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff98bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1738354a0 0x55f1739b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff991067700 1 -- 172.21.15.46:0/324094081 learned_addr learned my addr 172.21.15.46:0/324094081 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.005+0000 7ff990866700 1 --2- 172.21.15.46:0/324094081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f17391f160 0x55f1739b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.006+0000 7ff991067700 1 -- 172.21.15.46:0/324094081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1738354a0 msgr2=0x55f1739b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.006+0000 7ff991067700 1 --2- 172.21.15.46:0/324094081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1738354a0 0x55f1739b72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.006+0000 7ff991067700 1 -- 172.21.15.46:0/324094081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f17391f160 msgr2=0x55f1739b6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:36.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.006+0000 7ff991067700 1 --2- 172.21.15.46:0/324094081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f17391f160 0x55f1739b6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.006+0000 7ff991067700 1 -- 172.21.15.46:0/324094081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17386b370 con 0x55f1739bb400 2024-05-09T00:19:36.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.006+0000 7ff990866700 1 --2- 172.21.15.46:0/324094081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f17391f160 0x55f1739b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:36.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.006+0000 7ff991067700 1 --2- 172.21.15.46:0/324094081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 0x55f1739bb810 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff984009f20 tx=0x7ff984009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d61234c73262277 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:36.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.006+0000 7ff976ffd700 1 -- 172.21.15.46:0/324094081 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98401a070 con 0x55f1739bb400 2024-05-09T00:19:36.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.006+0000 7ff976ffd700 1 -- 172.21.15.46:0/324094081 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff984015410 con 0x55f1739bb400 2024-05-09T00:19:36.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.006+0000 7ff976ffd700 1 -- 172.21.15.46:0/324094081 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98400a600 con 0x55f1739bb400 2024-05-09T00:19:36.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.007+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/324094081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 msgr2=0x55f1739bb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:36.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.007+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/324094081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 0x55f1739bb810 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff984009f20 tx=0x7ff984009f50 comp rx=0 tx=0).stop 2024-05-09T00:19:36.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.007+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/324094081 shutdown_connections 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.007+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/324094081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 0x55f1739bb810 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.007+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/324094081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f17391f160 0x55f1739b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.007+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/324094081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1738354a0 0x55f1739b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.007+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/324094081 >> 172.21.15.46:0/324094081 conn(0x55f1738956a0 msgr2=0x55f1739ad400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.007+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/324094081 shutdown_connections 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.007+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/324094081 wait complete. 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.007+0000 7ff9a14e0b80 1 -- start start 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff9a14e0b80 1 Processor -- start 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff9a14e0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1738354a0 0x55f1739c05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff9a14e0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f17391f160 0x55f1739c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff9a14e0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 0x55f1739c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff9a14e0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f173750ba0 con 0x55f1739bb400 2024-05-09T00:19:36.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff9a14e0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f17388f150 con 0x55f17391f160 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff9a14e0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f1739bc1b0 con 0x55f1738354a0 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff991067700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 0x55f1739c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff991067700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 0x55f1739c1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58560/0 (socket says 172.21.15.46:58560) 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff991067700 1 -- 172.21.15.46:0/2638596486 learned_addr learned my addr 172.21.15.46:0/2638596486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff9a14e0b80 10 client.0 _ll_register_callbacks cb 0x55f1737466c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff98bfff700 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f17391f160 0x55f1739c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.008+0000 7ff990866700 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1738354a0 0x55f1739c05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff991067700 1 -- 172.21.15.46:0/2638596486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1738354a0 msgr2=0x55f1739c05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff991067700 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1738354a0 0x55f1739c05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff991067700 1 -- 172.21.15.46:0/2638596486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f17391f160 msgr2=0x55f1739c0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff991067700 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f17391f160 0x55f1739c0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff991067700 1 -- 172.21.15.46:0/2638596486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f173750490 con 0x55f1739bb400 2024-05-09T00:19:36.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff990866700 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1738354a0 0x55f1739c05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff991067700 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 0x55f1739c1050 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff984018040 tx=0x7ff98400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf42d246faedeee6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff96bfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff96bfff700 1 -- 172.21.15.46:0/2638596486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98401a040 con 0x55f1739bb400 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff96bfff700 1 -- 172.21.15.46:0/2638596486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff984020450 con 0x55f1739bb400 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2638596486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff984015410 con 0x55f1739bb400 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff96bfff700 1 -- 172.21.15.46:0/2638596486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff984020650 con 0x55f1739bb400 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff9a14e0b80 10 client.16698 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2638596486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1739cded0 con 0x55f1739bb400 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.009+0000 7ff96bfff700 1 -- 172.21.15.46:0/2638596486 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff984015410 con 0x55f1739bb400 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.010+0000 7ff96bfff700 1 -- 172.21.15.46:0/2638596486 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff984029400 con 0x55f1739bb400 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.010+0000 7ff9a14e0b80 10 client.16698 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.010+0000 7ff9a14e0b80 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.010+0000 7ff9a14e0b80 20 client.16698 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:36.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.010+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f1739d0450 0x55f1739d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.010+0000 7ff9a14e0b80 4 client.16698 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.010+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2638596486 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1738e29d0 con 0x55f1739d0450 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.010+0000 7ff990866700 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f1739d0450 0x55f1739d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.010+0000 7ff96bfff700 10 client.16698 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.010+0000 7ff990866700 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f1739d0450 0x55f1739d2870 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f176561d5c9b2e server_cookie=a62422890420ce36 in_seq=0 out_seq=0 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.011+0000 7ff96bfff700 1 -- 172.21.15.46:0/2638596486 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f1738e29d0 con 0x55f1739d0450 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.011+0000 7ff96bfff700 10 client.16698 handle_command_reply: tid=0 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.011+0000 7ff9a14e0b80 1 client.16698 shutdown 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.011+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2638596486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 msgr2=0x55f1739c1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.011+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 0x55f1739c1050 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff984018040 tx=0x7ff98400cc60 comp rx=0 tx=0).stop 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.011+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2638596486 shutdown_connections 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.011+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f1739d0450 0x55f1739d2870 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.011+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1739bb400 0x55f1739c1050 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.011+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1738354a0 0x55f1739c05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.011+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/2638596486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f17391f160 0x55f1739c0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.011+0000 7ff96bfff700 0 client.16698 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:36.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.011+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2638596486 >> 172.21.15.46:0/2638596486 conn(0x55f1738956a0 msgr2=0x55f1739ad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:36.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.012+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2638596486 shutdown_connections 2024-05-09T00:19:36.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.012+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2638596486 wait complete. 2024-05-09T00:19:36.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.012+0000 7ff9a14e0b80 20 client.16698 trim_cache size 0 max 16384 2024-05-09T00:19:36.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.027+0000 7ff9a14e0b80 1 Processor -- start 2024-05-09T00:19:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.027+0000 7ff9a14e0b80 1 -- start start 2024-05-09T00:19:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.027+0000 7ff9a14e0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1738354a0 0x55f1739cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.027+0000 7ff9a14e0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f17391f160 0x55f1739bcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.027+0000 7ff9a14e0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1739bb400 0x55f1739d5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.027+0000 7ff9a14e0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c1af400 con 0x55f17391f160 2024-05-09T00:19:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.027+0000 7ff9a14e0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c1acc90 con 0x55f1738354a0 2024-05-09T00:19:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.027+0000 7ff9a14e0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c1aaac0 con 0x55f1739bb400 2024-05-09T00:19:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff98bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f17391f160 0x55f1739bcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff98bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f17391f160 0x55f1739bcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58572/0 (socket says 172.21.15.46:58572) 2024-05-09T00:19:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff98bfff700 1 -- 172.21.15.46:0/1315618602 learned_addr learned my addr 172.21.15.46:0/1315618602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff990866700 1 --2- 172.21.15.46:0/1315618602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1738354a0 0x55f1739cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff98bfff700 1 -- 172.21.15.46:0/1315618602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1739bb400 msgr2=0x55f1739d5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff98bfff700 1 --2- 172.21.15.46:0/1315618602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1739bb400 0x55f1739d5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff98bfff700 1 -- 172.21.15.46:0/1315618602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1738354a0 msgr2=0x55f1739cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff98bfff700 1 --2- 172.21.15.46:0/1315618602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1738354a0 0x55f1739cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff98bfff700 1 -- 172.21.15.46:0/1315618602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1739a6f60 con 0x55f17391f160 2024-05-09T00:19:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff98bfff700 1 --2- 172.21.15.46:0/1315618602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f17391f160 0x55f1739bcb20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff9780df620 tx=0x7ff9780a91f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f689be734049f6e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff9767fc700 1 -- 172.21.15.46:0/1315618602 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978020040 con 0x55f17391f160 2024-05-09T00:19:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff9767fc700 1 -- 172.21.15.46:0/1315618602 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9780b1fa0 con 0x55f17391f160 2024-05-09T00:19:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.028+0000 7ff9767fc700 1 -- 172.21.15.46:0/1315618602 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9780a9600 con 0x55f17391f160 2024-05-09T00:19:36.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.029+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/1315618602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f17391f160 msgr2=0x55f1739bcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.029+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/1315618602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f17391f160 0x55f1739bcb20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff9780df620 tx=0x7ff9780a91f0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.029+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/1315618602 shutdown_connections 2024-05-09T00:19:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.029+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/1315618602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f1739bb400 0x55f1739d5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.029+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/1315618602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1738354a0 0x55f1739cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.029+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/1315618602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f17391f160 0x55f1739bcb20 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.029+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/1315618602 >> 172.21.15.46:0/1315618602 conn(0x55f1738956a0 msgr2=0x55f1739bc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.029+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/1315618602 shutdown_connections 2024-05-09T00:19:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.029+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/1315618602 wait complete. 2024-05-09T00:19:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff9a14e0b80 1 -- start start 2024-05-09T00:19:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff9a14e0b80 1 Processor -- start 2024-05-09T00:19:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff9a14e0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1738354a0 0x55f1739d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff9a14e0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f17391f160 0x55f1739d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff9a14e0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1739bb400 0x55f1739d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff9a14e0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c1ab150 con 0x55f1738354a0 2024-05-09T00:19:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff9a14e0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c1ac5c0 con 0x55f1739bb400 2024-05-09T00:19:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff9a14e0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c1ab820 con 0x55f17391f160 2024-05-09T00:19:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff9a14e0b80 10 client.0 _ll_register_callbacks cb 0x55f1737466c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff990866700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1738354a0 0x55f1739d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff990866700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1738354a0 0x55f1739d5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58574/0 (socket says 172.21.15.46:58574) 2024-05-09T00:19:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff990866700 1 -- 172.21.15.46:0/2874043182 learned_addr learned my addr 172.21.15.46:0/2874043182 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff991067700 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1739bb400 0x55f1739d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.030+0000 7ff98bfff700 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f17391f160 0x55f1739d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.031+0000 7ff990866700 1 -- 172.21.15.46:0/2874043182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f17391f160 msgr2=0x55f1739d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.031+0000 7ff990866700 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f17391f160 0x55f1739d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.031+0000 7ff990866700 1 -- 172.21.15.46:0/2874043182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1739bb400 msgr2=0x55f1739d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.031+0000 7ff990866700 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1739bb400 0x55f1739d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.031+0000 7ff990866700 1 -- 172.21.15.46:0/2874043182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1739d38d0 con 0x55f1738354a0 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.031+0000 7ff990866700 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1738354a0 0x55f1739d5ad0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff9800099c0 tx=0x7ff980009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f323822f741fb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.031+0000 7ff96bfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.031+0000 7ff96bfff700 1 -- 172.21.15.46:0/2874043182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980031430 con 0x55f1738354a0 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.031+0000 7ff96bfff700 1 -- 172.21.15.46:0/2874043182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff9800315d0 con 0x55f1738354a0 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.031+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2874043182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9780b1fa0 con 0x55f1738354a0 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.032+0000 7ff96bfff700 1 -- 172.21.15.46:0/2874043182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9800317d0 con 0x55f1738354a0 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.032+0000 7ff9a14e0b80 10 client.16704 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.032+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2874043182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f173a3a560 con 0x55f1738354a0 2024-05-09T00:19:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.032+0000 7ff96bfff700 1 -- 172.21.15.46:0/2874043182 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9780b1fa0 con 0x55f1738354a0 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.032+0000 7ff96bfff700 1 -- 172.21.15.46:0/2874043182 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff98000f580 con 0x55f1738354a0 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.032+0000 7ff9a14e0b80 10 client.16704 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.032+0000 7ff9a14e0b80 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.032+0000 7ff9a14e0b80 20 client.16704 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.032+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f173a3acf0 0x55f173a3d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.032+0000 7ff9a14e0b80 4 client.16704 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.032+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2874043182 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1738dca70 con 0x55f173a3acf0 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.032+0000 7ff98bfff700 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f173a3acf0 0x55f173a3d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff96bfff700 10 client.16704 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff98bfff700 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f173a3acf0 0x55f173a3d110 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f7eeb544d24783 server_cookie=ac2c876e0e2f2d98 in_seq=0 out_seq=0 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff96bfff700 1 -- 172.21.15.46:0/2874043182 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1738dca70 con 0x55f173a3acf0 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff96bfff700 10 client.16704 handle_command_reply: tid=0 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff9a14e0b80 1 client.16704 shutdown 2024-05-09T00:19:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2874043182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1738354a0 msgr2=0x55f1739d5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1738354a0 0x55f1739d5ad0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff9800099c0 tx=0x7ff980009cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2874043182 shutdown_connections 2024-05-09T00:19:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f173a3acf0 0x55f173a3d110 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1739bb400 0x55f1739d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff96bfff700 0 client.16704 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1738354a0 0x55f1739d5ad0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff9a14e0b80 1 --2- 172.21.15.46:0/2874043182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f17391f160 0x55f1739d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2874043182 >> 172.21.15.46:0/2874043182 conn(0x55f1738956a0 msgr2=0x55f1739b6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.033+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2874043182 shutdown_connections 2024-05-09T00:19:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff9a14e0b80 1 -- 172.21.15.46:0/2874043182 wait complete. 2024-05-09T00:19:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff9a14e0b80 20 client.16704 trim_cache size 0 max 16384 2024-05-09T00:19:36.039 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:19:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff992aca700 1 -- 172.21.15.46:0/2783667092 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff97c0b2880 msgr2=0x7ff97c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff992aca700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff97c0b2880 0x7ff97c0b4d90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff980003800 tx=0x7ff980000bc0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff992aca700 1 -- 172.21.15.46:0/2783667092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98c107df0 msgr2=0x7ff98c19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff992aca700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98c107df0 0x7ff98c19c2c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff97800d950 tx=0x7ff97800dd10 comp rx=0 tx=0).stop 2024-05-09T00:19:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff992aca700 1 -- 172.21.15.46:0/2783667092 shutdown_connections 2024-05-09T00:19:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff992aca700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff97c0b2880 0x7ff97c0b4d90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff992aca700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff98c108740 0x7ff98c19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff992aca700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff98c1034c0 0x7ff98c19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff992aca700 1 --2- 172.21.15.46:0/2783667092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff98c107df0 0x7ff98c19c2c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff992aca700 1 -- 172.21.15.46:0/2783667092 >> 172.21.15.46:0/2783667092 conn(0x7ff98c0fedc0 msgr2=0x7ff98c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff992aca700 1 -- 172.21.15.46:0/2783667092 shutdown_connections 2024-05-09T00:19:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:36.034+0000 7ff992aca700 1 -- 172.21.15.46:0/2783667092 wait complete. 2024-05-09T00:19:36.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:36 smithi046 ceph-mon[30151]: pgmap v623: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-05-09T00:19:36.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:36 smithi169 ceph-mon[34777]: pgmap v623: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-05-09T00:19:36.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:36 smithi142 ceph-mon[34533]: pgmap v623: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-05-09T00:19:37.945 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:19:37.945 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:19:38.214 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:38.646 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:38 smithi046 ceph-mon[30151]: pgmap v624: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:19:38.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:38 smithi169 ceph-mon[34777]: pgmap v624: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:19:38.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:38 smithi142 ceph-mon[34533]: pgmap v624: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:19:40.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.137+0000 7f8f7fce1700 1 -- 172.21.15.46:0/755018772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f78103310 msgr2=0x7f8f7810f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:40.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.137+0000 7f8f7fce1700 1 --2- 172.21.15.46:0/755018772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f78103310 0x7f8f7810f890 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000aaa0 tx=0x7f8f7000adb0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.137+0000 7f8f7fce1700 1 -- 172.21.15.46:0/755018772 shutdown_connections 2024-05-09T00:19:40.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.137+0000 7f8f7fce1700 1 --2- 172.21.15.46:0/755018772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f78103310 0x7f8f7810f890 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.137+0000 7f8f7fce1700 1 --2- 172.21.15.46:0/755018772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f781089f0 0x7f8f78108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.137+0000 7f8f7fce1700 1 --2- 172.21.15.46:0/755018772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f781029c0 0x7f8f78102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.137+0000 7f8f7fce1700 1 -- 172.21.15.46:0/755018772 >> 172.21.15.46:0/755018772 conn(0x7f8f780fe4e0 msgr2=0x7f8f78100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:40.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.137+0000 7f8f7fce1700 1 -- 172.21.15.46:0/755018772 shutdown_connections 2024-05-09T00:19:40.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.137+0000 7f8f7fce1700 1 -- 172.21.15.46:0/755018772 wait complete. 2024-05-09T00:19:40.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.138+0000 7f8f7fce1700 1 Processor -- start 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.138+0000 7f8f7fce1700 1 -- start start 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.138+0000 7f8f7fce1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f781029c0 0x7f8f7819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.138+0000 7f8f7fce1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f78103310 0x7f8f7819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.138+0000 7f8f7fce1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f781089f0 0x7f8f7819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.138+0000 7f8f7fce1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781122f0 con 0x7f8f78103310 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.138+0000 7f8f7fce1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78112170 con 0x7f8f781029c0 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.138+0000 7f8f7fce1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78112470 con 0x7f8f781089f0 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f7d27c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f78103310 0x7f8f7819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f7d27c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f78103310 0x7f8f7819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59256/0 (socket says 172.21.15.46:59256) 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f7d27c700 1 -- 172.21.15.46:0/2606062386 learned_addr learned my addr 172.21.15.46:0/2606062386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f781029c0 0x7f8f7819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f7e27e700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f781089f0 0x7f8f7819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f7d27c700 1 -- 172.21.15.46:0/2606062386 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f781089f0 msgr2=0x7f8f7819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f781089f0 0x7f8f7819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f7d27c700 1 -- 172.21.15.46:0/2606062386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f781029c0 msgr2=0x7f8f7819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:40.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f781029c0 0x7f8f7819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f7d27c700 1 -- 172.21.15.46:0/2606062386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7819fc40 con 0x7f8f78103310 2024-05-09T00:19:40.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f78103310 0x7f8f7819b3c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00c9e0 tx=0x7f8f6c00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:40.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f76ffd700 1 -- 172.21.15.46:0/2606062386 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c007aa0 con 0x7f8f78103310 2024-05-09T00:19:40.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f76ffd700 1 -- 172.21.15.46:0/2606062386 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f6c007c40 con 0x7f8f78103310 2024-05-09T00:19:40.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f7fce1700 1 -- 172.21.15.46:0/2606062386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7819ff30 con 0x7f8f78103310 2024-05-09T00:19:40.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.140+0000 7f8f7fce1700 1 -- 172.21.15.46:0/2606062386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f781a0470 con 0x7f8f78103310 2024-05-09T00:19:40.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.139+0000 7f8f76ffd700 1 -- 172.21.15.46:0/2606062386 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c018660 con 0x7f8f78103310 2024-05-09T00:19:40.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.142+0000 7f8f76ffd700 1 -- 172.21.15.46:0/2606062386 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8f6c018800 con 0x7f8f78103310 2024-05-09T00:19:40.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.143+0000 7f8f76ffd700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8f600b2880 0x7f8f600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.143+0000 7f8f8e6f7b80 1 Processor -- start 2024-05-09T00:19:40.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.143+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8f600b2880 0x7f8f600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.143+0000 7f8f76ffd700 1 -- 172.21.15.46:0/2606062386 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8f6c0e2240 con 0x7f8f78103310 2024-05-09T00:19:40.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.143+0000 7f8f8e6f7b80 1 -- start start 2024-05-09T00:19:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f8e6f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40efe6d80 0x55d40f0ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8f600b2880 0x7f8f600b4d90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8f78103b00 tx=0x7f8f68009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f8e6f7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 0x55d40f0ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f8e6f7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40f0e9480 0x55d40f0e9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f8e6f7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d40f00c120 con 0x55d40ef6c520 2024-05-09T00:19:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f8e6f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d40efb1a20 con 0x55d40efe6d80 2024-05-09T00:19:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f8e6f7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d40f0080e0 con 0x55d40f0e9480 2024-05-09T00:19:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f7d27c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 0x55d40f0ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f7d27c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 0x55d40f0ef800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59270/0 (socket says 172.21.15.46:59270) 2024-05-09T00:19:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f7d27c700 1 -- 172.21.15.46:0/3879489664 learned_addr learned my addr 172.21.15.46:0/3879489664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f7e27e700 1 --2- 172.21.15.46:0/3879489664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40f0e9480 0x55d40f0e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/3879489664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40efe6d80 0x55d40f0ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f7d27c700 1 -- 172.21.15.46:0/3879489664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40f0e9480 msgr2=0x55d40f0e9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/3879489664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40f0e9480 0x55d40f0e9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f7d27c700 1 -- 172.21.15.46:0/3879489664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40efe6d80 msgr2=0x55d40f0ef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/3879489664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40efe6d80 0x55d40f0ef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.144+0000 7f8f7d27c700 1 -- 172.21.15.46:0/3879489664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40efa2580 con 0x55d40ef6c520 2024-05-09T00:19:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.145+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/3879489664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 0x55d40f0ef800 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0a97b0 tx=0x7f8f6c0a93b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c6747a10fb7085 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.145+0000 7f8f5ffff700 1 -- 172.21.15.46:0/3879489664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0a9820 con 0x55d40ef6c520 2024-05-09T00:19:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.145+0000 7f8f5ffff700 1 -- 172.21.15.46:0/3879489664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f6c0adbd0 con 0x55d40ef6c520 2024-05-09T00:19:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.145+0000 7f8f5ffff700 1 -- 172.21.15.46:0/3879489664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0a9a80 con 0x55d40ef6c520 2024-05-09T00:19:40.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.146+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3879489664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 msgr2=0x55d40f0ef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:40.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.146+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3879489664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 0x55d40f0ef800 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0a97b0 tx=0x7f8f6c0a93b0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.146+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3879489664 shutdown_connections 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.146+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3879489664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40f0e9480 0x55d40f0e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.146+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3879489664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40efe6d80 0x55d40f0ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.146+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3879489664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 0x55d40f0ef800 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.146+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3879489664 >> 172.21.15.46:0/3879489664 conn(0x55d40ef6fff0 msgr2=0x55d40f0e42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.146+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3879489664 shutdown_connections 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.146+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3879489664 wait complete. 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.146+0000 7f8f8e6f7b80 1 -- start start 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.146+0000 7f8f8e6f7b80 1 Processor -- start 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f8e6f7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40ef6c520 0x55d40f0f3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f8e6f7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 0x55d40f0f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f8e6f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40f0e9480 0x55d40f0f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f8e6f7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d40f0ea180 con 0x55d40efe6d80 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f8e6f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d40efa6ba0 con 0x55d40f0e9480 2024-05-09T00:19:40.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f8e6f7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d40f0ff300 con 0x55d40ef6c520 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f8e6f7b80 10 client.0 _ll_register_callbacks cb 0x55d40ee7d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f7d27c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 0x55d40f0f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f7d27c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 0x55d40f0f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59278/0 (socket says 172.21.15.46:59278) 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f7d27c700 1 -- 172.21.15.46:0/1018290833 learned_addr learned my addr 172.21.15.46:0/1018290833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f7e27e700 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40f0e9480 0x55d40f0f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40ef6c520 0x55d40f0f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f7d27c700 1 -- 172.21.15.46:0/1018290833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40ef6c520 msgr2=0x55d40f0f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40ef6c520 0x55d40f0f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f7d27c700 1 -- 172.21.15.46:0/1018290833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40f0e9480 msgr2=0x55d40f0f3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.147+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40f0e9480 0x55d40f0f3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.148+0000 7f8f7d27c700 1 -- 172.21.15.46:0/1018290833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d40ee87510 con 0x55d40efe6d80 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.148+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40ef6c520 0x55d40f0f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.148+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 0x55d40f0f36d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0a9520 tx=0x7f8f6c01f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0271f70506817d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.148+0000 7f8f5d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.148+0000 7f8f5d7fa700 1 -- 172.21.15.46:0/1018290833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0b2680 con 0x55d40efe6d80 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.148+0000 7f8f5d7fa700 1 -- 172.21.15.46:0/1018290833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8f6c06f170 con 0x55d40efe6d80 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.148+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/1018290833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f6c0adbd0 con 0x55d40efe6d80 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.148+0000 7f8f5d7fa700 1 -- 172.21.15.46:0/1018290833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f6c0b21f0 con 0x55d40efe6d80 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.148+0000 7f8f8e6f7b80 10 client.16716 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.148+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/1018290833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d40f100ca0 con 0x55d40efe6d80 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.148+0000 7f8f5d7fa700 1 -- 172.21.15.46:0/1018290833 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f6c0adbd0 con 0x55d40efe6d80 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.148+0000 7f8f5d7fa700 1 -- 172.21.15.46:0/1018290833 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8f6c020030 con 0x55d40efe6d80 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.149+0000 7f8f8e6f7b80 10 client.16716 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.149+0000 7f8f8e6f7b80 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.149+0000 7f8f8e6f7b80 20 client.16716 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.149+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d40f103170 0x55d40f105590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.149+0000 7f8f8e6f7b80 4 client.16716 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.149+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/1018290833 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d40f019b00 con 0x55d40f103170 2024-05-09T00:19:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.149+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d40f103170 0x55d40f105590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.149+0000 7f8f5d7fa700 10 client.16716 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.149+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d40f103170 0x55d40f105590 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1be3191f45b71d6 server_cookie=b155dd5cb31b1554 in_seq=0 out_seq=0 2024-05-09T00:19:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.150+0000 7f8f5d7fa700 1 -- 172.21.15.46:0/1018290833 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d40f019b00 con 0x55d40f103170 2024-05-09T00:19:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.150+0000 7f8f5d7fa700 10 client.16716 handle_command_reply: tid=0 2024-05-09T00:19:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.150+0000 7f8f8e6f7b80 1 client.16716 shutdown 2024-05-09T00:19:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.151+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/1018290833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 msgr2=0x55d40f0f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.151+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 0x55d40f0f36d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0a9520 tx=0x7f8f6c01f5c0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.151+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/1018290833 shutdown_connections 2024-05-09T00:19:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.151+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d40f103170 0x55d40f105590 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.151+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40f0e9480 0x55d40f0f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.151+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40ef6c520 0x55d40f0f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.151+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/1018290833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 0x55d40f0f36d0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.151+0000 7f8f5d7fa700 0 client.16716 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.151+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/1018290833 >> 172.21.15.46:0/1018290833 conn(0x55d40ef6fff0 msgr2=0x55d40f0de070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.151+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/1018290833 shutdown_connections 2024-05-09T00:19:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.151+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/1018290833 wait complete. 2024-05-09T00:19:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.151+0000 7f8f8e6f7b80 20 client.16716 trim_cache size 0 max 16384 2024-05-09T00:19:40.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f8e6f7b80 1 Processor -- start 2024-05-09T00:19:40.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f8e6f7b80 1 -- start start 2024-05-09T00:19:40.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f8e6f7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 0x55d40f0f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f8e6f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40efe6d80 0x55d40f0e8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f8e6f7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40f0e9480 0x55d40f0de930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f8e6f7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781b14c0 con 0x55d40ef6c520 2024-05-09T00:19:40.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f8e6f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781aed50 con 0x55d40efe6d80 2024-05-09T00:19:40.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f8e6f7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781acb80 con 0x55d40f0e9480 2024-05-09T00:19:40.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f7da7d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 0x55d40f0f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f7da7d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 0x55d40f0f0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59294/0 (socket says 172.21.15.46:59294) 2024-05-09T00:19:40.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f7da7d700 1 -- 172.21.15.46:0/3105268084 learned_addr learned my addr 172.21.15.46:0/3105268084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:40.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f7e27e700 1 --2- 172.21.15.46:0/3105268084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40f0e9480 0x55d40f0de930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/3105268084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40efe6d80 0x55d40f0e8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f7da7d700 1 -- 172.21.15.46:0/3105268084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40f0e9480 msgr2=0x55d40f0de930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:40.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/3105268084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40f0e9480 0x55d40f0de930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f7da7d700 1 -- 172.21.15.46:0/3105268084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40efe6d80 msgr2=0x55d40f0e8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/3105268084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40efe6d80 0x55d40f0e8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.172+0000 7f8f7da7d700 1 -- 172.21.15.46:0/3105268084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40f0e0a50 con 0x55d40ef6c520 2024-05-09T00:19:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.173+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/3105268084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 0x55d40f0f0d70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8f68007b70 tx=0x7f8f68015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e482a39ce78958a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.173+0000 7f8f5f7fe700 1 -- 172.21.15.46:0/3105268084 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68029150 con 0x55d40ef6c520 2024-05-09T00:19:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.173+0000 7f8f5f7fe700 1 -- 172.21.15.46:0/3105268084 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f680297a0 con 0x55d40ef6c520 2024-05-09T00:19:40.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.173+0000 7f8f5f7fe700 1 -- 172.21.15.46:0/3105268084 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68025cf0 con 0x55d40ef6c520 2024-05-09T00:19:40.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.173+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3105268084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 msgr2=0x55d40f0f0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:40.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.173+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3105268084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 0x55d40f0f0d70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8f68007b70 tx=0x7f8f68015000 comp rx=0 tx=0).stop 2024-05-09T00:19:40.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.174+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3105268084 shutdown_connections 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.174+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3105268084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40f0e9480 0x55d40f0de930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.174+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3105268084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40ef6c520 0x55d40f0f0d70 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.174+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3105268084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40efe6d80 0x55d40f0e8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.174+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3105268084 >> 172.21.15.46:0/3105268084 conn(0x55d40ef6fff0 msgr2=0x55d40f0f4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.174+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3105268084 shutdown_connections 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.174+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3105268084 wait complete. 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.174+0000 7f8f8e6f7b80 1 -- start start 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.174+0000 7f8f8e6f7b80 1 Processor -- start 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.174+0000 7f8f8e6f7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40ef6c520 0x55d40f107940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f8e6f7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 0x55d40f107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f8e6f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40f0e9480 0x55d40f16d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f8e6f7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781ad210 con 0x55d40efe6d80 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f8e6f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781ae680 con 0x55d40f0e9480 2024-05-09T00:19:40.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f8e6f7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781ad8e0 con 0x55d40ef6c520 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f8e6f7b80 10 client.0 _ll_register_callbacks cb 0x55d40ee7d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f7d27c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 0x55d40f107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f7d27c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 0x55d40f107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59298/0 (socket says 172.21.15.46:59298) 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f7da7d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40ef6c520 0x55d40f107940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f7d27c700 1 -- 172.21.15.46:0/3867864929 learned_addr learned my addr 172.21.15.46:0/3867864929 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f7e27e700 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40f0e9480 0x55d40f16d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f7d27c700 1 -- 172.21.15.46:0/3867864929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40ef6c520 msgr2=0x55d40f107940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40ef6c520 0x55d40f107940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f7d27c700 1 -- 172.21.15.46:0/3867864929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40f0e9480 msgr2=0x55d40f16d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40f0e9480 0x55d40f16d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f7d27c700 1 -- 172.21.15.46:0/3867864929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d40f0ddd40 con 0x55d40efe6d80 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f7e27e700 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40f0e9480 0x55d40f16d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f7d27c700 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 0x55d40f107e80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0a5850 tx=0x7f8f6c0b25a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6924b2f82f98f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f5d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:40.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f5d7fa700 1 -- 172.21.15.46:0/3867864929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0a95f0 con 0x55d40efe6d80 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f5d7fa700 1 -- 172.21.15.46:0/3867864929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8f6c018b70 con 0x55d40efe6d80 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.175+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3867864929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f680297a0 con 0x55d40efe6d80 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.176+0000 7f8f5d7fa700 1 -- 172.21.15.46:0/3867864929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f6c0b2680 con 0x55d40efe6d80 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.176+0000 7f8f8e6f7b80 10 client.16722 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.176+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3867864929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d40f0efb70 con 0x55d40efe6d80 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.176+0000 7f8f5d7fa700 1 -- 172.21.15.46:0/3867864929 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f680297a0 con 0x55d40efe6d80 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.176+0000 7f8f5d7fa700 1 -- 172.21.15.46:0/3867864929 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8f6c0a5190 con 0x55d40efe6d80 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.176+0000 7f8f8e6f7b80 10 client.16722 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.176+0000 7f8f8e6f7b80 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.176+0000 7f8f8e6f7b80 20 client.16722 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.176+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d40f16eea0 0x55d40f1712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.176+0000 7f8f8e6f7b80 4 client.16722 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.176+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3867864929 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d40f013ba0 con 0x55d40f16eea0 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.176+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d40f16eea0 0x55d40f1712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.177+0000 7f8f7da7d700 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d40f16eea0 0x55d40f1712c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7db7fcb37db813b server_cookie=787fdb0cc20ebc9b in_seq=0 out_seq=0 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.177+0000 7f8f5d7fa700 10 client.16722 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f5d7fa700 1 -- 172.21.15.46:0/3867864929 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d40f013ba0 con 0x55d40f16eea0 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f5d7fa700 10 client.16722 handle_command_reply: tid=0 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f8e6f7b80 1 client.16722 shutdown 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3867864929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 msgr2=0x55d40f107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 0x55d40f107e80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0a5850 tx=0x7f8f6c0b25a0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3867864929 shutdown_connections 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d40f16eea0 0x55d40f1712c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d40f0e9480 0x55d40f16d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d40ef6c520 0x55d40f107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f8e6f7b80 1 --2- 172.21.15.46:0/3867864929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40efe6d80 0x55d40f107e80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f5d7fa700 0 client.16722 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3867864929 >> 172.21.15.46:0/3867864929 conn(0x55d40ef6fff0 msgr2=0x55d40f0e8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3867864929 shutdown_connections 2024-05-09T00:19:40.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.178+0000 7f8f8e6f7b80 1 -- 172.21.15.46:0/3867864929 wait complete. 2024-05-09T00:19:40.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f8e6f7b80 20 client.16722 trim_cache size 0 max 16384 2024-05-09T00:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "63f0e766-0a30-411b-a24c-8a0710920f5e", 2024-05-09T00:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:19:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f7fce1700 1 -- 172.21.15.46:0/2606062386 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8f600b2880 msgr2=0x7f8f600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f7fce1700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8f600b2880 0x7f8f600b4d90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8f78103b00 tx=0x7f8f68009040 comp rx=0 tx=0).stop 2024-05-09T00:19:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f7fce1700 1 -- 172.21.15.46:0/2606062386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f78103310 msgr2=0x7f8f7819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f7fce1700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f78103310 0x7f8f7819b3c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00c9e0 tx=0x7f8f6c00cda0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f7fce1700 1 -- 172.21.15.46:0/2606062386 shutdown_connections 2024-05-09T00:19:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f7fce1700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8f600b2880 0x7f8f600b4d90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f7fce1700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f781089f0 0x7f8f7819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f7fce1700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f781029c0 0x7f8f7819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f7fce1700 1 --2- 172.21.15.46:0/2606062386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f78103310 0x7f8f7819b3c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f7fce1700 1 -- 172.21.15.46:0/2606062386 >> 172.21.15.46:0/2606062386 conn(0x7f8f780fe4e0 msgr2=0x7f8f78105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f7fce1700 1 -- 172.21.15.46:0/2606062386 shutdown_connections 2024-05-09T00:19:40.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:40.179+0000 7f8f7fce1700 1 -- 172.21.15.46:0/2606062386 wait complete. 2024-05-09T00:19:40.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:40 smithi169 ceph-mon[34777]: pgmap v625: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:40.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:40 smithi046 ceph-mon[30151]: pgmap v625: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:40.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:19:40.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:40 smithi142 ceph-mon[34533]: pgmap v625: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:41.196 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:41.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:41 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:19:41.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:41 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:41.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:41 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:19:41.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:41 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:41.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:41 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:19:41.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:41 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:41.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:41 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:19:41.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:41 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:42.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:41 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:19:42.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:41 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:42.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:41 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:19:42.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:41 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:42.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:42 smithi046 ceph-mon[30151]: pgmap v626: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:19:42.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:42 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:19:42.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:42 smithi169 ceph-mon[34777]: pgmap v626: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:19:42.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:42 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:19:42.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:42 smithi142 ceph-mon[34533]: pgmap v626: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:19:42.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:42 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:19:43.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.084+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/345210962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e81034e0 msgr2=0x7fd4e81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:43.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.084+0000 7fd4ef0ba700 1 --2- 172.21.15.46:0/345210962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e81034e0 0x7fd4e81038f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009a40 tx=0x7fd4d8009d50 comp rx=0 tx=0).stop 2024-05-09T00:19:43.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.084+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/345210962 shutdown_connections 2024-05-09T00:19:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.084+0000 7fd4ef0ba700 1 --2- 172.21.15.46:0/345210962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4e8108760 0x7fd4e81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.084+0000 7fd4ef0ba700 1 --2- 172.21.15.46:0/345210962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e81034e0 0x7fd4e81038f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.084+0000 7fd4ef0ba700 1 --2- 172.21.15.46:0/345210962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4e8107e10 0x7fd4e8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.084+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/345210962 >> 172.21.15.46:0/345210962 conn(0x7fd4e80fee50 msgr2=0x7fd4e8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.085+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/345210962 shutdown_connections 2024-05-09T00:19:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.085+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/345210962 wait complete. 2024-05-09T00:19:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.085+0000 7fd4ef0ba700 1 Processor -- start 2024-05-09T00:19:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.085+0000 7fd4ef0ba700 1 -- start start 2024-05-09T00:19:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4ef0ba700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4e81034e0 0x7fd4e819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4ef0ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8107e10 0x7fd4e819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4ef0ba700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4e8108760 0x7fd4e81a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4ef0ba700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81122f0 con 0x7fd4e8107e10 2024-05-09T00:19:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4ef0ba700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8112170 con 0x7fd4e8108760 2024-05-09T00:19:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4ef0ba700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8112470 con 0x7fd4e81034e0 2024-05-09T00:19:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8107e10 0x7fd4e819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8107e10 0x7fd4e819d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59326/0 (socket says 172.21.15.46:59326) 2024-05-09T00:19:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4e7fff700 1 -- 172.21.15.46:0/3556354749 learned_addr learned my addr 172.21.15.46:0/3556354749 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:43.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4ece56700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4e81034e0 0x7fd4e819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4ed657700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4e8108760 0x7fd4e81a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4e7fff700 1 -- 172.21.15.46:0/3556354749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4e81034e0 msgr2=0x7fd4e819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4e7fff700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4e81034e0 0x7fd4e819cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4e7fff700 1 -- 172.21.15.46:0/3556354749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4e8108760 msgr2=0x7fd4e81a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4e7fff700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4e8108760 0x7fd4e81a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.086+0000 7fd4e7fff700 1 -- 172.21.15.46:0/3556354749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e81a1ec0 con 0x7fd4e8107e10 2024-05-09T00:19:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.087+0000 7fd4e7fff700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8107e10 0x7fd4e819d590 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00c970 tx=0x7fd4dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.087+0000 7fd4e5ffb700 1 -- 172.21.15.46:0/3556354749 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc004a30 con 0x7fd4e8107e10 2024-05-09T00:19:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.087+0000 7fd4e5ffb700 1 -- 172.21.15.46:0/3556354749 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4dc0054d0 con 0x7fd4e8107e10 2024-05-09T00:19:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.087+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/3556354749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e81a21b0 con 0x7fd4e8107e10 2024-05-09T00:19:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.087+0000 7fd4e5ffb700 1 -- 172.21.15.46:0/3556354749 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc007c10 con 0x7fd4e8107e10 2024-05-09T00:19:43.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.087+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/3556354749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e81a9ad0 con 0x7fd4e8107e10 2024-05-09T00:19:43.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.090+0000 7fd4e5ffb700 1 -- 172.21.15.46:0/3556354749 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd4dc00f450 con 0x7fd4e8107e10 2024-05-09T00:19:43.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4e5ffb700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4d00b2880 0x7fd4d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4ece56700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4d00b2880 0x7fd4d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4fdad0b80 1 Processor -- start 2024-05-09T00:19:43.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4e5ffb700 1 -- 172.21.15.46:0/3556354749 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd4dc0e1dd0 con 0x7fd4e8107e10 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4fdad0b80 1 -- start start 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4fdad0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 0x55c3c5483240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4fdad0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c5483780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4fdad0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c547d400 0x55c3c547d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4fdad0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c3c53a00a0 con 0x55c3c537ad00 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4fdad0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3c53459a0 con 0x55c3c547d400 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4fdad0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c3c539c060 con 0x55c3c53004a0 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4ece56700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 0x55c3c5483240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4ece56700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 0x55c3c5483240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59330/0 (socket says 172.21.15.46:59330) 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.091+0000 7fd4ece56700 1 -- 172.21.15.46:0/1282690792 learned_addr learned my addr 172.21.15.46:0/1282690792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4ed657700 1 --2- 172.21.15.46:0/1282690792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c547d400 0x55c3c547d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4e7fff700 1 --2- 172.21.15.46:0/1282690792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c5483780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4ece56700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4d00b2880 0x7fd4d00b4d90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800b540 tx=0x7fd4d8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4ece56700 1 -- 172.21.15.46:0/1282690792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 msgr2=0x55c3c5483780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4ece56700 1 --2- 172.21.15.46:0/1282690792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c5483780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4ece56700 1 -- 172.21.15.46:0/1282690792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c547d400 msgr2=0x55c3c547d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4ece56700 1 --2- 172.21.15.46:0/1282690792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c547d400 0x55c3c547d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4ece56700 1 -- 172.21.15.46:0/1282690792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c5336500 con 0x55c3c537ad00 2024-05-09T00:19:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4e7fff700 1 --2- 172.21.15.46:0/1282690792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c5483780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4ece56700 1 --2- 172.21.15.46:0/1282690792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 0x55c3c5483240 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8026000 tx=0x7fd4d8027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bab25eb13e3258 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4ceffd700 1 -- 172.21.15.46:0/1282690792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d800bcb0 con 0x55c3c537ad00 2024-05-09T00:19:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4ceffd700 1 -- 172.21.15.46:0/1282690792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4d8031070 con 0x55c3c537ad00 2024-05-09T00:19:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.092+0000 7fd4ceffd700 1 -- 172.21.15.46:0/1282690792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8022690 con 0x55c3c537ad00 2024-05-09T00:19:43.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.093+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/1282690792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 msgr2=0x55c3c5483240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.093+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/1282690792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 0x55c3c5483240 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8026000 tx=0x7fd4d8027000 comp rx=0 tx=0).stop 2024-05-09T00:19:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.093+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/1282690792 shutdown_connections 2024-05-09T00:19:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.093+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/1282690792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c547d400 0x55c3c547d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.093+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/1282690792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 0x55c3c5483240 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.093+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/1282690792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c5483780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.093+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/1282690792 >> 172.21.15.46:0/1282690792 conn(0x55c3c5303f70 msgr2=0x55c3c5478220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.093+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/1282690792 shutdown_connections 2024-05-09T00:19:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.093+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/1282690792 wait complete. 2024-05-09T00:19:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.094+0000 7fd4fdad0b80 1 -- start start 2024-05-09T00:19:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.094+0000 7fd4fdad0b80 1 Processor -- start 2024-05-09T00:19:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.094+0000 7fd4fdad0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c5486f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.094+0000 7fd4fdad0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 0x55c3c54874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.094+0000 7fd4fdad0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c547d400 0x55c3c5487a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.094+0000 7fd4fdad0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c3c547e100 con 0x55c3c537ad00 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.094+0000 7fd4fdad0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3c533ab20 con 0x55c3c547d400 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.094+0000 7fd4fdad0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c3c54931d0 con 0x55c3c53004a0 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.094+0000 7fd4fdad0b80 10 client.0 _ll_register_callbacks cb 0x55c3c52116c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.094+0000 7fd4e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 0x55c3c54874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.094+0000 7fd4e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 0x55c3c54874d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59340/0 (socket says 172.21.15.46:59340) 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.094+0000 7fd4e7fff700 1 -- 172.21.15.46:0/2101305681 learned_addr learned my addr 172.21.15.46:0/2101305681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4ed657700 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c547d400 0x55c3c5487a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4ece56700 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c5486f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4e7fff700 1 -- 172.21.15.46:0/2101305681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 msgr2=0x55c3c5486f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4e7fff700 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c5486f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4e7fff700 1 -- 172.21.15.46:0/2101305681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c547d400 msgr2=0x55c3c5487a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4e7fff700 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c547d400 0x55c3c5487a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4e7fff700 1 -- 172.21.15.46:0/2101305681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c521b490 con 0x55c3c537ad00 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4ece56700 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c5486f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4e7fff700 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 0x55c3c54874d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0df8d0 tx=0x7fd4dc0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9a553f483ce558 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4c7fff700 1 -- 172.21.15.46:0/2101305681 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc01d030 con 0x55c3c537ad00 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4c7fff700 1 -- 172.21.15.46:0/2101305681 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd4dc0a9650 con 0x55c3c537ad00 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2101305681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4d8031070 con 0x55c3c537ad00 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4c7fff700 1 -- 172.21.15.46:0/2101305681 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4dc0a9af0 con 0x55c3c537ad00 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4fdad0b80 10 client.16734 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.095+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2101305681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c3c5494ad0 con 0x55c3c537ad00 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.096+0000 7fd4c7fff700 1 -- 172.21.15.46:0/2101305681 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4d8031070 con 0x55c3c537ad00 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.096+0000 7fd4c7fff700 1 -- 172.21.15.46:0/2101305681 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd4dc0c3ce0 con 0x55c3c537ad00 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.096+0000 7fd4fdad0b80 10 client.16734 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.096+0000 7fd4fdad0b80 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.096+0000 7fd4fdad0b80 20 client.16734 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.096+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c3c5497060 0x55c3c5499480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.096+0000 7fd4fdad0b80 4 client.16734 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.096+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2101305681 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3c53ada80 con 0x55c3c5497060 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.096+0000 7fd4ece56700 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c3c5497060 0x55c3c5499480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.097+0000 7fd4ece56700 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c3c5497060 0x55c3c5499480 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418ce5ad0c4334d8 server_cookie=4eb067f0842f7455 in_seq=0 out_seq=0 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.097+0000 7fd4c7fff700 10 client.16734 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.097+0000 7fd4c7fff700 1 -- 172.21.15.46:0/2101305681 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c3c53ada80 con 0x55c3c5497060 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.097+0000 7fd4c7fff700 10 client.16734 handle_command_reply: tid=0 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.097+0000 7fd4fdad0b80 1 client.16734 shutdown 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.098+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2101305681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 msgr2=0x55c3c54874d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.098+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 0x55c3c54874d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0df8d0 tx=0x7fd4dc0050b0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.098+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2101305681 shutdown_connections 2024-05-09T00:19:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.098+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c3c5497060 0x55c3c5499480 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.098+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c547d400 0x55c3c5487a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.098+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c5486f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.098+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/2101305681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c537ad00 0x55c3c54874d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.098+0000 7fd4c7fff700 0 client.16734 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.098+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2101305681 >> 172.21.15.46:0/2101305681 conn(0x55c3c5303f70 msgr2=0x55c3c5471ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.098+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2101305681 shutdown_connections 2024-05-09T00:19:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.098+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2101305681 wait complete. 2024-05-09T00:19:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.098+0000 7fd4fdad0b80 20 client.16734 trim_cache size 0 max 16384 2024-05-09T00:19:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.118+0000 7fd4fdad0b80 1 Processor -- start 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.118+0000 7fd4fdad0b80 1 -- start start 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.118+0000 7fd4fdad0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c53004a0 0x55c3c5481700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.118+0000 7fd4fdad0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c537ad00 0x55c3c5472700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4fdad0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 0x55c3c549ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4fdad0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81b3750 con 0x55c3c547d400 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4fdad0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81b0fe0 con 0x55c3c53004a0 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4fdad0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81aee10 con 0x55c3c537ad00 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ed657700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 0x55c3c549ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ed657700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 0x55c3c549ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59342/0 (socket says 172.21.15.46:59342) 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ed657700 1 -- 172.21.15.46:0/3411550031 learned_addr learned my addr 172.21.15.46:0/3411550031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ece56700 1 --2- 172.21.15.46:0/3411550031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c53004a0 0x55c3c5481700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4e7fff700 1 --2- 172.21.15.46:0/3411550031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c537ad00 0x55c3c5472700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ed657700 1 -- 172.21.15.46:0/3411550031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c537ad00 msgr2=0x55c3c5472700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ed657700 1 --2- 172.21.15.46:0/3411550031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c537ad00 0x55c3c5472700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ed657700 1 -- 172.21.15.46:0/3411550031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c53004a0 msgr2=0x55c3c5481700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ed657700 1 --2- 172.21.15.46:0/3411550031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c53004a0 0x55c3c5481700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ed657700 1 -- 172.21.15.46:0/3411550031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c5474510 con 0x55c3c547d400 2024-05-09T00:19:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ed657700 1 --2- 172.21.15.46:0/3411550031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 0x55c3c549ebf0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009ee0 tx=0x7fd4e0009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88105739b15dfa3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ce7fc700 1 -- 172.21.15.46:0/3411550031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0007e20 con 0x55c3c547d400 2024-05-09T00:19:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ce7fc700 1 -- 172.21.15.46:0/3411550031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e001b070 con 0x55c3c547d400 2024-05-09T00:19:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.119+0000 7fd4ce7fc700 1 -- 172.21.15.46:0/3411550031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0005020 con 0x55c3c547d400 2024-05-09T00:19:43.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.120+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/3411550031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 msgr2=0x55c3c549ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:43.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.120+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/3411550031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 0x55c3c549ebf0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009ee0 tx=0x7fd4e0009f10 comp rx=0 tx=0).stop 2024-05-09T00:19:43.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.120+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/3411550031 shutdown_connections 2024-05-09T00:19:43.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.120+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/3411550031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 0x55c3c549ebf0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.120+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/3411550031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c53004a0 0x55c3c5481700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.120+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/3411550031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c537ad00 0x55c3c5472700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.120+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/3411550031 >> 172.21.15.46:0/3411550031 conn(0x55c3c5303f70 msgr2=0x55c3c5471ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:43.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.120+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/3411550031 shutdown_connections 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.120+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/3411550031 wait complete. 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.121+0000 7fd4fdad0b80 1 -- start start 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.121+0000 7fd4fdad0b80 1 Processor -- start 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.121+0000 7fd4fdad0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c548f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.121+0000 7fd4fdad0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c537ad00 0x55c3c548f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.121+0000 7fd4fdad0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 0x55c3c548c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.121+0000 7fd4fdad0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81af4a0 con 0x55c3c547d400 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.121+0000 7fd4fdad0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81b0910 con 0x55c3c537ad00 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.121+0000 7fd4fdad0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81afb70 con 0x55c3c53004a0 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.121+0000 7fd4fdad0b80 10 client.0 _ll_register_callbacks cb 0x55c3c52116c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4ed657700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 0x55c3c548c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4ece56700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c548f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4ece56700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c548f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:43532/0 (socket says 172.21.15.46:43532) 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4ed657700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 0x55c3c548c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59356/0 (socket says 172.21.15.46:59356) 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4ece56700 1 -- 172.21.15.46:0/2389433850 learned_addr learned my addr 172.21.15.46:0/2389433850 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4e7fff700 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c537ad00 0x55c3c548f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4ed657700 1 -- 172.21.15.46:0/2389433850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 msgr2=0x55c3c548f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4ed657700 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c548f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4ed657700 1 -- 172.21.15.46:0/2389433850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c537ad00 msgr2=0x55c3c548f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4ed657700 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c537ad00 0x55c3c548f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4ed657700 1 -- 172.21.15.46:0/2389433850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c548c740 con 0x55c3c547d400 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4ece56700 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c548f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4ed657700 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 0x55c3c548c100 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009710 tx=0x7fd4e001ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373725fd2e626fb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4c7fff700 1 -- 172.21.15.46:0/2389433850 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0004e60 con 0x55c3c547d400 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2389433850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e001b070 con 0x55c3c547d400 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.122+0000 7fd4c7fff700 1 -- 172.21.15.46:0/2389433850 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd4e001f620 con 0x55c3c547d400 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.123+0000 7fd4c7fff700 1 -- 172.21.15.46:0/2389433850 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4e001f820 con 0x55c3c547d400 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.123+0000 7fd4fdad0b80 10 client.16740 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:43.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.123+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2389433850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c3c5502040 con 0x55c3c547d400 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.123+0000 7fd4c7fff700 1 -- 172.21.15.46:0/2389433850 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e001b070 con 0x55c3c547d400 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.123+0000 7fd4c7fff700 1 -- 172.21.15.46:0/2389433850 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd4e00050d0 con 0x55c3c547d400 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.123+0000 7fd4fdad0b80 10 client.16740 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.123+0000 7fd4fdad0b80 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.123+0000 7fd4fdad0b80 20 client.16740 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.123+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c3c55025f0 0x55c3c5504a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.123+0000 7fd4fdad0b80 4 client.16740 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.123+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2389433850 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c3c5471d40 con 0x55c3c55025f0 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.123+0000 7fd4ece56700 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c3c55025f0 0x55c3c5504a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4ece56700 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c3c55025f0 0x55c3c5504a10 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1af71c8c3f3d49e server_cookie=b32dc2f6360143bd in_seq=0 out_seq=0 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4c7fff700 10 client.16740 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4c7fff700 1 -- 172.21.15.46:0/2389433850 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c3c5471d40 con 0x55c3c55025f0 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4c7fff700 10 client.16740 handle_command_reply: tid=0 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4fdad0b80 1 client.16740 shutdown 2024-05-09T00:19:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2389433850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 msgr2=0x55c3c548c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:43.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 0x55c3c548c100 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009710 tx=0x7fd4e001ff90 comp rx=0 tx=0).stop 2024-05-09T00:19:43.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2389433850 shutdown_connections 2024-05-09T00:19:43.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c3c55025f0 0x55c3c5504a10 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c547d400 0x55c3c548c100 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c3c53004a0 0x55c3c548f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4fdad0b80 1 --2- 172.21.15.46:0/2389433850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3c537ad00 0x55c3c548f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2389433850 >> 172.21.15.46:0/2389433850 conn(0x55c3c5303f70 msgr2=0x55c3c5481460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:43.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4c7fff700 0 client.16740 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:43.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2389433850 shutdown_connections 2024-05-09T00:19:43.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.124+0000 7fd4fdad0b80 1 -- 172.21.15.46:0/2389433850 wait complete. 2024-05-09T00:19:43.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4fdad0b80 20 client.16740 trim_cache size 0 max 16384 2024-05-09T00:19:43.129 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:19:43.129 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:19:43.129 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:19:43.129 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:19:43.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/3556354749 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4d00b2880 msgr2=0x7fd4d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:43.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4ef0ba700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4d00b2880 0x7fd4d00b4d90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800b540 tx=0x7fd4d8000bc0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/3556354749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8107e10 msgr2=0x7fd4e819d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4ef0ba700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8107e10 0x7fd4e819d590 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00c970 tx=0x7fd4dc00cc80 comp rx=0 tx=0).stop 2024-05-09T00:19:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/3556354749 shutdown_connections 2024-05-09T00:19:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4ef0ba700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd4d00b2880 0x7fd4d00b4d90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4ef0ba700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4e8108760 0x7fd4e81a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4ef0ba700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4e81034e0 0x7fd4e819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4ef0ba700 1 --2- 172.21.15.46:0/3556354749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8107e10 0x7fd4e819d590 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/3556354749 >> 172.21.15.46:0/3556354749 conn(0x7fd4e80fee50 msgr2=0x7fd4e8107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/3556354749 shutdown_connections 2024-05-09T00:19:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:43.125+0000 7fd4ef0ba700 1 -- 172.21.15.46:0/3556354749 wait complete. 2024-05-09T00:19:43.884 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:19:43.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:19:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:44.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:44 smithi046 ceph-mon[30151]: pgmap v627: 97 pgs: 97 active+clean; 4.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:19:44.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:44 smithi169 ceph-mon[34777]: pgmap v627: 97 pgs: 97 active+clean; 4.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:19:44.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:44 smithi142 ceph-mon[34533]: pgmap v627: 97 pgs: 97 active+clean; 4.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:19:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.875+0000 7f6dd3026700 1 -- 172.21.15.46:0/2194961556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc103960 msgr2=0x7f6dcc103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.875+0000 7f6dd3026700 1 --2- 172.21.15.46:0/2194961556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc103960 0x7f6dcc103d70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009990 tx=0x7f6db8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:19:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.875+0000 7f6dd3026700 1 -- 172.21.15.46:0/2194961556 shutdown_connections 2024-05-09T00:19:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.875+0000 7f6dd3026700 1 --2- 172.21.15.46:0/2194961556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dcc1042b0 0x7f6dcc110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.875+0000 7f6dd3026700 1 --2- 172.21.15.46:0/2194961556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dcc108200 0x7f6dcc108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.875+0000 7f6dd3026700 1 --2- 172.21.15.46:0/2194961556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc103960 0x7f6dcc103d70 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.875+0000 7f6dd3026700 1 -- 172.21.15.46:0/2194961556 >> 172.21.15.46:0/2194961556 conn(0x7f6dcc0fe520 msgr2=0x7f6dcc100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:45.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.875+0000 7f6dd3026700 1 -- 172.21.15.46:0/2194961556 shutdown_connections 2024-05-09T00:19:45.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.875+0000 7f6dd3026700 1 -- 172.21.15.46:0/2194961556 wait complete. 2024-05-09T00:19:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.876+0000 7f6dd3026700 1 Processor -- start 2024-05-09T00:19:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.876+0000 7f6dd3026700 1 -- start start 2024-05-09T00:19:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd3026700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dcc103960 0x7f6dcc19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd3026700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dcc1042b0 0x7f6dcc19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd3026700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc108200 0x7f6dcc19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd3026700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc113380 con 0x7f6dcc108200 2024-05-09T00:19:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd3026700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc113200 con 0x7f6dcc103960 2024-05-09T00:19:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd3026700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc113500 con 0x7f6dcc1042b0 2024-05-09T00:19:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd15c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc108200 0x7f6dcc19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd15c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc108200 0x7f6dcc19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59372/0 (socket says 172.21.15.46:59372) 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd15c3700 1 -- 172.21.15.46:0/3087768423 learned_addr learned my addr 172.21.15.46:0/3087768423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dcbfff700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dcc1042b0 0x7f6dcc19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd0dc2700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dcc103960 0x7f6dcc19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd15c3700 1 -- 172.21.15.46:0/3087768423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dcc1042b0 msgr2=0x7f6dcc19c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd15c3700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dcc1042b0 0x7f6dcc19c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd15c3700 1 -- 172.21.15.46:0/3087768423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dcc103960 msgr2=0x7f6dcc19bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd15c3700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dcc103960 0x7f6dcc19bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd15c3700 1 -- 172.21.15.46:0/3087768423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc1a6fd0 con 0x7f6dcc108200 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.877+0000 7f6dd15c3700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc108200 0x7f6dcc19c720 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4007ee0 tx=0x7f6dc400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.878+0000 7f6dc9ffb700 1 -- 172.21.15.46:0/3087768423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400dbd0 con 0x7f6dcc108200 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.878+0000 7f6dc9ffb700 1 -- 172.21.15.46:0/3087768423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc400dd70 con 0x7f6dcc108200 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.878+0000 7f6dd3026700 1 -- 172.21.15.46:0/3087768423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dcc1a72c0 con 0x7f6dcc108200 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.878+0000 7f6dc9ffb700 1 -- 172.21.15.46:0/3087768423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4005990 con 0x7f6dcc108200 2024-05-09T00:19:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.878+0000 7f6dd3026700 1 -- 172.21.15.46:0/3087768423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dcc1a77d0 con 0x7f6dcc108200 2024-05-09T00:19:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.879+0000 7f6dd3026700 1 -- 172.21.15.46:0/3087768423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dcc10b6f0 con 0x7f6dcc108200 2024-05-09T00:19:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.880+0000 7f6dc9ffb700 1 -- 172.21.15.46:0/3087768423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6dc401a020 con 0x7f6dcc108200 2024-05-09T00:19:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.881+0000 7f6dc9ffb700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6dbc0b2900 0x7f6dbc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.881+0000 7f6dd0dc2700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6dbc0b2900 0x7f6dbc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.881+0000 7f6dc9ffb700 1 -- 172.21.15.46:0/3087768423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f6dc40e1970 con 0x7f6dcc108200 2024-05-09T00:19:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.882+0000 7f6dd0dc2700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6dbc0b2900 0x7f6dbc0b4e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005950 tx=0x7f6dc00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:45.886+0000 7f6dc9ffb700 1 -- 172.21.15.46:0/3087768423 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6dc4010070 con 0x7f6dcc108200 2024-05-09T00:19:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.149+0000 7f6dd3026700 1 -- 172.21.15.46:0/3087768423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dcc10b0d0 con 0x7f6dcc108200 2024-05-09T00:19:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.150+0000 7f6dc9ffb700 1 -- 172.21.15.46:0/3087768423 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f6dc40a9650 con 0x7f6dcc108200 2024-05-09T00:19:46.149 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:19:46.149 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:19:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.152+0000 7f6dd3026700 1 -- 172.21.15.46:0/3087768423 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6dbc0b2900 msgr2=0x7f6dbc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:46.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.152+0000 7f6dd3026700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6dbc0b2900 0x7f6dbc0b4e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005950 tx=0x7f6dc00058e0 comp rx=0 tx=0).stop 2024-05-09T00:19:46.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.152+0000 7f6dd3026700 1 -- 172.21.15.46:0/3087768423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc108200 msgr2=0x7f6dcc19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:46.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.152+0000 7f6dd3026700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc108200 0x7f6dcc19c720 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4007ee0 tx=0x7f6dc400d3b0 comp rx=0 tx=0).stop 2024-05-09T00:19:46.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.152+0000 7f6dd3026700 1 -- 172.21.15.46:0/3087768423 shutdown_connections 2024-05-09T00:19:46.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.152+0000 7f6dd3026700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f6dbc0b2900 0x7f6dbc0b4e10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:46.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.152+0000 7f6dd3026700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dcc108200 0x7f6dcc19c720 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:46.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.152+0000 7f6dd3026700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dcc103960 0x7f6dcc19bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:46.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.152+0000 7f6dd3026700 1 --2- 172.21.15.46:0/3087768423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dcc1042b0 0x7f6dcc19c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:46.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.152+0000 7f6dd3026700 1 -- 172.21.15.46:0/3087768423 >> 172.21.15.46:0/3087768423 conn(0x7f6dcc0fe520 msgr2=0x7f6dcc100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:46.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.152+0000 7f6dd3026700 1 -- 172.21.15.46:0/3087768423 shutdown_connections 2024-05-09T00:19:46.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:46.152+0000 7f6dd3026700 1 -- 172.21.15.46:0/3087768423 wait complete. 2024-05-09T00:19:46.153 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:19:46.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:46 smithi046 ceph-mon[30151]: pgmap v628: 97 pgs: 97 active+clean; 4.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:46.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:46 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3087768423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:19:46.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:46 smithi169 ceph-mon[34777]: pgmap v628: 97 pgs: 97 active+clean; 4.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:46.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:46 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3087768423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:19:46.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:19:46.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:46 smithi142 ceph-mon[34533]: pgmap v628: 97 pgs: 97 active+clean; 4.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:46.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:46 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3087768423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:19:47.158 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:48.851 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:48 smithi046 ceph-mon[30151]: pgmap v629: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:19:48.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:48 smithi169 ceph-mon[34777]: pgmap v629: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:19:48.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.877+0000 7f981d641700 1 -- 172.21.15.46:0/4082108848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98181027e0 msgr2=0x7f9818102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:48.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.877+0000 7f981d641700 1 --2- 172.21.15.46:0/4082108848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98181027e0 0x7f9818102bf0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f980c009990 tx=0x7f980c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.877+0000 7f981d641700 1 -- 172.21.15.46:0/4082108848 shutdown_connections 2024-05-09T00:19:48.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.877+0000 7f981d641700 1 --2- 172.21.15.46:0/4082108848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9818103130 0x7f9818111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.877+0000 7f981d641700 1 --2- 172.21.15.46:0/4082108848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f981810a3e0 0x7f981810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.877+0000 7f981d641700 1 --2- 172.21.15.46:0/4082108848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98181027e0 0x7f9818102bf0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.877+0000 7f981d641700 1 -- 172.21.15.46:0/4082108848 >> 172.21.15.46:0/4082108848 conn(0x7f98180fe6f0 msgr2=0x7f9818100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:48.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.877+0000 7f981d641700 1 -- 172.21.15.46:0/4082108848 shutdown_connections 2024-05-09T00:19:48.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.877+0000 7f981d641700 1 -- 172.21.15.46:0/4082108848 wait complete. 2024-05-09T00:19:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.878+0000 7f981d641700 1 Processor -- start 2024-05-09T00:19:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.878+0000 7f981d641700 1 -- start start 2024-05-09T00:19:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f981d641700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98181027e0 0x7f98181a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f981d641700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9818103130 0x7f98181a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f981d641700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810a3e0 0x7f981819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f981d641700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9818103ee0 con 0x7f981810a3e0 2024-05-09T00:19:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f981d641700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9818103d60 con 0x7f9818103130 2024-05-09T00:19:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f981d641700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9818104060 con 0x7f98181027e0 2024-05-09T00:19:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f98177fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810a3e0 0x7f981819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f98177fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810a3e0 0x7f981819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40792/0 (socket says 172.21.15.46:40792) 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f98177fe700 1 -- 172.21.15.46:0/2804215578 learned_addr learned my addr 172.21.15.46:0/2804215578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f9816ffd700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98181027e0 0x7f98181a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f98167fc700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9818103130 0x7f98181a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f98177fe700 1 -- 172.21.15.46:0/2804215578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98181027e0 msgr2=0x7f98181a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f98177fe700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98181027e0 0x7f98181a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f98177fe700 1 -- 172.21.15.46:0/2804215578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9818103130 msgr2=0x7f98181a31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f98177fe700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9818103130 0x7f98181a31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f98177fe700 1 -- 172.21.15.46:0/2804215578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981819d6c0 con 0x7f981810a3e0 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f9816ffd700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98181027e0 0x7f98181a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.879+0000 7f98177fe700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810a3e0 0x7f981819cf90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f981000d980 tx=0x7f981000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.880+0000 7f97fffff700 1 -- 172.21.15.46:0/2804215578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810017070 con 0x7f981810a3e0 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.880+0000 7f981d641700 1 -- 172.21.15.46:0/2804215578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981819d9b0 con 0x7f981810a3e0 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.880+0000 7f97fffff700 1 -- 172.21.15.46:0/2804215578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9810009df0 con 0x7f981810a3e0 2024-05-09T00:19:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.880+0000 7f97fffff700 1 -- 172.21.15.46:0/2804215578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98100053a0 con 0x7f981810a3e0 2024-05-09T00:19:48.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.880+0000 7f981d641700 1 -- 172.21.15.46:0/2804215578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98181a9ac0 con 0x7f981810a3e0 2024-05-09T00:19:48.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.882+0000 7f97fffff700 1 -- 172.21.15.46:0/2804215578 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9810004750 con 0x7f981810a3e0 2024-05-09T00:19:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.883+0000 7f97fffff700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f98000b2880 0x7f98000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.883+0000 7f9816ffd700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f98000b2880 0x7f98000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.883+0000 7f97fffff700 1 -- 172.21.15.46:0/2804215578 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f98100e2230 con 0x7f981810a3e0 2024-05-09T00:19:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.883+0000 7f982c057b80 1 Processor -- start 2024-05-09T00:19:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f9816ffd700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f98000b2880 0x7f98000b4d90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9808009d50 tx=0x7f9808009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f982c057b80 1 -- start start 2024-05-09T00:19:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f982c057b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 0x55f37543e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f982c057b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3752bb4a0 0x55f37543ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f982c057b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f375438ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f982c057b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f37535aff0 con 0x55f3753a5160 2024-05-09T00:19:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f982c057b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f3752f97f0 con 0x55f3754386a0 2024-05-09T00:19:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f982c057b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f375356fb0 con 0x55f3752bb4a0 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 0x55f37543e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 0x55f37543e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40804/0 (socket says 172.21.15.46:40804) 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f9816ffd700 1 -- 172.21.15.46:0/1089261017 learned_addr learned my addr 172.21.15.46:0/1089261017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f98167fc700 1 --2- 172.21.15.46:0/1089261017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3752bb4a0 0x55f37543ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f98177fe700 1 --2- 172.21.15.46:0/1089261017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f375438ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f9816ffd700 1 -- 172.21.15.46:0/1089261017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3752bb4a0 msgr2=0x55f37543ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f9816ffd700 1 --2- 172.21.15.46:0/1089261017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3752bb4a0 0x55f37543ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f9816ffd700 1 -- 172.21.15.46:0/1089261017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 msgr2=0x55f375438ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f9816ffd700 1 --2- 172.21.15.46:0/1089261017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f375438ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f9816ffd700 1 -- 172.21.15.46:0/1089261017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3752f1370 con 0x55f3753a5160 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.884+0000 7f98167fc700 1 --2- 172.21.15.46:0/1089261017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3752bb4a0 0x55f37543ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.885+0000 7f9816ffd700 1 --2- 172.21.15.46:0/1089261017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 0x55f37543e4f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9808016470 tx=0x7f9808012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abb4a95f150ab35 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.885+0000 7f97fd7fa700 1 -- 172.21.15.46:0/1089261017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808026070 con 0x55f3753a5160 2024-05-09T00:19:48.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.885+0000 7f97fd7fa700 1 -- 172.21.15.46:0/1089261017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9808021ab0 con 0x55f3753a5160 2024-05-09T00:19:48.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.885+0000 7f97fd7fa700 1 -- 172.21.15.46:0/1089261017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808014ab0 con 0x55f3753a5160 2024-05-09T00:19:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.886+0000 7f982c057b80 1 -- 172.21.15.46:0/1089261017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 msgr2=0x55f37543e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.886+0000 7f982c057b80 1 --2- 172.21.15.46:0/1089261017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 0x55f37543e4f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9808016470 tx=0x7f9808012000 comp rx=0 tx=0).stop 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.886+0000 7f982c057b80 1 -- 172.21.15.46:0/1089261017 shutdown_connections 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.886+0000 7f982c057b80 1 --2- 172.21.15.46:0/1089261017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f375438ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.886+0000 7f982c057b80 1 --2- 172.21.15.46:0/1089261017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 0x55f37543e4f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.886+0000 7f982c057b80 1 --2- 172.21.15.46:0/1089261017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3752bb4a0 0x55f37543ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.886+0000 7f982c057b80 1 -- 172.21.15.46:0/1089261017 >> 172.21.15.46:0/1089261017 conn(0x55f37531b6a0 msgr2=0x55f375433400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.886+0000 7f982c057b80 1 -- 172.21.15.46:0/1089261017 shutdown_connections 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.886+0000 7f982c057b80 1 -- 172.21.15.46:0/1089261017 wait complete. 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.886+0000 7f982c057b80 1 -- start start 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.886+0000 7f982c057b80 1 Processor -- start 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f982c057b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 0x55f3754421d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f982c057b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3753a5160 0x55f375442710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f982c057b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f375442c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f982c057b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f3751d6ba0 con 0x55f3752bb4a0 2024-05-09T00:19:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f982c057b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f375315150 con 0x55f3754386a0 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f982c057b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f3754451d0 con 0x55f3753a5160 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f982c057b80 10 client.0 _ll_register_callbacks cb 0x55f3751cc6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 0x55f3754421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 0x55f3754421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40816/0 (socket says 172.21.15.46:40816) 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f9816ffd700 1 -- 172.21.15.46:0/892359591 learned_addr learned my addr 172.21.15.46:0/892359591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f98177fe700 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f375442c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f98167fc700 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3753a5160 0x55f375442710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f9816ffd700 1 -- 172.21.15.46:0/892359591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3753a5160 msgr2=0x55f375442710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f9816ffd700 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3753a5160 0x55f375442710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f9816ffd700 1 -- 172.21.15.46:0/892359591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 msgr2=0x55f375442c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f9816ffd700 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f375442c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f9816ffd700 1 -- 172.21.15.46:0/892359591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3751d6490 con 0x55f3752bb4a0 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f9816ffd700 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 0x55f3754421d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9808014690 tx=0x7f9808022c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d2651e6c9333c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.887+0000 7f97faff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f97faff5700 1 -- 172.21.15.46:0/892359591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808030030 con 0x55f3752bb4a0 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f97faff5700 1 -- 172.21.15.46:0/892359591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9808014790 con 0x55f3752bb4a0 2024-05-09T00:19:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f982c057b80 1 -- 172.21.15.46:0/892359591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9808021ab0 con 0x55f3752bb4a0 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f97faff5700 1 -- 172.21.15.46:0/892359591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9808026040 con 0x55f3752bb4a0 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f982c057b80 10 client.16758 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f982c057b80 1 -- 172.21.15.46:0/892359591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f37544fba0 con 0x55f3752bb4a0 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f97faff5700 1 -- 172.21.15.46:0/892359591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9808021ab0 con 0x55f3752bb4a0 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f97faff5700 1 -- 172.21.15.46:0/892359591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9808029030 con 0x55f3752bb4a0 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f982c057b80 10 client.16758 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f982c057b80 10 client.16758 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f982c057b80 20 client.16758 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f982c057b80 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f375452070 0x55f375454490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f982c057b80 4 client.16758 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.888+0000 7f982c057b80 1 -- 172.21.15.46:0/892359591 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3753689d0 con 0x55f375452070 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.889+0000 7f98167fc700 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f375452070 0x55f375454490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.889+0000 7f97faff5700 10 client.16758 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:19:48.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.889+0000 7f98167fc700 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f375452070 0x55f375454490 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66f2f1c793f4ac03 server_cookie=ae120e130e6e8a3c in_seq=0 out_seq=0 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f97faff5700 1 -- 172.21.15.46:0/892359591 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f3753689d0 con 0x55f375452070 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f97faff5700 10 client.16758 handle_command_reply: tid=0 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f982c057b80 1 client.16758 shutdown 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f982c057b80 1 -- 172.21.15.46:0/892359591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 msgr2=0x55f3754421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f982c057b80 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 0x55f3754421d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9808014690 tx=0x7f9808022c70 comp rx=0 tx=0).stop 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f982c057b80 1 -- 172.21.15.46:0/892359591 shutdown_connections 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f982c057b80 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f375452070 0x55f375454490 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f982c057b80 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f375442c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f982c057b80 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 0x55f3754421d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f982c057b80 1 --2- 172.21.15.46:0/892359591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3753a5160 0x55f375442710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f97faff5700 0 client.16758 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f982c057b80 1 -- 172.21.15.46:0/892359591 >> 172.21.15.46:0/892359591 conn(0x55f37531b6a0 msgr2=0x55f3754331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f982c057b80 1 -- 172.21.15.46:0/892359591 shutdown_connections 2024-05-09T00:19:48.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f982c057b80 1 -- 172.21.15.46:0/892359591 wait complete. 2024-05-09T00:19:48.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.890+0000 7f982c057b80 20 client.16758 trim_cache size 0 max 16384 2024-05-09T00:19:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.905+0000 7f982c057b80 1 Processor -- start 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f982c057b80 1 -- start start 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f982c057b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 0x55f375442a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f982c057b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3753a5160 0x55f37544df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f982c057b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f37543d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f982c057b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f98181b3680 con 0x55f3752bb4a0 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f982c057b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98181b0f10 con 0x55f3754386a0 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f982c057b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f98181aed40 con 0x55f3753a5160 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 0x55f375442a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 0x55f375442a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40824/0 (socket says 172.21.15.46:40824) 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f9816ffd700 1 -- 172.21.15.46:0/963176246 learned_addr learned my addr 172.21.15.46:0/963176246 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f98167fc700 1 --2- 172.21.15.46:0/963176246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3753a5160 0x55f37544df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f9816ffd700 1 -- 172.21.15.46:0/963176246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3753a5160 msgr2=0x55f37544df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f9816ffd700 1 --2- 172.21.15.46:0/963176246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3753a5160 0x55f37544df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f9816ffd700 1 -- 172.21.15.46:0/963176246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 msgr2=0x55f37543d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f9816ffd700 1 --2- 172.21.15.46:0/963176246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f37543d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f9816ffd700 1 -- 172.21.15.46:0/963176246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f37543e050 con 0x55f3752bb4a0 2024-05-09T00:19:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.906+0000 7f9816ffd700 1 --2- 172.21.15.46:0/963176246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 0x55f375442a90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9808007ca0 tx=0x7f98080214a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7314f83fff1a40ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.907+0000 7f97fcff9700 1 -- 172.21.15.46:0/963176246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808030070 con 0x55f3752bb4a0 2024-05-09T00:19:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.907+0000 7f97fcff9700 1 -- 172.21.15.46:0/963176246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9808005c60 con 0x55f3752bb4a0 2024-05-09T00:19:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.907+0000 7f97fcff9700 1 -- 172.21.15.46:0/963176246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808004cf0 con 0x55f3752bb4a0 2024-05-09T00:19:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.907+0000 7f982c057b80 1 -- 172.21.15.46:0/963176246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 msgr2=0x55f375442a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.907+0000 7f982c057b80 1 --2- 172.21.15.46:0/963176246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 0x55f375442a90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9808007ca0 tx=0x7f98080214a0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.907+0000 7f982c057b80 1 -- 172.21.15.46:0/963176246 shutdown_connections 2024-05-09T00:19:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.907+0000 7f982c057b80 1 --2- 172.21.15.46:0/963176246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f37543d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.907+0000 7f982c057b80 1 --2- 172.21.15.46:0/963176246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3752bb4a0 0x55f375442a90 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.907+0000 7f982c057b80 1 --2- 172.21.15.46:0/963176246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3753a5160 0x55f37544df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.907+0000 7f982c057b80 1 -- 172.21.15.46:0/963176246 >> 172.21.15.46:0/963176246 conn(0x55f37531b6a0 msgr2=0x55f375440ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f982c057b80 1 -- 172.21.15.46:0/963176246 shutdown_connections 2024-05-09T00:19:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f982c057b80 1 -- 172.21.15.46:0/963176246 wait complete. 2024-05-09T00:19:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f982c057b80 1 -- start start 2024-05-09T00:19:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f982c057b80 1 Processor -- start 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f982c057b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3752bb4a0 0x55f37545a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f982c057b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 0x55f37544a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f982c057b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f37544a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f982c057b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f98181af3d0 con 0x55f3753a5160 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f982c057b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98181b0840 con 0x55f3754386a0 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f982c057b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f98181afaa0 con 0x55f3752bb4a0 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f982c057b80 10 client.0 _ll_register_callbacks cb 0x55f3751cc6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f98167fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 0x55f37544a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f98167fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 0x55f37544a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40828/0 (socket says 172.21.15.46:40828) 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f98167fc700 1 -- 172.21.15.46:0/238280661 learned_addr learned my addr 172.21.15.46:0/238280661 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.908+0000 7f9816ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3752bb4a0 0x55f37545a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f98177fe700 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f37544a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f98167fc700 1 -- 172.21.15.46:0/238280661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3752bb4a0 msgr2=0x55f37545a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f98167fc700 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3752bb4a0 0x55f37545a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f98167fc700 1 -- 172.21.15.46:0/238280661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 msgr2=0x55f37544a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f98167fc700 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f37544a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f98167fc700 1 -- 172.21.15.46:0/238280661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f375458a80 con 0x55f3753a5160 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f98177fe700 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f37544a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f98167fc700 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 0x55f37544a300 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f980c009750 tx=0x7f980c009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a62f871a5e9469 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f97faff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f97faff5700 1 -- 172.21.15.46:0/238280661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c003bf0 con 0x55f3753a5160 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f97faff5700 1 -- 172.21.15.46:0/238280661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f980c003d90 con 0x55f3753a5160 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f982c057b80 1 -- 172.21.15.46:0/238280661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9808005c60 con 0x55f3753a5160 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f97faff5700 1 -- 172.21.15.46:0/238280661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f980c00b980 con 0x55f3753a5160 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f982c057b80 10 client.16764 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.909+0000 7f982c057b80 1 -- 172.21.15.46:0/238280661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3754bb890 con 0x55f3753a5160 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.910+0000 7f97faff5700 1 -- 172.21.15.46:0/238280661 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9808005c60 con 0x55f3753a5160 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.910+0000 7f97faff5700 1 -- 172.21.15.46:0/238280661 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f980c030030 con 0x55f3753a5160 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.910+0000 7f982c057b80 10 client.16764 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.910+0000 7f982c057b80 10 client.16764 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.910+0000 7f982c057b80 20 client.16764 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.910+0000 7f982c057b80 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f3754bbf80 0x55f3754be3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.910+0000 7f982c057b80 4 client.16764 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.910+0000 7f982c057b80 1 -- 172.21.15.46:0/238280661 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f375362a70 con 0x55f3754bbf80 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.910+0000 7f9816ffd700 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f3754bbf80 0x55f3754be3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.910+0000 7f97faff5700 10 client.16764 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.910+0000 7f9816ffd700 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f3754bbf80 0x55f3754be3a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b48c54ec80085840 server_cookie=24bdb8efb47aacb7 in_seq=0 out_seq=0 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f97faff5700 1 -- 172.21.15.46:0/238280661 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f375362a70 con 0x55f3754bbf80 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f97faff5700 10 client.16764 handle_command_reply: tid=0 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f982c057b80 1 client.16764 shutdown 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f982c057b80 1 -- 172.21.15.46:0/238280661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 msgr2=0x55f37544a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f982c057b80 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 0x55f37544a300 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f980c009750 tx=0x7f980c009fb0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f982c057b80 1 -- 172.21.15.46:0/238280661 shutdown_connections 2024-05-09T00:19:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f982c057b80 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55f3754bbf80 0x55f3754be3a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f982c057b80 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3754386a0 0x55f37544a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f982c057b80 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3752bb4a0 0x55f37545a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f982c057b80 1 --2- 172.21.15.46:0/238280661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3753a5160 0x55f37544a300 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f97faff5700 0 client.16764 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:19:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f982c057b80 1 -- 172.21.15.46:0/238280661 >> 172.21.15.46:0/238280661 conn(0x55f37531b6a0 msgr2=0x55f37543d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f982c057b80 1 -- 172.21.15.46:0/238280661 shutdown_connections 2024-05-09T00:19:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f982c057b80 1 -- 172.21.15.46:0/238280661 wait complete. 2024-05-09T00:19:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.911+0000 7f982c057b80 20 client.16764 trim_cache size 0 max 16384 2024-05-09T00:19:48.915 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:19:48.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.912+0000 7f981d641700 1 -- 172.21.15.46:0/2804215578 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f98000b2880 msgr2=0x7f98000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:48.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.912+0000 7f981d641700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f98000b2880 0x7f98000b4d90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9808009d50 tx=0x7f9808009040 comp rx=0 tx=0).stop 2024-05-09T00:19:48.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.912+0000 7f981d641700 1 -- 172.21.15.46:0/2804215578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810a3e0 msgr2=0x7f981819cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:48.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.912+0000 7f981d641700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810a3e0 0x7f981819cf90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f981000d980 tx=0x7f981000dd40 comp rx=0 tx=0).stop 2024-05-09T00:19:48.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.912+0000 7f981d641700 1 -- 172.21.15.46:0/2804215578 shutdown_connections 2024-05-09T00:19:48.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.912+0000 7f981d641700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f98000b2880 0x7f98000b4d90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.912+0000 7f981d641700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981810a3e0 0x7f981819cf90 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.912+0000 7f981d641700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98181027e0 0x7f98181a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.912+0000 7f981d641700 1 --2- 172.21.15.46:0/2804215578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9818103130 0x7f98181a31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:48.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.912+0000 7f981d641700 1 -- 172.21.15.46:0/2804215578 >> 172.21.15.46:0/2804215578 conn(0x7f98180fe6f0 msgr2=0x7f98180ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:48.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.912+0000 7f981d641700 1 -- 172.21.15.46:0/2804215578 shutdown_connections 2024-05-09T00:19:48.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:48.912+0000 7f981d641700 1 -- 172.21.15.46:0/2804215578 wait complete. 2024-05-09T00:19:48.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:48 smithi142 ceph-mon[34533]: pgmap v629: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:19:49.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:19:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:50.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:50 smithi046 ceph-mon[30151]: pgmap v630: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:50.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:50 smithi169 ceph-mon[34777]: pgmap v630: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:50.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:50 smithi142 ceph-mon[34533]: pgmap v630: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:19:51.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.601+0000 7fdf1c398700 1 -- 172.21.15.46:0/3923396661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf14103310 msgr2=0x7fdf1410f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:51.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.601+0000 7fdf1c398700 1 --2- 172.21.15.46:0/3923396661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf14103310 0x7fdf1410f890 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c210 tx=0x7fdf0c00c520 comp rx=0 tx=0).stop 2024-05-09T00:19:51.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.601+0000 7fdf1c398700 1 -- 172.21.15.46:0/3923396661 shutdown_connections 2024-05-09T00:19:51.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.601+0000 7fdf1c398700 1 --2- 172.21.15.46:0/3923396661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf14103310 0x7fdf1410f890 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.601+0000 7fdf1c398700 1 --2- 172.21.15.46:0/3923396661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf141089f0 0x7fdf14108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.601+0000 7fdf1c398700 1 --2- 172.21.15.46:0/3923396661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf141029c0 0x7fdf14102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.601+0000 7fdf1c398700 1 -- 172.21.15.46:0/3923396661 >> 172.21.15.46:0/3923396661 conn(0x7fdf140fe6f0 msgr2=0x7fdf14100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:51.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.601+0000 7fdf1c398700 1 -- 172.21.15.46:0/3923396661 shutdown_connections 2024-05-09T00:19:51.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.601+0000 7fdf1c398700 1 -- 172.21.15.46:0/3923396661 wait complete. 2024-05-09T00:19:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.602+0000 7fdf1c398700 1 Processor -- start 2024-05-09T00:19:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.602+0000 7fdf1c398700 1 -- start start 2024-05-09T00:19:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf1c398700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf141029c0 0x7fdf1419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf1c398700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf14103310 0x7fdf1419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf1c398700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf141089f0 0x7fdf1419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf1c398700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141122e0 con 0x7fdf141089f0 2024-05-09T00:19:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf1c398700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14112160 con 0x7fdf14103310 2024-05-09T00:19:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf1c398700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14112460 con 0x7fdf141029c0 2024-05-09T00:19:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf1a935700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf141089f0 0x7fdf1419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf1a935700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf141089f0 0x7fdf1419f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40852/0 (socket says 172.21.15.46:40852) 2024-05-09T00:19:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf1a935700 1 -- 172.21.15.46:0/2242267992 learned_addr learned my addr 172.21.15.46:0/2242267992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf19933700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf14103310 0x7fdf1419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf1a134700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf141029c0 0x7fdf1419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf19933700 1 -- 172.21.15.46:0/2242267992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf141029c0 msgr2=0x7fdf1419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:51.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf19933700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf141029c0 0x7fdf1419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf19933700 1 -- 172.21.15.46:0/2242267992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf141089f0 msgr2=0x7fdf1419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:51.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf19933700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf141089f0 0x7fdf1419f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.603+0000 7fdf19933700 1 -- 172.21.15.46:0/2242267992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1419fcb0 con 0x7fdf14103310 2024-05-09T00:19:51.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.604+0000 7fdf19933700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf14103310 0x7fdf1419b380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400e9c0 tx=0x7fdf0400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:51.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.604+0000 7fdf0b7fe700 1 -- 172.21.15.46:0/2242267992 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0400c9c0 con 0x7fdf14103310 2024-05-09T00:19:51.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.604+0000 7fdf0b7fe700 1 -- 172.21.15.46:0/2242267992 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf0400cb60 con 0x7fdf14103310 2024-05-09T00:19:51.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.604+0000 7fdf1c398700 1 -- 172.21.15.46:0/2242267992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1419ffa0 con 0x7fdf14103310 2024-05-09T00:19:51.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.604+0000 7fdf0b7fe700 1 -- 172.21.15.46:0/2242267992 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04018680 con 0x7fdf14103310 2024-05-09T00:19:51.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.604+0000 7fdf1c398700 1 -- 172.21.15.46:0/2242267992 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf141a78c0 con 0x7fdf14103310 2024-05-09T00:19:51.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.606+0000 7fdf0b7fe700 1 -- 172.21.15.46:0/2242267992 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdf04018820 con 0x7fdf14103310 2024-05-09T00:19:51.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.608+0000 7fdf0b7fe700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdf000b2900 0x7fdf000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.608+0000 7fdf0b7fe700 1 -- 172.21.15.46:0/2242267992 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fdf040e3250 con 0x7fdf14103310 2024-05-09T00:19:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.608+0000 7fdf1a134700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdf000b2900 0x7fdf000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.609+0000 7fdf1a134700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdf000b2900 0x7fdf000b4e10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdf14103b00 tx=0x7fdf10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:51.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.609+0000 7fdf2adaeb80 1 Processor -- start 2024-05-09T00:19:51.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.609+0000 7fdf2adaeb80 1 -- start start 2024-05-09T00:19:51.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.609+0000 7fdf2adaeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 0x556f6662c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.609+0000 7fdf2adaeb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f664a94a0 0x556f6662c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.609+0000 7fdf2adaeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66626400 0x556f66626810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.609+0000 7fdf2adaeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556f665490a0 con 0x556f66523d00 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.609+0000 7fdf2adaeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556f664ee9a0 con 0x556f66626400 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.609+0000 7fdf2adaeb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556f66545060 con 0x556f664a94a0 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf1a134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 0x556f6662c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf19933700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f664a94a0 0x556f6662c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf1a134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 0x556f6662c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40868/0 (socket says 172.21.15.46:40868) 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf1a134700 1 -- 172.21.15.46:0/2960981091 learned_addr learned my addr 172.21.15.46:0/2960981091 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf1a935700 1 --2- 172.21.15.46:0/2960981091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66626400 0x556f66626810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf19933700 1 -- 172.21.15.46:0/2960981091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66626400 msgr2=0x556f66626810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf19933700 1 --2- 172.21.15.46:0/2960981091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66626400 0x556f66626810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf19933700 1 -- 172.21.15.46:0/2960981091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 msgr2=0x556f6662c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf19933700 1 --2- 172.21.15.46:0/2960981091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 0x556f6662c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf19933700 1 -- 172.21.15.46:0/2960981091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f664df500 con 0x556f664a94a0 2024-05-09T00:19:51.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf1a134700 1 --2- 172.21.15.46:0/2960981091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 0x556f6662c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:19:51.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf19933700 1 --2- 172.21.15.46:0/2960981091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f664a94a0 0x556f6662c780 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdf040aa800 tx=0x7fdf04009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a47eface070e9f78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:51.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.610+0000 7fdf08d47700 1 -- 172.21.15.46:0/2960981091 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf040aa870 con 0x556f664a94a0 2024-05-09T00:19:51.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.611+0000 7fdf08d47700 1 -- 172.21.15.46:0/2960981091 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf040aec20 con 0x556f664a94a0 2024-05-09T00:19:51.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.611+0000 7fdf08d47700 1 -- 172.21.15.46:0/2960981091 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf040aacd0 con 0x556f664a94a0 2024-05-09T00:19:51.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.612+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/2960981091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f664a94a0 msgr2=0x556f6662c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:51.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.612+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/2960981091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f664a94a0 0x556f6662c780 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdf040aa800 tx=0x7fdf04009790 comp rx=0 tx=0).stop 2024-05-09T00:19:51.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.612+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/2960981091 shutdown_connections 2024-05-09T00:19:51.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.612+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/2960981091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66626400 0x556f66626810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.612+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/2960981091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 0x556f6662c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.612+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/2960981091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f664a94a0 0x556f6662c780 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.612+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/2960981091 >> 172.21.15.46:0/2960981091 conn(0x556f664acf70 msgr2=0x556f66621220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:51.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.612+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/2960981091 shutdown_connections 2024-05-09T00:19:51.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.612+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/2960981091 wait complete. 2024-05-09T00:19:51.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.613+0000 7fdf2adaeb80 1 -- start start 2024-05-09T00:19:51.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.613+0000 7fdf2adaeb80 1 Processor -- start 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.613+0000 7fdf2adaeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f664a94a0 0x556f666300b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.613+0000 7fdf2adaeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 0x556f666305f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.613+0000 7fdf2adaeb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f66630b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.613+0000 7fdf2adaeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556f66627160 con 0x556f66523d00 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.613+0000 7fdf2adaeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556f664e3b20 con 0x556f664a94a0 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.613+0000 7fdf2adaeb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556f66633020 con 0x556f66626400 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.613+0000 7fdf19933700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 0x556f666305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.613+0000 7fdf19933700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 0x556f666305f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40870/0 (socket says 172.21.15.46:40870) 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.613+0000 7fdf19933700 1 -- 172.21.15.46:0/3900273469 learned_addr learned my addr 172.21.15.46:0/3900273469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.613+0000 7fdf1a935700 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f66630b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdf1a134700 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f664a94a0 0x556f666300b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdf2adaeb80 10 client.0 _ll_register_callbacks cb 0x556f663ba6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdf1a935700 1 -- 172.21.15.46:0/3900273469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f664a94a0 msgr2=0x556f666300b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:51.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdf1a935700 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f664a94a0 0x556f666300b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdf1a935700 1 -- 172.21.15.46:0/3900273469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 msgr2=0x556f666305f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdf1a935700 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 0x556f666305f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdf1a935700 1 -- 172.21.15.46:0/3900273469 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f663c4490 con 0x556f66626400 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdf19933700 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 0x556f666305f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdf1a935700 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f66630b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c8f0 tx=0x7fdf0c004a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c61df8861ad9c471 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdefdffb700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdefdffb700 1 -- 172.21.15.46:0/3900273469 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c02e060 con 0x556f66626400 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdefdffb700 1 -- 172.21.15.46:0/3900273469 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fdf0c00a0d0 con 0x556f66626400 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.614+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3900273469 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf040aec20 con 0x556f66626400 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.615+0000 7fdefdffb700 1 -- 172.21.15.46:0/3900273469 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf0c004e10 con 0x556f66626400 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.615+0000 7fdf2adaeb80 10 client.26314 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.615+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3900273469 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556f6663da30 con 0x556f66626400 2024-05-09T00:19:51.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.615+0000 7fdefdffb700 1 -- 172.21.15.46:0/3900273469 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf040aec20 con 0x556f66626400 2024-05-09T00:19:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.615+0000 7fdefdffb700 1 -- 172.21.15.46:0/3900273469 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdf0c033070 con 0x556f66626400 2024-05-09T00:19:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.615+0000 7fdf2adaeb80 10 client.26314 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.615+0000 7fdf2adaeb80 10 client.26314 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:19:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.615+0000 7fdf2adaeb80 20 client.26314 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.615+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556f666400c0 0x556f66642540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.615+0000 7fdf2adaeb80 4 client.26314 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.615+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3900273469 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f66556a80 con 0x556f666400c0 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.615+0000 7fdf1a134700 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556f666400c0 0x556f66642540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.616+0000 7fdf1a134700 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556f666400c0 0x556f66642540 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce0b36d34c4f777a server_cookie=bcfcd36601451c87 in_seq=0 out_seq=0 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.616+0000 7fdefdffb700 10 client.26314 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdefdffb700 1 -- 172.21.15.46:0/3900273469 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f66556a80 con 0x556f666400c0 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdefdffb700 10 client.26314 handle_command_reply: tid=0 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdf2adaeb80 1 client.26314 shutdown 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3900273469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 msgr2=0x556f66630b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f66630b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c8f0 tx=0x7fdf0c004a10 comp rx=0 tx=0).stop 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3900273469 shutdown_connections 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556f666400c0 0x556f66642540 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f66630b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f664a94a0 0x556f666300b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdefdffb700 0 client.26314 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:19:51.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/3900273469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f66523d00 0x556f666305f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3900273469 >> 172.21.15.46:0/3900273469 conn(0x556f664acf70 msgr2=0x556f6661aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:51.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3900273469 shutdown_connections 2024-05-09T00:19:51.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3900273469 wait complete. 2024-05-09T00:19:51.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.617+0000 7fdf2adaeb80 20 client.26314 trim_cache size 0 max 16384 2024-05-09T00:19:51.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf2adaeb80 1 Processor -- start 2024-05-09T00:19:51.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf2adaeb80 1 -- start start 2024-05-09T00:19:51.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf2adaeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 0x556f66630970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf2adaeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66523d00 0x556f6661b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf2adaeb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f6663bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf2adaeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141af270 con 0x556f664a94a0 2024-05-09T00:19:51.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf2adaeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141acb00 con 0x556f66523d00 2024-05-09T00:19:51.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf2adaeb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141aa950 con 0x556f66626400 2024-05-09T00:19:51.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf1a134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 0x556f66630970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf1a134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 0x556f66630970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40882/0 (socket says 172.21.15.46:40882) 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf1a134700 1 -- 172.21.15.46:0/1366726720 learned_addr learned my addr 172.21.15.46:0/1366726720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf1a935700 1 --2- 172.21.15.46:0/1366726720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f6663bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.633+0000 7fdf19933700 1 --2- 172.21.15.46:0/1366726720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66523d00 0x556f6661b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.634+0000 7fdf1a935700 1 -- 172.21.15.46:0/1366726720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66523d00 msgr2=0x556f6661b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.634+0000 7fdf1a935700 1 --2- 172.21.15.46:0/1366726720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66523d00 0x556f6661b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.634+0000 7fdf1a935700 1 -- 172.21.15.46:0/1366726720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 msgr2=0x556f66630970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.634+0000 7fdf1a935700 1 --2- 172.21.15.46:0/1366726720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 0x556f66630970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.634+0000 7fdf1a935700 1 -- 172.21.15.46:0/1366726720 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f6661d9d0 con 0x556f66626400 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.634+0000 7fdf1a134700 1 --2- 172.21.15.46:0/1366726720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 0x556f66630970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.634+0000 7fdf1a935700 1 --2- 172.21.15.46:0/1366726720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f6663bde0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0062a0 tx=0x7fdf0c004b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffb3b4f75f90ce77 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.634+0000 7fdefffff700 1 -- 172.21.15.46:0/1366726720 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c039070 con 0x556f66626400 2024-05-09T00:19:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.634+0000 7fdefffff700 1 -- 172.21.15.46:0/1366726720 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf0c0086d0 con 0x556f66626400 2024-05-09T00:19:51.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.634+0000 7fdefffff700 1 -- 172.21.15.46:0/1366726720 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c02e040 con 0x556f66626400 2024-05-09T00:19:51.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.635+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/1366726720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 msgr2=0x556f6663bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:51.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.635+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/1366726720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f6663bde0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0062a0 tx=0x7fdf0c004b20 comp rx=0 tx=0).stop 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.635+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/1366726720 shutdown_connections 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.635+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/1366726720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f6663bde0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.635+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/1366726720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 0x556f66630970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.635+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/1366726720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66523d00 0x556f6661b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.635+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/1366726720 >> 172.21.15.46:0/1366726720 conn(0x556f664acf70 msgr2=0x556f6661aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.635+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/1366726720 shutdown_connections 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.635+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/1366726720 wait complete. 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf2adaeb80 1 -- start start 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf2adaeb80 1 Processor -- start 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf2adaeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 0x556f66638160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf2adaeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66523d00 0x556f666386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf2adaeb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f66646cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf2adaeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141aafc0 con 0x556f664a94a0 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf2adaeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141ac430 con 0x556f66523d00 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf2adaeb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141ab690 con 0x556f66626400 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf1a134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 0x556f66638160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf2adaeb80 10 client.0 _ll_register_callbacks cb 0x556f663ba6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf1a134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 0x556f66638160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40894/0 (socket says 172.21.15.46:40894) 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf19933700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66523d00 0x556f666386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.636+0000 7fdf1a134700 1 -- 172.21.15.46:0/3120409722 learned_addr learned my addr 172.21.15.46:0/3120409722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdf1a935700 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f66646cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdf1a134700 1 -- 172.21.15.46:0/3120409722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 msgr2=0x556f66646cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdf1a134700 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f66646cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdf1a134700 1 -- 172.21.15.46:0/3120409722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66523d00 msgr2=0x556f666386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdf1a134700 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66523d00 0x556f666386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdf1a134700 1 -- 172.21.15.46:0/3120409722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f66627600 con 0x556f664a94a0 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdf19933700 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66523d00 0x556f666386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:19:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdf1a134700 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 0x556f66638160 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fdf100117d0 tx=0x7fdf10013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f558c713598c53f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdefdffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdefdffb700 1 -- 172.21.15.46:0/3120409722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10031070 con 0x556f664a94a0 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdefdffb700 1 -- 172.21.15.46:0/3120409722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fdf10003c70 con 0x556f664a94a0 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3120409722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf0c0086d0 con 0x556f664a94a0 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdefdffb700 1 -- 172.21.15.46:0/3120409722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf1002c420 con 0x556f664a94a0 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdf2adaeb80 10 client.16782 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.637+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3120409722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556f666a9e10 con 0x556f664a94a0 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.638+0000 7fdefdffb700 1 -- 172.21.15.46:0/3120409722 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf0c0086d0 con 0x556f664a94a0 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.638+0000 7fdefdffb700 1 -- 172.21.15.46:0/3120409722 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdf10034030 con 0x556f664a94a0 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.638+0000 7fdf2adaeb80 10 client.16782 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.638+0000 7fdf2adaeb80 10 client.16782 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.638+0000 7fdf2adaeb80 20 client.16782 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.638+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556f666aa500 0x556f666ac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.638+0000 7fdf2adaeb80 4 client.16782 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.638+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3120409722 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f66550b20 con 0x556f666aa500 2024-05-09T00:19:51.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.638+0000 7fdf19933700 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556f666aa500 0x556f666ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf19933700 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556f666aa500 0x556f666ac920 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=581ba1981baedee9 server_cookie=bfed3c0f60521d0a in_seq=0 out_seq=0 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdefdffb700 10 client.16782 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdefdffb700 1 -- 172.21.15.46:0/3120409722 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f66550b20 con 0x556f666aa500 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdefdffb700 10 client.16782 handle_command_reply: tid=0 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf2adaeb80 1 client.16782 shutdown 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3120409722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 msgr2=0x556f66638160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 0x556f66638160 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fdf100117d0 tx=0x7fdf10013040 comp rx=0 tx=0).stop 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3120409722 shutdown_connections 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x556f666aa500 0x556f666ac920 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f66626400 0x556f66646cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f664a94a0 0x556f66638160 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf2adaeb80 1 --2- 172.21.15.46:0/3120409722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556f66523d00 0x556f666386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3120409722 >> 172.21.15.46:0/3120409722 conn(0x556f664acf70 msgr2=0x556f666253f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdefdffb700 0 client.16782 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:19:51.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3120409722 shutdown_connections 2024-05-09T00:19:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf2adaeb80 1 -- 172.21.15.46:0/3120409722 wait complete. 2024-05-09T00:19:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.639+0000 7fdf2adaeb80 20 client.16782 trim_cache size 0 max 16384 2024-05-09T00:19:51.643 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:19:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.640+0000 7fdf1c398700 1 -- 172.21.15.46:0/2242267992 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdf000b2900 msgr2=0x7fdf000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.640+0000 7fdf1c398700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdf000b2900 0x7fdf000b4e10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdf14103b00 tx=0x7fdf10009260 comp rx=0 tx=0).stop 2024-05-09T00:19:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.640+0000 7fdf1c398700 1 -- 172.21.15.46:0/2242267992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf14103310 msgr2=0x7fdf1419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.640+0000 7fdf1c398700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf14103310 0x7fdf1419b380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400e9c0 tx=0x7fdf0400ecd0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.640+0000 7fdf1c398700 1 -- 172.21.15.46:0/2242267992 shutdown_connections 2024-05-09T00:19:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.640+0000 7fdf1c398700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdf000b2900 0x7fdf000b4e10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.640+0000 7fdf1c398700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf141089f0 0x7fdf1419f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.640+0000 7fdf1c398700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf141029c0 0x7fdf1419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.640+0000 7fdf1c398700 1 --2- 172.21.15.46:0/2242267992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf14103310 0x7fdf1419b380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.640+0000 7fdf1c398700 1 -- 172.21.15.46:0/2242267992 >> 172.21.15.46:0/2242267992 conn(0x7fdf140fe6f0 msgr2=0x7fdf14106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.640+0000 7fdf1c398700 1 -- 172.21.15.46:0/2242267992 shutdown_connections 2024-05-09T00:19:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:51.640+0000 7fdf1c398700 1 -- 172.21.15.46:0/2242267992 wait complete. 2024-05-09T00:19:51.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:51 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:19:51.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:51 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:19:52.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:51 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:19:52.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:19:52.747 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:52.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:52 smithi046 ceph-mon[30151]: pgmap v631: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:52.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:52 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:19:52.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:52 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:19:52.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:52 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:19:52.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:52 smithi169 ceph-mon[34777]: pgmap v631: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:52.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:52 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:19:52.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:52 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:19:52.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:52 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:19:52.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:52 smithi142 ceph-mon[34533]: pgmap v631: 97 pgs: 97 active+clean; 4.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:52.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:52 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:19:52.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:52 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:19:52.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:52 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:19:54.447 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.447 INFO:tasks.workunit.client.0.smithi046.stdout:313/600: open fd = 3 2024-05-09T00:19:54.447 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.447 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.447 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.447 INFO:tasks.workunit.client.0.smithi046.stdout:314/600: open fd = 3 2024-05-09T00:19:54.447 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:315/600: open fd = 3 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:316/600: open fd = 3 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:317/600: open fd = 3 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.448 INFO:tasks.workunit.client.0.smithi046.stdout:318/600: open fd = 3 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:319/600: open fd = 3 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:320/600: open fd = 3 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:321/600: open fd = 3 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:322/600: open fd = 3 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:323/600: open fd = 3 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:324/600: open fd = 3 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:325/600: open fd = 3 2024-05-09T00:19:54.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:326/600: open fd = 3 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:327/600: open fd = 3 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:328/600: open fd = 3 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.451 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:329/600: open fd = 3 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:330/600: open fd = 3 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:331/600: open fd = 3 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:332/600: open fd = 3 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.452 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:333/600: open fd = 3 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:334/600: open fd = 3 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:335/600: open fd = 3 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:336/600: open fd = 3 2024-05-09T00:19:54.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:337/600: open fd = 3 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:338/600: open fd = 3 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:339/600: open fd = 3 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.454 INFO:tasks.workunit.client.0.smithi046.stdout:340/600: open fd = 3 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:341/600: open fd = 3 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:342/600: open fd = 3 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:343/600: open fd = 3 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:344/600: open fd = 3 2024-05-09T00:19:54.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:345/600: open fd = 3 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:346/600: open fd = 3 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:347/600: open fd = 3 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.456 INFO:tasks.workunit.client.0.smithi046.stdout:348/600: open fd = 3 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:349/600: open fd = 3 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:350/600: open fd = 3 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:351/600: open fd = 3 2024-05-09T00:19:54.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:352/600: open fd = 3 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:353/600: open fd = 3 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:354/600: open fd = 3 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.458 INFO:tasks.workunit.client.0.smithi046.stdout:355/600: open fd = 3 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:356/600: open fd = 3 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:357/600: open fd = 3 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:358/600: open fd = 3 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.459 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:359/600: open fd = 3 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:360/600: open fd = 3 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:361/600: open fd = 3 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:362/600: open fd = 3 2024-05-09T00:19:54.460 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:363/600: open fd = 3 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:364/600: open fd = 3 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:365/600: open fd = 3 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.461 INFO:tasks.workunit.client.0.smithi046.stdout:366/600: open fd = 3 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:367/600: open fd = 3 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:368/600: open fd = 3 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:369/600: open fd = 3 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.462 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:370/600: open fd = 3 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:371/600: open fd = 3 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:372/600: open fd = 3 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:373/600: open fd = 3 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.463 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.464 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.464 INFO:tasks.workunit.client.0.smithi046.stdout:374/600: open fd = 3 2024-05-09T00:19:54.464 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.464 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:19:54.464 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:19:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.531+0000 7f447009c700 1 -- 172.21.15.46:0/393040941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4468075b60 msgr2=0x7f4468075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.531+0000 7f447009c700 1 --2- 172.21.15.46:0/393040941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4468075b60 0x7f4468075f70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f445c009990 tx=0x7f445c009ca0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.531+0000 7f447009c700 1 -- 172.21.15.46:0/393040941 shutdown_connections 2024-05-09T00:19:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.531+0000 7f447009c700 1 --2- 172.21.15.46:0/393040941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4468110c00 0x7f4468113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.531+0000 7f447009c700 1 --2- 172.21.15.46:0/393040941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4468075b60 0x7f4468075f70 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.531+0000 7f447009c700 1 --2- 172.21.15.46:0/393040941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44680764b0 0x7f44681106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.531+0000 7f447009c700 1 -- 172.21.15.46:0/393040941 >> 172.21.15.46:0/393040941 conn(0x7f4468079f00 msgr2=0x7f446807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.531+0000 7f447009c700 1 -- 172.21.15.46:0/393040941 shutdown_connections 2024-05-09T00:19:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.531+0000 7f447009c700 1 -- 172.21.15.46:0/393040941 wait complete. 2024-05-09T00:19:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.532+0000 7f447009c700 1 Processor -- start 2024-05-09T00:19:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.532+0000 7f447009c700 1 -- start start 2024-05-09T00:19:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.532+0000 7f447009c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4468075b60 0x7f446819d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.532+0000 7f447009c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44680764b0 0x7f446819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.532+0000 7f447009c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4468110c00 0x7f44681a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.532+0000 7f447009c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44681167b0 con 0x7f4468075b60 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.532+0000 7f447009c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4468116630 con 0x7f4468110c00 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f447009c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4468116930 con 0x7f44680764b0 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f446de38700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4468075b60 0x7f446819d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f446de38700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4468075b60 0x7f446819d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40918/0 (socket says 172.21.15.46:40918) 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f446de38700 1 -- 172.21.15.46:0/224495201 learned_addr learned my addr 172.21.15.46:0/224495201 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f446d637700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44680764b0 0x7f446819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f446e639700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4468110c00 0x7f44681a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f446de38700 1 -- 172.21.15.46:0/224495201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44680764b0 msgr2=0x7f446819d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f446de38700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44680764b0 0x7f446819d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f446de38700 1 -- 172.21.15.46:0/224495201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4468110c00 msgr2=0x7f44681a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f446de38700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4468110c00 0x7f44681a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f446de38700 1 -- 172.21.15.46:0/224495201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44681a1fb0 con 0x7f4468075b60 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f446d637700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44680764b0 0x7f446819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f446de38700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4468075b60 0x7f446819d180 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f445c000c00 tx=0x7f445c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f445affd700 1 -- 172.21.15.46:0/224495201 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c027070 con 0x7f4468075b60 2024-05-09T00:19:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f445affd700 1 -- 172.21.15.46:0/224495201 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f445c004020 con 0x7f4468075b60 2024-05-09T00:19:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f447009c700 1 -- 172.21.15.46:0/224495201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44681a2240 con 0x7f4468075b60 2024-05-09T00:19:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.533+0000 7f445affd700 1 -- 172.21.15.46:0/224495201 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c022e00 con 0x7f4468075b60 2024-05-09T00:19:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.534+0000 7f447009c700 1 -- 172.21.15.46:0/224495201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44681a2720 con 0x7f4468075b60 2024-05-09T00:19:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.536+0000 7f445affd700 1 -- 172.21.15.46:0/224495201 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f445c02a070 con 0x7f4468075b60 2024-05-09T00:19:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.537+0000 7f445affd700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f44540b2880 0x7f44540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.537+0000 7f447eab2b80 1 Processor -- start 2024-05-09T00:19:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.537+0000 7f446d637700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f44540b2880 0x7f44540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.537+0000 7f445affd700 1 -- 172.21.15.46:0/224495201 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f445c0e3280 con 0x7f4468075b60 2024-05-09T00:19:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.537+0000 7f447eab2b80 1 -- start start 2024-05-09T00:19:54.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f447eab2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df18f3ed00 0x55df19047240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f447eab2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18ec44a0 0x55df19047780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f447eab2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df19041400 0x55df19041810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f447eab2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55df18f640a0 con 0x55df19041400 2024-05-09T00:19:54.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f447eab2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55df18f099a0 con 0x55df18f3ed00 2024-05-09T00:19:54.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f447eab2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55df18f60060 con 0x55df18ec44a0 2024-05-09T00:19:54.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446d637700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f44540b2880 0x7f44540b4d90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4468112b10 tx=0x7f4464005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:54.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446e639700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df19041400 0x55df19041810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446e639700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df19041400 0x55df19041810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40930/0 (socket says 172.21.15.46:40930) 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446e639700 1 -- 172.21.15.46:0/1536910073 learned_addr learned my addr 172.21.15.46:0/1536910073 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446de38700 1 --2- 172.21.15.46:0/1536910073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df18f3ed00 0x55df19047240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446d637700 1 --2- 172.21.15.46:0/1536910073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18ec44a0 0x55df19047780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446e639700 1 -- 172.21.15.46:0/1536910073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18ec44a0 msgr2=0x55df19047780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446e639700 1 --2- 172.21.15.46:0/1536910073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18ec44a0 0x55df19047780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446e639700 1 -- 172.21.15.46:0/1536910073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df18f3ed00 msgr2=0x55df19047240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446e639700 1 --2- 172.21.15.46:0/1536910073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df18f3ed00 0x55df19047240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446e639700 1 -- 172.21.15.46:0/1536910073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df18efa500 con 0x55df19041400 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446de38700 1 --2- 172.21.15.46:0/1536910073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df18f3ed00 0x55df19047240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.538+0000 7f446e639700 1 --2- 172.21.15.46:0/1536910073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df19041400 0x55df19041810 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4460009ef0 tx=0x7f4460009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16fb57694296ea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.539+0000 7f4453fff700 1 -- 172.21.15.46:0/1536910073 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446001a070 con 0x55df19041400 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.539+0000 7f4453fff700 1 -- 172.21.15.46:0/1536910073 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4460015410 con 0x55df19041400 2024-05-09T00:19:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.539+0000 7f4453fff700 1 -- 172.21.15.46:0/1536910073 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446000a660 con 0x55df19041400 2024-05-09T00:19:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.540+0000 7f447eab2b80 1 -- 172.21.15.46:0/1536910073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df19041400 msgr2=0x55df19041810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.540+0000 7f447eab2b80 1 --2- 172.21.15.46:0/1536910073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df19041400 0x55df19041810 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4460009ef0 tx=0x7f4460009f20 comp rx=0 tx=0).stop 2024-05-09T00:19:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.540+0000 7f447eab2b80 1 -- 172.21.15.46:0/1536910073 shutdown_connections 2024-05-09T00:19:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.540+0000 7f447eab2b80 1 --2- 172.21.15.46:0/1536910073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df19041400 0x55df19041810 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.540+0000 7f447eab2b80 1 --2- 172.21.15.46:0/1536910073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df18f3ed00 0x55df19047240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.540+0000 7f447eab2b80 1 --2- 172.21.15.46:0/1536910073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18ec44a0 0x55df19047780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.540+0000 7f447eab2b80 1 -- 172.21.15.46:0/1536910073 >> 172.21.15.46:0/1536910073 conn(0x55df18ec7f70 msgr2=0x55df1903c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.540+0000 7f447eab2b80 1 -- 172.21.15.46:0/1536910073 shutdown_connections 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.540+0000 7f447eab2b80 1 -- 172.21.15.46:0/1536910073 wait complete. 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.540+0000 7f447eab2b80 1 -- start start 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.540+0000 7f447eab2b80 1 Processor -- start 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f447eab2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 0x55df1904b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f447eab2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df18f3ed00 0x55df1904b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f447eab2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df19041400 0x55df1904bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f447eab2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55df19042160 con 0x55df18ec44a0 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f447eab2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55df18efeb20 con 0x55df18f3ed00 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f447eab2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55df1904e020 con 0x55df19041400 2024-05-09T00:19:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f447eab2b80 10 client.0 _ll_register_callbacks cb 0x55df18dd56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f446de38700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 0x55df1904b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f446de38700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 0x55df1904b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40934/0 (socket says 172.21.15.46:40934) 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f446de38700 1 -- 172.21.15.46:0/3349810804 learned_addr learned my addr 172.21.15.46:0/3349810804 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f446e639700 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df19041400 0x55df1904bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f446d637700 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df18f3ed00 0x55df1904b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f446de38700 1 -- 172.21.15.46:0/3349810804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df19041400 msgr2=0x55df1904bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f446de38700 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df19041400 0x55df1904bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f446de38700 1 -- 172.21.15.46:0/3349810804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df18f3ed00 msgr2=0x55df1904b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f446de38700 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df18f3ed00 0x55df1904b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f446de38700 1 -- 172.21.15.46:0/3349810804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df18ddf490 con 0x55df18ec44a0 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f446de38700 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 0x55df1904b110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f445c003800 tx=0x7f445c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96a4dcfde8ea918 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.541+0000 7f44517fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f44517fa700 1 -- 172.21.15.46:0/3349810804 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c0aa440 con 0x55df18ec44a0 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f44517fa700 1 -- 172.21.15.46:0/3349810804 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f445c0a6f20 con 0x55df18ec44a0 2024-05-09T00:19:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f447eab2b80 1 -- 172.21.15.46:0/3349810804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4460015410 con 0x55df18ec44a0 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f44517fa700 1 -- 172.21.15.46:0/3349810804 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f445c0ab030 con 0x55df18ec44a0 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f447eab2b80 10 client.16794 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f447eab2b80 1 -- 172.21.15.46:0/3349810804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df19058a90 con 0x55df18ec44a0 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f44517fa700 1 -- 172.21.15.46:0/3349810804 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4460015410 con 0x55df18ec44a0 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f44517fa700 1 -- 172.21.15.46:0/3349810804 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f445c0aab90 con 0x55df18ec44a0 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f447eab2b80 10 client.16794 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f447eab2b80 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f447eab2b80 20 client.16794 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55df1905b050 0x55df1905d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f447eab2b80 4 client.16794 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.542+0000 7f447eab2b80 1 -- 172.21.15.46:0/3349810804 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df18f71a80 con 0x55df1905b050 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.543+0000 7f446d637700 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55df1905b050 0x55df1905d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.543+0000 7f446d637700 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55df1905b050 0x55df1905d4e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3dd2f3ced1f4871 server_cookie=393f83f40ce2be8d in_seq=0 out_seq=0 2024-05-09T00:19:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.543+0000 7f44517fa700 10 client.16794 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f44517fa700 1 -- 172.21.15.46:0/3349810804 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df18f71a80 con 0x55df1905b050 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f44517fa700 10 client.16794 handle_command_reply: tid=0 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f447eab2b80 1 client.16794 shutdown 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f447eab2b80 1 -- 172.21.15.46:0/3349810804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 msgr2=0x55df1904b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 0x55df1904b110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f445c003800 tx=0x7f445c005fd0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f447eab2b80 1 -- 172.21.15.46:0/3349810804 shutdown_connections 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55df1905b050 0x55df1905d4e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df19041400 0x55df1904bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 0x55df1904b110 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f44517fa700 0 client.16794 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3349810804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df18f3ed00 0x55df1904b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f447eab2b80 1 -- 172.21.15.46:0/3349810804 >> 172.21.15.46:0/3349810804 conn(0x55df18ec7f70 msgr2=0x55df19035ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f447eab2b80 1 -- 172.21.15.46:0/3349810804 shutdown_connections 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f447eab2b80 1 -- 172.21.15.46:0/3349810804 wait complete. 2024-05-09T00:19:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.544+0000 7f447eab2b80 20 client.16794 trim_cache size 0 max 16384 2024-05-09T00:19:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.559+0000 7f447eab2b80 1 Processor -- start 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.559+0000 7f447eab2b80 1 -- start start 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f447eab2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 0x55df1904b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f447eab2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18f3ed00 0x55df19035ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f447eab2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df19041400 0x55df19036530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f447eab2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44681b3740 con 0x55df18ec44a0 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f447eab2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44681b0fd0 con 0x55df19041400 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f447eab2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44681aee00 con 0x55df18f3ed00 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f446de38700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 0x55df1904b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f446de38700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 0x55df1904b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40936/0 (socket says 172.21.15.46:40936) 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f446de38700 1 -- 172.21.15.46:0/3777720410 learned_addr learned my addr 172.21.15.46:0/3777720410 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f446d637700 1 --2- 172.21.15.46:0/3777720410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18f3ed00 0x55df19035ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f446de38700 1 -- 172.21.15.46:0/3777720410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18f3ed00 msgr2=0x55df19035ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f446de38700 1 --2- 172.21.15.46:0/3777720410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18f3ed00 0x55df19035ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f446de38700 1 -- 172.21.15.46:0/3777720410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df19041400 msgr2=0x55df19036530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f446de38700 1 --2- 172.21.15.46:0/3777720410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df19041400 0x55df19036530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f446de38700 1 -- 172.21.15.46:0/3777720410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df1904d960 con 0x55df18ec44a0 2024-05-09T00:19:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f446d637700 1 --2- 172.21.15.46:0/3777720410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18f3ed00 0x55df19035ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:19:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f446de38700 1 --2- 172.21.15.46:0/3777720410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 0x55df1904b9d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f445c0a69b0 tx=0x7f445c0b2fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddc3e05556a3227 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f44537fe700 1 -- 172.21.15.46:0/3777720410 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c0a60c0 con 0x55df18ec44a0 2024-05-09T00:19:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.560+0000 7f44537fe700 1 -- 172.21.15.46:0/3777720410 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f445c0aa440 con 0x55df18ec44a0 2024-05-09T00:19:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.561+0000 7f44537fe700 1 -- 172.21.15.46:0/3777720410 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c0aa930 con 0x55df18ec44a0 2024-05-09T00:19:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.561+0000 7f447eab2b80 1 -- 172.21.15.46:0/3777720410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 msgr2=0x55df1904b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.561+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3777720410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 0x55df1904b9d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f445c0a69b0 tx=0x7f445c0b2fe0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.561+0000 7f447eab2b80 1 -- 172.21.15.46:0/3777720410 shutdown_connections 2024-05-09T00:19:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.561+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3777720410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df19041400 0x55df19036530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.561+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3777720410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18ec44a0 0x55df1904b9d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.561+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3777720410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18f3ed00 0x55df19035ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.561+0000 7f447eab2b80 1 -- 172.21.15.46:0/3777720410 >> 172.21.15.46:0/3777720410 conn(0x55df18ec7f70 msgr2=0x55df19049a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.561+0000 7f447eab2b80 1 -- 172.21.15.46:0/3777720410 shutdown_connections 2024-05-09T00:19:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.561+0000 7f447eab2b80 1 -- 172.21.15.46:0/3777720410 wait complete. 2024-05-09T00:19:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.562+0000 7f447eab2b80 1 -- start start 2024-05-09T00:19:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.562+0000 7f447eab2b80 1 Processor -- start 2024-05-09T00:19:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.562+0000 7f447eab2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18ec44a0 0x55df190533f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.562+0000 7f447eab2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18f3ed00 0x55df19062270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.562+0000 7f447eab2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df19041400 0x55df190627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.562+0000 7f447eab2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44681af490 con 0x55df18f3ed00 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.562+0000 7f447eab2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44681b0900 con 0x55df19041400 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.562+0000 7f447eab2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44681afb60 con 0x55df18ec44a0 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.562+0000 7f447eab2b80 10 client.0 _ll_register_callbacks cb 0x55df18dd56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f446e639700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df19041400 0x55df190627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f446d637700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18f3ed00 0x55df19062270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f446e639700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df19041400 0x55df190627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:55830/0 (socket says 172.21.15.46:55830) 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f446d637700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18f3ed00 0x55df19062270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40950/0 (socket says 172.21.15.46:40950) 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f446e639700 1 -- 172.21.15.46:0/3207350931 learned_addr learned my addr 172.21.15.46:0/3207350931 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f446de38700 1 --2- 172.21.15.46:0/3207350931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18ec44a0 0x55df190533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f446d637700 1 -- 172.21.15.46:0/3207350931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18ec44a0 msgr2=0x55df190533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:54.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f446d637700 1 --2- 172.21.15.46:0/3207350931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18ec44a0 0x55df190533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f446d637700 1 -- 172.21.15.46:0/3207350931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df19041400 msgr2=0x55df190627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f446d637700 1 --2- 172.21.15.46:0/3207350931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df19041400 0x55df190627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f446d637700 1 -- 172.21.15.46:0/3207350931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df19062cf0 con 0x55df18f3ed00 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f446d637700 1 --2- 172.21.15.46:0/3207350931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18f3ed00 0x55df19062270 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x55df1903da00 tx=0x7f44640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8935b48a2513690 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f44517fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f44517fa700 1 -- 172.21.15.46:0/3207350931 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464004a30 con 0x55df18f3ed00 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f44517fa700 1 -- 172.21.15.46:0/3207350931 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4464004bd0 con 0x55df18f3ed00 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f447eab2b80 1 -- 172.21.15.46:0/3207350931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f445c0aa440 con 0x55df18f3ed00 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f44517fa700 1 -- 172.21.15.46:0/3207350931 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4464010e30 con 0x55df18f3ed00 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f447eab2b80 10 client.16800 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.563+0000 7f447eab2b80 1 -- 172.21.15.46:0/3207350931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df190c53c0 con 0x55df18f3ed00 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.564+0000 7f44517fa700 1 -- 172.21.15.46:0/3207350931 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f445c0aa440 con 0x55df18f3ed00 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.564+0000 7f44517fa700 1 -- 172.21.15.46:0/3207350931 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f44640053f0 con 0x55df18f3ed00 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.564+0000 7f447eab2b80 10 client.16800 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.564+0000 7f447eab2b80 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:54.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.564+0000 7f447eab2b80 20 client.16800 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.564+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3207350931 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55df190c5af0 0x55df190c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.564+0000 7f447eab2b80 4 client.16800 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.564+0000 7f447eab2b80 1 -- 172.21.15.46:0/3207350931 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df18f6bb20 con 0x55df190c5af0 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.564+0000 7f446de38700 1 --2- 172.21.15.46:0/3207350931 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55df190c5af0 0x55df190c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f44517fa700 10 client.16800 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f446de38700 1 --2- 172.21.15.46:0/3207350931 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55df190c5af0 0x55df190c7f10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb563ee1494bc2d server_cookie=d2e43fbb429ac783 in_seq=0 out_seq=0 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f44517fa700 1 -- 172.21.15.46:0/3207350931 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df18f6bb20 con 0x55df190c5af0 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f44517fa700 10 client.16800 handle_command_reply: tid=0 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f447eab2b80 1 client.16800 shutdown 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f447eab2b80 1 -- 172.21.15.46:0/3207350931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18f3ed00 msgr2=0x55df19062270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3207350931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18f3ed00 0x55df19062270 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x55df1903da00 tx=0x7f44640047c0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f447eab2b80 1 -- 172.21.15.46:0/3207350931 shutdown_connections 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3207350931 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55df190c5af0 0x55df190c7f10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3207350931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df19041400 0x55df190627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f44517fa700 0 client.16800 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3207350931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df18ec44a0 0x55df190533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f447eab2b80 1 --2- 172.21.15.46:0/3207350931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55df18f3ed00 0x55df19062270 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f447eab2b80 1 -- 172.21.15.46:0/3207350931 >> 172.21.15.46:0/3207350931 conn(0x55df18ec7f70 msgr2=0x55df19035ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f447eab2b80 1 -- 172.21.15.46:0/3207350931 shutdown_connections 2024-05-09T00:19:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f447eab2b80 1 -- 172.21.15.46:0/3207350931 wait complete. 2024-05-09T00:19:54.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.565+0000 7f447eab2b80 20 client.16800 trim_cache size 0 max 16384 2024-05-09T00:19:54.569 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:19:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.566+0000 7f447009c700 1 -- 172.21.15.46:0/224495201 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f44540b2880 msgr2=0x7f44540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.566+0000 7f447009c700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f44540b2880 0x7f44540b4d90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4468112b10 tx=0x7f4464005f50 comp rx=0 tx=0).stop 2024-05-09T00:19:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.566+0000 7f447009c700 1 -- 172.21.15.46:0/224495201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4468075b60 msgr2=0x7f446819d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.566+0000 7f447009c700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4468075b60 0x7f446819d180 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f445c000c00 tx=0x7f445c0057c0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.566+0000 7f447009c700 1 -- 172.21.15.46:0/224495201 shutdown_connections 2024-05-09T00:19:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.566+0000 7f447009c700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f44540b2880 0x7f44540b4d90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.566+0000 7f447009c700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4468110c00 0x7f44681a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.566+0000 7f447009c700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4468075b60 0x7f446819d180 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.566+0000 7f447009c700 1 --2- 172.21.15.46:0/224495201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44680764b0 0x7f446819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:54.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.566+0000 7f447009c700 1 -- 172.21.15.46:0/224495201 >> 172.21.15.46:0/224495201 conn(0x7f4468079f00 msgr2=0x7f44681011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:54.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.566+0000 7f447009c700 1 -- 172.21.15.46:0/224495201 shutdown_connections 2024-05-09T00:19:54.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:54.566+0000 7f447009c700 1 -- 172.21.15.46:0/224495201 wait complete. 2024-05-09T00:19:54.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:54 smithi046 ceph-mon[30151]: pgmap v632: 97 pgs: 97 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:19:54.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:54 smithi169 ceph-mon[34777]: pgmap v632: 97 pgs: 97 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:19:54.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:54 smithi142 ceph-mon[34533]: pgmap v632: 97 pgs: 97 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:19:56.282 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:19:56.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:19:56.550 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:56.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:56 smithi169 ceph-mon[34777]: pgmap v633: 97 pgs: 97 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:56.936 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:56 smithi046 ceph-mon[30151]: pgmap v633: 97 pgs: 97 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:56.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:56 smithi142 ceph-mon[34533]: pgmap v633: 97 pgs: 97 active+clean; 4.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:19:58.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.307+0000 7ff986c4a700 1 -- 172.21.15.46:0/2249382057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff980107df0 msgr2=0x7ff980108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:58.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.307+0000 7ff986c4a700 1 --2- 172.21.15.46:0/2249382057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff980107df0 0x7ff980108200 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff9700099c0 tx=0x7ff970009cd0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.307+0000 7ff986c4a700 1 -- 172.21.15.46:0/2249382057 shutdown_connections 2024-05-09T00:19:58.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.307+0000 7ff986c4a700 1 --2- 172.21.15.46:0/2249382057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff980108740 0x7ff9801100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.307+0000 7ff986c4a700 1 --2- 172.21.15.46:0/2249382057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9801034c0 0x7ff9801038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.307+0000 7ff986c4a700 1 --2- 172.21.15.46:0/2249382057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff980107df0 0x7ff980108200 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.307+0000 7ff986c4a700 1 -- 172.21.15.46:0/2249382057 >> 172.21.15.46:0/2249382057 conn(0x7ff9800fedc0 msgr2=0x7ff9801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:58.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.307+0000 7ff986c4a700 1 -- 172.21.15.46:0/2249382057 shutdown_connections 2024-05-09T00:19:58.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.307+0000 7ff986c4a700 1 -- 172.21.15.46:0/2249382057 wait complete. 2024-05-09T00:19:58.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.308+0000 7ff986c4a700 1 Processor -- start 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.308+0000 7ff986c4a700 1 -- start start 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.308+0000 7ff986c4a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9801034c0 0x7ff980199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.308+0000 7ff986c4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff980107df0 0x7ff980199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.308+0000 7ff986c4a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff980108740 0x7ff98019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff986c4a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff980113370 con 0x7ff980107df0 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff986c4a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801131f0 con 0x7ff980108740 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff986c4a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801134f0 con 0x7ff9801034c0 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff977fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff980107df0 0x7ff980199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff977fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff980107df0 0x7ff980199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53212/0 (socket says 172.21.15.46:53212) 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff977fff700 1 -- 172.21.15.46:0/3500343244 learned_addr learned my addr 172.21.15.46:0/3500343244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff9849e6700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9801034c0 0x7ff980199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff9851e7700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff980108740 0x7ff98019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff977fff700 1 -- 172.21.15.46:0/3500343244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9801034c0 msgr2=0x7ff980199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff977fff700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9801034c0 0x7ff980199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff977fff700 1 -- 172.21.15.46:0/3500343244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff980108740 msgr2=0x7ff98019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff977fff700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff980108740 0x7ff98019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff977fff700 1 -- 172.21.15.46:0/3500343244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9801a9180 con 0x7ff980107df0 2024-05-09T00:19:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff9851e7700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff980108740 0x7ff98019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:19:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff977fff700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff980107df0 0x7ff980199fc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff970006010 tx=0x7ff9700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff975ffb700 1 -- 172.21.15.46:0/3500343244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970027070 con 0x7ff980107df0 2024-05-09T00:19:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff975ffb700 1 -- 172.21.15.46:0/3500343244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff970003780 con 0x7ff980107df0 2024-05-09T00:19:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff986c4a700 1 -- 172.21.15.46:0/3500343244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9801a9410 con 0x7ff980107df0 2024-05-09T00:19:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.309+0000 7ff975ffb700 1 -- 172.21.15.46:0/3500343244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970022be0 con 0x7ff980107df0 2024-05-09T00:19:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.310+0000 7ff986c4a700 1 -- 172.21.15.46:0/3500343244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9801a9860 con 0x7ff980107df0 2024-05-09T00:19:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.312+0000 7ff975ffb700 1 -- 172.21.15.46:0/3500343244 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff97002a070 con 0x7ff980107df0 2024-05-09T00:19:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.313+0000 7ff975ffb700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9680b2880 0x7ff9680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.313+0000 7ff975ffb700 1 -- 172.21.15.46:0/3500343244 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff9700e2e20 con 0x7ff980107df0 2024-05-09T00:19:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.313+0000 7ff9849e6700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9680b2880 0x7ff9680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.313+0000 7ff995660b80 1 Processor -- start 2024-05-09T00:19:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff995660b80 1 -- start start 2024-05-09T00:19:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff9849e6700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9680b2880 0x7ff9680b4d90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff97c005950 tx=0x7ff97c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff995660b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed5202c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff995660b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed520800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff995660b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 0x5649ed51a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff995660b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5649ed43d120 con 0x5649ed39d520 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff995660b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5649ed3e2a20 con 0x5649ed51a480 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff995660b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5649ed4390e0 con 0x5649ed417d80 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff977fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed520800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff977fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed520800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53216/0 (socket says 172.21.15.46:53216) 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff977fff700 1 -- 172.21.15.46:0/4126746950 learned_addr learned my addr 172.21.15.46:0/4126746950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff9849e6700 1 --2- 172.21.15.46:0/4126746950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed5202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff9851e7700 1 --2- 172.21.15.46:0/4126746950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 0x5649ed51a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff977fff700 1 -- 172.21.15.46:0/4126746950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 msgr2=0x5649ed5202c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff977fff700 1 --2- 172.21.15.46:0/4126746950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed5202c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff977fff700 1 -- 172.21.15.46:0/4126746950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 msgr2=0x5649ed51a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff977fff700 1 --2- 172.21.15.46:0/4126746950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 0x5649ed51a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.314+0000 7ff977fff700 1 -- 172.21.15.46:0/4126746950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ed3d3580 con 0x5649ed39d520 2024-05-09T00:19:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.315+0000 7ff977fff700 1 --2- 172.21.15.46:0/4126746950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed520800 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff970004360 tx=0x7ff9700226c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd45a6a1fa91f1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.315+0000 7ff966ffd700 1 -- 172.21.15.46:0/4126746950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970009590 con 0x5649ed39d520 2024-05-09T00:19:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.315+0000 7ff966ffd700 1 -- 172.21.15.46:0/4126746950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff970022470 con 0x5649ed39d520 2024-05-09T00:19:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.315+0000 7ff966ffd700 1 -- 172.21.15.46:0/4126746950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700a93d0 con 0x5649ed39d520 2024-05-09T00:19:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.316+0000 7ff995660b80 1 -- 172.21.15.46:0/4126746950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 msgr2=0x5649ed520800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.316+0000 7ff995660b80 1 --2- 172.21.15.46:0/4126746950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed520800 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff970004360 tx=0x7ff9700226c0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.316+0000 7ff995660b80 1 -- 172.21.15.46:0/4126746950 shutdown_connections 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.316+0000 7ff995660b80 1 --2- 172.21.15.46:0/4126746950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 0x5649ed51a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.316+0000 7ff995660b80 1 --2- 172.21.15.46:0/4126746950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed5202c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.316+0000 7ff995660b80 1 --2- 172.21.15.46:0/4126746950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed520800 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.316+0000 7ff995660b80 1 -- 172.21.15.46:0/4126746950 >> 172.21.15.46:0/4126746950 conn(0x5649ed3a0ff0 msgr2=0x5649ed5152a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.316+0000 7ff995660b80 1 -- 172.21.15.46:0/4126746950 shutdown_connections 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.316+0000 7ff995660b80 1 -- 172.21.15.46:0/4126746950 wait complete. 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.316+0000 7ff995660b80 1 -- start start 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.316+0000 7ff995660b80 1 Processor -- start 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff995660b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed39d520 0x5649ed529b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff995660b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed417d80 0x5649ed52a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff995660b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed51a480 0x5649ed523f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff995660b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5649ed51b030 con 0x5649ed417d80 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff995660b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5649ed3d7ba0 con 0x5649ed39d520 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff995660b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5649ed51b1b0 con 0x5649ed51a480 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff995660b80 10 client.0 _ll_register_callbacks cb 0x5649ed2ae740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff977fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed417d80 0x5649ed52a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff977fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed417d80 0x5649ed52a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53222/0 (socket says 172.21.15.46:53222) 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff977fff700 1 -- 172.21.15.46:0/3232617250 learned_addr learned my addr 172.21.15.46:0/3232617250 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff9851e7700 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed51a480 0x5649ed523f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff9849e6700 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed39d520 0x5649ed529b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff977fff700 1 -- 172.21.15.46:0/3232617250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed51a480 msgr2=0x5649ed523f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff977fff700 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed51a480 0x5649ed523f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff977fff700 1 -- 172.21.15.46:0/3232617250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed39d520 msgr2=0x5649ed529b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff977fff700 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed39d520 0x5649ed529b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff977fff700 1 -- 172.21.15.46:0/3232617250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ed2b8510 con 0x5649ed417d80 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff977fff700 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed417d80 0x5649ed52a0d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff970009990 tx=0x7ff9700a9c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac10c54251d3b6f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.317+0000 7ff95ffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff95ffff700 1 -- 172.21.15.46:0/3232617250 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700c3070 con 0x5649ed417d80 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff95ffff700 1 -- 172.21.15.46:0/3232617250 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff9700a6280 con 0x5649ed417d80 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff995660b80 1 -- 172.21.15.46:0/3232617250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff970022470 con 0x5649ed417d80 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff95ffff700 1 -- 172.21.15.46:0/3232617250 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff970009590 con 0x5649ed417d80 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff995660b80 10 client.16812 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff995660b80 1 -- 172.21.15.46:0/3232617250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5649ed51b030 con 0x5649ed417d80 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff95ffff700 1 -- 172.21.15.46:0/3232617250 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff970022470 con 0x5649ed417d80 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff95ffff700 1 -- 172.21.15.46:0/3232617250 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff9700aeb60 con 0x5649ed417d80 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff995660b80 10 client.16812 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff995660b80 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff995660b80 20 client.16812 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff995660b80 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5649ed533b00 0x5649ed535f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff995660b80 4 client.16812 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.318+0000 7ff995660b80 1 -- 172.21.15.46:0/3232617250 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649ed44ab00 con 0x5649ed533b00 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.319+0000 7ff9849e6700 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5649ed533b00 0x5649ed535f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.319+0000 7ff9849e6700 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5649ed533b00 0x5649ed535f50 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8366a807845378 server_cookie=c044af133974aa33 in_seq=0 out_seq=0 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.319+0000 7ff95ffff700 10 client.16812 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff95ffff700 1 -- 172.21.15.46:0/3232617250 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5649ed44ab00 con 0x5649ed533b00 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff95ffff700 10 client.16812 handle_command_reply: tid=0 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff995660b80 1 client.16812 shutdown 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff995660b80 1 -- 172.21.15.46:0/3232617250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed417d80 msgr2=0x5649ed52a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff995660b80 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed417d80 0x5649ed52a0d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff970009990 tx=0x7ff9700a9c10 comp rx=0 tx=0).stop 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff995660b80 1 -- 172.21.15.46:0/3232617250 shutdown_connections 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff995660b80 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5649ed533b00 0x5649ed535f50 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff95ffff700 0 client.16812 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff995660b80 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed51a480 0x5649ed523f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff995660b80 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed39d520 0x5649ed529b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff995660b80 1 --2- 172.21.15.46:0/3232617250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed417d80 0x5649ed52a0d0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff995660b80 1 -- 172.21.15.46:0/3232617250 >> 172.21.15.46:0/3232617250 conn(0x5649ed3a0ff0 msgr2=0x5649ed50f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff995660b80 1 -- 172.21.15.46:0/3232617250 shutdown_connections 2024-05-09T00:19:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.320+0000 7ff995660b80 1 -- 172.21.15.46:0/3232617250 wait complete. 2024-05-09T00:19:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.321+0000 7ff995660b80 20 client.16812 trim_cache size 0 max 16384 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.341+0000 7ff995660b80 1 Processor -- start 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.341+0000 7ff995660b80 1 -- start start 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.341+0000 7ff995660b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed50faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.341+0000 7ff995660b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed5238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.341+0000 7ff995660b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 0x5649ed523df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.341+0000 7ff995660b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801b3650 con 0x5649ed39d520 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.341+0000 7ff995660b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801b0ee0 con 0x5649ed51a480 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.341+0000 7ff995660b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801aed10 con 0x5649ed417d80 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.341+0000 7ff9849e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed50faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff9849e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed50faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53234/0 (socket says 172.21.15.46:53234) 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff9849e6700 1 -- 172.21.15.46:0/3644389251 learned_addr learned my addr 172.21.15.46:0/3644389251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff9851e7700 1 --2- 172.21.15.46:0/3644389251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 0x5649ed523df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff977fff700 1 --2- 172.21.15.46:0/3644389251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed5238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff9849e6700 1 -- 172.21.15.46:0/3644389251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 msgr2=0x5649ed5238b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff9849e6700 1 --2- 172.21.15.46:0/3644389251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed5238b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff9849e6700 1 -- 172.21.15.46:0/3644389251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 msgr2=0x5649ed523df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff9849e6700 1 --2- 172.21.15.46:0/3644389251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 0x5649ed523df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff9849e6700 1 -- 172.21.15.46:0/3644389251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ed516290 con 0x5649ed39d520 2024-05-09T00:19:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff977fff700 1 --2- 172.21.15.46:0/3644389251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed5238b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:19:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff9849e6700 1 --2- 172.21.15.46:0/3644389251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed50faa0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff97c010570 tx=0x7ff97c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8959234bd0d62ec0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff9667fc700 1 -- 172.21.15.46:0/3644389251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c016070 con 0x5649ed39d520 2024-05-09T00:19:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff9667fc700 1 -- 172.21.15.46:0/3644389251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff97c010da0 con 0x5649ed39d520 2024-05-09T00:19:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.342+0000 7ff9667fc700 1 -- 172.21.15.46:0/3644389251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c01cb60 con 0x5649ed39d520 2024-05-09T00:19:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.343+0000 7ff995660b80 1 -- 172.21.15.46:0/3644389251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 msgr2=0x5649ed50faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.343+0000 7ff995660b80 1 --2- 172.21.15.46:0/3644389251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed50faa0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff97c010570 tx=0x7ff97c012000 comp rx=0 tx=0).stop 2024-05-09T00:19:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.343+0000 7ff995660b80 1 -- 172.21.15.46:0/3644389251 shutdown_connections 2024-05-09T00:19:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.343+0000 7ff995660b80 1 --2- 172.21.15.46:0/3644389251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 0x5649ed523df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.343+0000 7ff995660b80 1 --2- 172.21.15.46:0/3644389251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed50faa0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.343+0000 7ff995660b80 1 --2- 172.21.15.46:0/3644389251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed5238b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.343+0000 7ff995660b80 1 -- 172.21.15.46:0/3644389251 >> 172.21.15.46:0/3644389251 conn(0x5649ed3a0ff0 msgr2=0x5649ed51f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.343+0000 7ff995660b80 1 -- 172.21.15.46:0/3644389251 shutdown_connections 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.343+0000 7ff995660b80 1 -- 172.21.15.46:0/3644389251 wait complete. 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.343+0000 7ff995660b80 1 -- start start 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff995660b80 1 Processor -- start 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff995660b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed52efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff995660b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed52f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff995660b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 0x5649ed59e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff995660b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801af3a0 con 0x5649ed39d520 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff995660b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801b0810 con 0x5649ed51a480 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff995660b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801afa70 con 0x5649ed417d80 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff995660b80 10 client.0 _ll_register_callbacks cb 0x5649ed2ae740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff9849e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed52efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff9849e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed52efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53250/0 (socket says 172.21.15.46:53250) 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff9849e6700 1 -- 172.21.15.46:0/2430957450 learned_addr learned my addr 172.21.15.46:0/2430957450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff9851e7700 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 0x5649ed59e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.344+0000 7ff977fff700 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed52f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff9849e6700 1 -- 172.21.15.46:0/2430957450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 msgr2=0x5649ed52f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff9849e6700 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed52f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff9849e6700 1 -- 172.21.15.46:0/2430957450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 msgr2=0x5649ed59e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff9849e6700 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 0x5649ed59e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff9849e6700 1 -- 172.21.15.46:0/2430957450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ed59e800 con 0x5649ed39d520 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff9849e6700 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed52efe0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00f750 tx=0x7ff97c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5977748dfd54b80 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff95ffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff95ffff700 1 -- 172.21.15.46:0/2430957450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c03c060 con 0x5649ed39d520 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff995660b80 1 -- 172.21.15.46:0/2430957450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff97c010da0 con 0x5649ed39d520 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff95ffff700 1 -- 172.21.15.46:0/2430957450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff97c01cd70 con 0x5649ed39d520 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff95ffff700 1 -- 172.21.15.46:0/2430957450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff97c027420 con 0x5649ed39d520 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff995660b80 10 client.16818 fetch_fsmap learned FSMap version 26 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff995660b80 1 -- 172.21.15.46:0/2430957450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5649ed59f830 con 0x5649ed39d520 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.345+0000 7ff95ffff700 1 -- 172.21.15.46:0/2430957450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff97c010da0 con 0x5649ed39d520 2024-05-09T00:19:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.346+0000 7ff95ffff700 1 -- 172.21.15.46:0/2430957450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff97c041070 con 0x5649ed39d520 2024-05-09T00:19:58.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.346+0000 7ff995660b80 10 client.16818 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:19:58.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.346+0000 7ff995660b80 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:19:58.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.346+0000 7ff995660b80 20 client.16818 populate_metadata read hostname 'smithi046' 2024-05-09T00:19:58.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.346+0000 7ff995660b80 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5649ed533b00 0x5649ed5a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:19:58.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.346+0000 7ff995660b80 4 client.16818 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:19:58.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.346+0000 7ff995660b80 1 -- 172.21.15.46:0/2430957450 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5649ed444ba0 con 0x5649ed533b00 2024-05-09T00:19:58.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.346+0000 7ff977fff700 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5649ed533b00 0x5649ed5a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:19:58.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.346+0000 7ff977fff700 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5649ed533b00 0x5649ed5a1ef0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8086e5062e5ce52 server_cookie=1b4bf47ec212075d in_seq=0 out_seq=0 2024-05-09T00:19:58.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.346+0000 7ff95ffff700 10 client.16818 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:58.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff95ffff700 1 -- 172.21.15.46:0/2430957450 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5649ed444ba0 con 0x5649ed533b00 2024-05-09T00:19:58.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff95ffff700 10 client.16818 handle_command_reply: tid=0 2024-05-09T00:19:58.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff995660b80 1 client.16818 shutdown 2024-05-09T00:19:58.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff995660b80 1 -- 172.21.15.46:0/2430957450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 msgr2=0x5649ed52efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff995660b80 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed52efe0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00f750 tx=0x7ff97c010c20 comp rx=0 tx=0).stop 2024-05-09T00:19:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff995660b80 1 -- 172.21.15.46:0/2430957450 shutdown_connections 2024-05-09T00:19:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff995660b80 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5649ed533b00 0x5649ed5a1ef0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff995660b80 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649ed51a480 0x5649ed59e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff995660b80 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649ed39d520 0x5649ed52efe0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff995660b80 1 --2- 172.21.15.46:0/2430957450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649ed417d80 0x5649ed52f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff995660b80 1 -- 172.21.15.46:0/2430957450 >> 172.21.15.46:0/2430957450 conn(0x5649ed3a0ff0 msgr2=0x5649ed519470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff95ffff700 0 client.16818 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:19:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff995660b80 1 -- 172.21.15.46:0/2430957450 shutdown_connections 2024-05-09T00:19:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff995660b80 1 -- 172.21.15.46:0/2430957450 wait complete. 2024-05-09T00:19:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.348+0000 7ff995660b80 20 client.16818 trim_cache size 0 max 16384 2024-05-09T00:19:58.352 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:19:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:19:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "a8be3d77-b486-468f-96eb-fc39abb4879e", 2024-05-09T00:19:58.353 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:19:58.353 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:19:58.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.349+0000 7ff986c4a700 1 -- 172.21.15.46:0/3500343244 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9680b2880 msgr2=0x7ff9680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.349+0000 7ff986c4a700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9680b2880 0x7ff9680b4d90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff97c005950 tx=0x7ff97c00a300 comp rx=0 tx=0).stop 2024-05-09T00:19:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.349+0000 7ff986c4a700 1 -- 172.21.15.46:0/3500343244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff980107df0 msgr2=0x7ff980199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:19:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.349+0000 7ff986c4a700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff980107df0 0x7ff980199fc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff970006010 tx=0x7ff9700047b0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.349+0000 7ff986c4a700 1 -- 172.21.15.46:0/3500343244 shutdown_connections 2024-05-09T00:19:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.349+0000 7ff986c4a700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9680b2880 0x7ff9680b4d90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.349+0000 7ff986c4a700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff980108740 0x7ff98019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.349+0000 7ff986c4a700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9801034c0 0x7ff980199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.349+0000 7ff986c4a700 1 --2- 172.21.15.46:0/3500343244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff980107df0 0x7ff980199fc0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:19:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.349+0000 7ff986c4a700 1 -- 172.21.15.46:0/3500343244 >> 172.21.15.46:0/3500343244 conn(0x7ff9800fedc0 msgr2=0x7ff9801096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:19:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.349+0000 7ff986c4a700 1 -- 172.21.15.46:0/3500343244 shutdown_connections 2024-05-09T00:19:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:19:58.349+0000 7ff986c4a700 1 -- 172.21.15.46:0/3500343244 wait complete. 2024-05-09T00:19:58.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:58 smithi142 ceph-mon[34533]: pgmap v634: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:19:59.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:58 smithi046 ceph-mon[30151]: pgmap v634: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:19:59.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:19:59.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:58 smithi169 ceph-mon[34777]: pgmap v634: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:19:59.313 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:19:59.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:59 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:19:59.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:59 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:59.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:59 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:19:59.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:19:59 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:59.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:59 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:19:59.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:59 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:19:59.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:59 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:19:59.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:19:59 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:20:00.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:59 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:20:00.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:59 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:20:00.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:59 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:20:00.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:19:59 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:20:00.762 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:00 smithi046 ceph-mon[30151]: pgmap v635: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:00.762 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:00 smithi046 ceph-mon[30151]: overall HEALTH_OK 2024-05-09T00:20:00.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:00 smithi142 ceph-mon[34533]: pgmap v635: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:00.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:00 smithi142 ceph-mon[34533]: overall HEALTH_OK 2024-05-09T00:20:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.014+0000 7f99a200f700 1 -- 172.21.15.46:0/4199468116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f999c108760 msgr2=0x7f999c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.014+0000 7f99a200f700 1 --2- 172.21.15.46:0/4199468116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f999c108760 0x7f999c1100c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f999400b0d0 tx=0x7f999400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.014+0000 7f99a200f700 1 -- 172.21.15.46:0/4199468116 shutdown_connections 2024-05-09T00:20:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.014+0000 7f99a200f700 1 --2- 172.21.15.46:0/4199468116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f999c108760 0x7f999c1100c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.014+0000 7f99a200f700 1 --2- 172.21.15.46:0/4199468116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f999c1034e0 0x7f999c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.014+0000 7f99a200f700 1 --2- 172.21.15.46:0/4199468116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f999c107e10 0x7f999c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.014+0000 7f99a200f700 1 -- 172.21.15.46:0/4199468116 >> 172.21.15.46:0/4199468116 conn(0x7f999c0fef30 msgr2=0x7f999c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.014+0000 7f99a200f700 1 -- 172.21.15.46:0/4199468116 shutdown_connections 2024-05-09T00:20:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.014+0000 7f99a200f700 1 -- 172.21.15.46:0/4199468116 wait complete. 2024-05-09T00:20:01.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.015+0000 7f99a200f700 1 Processor -- start 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.015+0000 7f99a200f700 1 -- start start 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.015+0000 7f99a200f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f999c1034e0 0x7f999c0ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.015+0000 7f99a200f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f999c107e10 0x7f999c0ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.015+0000 7f99a200f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f999c108760 0x7f999c0ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.015+0000 7f99a200f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f999c113390 con 0x7f999c1034e0 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.015+0000 7f99a200f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f999c113210 con 0x7f999c107e10 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.015+0000 7f99a200f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f999c113510 con 0x7f999c108760 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.015+0000 7f999b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f999c1034e0 0x7f999c0ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f999b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f999c1034e0 0x7f999c0ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53270/0 (socket says 172.21.15.46:53270) 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f999b7fe700 1 -- 172.21.15.46:0/3325833529 learned_addr learned my addr 172.21.15.46:0/3325833529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f999affd700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f999c107e10 0x7f999c0ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f999bfff700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f999c108760 0x7f999c0ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f999b7fe700 1 -- 172.21.15.46:0/3325833529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f999c108760 msgr2=0x7f999c0ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f999b7fe700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f999c108760 0x7f999c0ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f999b7fe700 1 -- 172.21.15.46:0/3325833529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f999c107e10 msgr2=0x7f999c0ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f999b7fe700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f999c107e10 0x7f999c0ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f999b7fe700 1 -- 172.21.15.46:0/3325833529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c1a9190 con 0x7f999c1034e0 2024-05-09T00:20:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f999b7fe700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f999c1034e0 0x7f999c0ff410 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f998c00d950 tx=0x7f998c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f9998ff9700 1 -- 172.21.15.46:0/3325833529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c016070 con 0x7f999c1034e0 2024-05-09T00:20:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f99a200f700 1 -- 172.21.15.46:0/3325833529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c1a9480 con 0x7f999c1034e0 2024-05-09T00:20:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f9998ff9700 1 -- 172.21.15.46:0/3325833529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f998c00fd40 con 0x7f999c1034e0 2024-05-09T00:20:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f9998ff9700 1 -- 172.21.15.46:0/3325833529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c00e3b0 con 0x7f999c1034e0 2024-05-09T00:20:01.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.016+0000 7f99a200f700 1 -- 172.21.15.46:0/3325833529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c1a99c0 con 0x7f999c1034e0 2024-05-09T00:20:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.019+0000 7f9998ff9700 1 -- 172.21.15.46:0/3325833529 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f998c019070 con 0x7f999c1034e0 2024-05-09T00:20:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.020+0000 7f9998ff9700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f99880b2880 0x7f99880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.020+0000 7f99b0a25b80 1 Processor -- start 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.020+0000 7f999affd700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f99880b2880 0x7f99880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.020+0000 7f9998ff9700 1 -- 172.21.15.46:0/3325833529 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f998c0e1d10 con 0x7f999c1034e0 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.020+0000 7f99b0a25b80 1 -- start start 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.020+0000 7f99b0a25b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71615fd00 0x55e716268240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.020+0000 7f99b0a25b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7160e54a0 0x55e716268780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.020+0000 7f99b0a25b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e716262400 0x55e716262810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.020+0000 7f99b0a25b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e7161850a0 con 0x55e7160e54a0 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.020+0000 7f99b0a25b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e71612a9a0 con 0x55e71615fd00 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.020+0000 7f99b0a25b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e716181060 con 0x55e716262400 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f999affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7160e54a0 0x55e716268780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f999affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7160e54a0 0x55e716268780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53282/0 (socket says 172.21.15.46:53282) 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f999affd700 1 -- 172.21.15.46:0/1377389821 learned_addr learned my addr 172.21.15.46:0/1377389821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f999affd700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f99880b2880 0x7f99880b4d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f999c109a00 tx=0x7f9984009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f999b7fe700 1 --2- 172.21.15.46:0/1377389821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71615fd00 0x55e716268240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f999affd700 1 -- 172.21.15.46:0/1377389821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e716262400 msgr2=0x55e716262810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f999affd700 1 --2- 172.21.15.46:0/1377389821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e716262400 0x55e716262810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f999affd700 1 -- 172.21.15.46:0/1377389821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71615fd00 msgr2=0x55e716268240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f999affd700 1 --2- 172.21.15.46:0/1377389821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71615fd00 0x55e716268240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f999affd700 1 -- 172.21.15.46:0/1377389821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e71611b500 con 0x55e7160e54a0 2024-05-09T00:20:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f999b7fe700 1 --2- 172.21.15.46:0/1377389821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71615fd00 0x55e716268240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:20:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f999affd700 1 --2- 172.21.15.46:0/1377389821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7160e54a0 0x55e716268780 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9984011c70 tx=0x7f9984014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333e2e8d21cc539e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f9991ffb700 1 -- 172.21.15.46:0/1377389821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998400e870 con 0x55e7160e54a0 2024-05-09T00:20:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f9991ffb700 1 -- 172.21.15.46:0/1377389821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9984004830 con 0x55e7160e54a0 2024-05-09T00:20:01.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.021+0000 7f9991ffb700 1 -- 172.21.15.46:0/1377389821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99840054e0 con 0x55e7160e54a0 2024-05-09T00:20:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.022+0000 7f99b0a25b80 1 -- 172.21.15.46:0/1377389821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7160e54a0 msgr2=0x55e716268780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.022+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/1377389821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7160e54a0 0x55e716268780 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9984011c70 tx=0x7f9984014000 comp rx=0 tx=0).stop 2024-05-09T00:20:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.022+0000 7f99b0a25b80 1 -- 172.21.15.46:0/1377389821 shutdown_connections 2024-05-09T00:20:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.022+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/1377389821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e716262400 0x55e716262810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.022+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/1377389821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71615fd00 0x55e716268240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.022+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/1377389821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7160e54a0 0x55e716268780 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.022+0000 7f99b0a25b80 1 -- 172.21.15.46:0/1377389821 >> 172.21.15.46:0/1377389821 conn(0x55e7160e8f70 msgr2=0x55e71625d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.022+0000 7f99b0a25b80 1 -- 172.21.15.46:0/1377389821 shutdown_connections 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.022+0000 7f99b0a25b80 1 -- 172.21.15.46:0/1377389821 wait complete. 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.023+0000 7f99b0a25b80 1 -- start start 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.023+0000 7f99b0a25b80 1 Processor -- start 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.023+0000 7f99b0a25b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e7160e54a0 0x55e71626c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.023+0000 7f99b0a25b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 0x55e71626c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.023+0000 7f99b0a25b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e716262400 0x55e71626cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.023+0000 7f99b0a25b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e716263160 con 0x55e71615fd00 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.023+0000 7f99b0a25b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e71611fb20 con 0x55e716262400 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.023+0000 7f99b0a25b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e71626f020 con 0x55e7160e54a0 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f99b0a25b80 10 client.0 _ll_register_callbacks cb 0x55e715ff66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f999affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 0x55e71626c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f999affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 0x55e71626c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53290/0 (socket says 172.21.15.46:53290) 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f999affd700 1 -- 172.21.15.46:0/371956194 learned_addr learned my addr 172.21.15.46:0/371956194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f999b7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e7160e54a0 0x55e71626c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f999bfff700 1 --2- 172.21.15.46:0/371956194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e716262400 0x55e71626cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f999affd700 1 -- 172.21.15.46:0/371956194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e7160e54a0 msgr2=0x55e71626c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f999affd700 1 --2- 172.21.15.46:0/371956194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e7160e54a0 0x55e71626c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f999affd700 1 -- 172.21.15.46:0/371956194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e716262400 msgr2=0x55e71626cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f999affd700 1 --2- 172.21.15.46:0/371956194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e716262400 0x55e71626cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f999affd700 1 -- 172.21.15.46:0/371956194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e716000490 con 0x55e71615fd00 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f999affd700 1 --2- 172.21.15.46:0/371956194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 0x55e71626c650 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f99840052c0 tx=0x7f99840052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74bb15d97a6d652 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f997affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f997affd700 1 -- 172.21.15.46:0/371956194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998402f060 con 0x55e71615fd00 2024-05-09T00:20:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f997affd700 1 -- 172.21.15.46:0/371956194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9984004be0 con 0x55e71615fd00 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.024+0000 7f99b0a25b80 1 -- 172.21.15.46:0/371956194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9984004830 con 0x55e71615fd00 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.025+0000 7f997affd700 1 -- 172.21.15.46:0/371956194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9984004de0 con 0x55e71615fd00 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.025+0000 7f99b0a25b80 10 client.16830 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.025+0000 7f99b0a25b80 1 -- 172.21.15.46:0/371956194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e716279a00 con 0x55e71615fd00 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.025+0000 7f997affd700 1 -- 172.21.15.46:0/371956194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9984004830 con 0x55e71615fd00 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.025+0000 7f997affd700 1 -- 172.21.15.46:0/371956194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9984034070 con 0x55e71615fd00 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.025+0000 7f99b0a25b80 10 client.16830 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.025+0000 7f99b0a25b80 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.025+0000 7f99b0a25b80 20 client.16830 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.025+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/371956194 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e71627bf20 0x55e71627e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.025+0000 7f99b0a25b80 4 client.16830 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.025+0000 7f99b0a25b80 1 -- 172.21.15.46:0/371956194 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e716192a80 con 0x55e71627bf20 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.025+0000 7f999b7fe700 1 --2- 172.21.15.46:0/371956194 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e71627bf20 0x55e71627e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.026+0000 7f999b7fe700 1 --2- 172.21.15.46:0/371956194 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e71627bf20 0x55e71627e340 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6360e733507bdec server_cookie=176c8ce6517b07cf in_seq=0 out_seq=0 2024-05-09T00:20:01.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.026+0000 7f997affd700 10 client.16830 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f997affd700 1 -- 172.21.15.46:0/371956194 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e716192a80 con 0x55e71627bf20 2024-05-09T00:20:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f997affd700 10 client.16830 handle_command_reply: tid=0 2024-05-09T00:20:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f99b0a25b80 1 client.16830 shutdown 2024-05-09T00:20:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f99b0a25b80 1 -- 172.21.15.46:0/371956194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 msgr2=0x55e71626c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/371956194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 0x55e71626c650 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f99840052c0 tx=0x7f99840052f0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f99b0a25b80 1 -- 172.21.15.46:0/371956194 shutdown_connections 2024-05-09T00:20:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/371956194 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e71627bf20 0x55e71627e340 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/371956194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e716262400 0x55e71626cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/371956194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e7160e54a0 0x55e71626c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/371956194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 0x55e71626c650 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f997affd700 0 client.16830 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:01.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f99b0a25b80 1 -- 172.21.15.46:0/371956194 >> 172.21.15.46:0/371956194 conn(0x55e7160e8f70 msgr2=0x55e716256ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:01.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f99b0a25b80 1 -- 172.21.15.46:0/371956194 shutdown_connections 2024-05-09T00:20:01.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f99b0a25b80 1 -- 172.21.15.46:0/371956194 wait complete. 2024-05-09T00:20:01.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.027+0000 7f99b0a25b80 20 client.16830 trim_cache size 0 max 16384 2024-05-09T00:20:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f99b0a25b80 1 Processor -- start 2024-05-09T00:20:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f99b0a25b80 1 -- start start 2024-05-09T00:20:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f99b0a25b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e7160e54a0 0x55e71626b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f99b0a25b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71615fd00 0x55e716269cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f99b0a25b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e716262400 0x55e71626c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f99b0a25b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1b3830 con 0x55e716262400 2024-05-09T00:20:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f99b0a25b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1b10c0 con 0x55e71615fd00 2024-05-09T00:20:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f99b0a25b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1aeef0 con 0x55e7160e54a0 2024-05-09T00:20:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f999bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e716262400 0x55e71626c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f999bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e716262400 0x55e71626c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53298/0 (socket says 172.21.15.46:53298) 2024-05-09T00:20:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f999bfff700 1 -- 172.21.15.46:0/116399979 learned_addr learned my addr 172.21.15.46:0/116399979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f999b7fe700 1 --2- 172.21.15.46:0/116399979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e7160e54a0 0x55e71626b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f999affd700 1 --2- 172.21.15.46:0/116399979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71615fd00 0x55e716269cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f999bfff700 1 -- 172.21.15.46:0/116399979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e7160e54a0 msgr2=0x55e71626b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f999bfff700 1 --2- 172.21.15.46:0/116399979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e7160e54a0 0x55e71626b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f999bfff700 1 -- 172.21.15.46:0/116399979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71615fd00 msgr2=0x55e716269cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.048+0000 7f999bfff700 1 --2- 172.21.15.46:0/116399979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71615fd00 0x55e716269cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.049+0000 7f999bfff700 1 -- 172.21.15.46:0/116399979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7162599d0 con 0x55e716262400 2024-05-09T00:20:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.049+0000 7f999bfff700 1 --2- 172.21.15.46:0/116399979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e716262400 0x55e71626c9d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f999400b0d0 tx=0x7f9994009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c05524432db0ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.049+0000 7f99917fa700 1 -- 172.21.15.46:0/116399979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994009360 con 0x55e716262400 2024-05-09T00:20:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.049+0000 7f99917fa700 1 -- 172.21.15.46:0/116399979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9994003a00 con 0x55e716262400 2024-05-09T00:20:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.049+0000 7f99917fa700 1 -- 172.21.15.46:0/116399979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99940056f0 con 0x55e716262400 2024-05-09T00:20:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.050+0000 7f99b0a25b80 1 -- 172.21.15.46:0/116399979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e716262400 msgr2=0x55e71626c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.050+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/116399979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e716262400 0x55e71626c9d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f999400b0d0 tx=0x7f9994009d10 comp rx=0 tx=0).stop 2024-05-09T00:20:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.050+0000 7f99b0a25b80 1 -- 172.21.15.46:0/116399979 shutdown_connections 2024-05-09T00:20:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.050+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/116399979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e716262400 0x55e71626c9d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.050+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/116399979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e7160e54a0 0x55e71626b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.050+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/116399979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71615fd00 0x55e716269cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.050+0000 7f99b0a25b80 1 -- 172.21.15.46:0/116399979 >> 172.21.15.46:0/116399979 conn(0x55e7160e8f70 msgr2=0x55e716256ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.050+0000 7f99b0a25b80 1 -- 172.21.15.46:0/116399979 shutdown_connections 2024-05-09T00:20:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.050+0000 7f99b0a25b80 1 -- 172.21.15.46:0/116399979 wait complete. 2024-05-09T00:20:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.050+0000 7f99b0a25b80 1 -- start start 2024-05-09T00:20:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.050+0000 7f99b0a25b80 1 Processor -- start 2024-05-09T00:20:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f99b0a25b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7160e54a0 0x55e7162814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f99b0a25b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 0x55e71627e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f99b0a25b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e716262400 0x55e71627ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f99b0a25b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1af580 con 0x55e71615fd00 2024-05-09T00:20:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f99b0a25b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1b09f0 con 0x55e7160e54a0 2024-05-09T00:20:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f99b0a25b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1afc50 con 0x55e716262400 2024-05-09T00:20:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f99b0a25b80 10 client.0 _ll_register_callbacks cb 0x55e715ff66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f999affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 0x55e71627e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f999affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 0x55e71627e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53300/0 (socket says 172.21.15.46:53300) 2024-05-09T00:20:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f999affd700 1 -- 172.21.15.46:0/2939849571 learned_addr learned my addr 172.21.15.46:0/2939849571 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f999b7fe700 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7160e54a0 0x55e7162814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f999bfff700 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e716262400 0x55e71627ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f999affd700 1 -- 172.21.15.46:0/2939849571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e716262400 msgr2=0x55e71627ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f999affd700 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e716262400 0x55e71627ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f999affd700 1 -- 172.21.15.46:0/2939849571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7160e54a0 msgr2=0x55e7162814a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f999affd700 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7160e54a0 0x55e7162814a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f999affd700 1 -- 172.21.15.46:0/2939849571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e71627f4a0 con 0x55e71615fd00 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f999b7fe700 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7160e54a0 0x55e7162814a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f999affd700 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 0x55e71627e820 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9984005290 tx=0x7f9984009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1a1f775ba7a099 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.051+0000 7f997affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f997affd700 1 -- 172.21.15.46:0/2939849571 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998403a030 con 0x55e71615fd00 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f997affd700 1 -- 172.21.15.46:0/2939849571 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9984026d80 con 0x55e71615fd00 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f99b0a25b80 1 -- 172.21.15.46:0/2939849571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9994003a00 con 0x55e71615fd00 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f997affd700 1 -- 172.21.15.46:0/2939849571 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f998402f050 con 0x55e71615fd00 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f99b0a25b80 10 client.16836 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f99b0a25b80 1 -- 172.21.15.46:0/2939849571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e7162e6350 con 0x55e71615fd00 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f997affd700 1 -- 172.21.15.46:0/2939849571 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9994003a00 con 0x55e71615fd00 2024-05-09T00:20:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f997affd700 1 -- 172.21.15.46:0/2939849571 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f998400ec00 con 0x55e71615fd00 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f99b0a25b80 10 client.16836 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f99b0a25b80 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f99b0a25b80 20 client.16836 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e7162e6a60 0x55e7162e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f99b0a25b80 4 client.16836 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f99b0a25b80 1 -- 172.21.15.46:0/2939849571 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e71618cb20 con 0x55e7162e6a60 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.052+0000 7f999b7fe700 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e7162e6a60 0x55e7162e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f997affd700 10 client.16836 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f999b7fe700 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e7162e6a60 0x55e7162e8e80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a324370eb0c06d7 server_cookie=b482aa4a0aebddbd in_seq=0 out_seq=0 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f997affd700 1 -- 172.21.15.46:0/2939849571 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e71618cb20 con 0x55e7162e6a60 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f997affd700 10 client.16836 handle_command_reply: tid=0 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f99b0a25b80 1 client.16836 shutdown 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f99b0a25b80 1 -- 172.21.15.46:0/2939849571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 msgr2=0x55e71627e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 0x55e71627e820 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9984005290 tx=0x7f9984009380 comp rx=0 tx=0).stop 2024-05-09T00:20:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f99b0a25b80 1 -- 172.21.15.46:0/2939849571 shutdown_connections 2024-05-09T00:20:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e7162e6a60 0x55e7162e8e80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e716262400 0x55e71627ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7160e54a0 0x55e7162814a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f99b0a25b80 1 --2- 172.21.15.46:0/2939849571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e71615fd00 0x55e71627e820 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f99b0a25b80 1 -- 172.21.15.46:0/2939849571 >> 172.21.15.46:0/2939849571 conn(0x55e7160e8f70 msgr2=0x55e71626b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.053+0000 7f997affd700 0 client.16836 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99b0a25b80 1 -- 172.21.15.46:0/2939849571 shutdown_connections 2024-05-09T00:20:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99b0a25b80 1 -- 172.21.15.46:0/2939849571 wait complete. 2024-05-09T00:20:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99b0a25b80 20 client.16836 trim_cache size 0 max 16384 2024-05-09T00:20:01.060 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:20:01.061 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:20:01.061 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T00:20:01.061 INFO:teuthology.orchestra.run.smithi046.stdout: "a8be3d77-b486-468f-96eb-fc39abb4879e": { 2024-05-09T00:20:01.061 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T00:20:01.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "a8be3d77-b486-468f-96eb-fc39abb4879e", 2024-05-09T00:20:01.061 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T00:20:01.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:20:01.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:20:01.061 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:20:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99a200f700 1 -- 172.21.15.46:0/3325833529 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f99880b2880 msgr2=0x7f99880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99a200f700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f99880b2880 0x7f99880b4d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f999c109a00 tx=0x7f9984009260 comp rx=0 tx=0).stop 2024-05-09T00:20:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99a200f700 1 -- 172.21.15.46:0/3325833529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f999c1034e0 msgr2=0x7f999c0ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99a200f700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f999c1034e0 0x7f999c0ff410 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f998c00d950 tx=0x7f998c00dc60 comp rx=0 tx=0).stop 2024-05-09T00:20:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99a200f700 1 -- 172.21.15.46:0/3325833529 shutdown_connections 2024-05-09T00:20:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99a200f700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f99880b2880 0x7f99880b4d90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99a200f700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f999c108760 0x7f999c0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99a200f700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f999c1034e0 0x7f999c0ff410 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99a200f700 1 --2- 172.21.15.46:0/3325833529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f999c107e10 0x7f999c0ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99a200f700 1 -- 172.21.15.46:0/3325833529 >> 172.21.15.46:0/3325833529 conn(0x7f999c0fef30 msgr2=0x7f999c06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99a200f700 1 -- 172.21.15.46:0/3325833529 shutdown_connections 2024-05-09T00:20:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:01.054+0000 7f99a200f700 1 -- 172.21.15.46:0/3325833529 wait complete. 2024-05-09T00:20:01.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:00 smithi169 ceph-mon[34777]: pgmap v635: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:01.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:00 smithi169 ceph-mon[34777]: overall HEALTH_OK 2024-05-09T00:20:01.793 INFO:tasks.cephfs.filesystem:scrub status for tag:a8be3d77-b486-468f-96eb-fc39abb4879e - {'path': '/', 'tag': 'a8be3d77-b486-468f-96eb-fc39abb4879e', 'options': 'recursive,force'} 2024-05-09T00:20:02.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:02 smithi142 ceph-mon[34533]: pgmap v636: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:02.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:02 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:20:03.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:02 smithi046 ceph-mon[30151]: pgmap v636: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:03.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:02 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:20:03.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:02 smithi169 ceph-mon[34777]: pgmap v636: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:03.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:02 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:20:04.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:04 smithi142 ceph-mon[34533]: pgmap v637: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:20:05.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:04 smithi046 ceph-mon[30151]: pgmap v637: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:20:05.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:04 smithi169 ceph-mon[34777]: pgmap v637: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:20:06.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:06 smithi142 ceph-mon[34533]: pgmap v638: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:07.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:06 smithi046 ceph-mon[30151]: pgmap v638: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:07.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:06 smithi169 ceph-mon[34777]: pgmap v638: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:08.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:08 smithi142 ceph-mon[34533]: pgmap v639: 97 pgs: 97 active+clean; 5.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:20:09.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:08 smithi046 ceph-mon[30151]: pgmap v639: 97 pgs: 97 active+clean; 5.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:20:09.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:08 smithi169 ceph-mon[34777]: pgmap v639: 97 pgs: 97 active+clean; 5.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:20:10.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:10 smithi142 ceph-mon[34533]: pgmap v640: 97 pgs: 97 active+clean; 5.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:11.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:10 smithi046 ceph-mon[30151]: pgmap v640: 97 pgs: 97 active+clean; 5.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:11.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:10 smithi169 ceph-mon[34777]: pgmap v640: 97 pgs: 97 active+clean; 5.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:12.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:12 smithi142 ceph-mon[34533]: pgmap v641: 97 pgs: 97 active+clean; 5.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:13.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:12 smithi046 ceph-mon[30151]: pgmap v641: 97 pgs: 97 active+clean; 5.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:13.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:12 smithi169 ceph-mon[34777]: pgmap v641: 97 pgs: 97 active+clean; 5.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:14.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:14 smithi142 ceph-mon[34533]: pgmap v642: 97 pgs: 97 active+clean; 5.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:20:15.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:14 smithi046 ceph-mon[30151]: pgmap v642: 97 pgs: 97 active+clean; 5.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:20:15.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:14 smithi169 ceph-mon[34777]: pgmap v642: 97 pgs: 97 active+clean; 5.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:20:16.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:16 smithi142 ceph-mon[34533]: pgmap v643: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:17.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:16 smithi046 ceph-mon[30151]: pgmap v643: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:17.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:16 smithi169 ceph-mon[34777]: pgmap v643: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:18.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:18 smithi142 ceph-mon[34533]: pgmap v644: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:20:19.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:18 smithi046 ceph-mon[30151]: pgmap v644: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:20:19.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:18 smithi169 ceph-mon[34777]: pgmap v644: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:20:20.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:20 smithi142 ceph-mon[34533]: pgmap v645: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:21.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:20 smithi046 ceph-mon[30151]: pgmap v645: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:21.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:20 smithi169 ceph-mon[34777]: pgmap v645: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:22.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:22 smithi142 ceph-mon[34533]: pgmap v646: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:23.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:22 smithi046 ceph-mon[30151]: pgmap v646: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:23.107 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:22 smithi169 ceph-mon[34777]: pgmap v646: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:24.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:24 smithi142 ceph-mon[34533]: pgmap v647: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:20:25.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:24 smithi046 ceph-mon[30151]: pgmap v647: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:20:25.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:24 smithi169 ceph-mon[34777]: pgmap v647: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:20:26.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:26 smithi142 ceph-mon[34533]: pgmap v648: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:20:27.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:26 smithi046 ceph-mon[30151]: pgmap v648: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:20:27.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:26 smithi169 ceph-mon[34777]: pgmap v648: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:20:28.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:28 smithi142 ceph-mon[34533]: pgmap v649: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:20:29.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:28 smithi046 ceph-mon[30151]: pgmap v649: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:20:29.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:28 smithi169 ceph-mon[34777]: pgmap v649: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:20:30.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:30 smithi142 ceph-mon[34533]: pgmap v650: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:31.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:30 smithi046 ceph-mon[30151]: pgmap v650: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:31.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:30 smithi169 ceph-mon[34777]: pgmap v650: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:31.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:20:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:20:32.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:32 smithi046 ceph-mon[30151]: pgmap v651: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:32.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:32 smithi142 ceph-mon[34533]: pgmap v651: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:33.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:32 smithi169 ceph-mon[34777]: pgmap v651: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.846+0000 7fc1b897b700 1 -- 172.21.15.46:0/2016194190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4107df0 msgr2=0x7fc1b4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.846+0000 7fc1b897b700 1 --2- 172.21.15.46:0/2016194190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4107df0 0x7fc1b4108200 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80099c0 tx=0x7fc1a8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.847+0000 7fc1b897b700 1 -- 172.21.15.46:0/2016194190 shutdown_connections 2024-05-09T00:20:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.847+0000 7fc1b897b700 1 --2- 172.21.15.46:0/2016194190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4108740 0x7fc1b41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.847+0000 7fc1b897b700 1 --2- 172.21.15.46:0/2016194190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b41034c0 0x7fc1b41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.847+0000 7fc1b897b700 1 --2- 172.21.15.46:0/2016194190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4107df0 0x7fc1b4108200 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.847+0000 7fc1b897b700 1 -- 172.21.15.46:0/2016194190 >> 172.21.15.46:0/2016194190 conn(0x7fc1b40fedf0 msgr2=0x7fc1b4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.847+0000 7fc1b897b700 1 -- 172.21.15.46:0/2016194190 shutdown_connections 2024-05-09T00:20:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.847+0000 7fc1b897b700 1 -- 172.21.15.46:0/2016194190 wait complete. 2024-05-09T00:20:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b897b700 1 Processor -- start 2024-05-09T00:20:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b897b700 1 -- start start 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b897b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b41034c0 0x7fc1b419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b897b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4107df0 0x7fc1b419ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b897b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4108740 0x7fc1b4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b897b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4113350 con 0x7fc1b4107df0 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b897b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41131d0 con 0x7fc1b4108740 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b897b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41134d0 con 0x7fc1b41034c0 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4107df0 0x7fc1b419ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4107df0 0x7fc1b419ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34756/0 (socket says 172.21.15.46:34756) 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b1d9b700 1 -- 172.21.15.46:0/67631564 learned_addr learned my addr 172.21.15.46:0/67631564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b259c700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b41034c0 0x7fc1b419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b2d9d700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4108740 0x7fc1b4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b1d9b700 1 -- 172.21.15.46:0/67631564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b41034c0 msgr2=0x7fc1b419f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b41034c0 0x7fc1b419f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b1d9b700 1 -- 172.21.15.46:0/67631564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4108740 msgr2=0x7fc1b4199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.848+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4108740 0x7fc1b4199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.849+0000 7fc1b1d9b700 1 -- 172.21.15.46:0/67631564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b419a280 con 0x7fc1b4107df0 2024-05-09T00:20:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.849+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4107df0 0x7fc1b419ff00 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8006010 tx=0x7fc1a80055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.849+0000 7fc1a37fe700 1 -- 172.21.15.46:0/67631564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8026070 con 0x7fc1b4107df0 2024-05-09T00:20:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.849+0000 7fc1a37fe700 1 -- 172.21.15.46:0/67631564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a800f460 con 0x7fc1b4107df0 2024-05-09T00:20:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.849+0000 7fc1b897b700 1 -- 172.21.15.46:0/67631564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b419a510 con 0x7fc1b4107df0 2024-05-09T00:20:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.849+0000 7fc1a37fe700 1 -- 172.21.15.46:0/67631564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8009e60 con 0x7fc1b4107df0 2024-05-09T00:20:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.849+0000 7fc1b897b700 1 -- 172.21.15.46:0/67631564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b41a6770 con 0x7fc1b4107df0 2024-05-09T00:20:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.852+0000 7fc1a37fe700 1 -- 172.21.15.46:0/67631564 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc1a8029070 con 0x7fc1b4107df0 2024-05-09T00:20:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.853+0000 7fc1a37fe700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc19c0b2880 0x7fc19c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.853+0000 7fc1b259c700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc19c0b2880 0x7fc19c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.853+0000 7fc1a37fe700 1 -- 172.21.15.46:0/67631564 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc1a80e2920 con 0x7fc1b4107df0 2024-05-09T00:20:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.853+0000 7fc1c7391b80 1 Processor -- start 2024-05-09T00:20:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.853+0000 7fc1c7391b80 1 -- start start 2024-05-09T00:20:33.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.853+0000 7fc1b259c700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc19c0b2880 0x7fc19c0b4d90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4007be0 tx=0x7fc1a40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:33.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1c7391b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee39141160 0x55ee391da4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1c7391b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391daa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1c7391b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee391d46a0 0x55ee391d4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1c7391b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ee390f6ff0 con 0x55ee390574a0 2024-05-09T00:20:33.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1c7391b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ee390957f0 con 0x55ee391d46a0 2024-05-09T00:20:33.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1c7391b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ee390f2fb0 con 0x55ee39141160 2024-05-09T00:20:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1b1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391daa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1b1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391daa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34770/0 (socket says 172.21.15.46:34770) 2024-05-09T00:20:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1b1d9b700 1 -- 172.21.15.46:0/115790055 learned_addr learned my addr 172.21.15.46:0/115790055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1b259c700 1 --2- 172.21.15.46:0/115790055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee39141160 0x55ee391da4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1b2d9d700 1 --2- 172.21.15.46:0/115790055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee391d46a0 0x55ee391d4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1b1d9b700 1 -- 172.21.15.46:0/115790055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee39141160 msgr2=0x55ee391da4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/115790055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee39141160 0x55ee391da4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1b1d9b700 1 -- 172.21.15.46:0/115790055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee391d46a0 msgr2=0x55ee391d4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:33.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/115790055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee391d46a0 0x55ee391d4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1b1d9b700 1 -- 172.21.15.46:0/115790055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee3908d370 con 0x55ee390574a0 2024-05-09T00:20:33.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1b2d9d700 1 --2- 172.21.15.46:0/115790055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee391d46a0 0x55ee391d4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:20:33.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/115790055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391daa30 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80a7240 tx=0x7fc1a800b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a28794ca5e54f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:33.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1a0ff9700 1 -- 172.21.15.46:0/115790055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a80a9fd0 con 0x55ee390574a0 2024-05-09T00:20:33.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1a0ff9700 1 -- 172.21.15.46:0/115790055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a80ae380 con 0x55ee390574a0 2024-05-09T00:20:33.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.854+0000 7fc1a0ff9700 1 -- 172.21.15.46:0/115790055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a80b2b10 con 0x55ee390574a0 2024-05-09T00:20:33.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.855+0000 7fc1c7391b80 1 -- 172.21.15.46:0/115790055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 msgr2=0x55ee391daa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:33.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.855+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/115790055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391daa30 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80a7240 tx=0x7fc1a800b540 comp rx=0 tx=0).stop 2024-05-09T00:20:33.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.855+0000 7fc1c7391b80 1 -- 172.21.15.46:0/115790055 shutdown_connections 2024-05-09T00:20:33.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.855+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/115790055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee391d46a0 0x55ee391d4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.855+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/115790055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee39141160 0x55ee391da4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.855+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/115790055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391daa30 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.855+0000 7fc1c7391b80 1 -- 172.21.15.46:0/115790055 >> 172.21.15.46:0/115790055 conn(0x55ee390b76a0 msgr2=0x55ee391cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:33.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1c7391b80 1 -- 172.21.15.46:0/115790055 shutdown_connections 2024-05-09T00:20:33.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1c7391b80 1 -- 172.21.15.46:0/115790055 wait complete. 2024-05-09T00:20:33.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1c7391b80 1 -- start start 2024-05-09T00:20:33.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1c7391b80 1 Processor -- start 2024-05-09T00:20:33.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1c7391b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391e3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1c7391b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee39141160 0x55ee391e4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1c7391b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 0x55ee391de120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1c7391b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ee38f72ba0 con 0x55ee390574a0 2024-05-09T00:20:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1c7391b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ee390b1150 con 0x55ee39141160 2024-05-09T00:20:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1c7391b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ee391d5360 con 0x55ee391d46a0 2024-05-09T00:20:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1c7391b80 10 client.0 _ll_register_callbacks cb 0x55ee38f686c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1b259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391e3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1b259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391e3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34778/0 (socket says 172.21.15.46:34778) 2024-05-09T00:20:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.856+0000 7fc1b259c700 1 -- 172.21.15.46:0/2179282988 learned_addr learned my addr 172.21.15.46:0/2179282988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee39141160 0x55ee391e4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc1b2d9d700 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 0x55ee391de120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc1b259c700 1 -- 172.21.15.46:0/2179282988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 msgr2=0x55ee391de120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:33.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc1b259c700 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 0x55ee391de120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc1b259c700 1 -- 172.21.15.46:0/2179282988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee39141160 msgr2=0x55ee391e4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:33.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc1b259c700 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee39141160 0x55ee391e4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc1b259c700 1 -- 172.21.15.46:0/2179282988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee38f72490 con 0x55ee390574a0 2024-05-09T00:20:33.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc1b259c700 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391e3d20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40042e0 tx=0x7fc1a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc2c85d12d6d631 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:33.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc199ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:33.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc199ffb700 1 -- 172.21.15.46:0/2179282988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4024070 con 0x55ee390574a0 2024-05-09T00:20:33.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc199ffb700 1 -- 172.21.15.46:0/2179282988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc1a4003a90 con 0x55ee390574a0 2024-05-09T00:20:33.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2179282988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a80ae380 con 0x55ee390574a0 2024-05-09T00:20:33.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc199ffb700 1 -- 172.21.15.46:0/2179282988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc1a4030050 con 0x55ee390574a0 2024-05-09T00:20:33.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc1c7391b80 10 client.16848 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:33.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.857+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2179282988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee391eb910 con 0x55ee390574a0 2024-05-09T00:20:33.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.858+0000 7fc199ffb700 1 -- 172.21.15.46:0/2179282988 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a80ae380 con 0x55ee390574a0 2024-05-09T00:20:33.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.858+0000 7fc199ffb700 1 -- 172.21.15.46:0/2179282988 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc1a4020410 con 0x55ee390574a0 2024-05-09T00:20:33.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.858+0000 7fc1c7391b80 10 client.16848 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:33.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.858+0000 7fc1c7391b80 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:20:33.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.858+0000 7fc1c7391b80 20 client.16848 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:33.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.858+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ee391edf40 0x55ee391f0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.858+0000 7fc1c7391b80 4 client.16848 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:20:33.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.858+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2179282988 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee391049d0 con 0x55ee391edf40 2024-05-09T00:20:33.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.858+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ee391edf40 0x55ee391f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.859+0000 7fc199ffb700 10 client.16848 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:33.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.859+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ee391edf40 0x55ee391f0380 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8769ab303751b25 server_cookie=9355ca65fd611628 in_seq=0 out_seq=0 2024-05-09T00:20:33.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc199ffb700 1 -- 172.21.15.46:0/2179282988 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee391049d0 con 0x55ee391edf40 2024-05-09T00:20:33.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc199ffb700 10 client.16848 handle_command_reply: tid=0 2024-05-09T00:20:33.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc1c7391b80 1 client.16848 shutdown 2024-05-09T00:20:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2179282988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 msgr2=0x55ee391e3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391e3d20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40042e0 tx=0x7fc1a40047c0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2179282988 shutdown_connections 2024-05-09T00:20:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ee391edf40 0x55ee391f0380 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 0x55ee391de120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391e3d20 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc199ffb700 0 client.16848 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/2179282988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee39141160 0x55ee391e4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2179282988 >> 172.21.15.46:0/2179282988 conn(0x55ee390b76a0 msgr2=0x55ee391cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:33.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2179282988 shutdown_connections 2024-05-09T00:20:33.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2179282988 wait complete. 2024-05-09T00:20:33.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.860+0000 7fc1c7391b80 20 client.16848 trim_cache size 0 max 16384 2024-05-09T00:20:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.881+0000 7fc1c7391b80 1 Processor -- start 2024-05-09T00:20:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.881+0000 7fc1c7391b80 1 -- start start 2024-05-09T00:20:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.881+0000 7fc1c7391b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee390574a0 0x55ee391de0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.881+0000 7fc1c7391b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee39141160 0x55ee391f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.881+0000 7fc1c7391b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 0x55ee391f6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.881+0000 7fc1c7391b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41b1490 con 0x55ee39141160 2024-05-09T00:20:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.881+0000 7fc1c7391b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41aed20 con 0x55ee390574a0 2024-05-09T00:20:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.881+0000 7fc1c7391b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41acb50 con 0x55ee391d46a0 2024-05-09T00:20:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.881+0000 7fc1b1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee39141160 0x55ee391f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.881+0000 7fc1b1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee39141160 0x55ee391f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34784/0 (socket says 172.21.15.46:34784) 2024-05-09T00:20:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.881+0000 7fc1b1d9b700 1 -- 172.21.15.46:0/1705015331 learned_addr learned my addr 172.21.15.46:0/1705015331 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.881+0000 7fc1b259c700 1 --2- 172.21.15.46:0/1705015331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee390574a0 0x55ee391de0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.882+0000 7fc1b2d9d700 1 --2- 172.21.15.46:0/1705015331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 0x55ee391f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.882+0000 7fc1b1d9b700 1 -- 172.21.15.46:0/1705015331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 msgr2=0x55ee391f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.882+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/1705015331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 0x55ee391f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.882+0000 7fc1b1d9b700 1 -- 172.21.15.46:0/1705015331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee390574a0 msgr2=0x55ee391de0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.882+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/1705015331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee390574a0 0x55ee391de0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.882+0000 7fc1b1d9b700 1 -- 172.21.15.46:0/1705015331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee391dc6b0 con 0x55ee39141160 2024-05-09T00:20:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.882+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/1705015331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee39141160 0x55ee391f5b40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80a9f60 tx=0x7fc1a80a5e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691687a667ef1dab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.882+0000 7fc19bfff700 1 -- 172.21.15.46:0/1705015331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a80aa230 con 0x55ee39141160 2024-05-09T00:20:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.882+0000 7fc19bfff700 1 -- 172.21.15.46:0/1705015331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a80c9070 con 0x55ee39141160 2024-05-09T00:20:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.882+0000 7fc19bfff700 1 -- 172.21.15.46:0/1705015331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a80b2b70 con 0x55ee39141160 2024-05-09T00:20:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.883+0000 7fc1c7391b80 1 -- 172.21.15.46:0/1705015331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee39141160 msgr2=0x55ee391f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.883+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/1705015331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee39141160 0x55ee391f5b40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80a9f60 tx=0x7fc1a80a5e90 comp rx=0 tx=0).stop 2024-05-09T00:20:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.883+0000 7fc1c7391b80 1 -- 172.21.15.46:0/1705015331 shutdown_connections 2024-05-09T00:20:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.883+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/1705015331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 0x55ee391f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.883+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/1705015331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee390574a0 0x55ee391de0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.883+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/1705015331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee39141160 0x55ee391f5b40 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.883+0000 7fc1c7391b80 1 -- 172.21.15.46:0/1705015331 >> 172.21.15.46:0/1705015331 conn(0x55ee390b76a0 msgr2=0x55ee391dd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.883+0000 7fc1c7391b80 1 -- 172.21.15.46:0/1705015331 shutdown_connections 2024-05-09T00:20:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.883+0000 7fc1c7391b80 1 -- 172.21.15.46:0/1705015331 wait complete. 2024-05-09T00:20:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.883+0000 7fc1c7391b80 1 -- start start 2024-05-09T00:20:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.883+0000 7fc1c7391b80 1 Processor -- start 2024-05-09T00:20:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1c7391b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1c7391b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee39141160 0x55ee391e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1c7391b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 0x55ee391e8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1c7391b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41ad1e0 con 0x55ee390574a0 2024-05-09T00:20:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1c7391b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41ae650 con 0x55ee39141160 2024-05-09T00:20:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1c7391b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b41ad8b0 con 0x55ee391d46a0 2024-05-09T00:20:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1c7391b80 10 client.0 _ll_register_callbacks cb 0x55ee391cf0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1b259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1b259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391ea880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34798/0 (socket says 172.21.15.46:34798) 2024-05-09T00:20:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1b259c700 1 -- 172.21.15.46:0/2563336116 learned_addr learned my addr 172.21.15.46:0/2563336116 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1b2d9d700 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 0x55ee391e8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee39141160 0x55ee391e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1b259c700 1 -- 172.21.15.46:0/2563336116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 msgr2=0x55ee391e8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1b259c700 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 0x55ee391e8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1b259c700 1 -- 172.21.15.46:0/2563336116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee39141160 msgr2=0x55ee391e7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1b259c700 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee39141160 0x55ee391e7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1b259c700 1 -- 172.21.15.46:0/2563336116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee391e89f0 con 0x55ee390574a0 2024-05-09T00:20:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee39141160 0x55ee391e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:20:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc1b259c700 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391ea880 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40042b0 tx=0x7fc1a40206c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d750e2e8a442f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.884+0000 7fc199ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc199ffb700 1 -- 172.21.15.46:0/2563336116 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4024040 con 0x55ee390574a0 2024-05-09T00:20:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc199ffb700 1 -- 172.21.15.46:0/2563336116 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc1a4003890 con 0x55ee390574a0 2024-05-09T00:20:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2563336116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a80c9070 con 0x55ee390574a0 2024-05-09T00:20:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc199ffb700 1 -- 172.21.15.46:0/2563336116 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc1a4020e30 con 0x55ee390574a0 2024-05-09T00:20:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc1c7391b80 10 client.16854 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2563336116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee39258750 con 0x55ee390574a0 2024-05-09T00:20:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc199ffb700 1 -- 172.21.15.46:0/2563336116 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a80c9070 con 0x55ee390574a0 2024-05-09T00:20:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc199ffb700 1 -- 172.21.15.46:0/2563336116 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc1a4029b60 con 0x55ee390574a0 2024-05-09T00:20:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc1c7391b80 10 client.16854 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc1c7391b80 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:20:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc1c7391b80 20 client.16854 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ee39258db0 0x55ee3925b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc1c7391b80 4 client.16854 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:20:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.885+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2563336116 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee390fea70 con 0x55ee39258db0 2024-05-09T00:20:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.886+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ee39258db0 0x55ee3925b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.886+0000 7fc199ffb700 10 client.16854 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.886+0000 7fc1b1d9b700 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ee39258db0 0x55ee3925b1d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60753c1bdfa71da2 server_cookie=3875b8d88f9ef54b in_seq=0 out_seq=0 2024-05-09T00:20:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.886+0000 7fc199ffb700 1 -- 172.21.15.46:0/2563336116 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ee390fea70 con 0x55ee39258db0 2024-05-09T00:20:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.886+0000 7fc199ffb700 10 client.16854 handle_command_reply: tid=0 2024-05-09T00:20:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.886+0000 7fc1c7391b80 1 client.16854 shutdown 2024-05-09T00:20:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.887+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2563336116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 msgr2=0x55ee391ea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.887+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391ea880 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40042b0 tx=0x7fc1a40206c0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.887+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2563336116 shutdown_connections 2024-05-09T00:20:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.887+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ee39258db0 0x55ee3925b1d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.887+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee391d46a0 0x55ee391e8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.887+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee390574a0 0x55ee391ea880 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.887+0000 7fc1c7391b80 1 --2- 172.21.15.46:0/2563336116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ee39141160 0x55ee391e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.887+0000 7fc199ffb700 0 client.16854 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.887+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2563336116 >> 172.21.15.46:0/2563336116 conn(0x55ee390b76a0 msgr2=0x55ee391d7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.887+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2563336116 shutdown_connections 2024-05-09T00:20:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.887+0000 7fc1c7391b80 1 -- 172.21.15.46:0/2563336116 wait complete. 2024-05-09T00:20:33.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.887+0000 7fc1c7391b80 20 client.16854 trim_cache size 0 max 16384 2024-05-09T00:20:33.899 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:20:33.899 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:20:33.899 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:20:33.899 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:20:33.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.888+0000 7fc1b897b700 1 -- 172.21.15.46:0/67631564 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc19c0b2880 msgr2=0x7fc19c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:33.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.888+0000 7fc1b897b700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc19c0b2880 0x7fc19c0b4d90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4007be0 tx=0x7fc1a40073d0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.888+0000 7fc1b897b700 1 -- 172.21.15.46:0/67631564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4107df0 msgr2=0x7fc1b419ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.888+0000 7fc1b897b700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4107df0 0x7fc1b419ff00 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8006010 tx=0x7fc1a80055f0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.888+0000 7fc1b897b700 1 -- 172.21.15.46:0/67631564 shutdown_connections 2024-05-09T00:20:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.888+0000 7fc1b897b700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc19c0b2880 0x7fc19c0b4d90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.888+0000 7fc1b897b700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1b4108740 0x7fc1b4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.888+0000 7fc1b897b700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b41034c0 0x7fc1b419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.888+0000 7fc1b897b700 1 --2- 172.21.15.46:0/67631564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b4107df0 0x7fc1b419ff00 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.888+0000 7fc1b897b700 1 -- 172.21.15.46:0/67631564 >> 172.21.15.46:0/67631564 conn(0x7fc1b40fedf0 msgr2=0x7fc1b4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.888+0000 7fc1b897b700 1 -- 172.21.15.46:0/67631564 shutdown_connections 2024-05-09T00:20:33.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:33.888+0000 7fc1b897b700 1 -- 172.21.15.46:0/67631564 wait complete. 2024-05-09T00:20:34.770 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:20:34.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:20:34.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:34 smithi142 ceph-mon[34533]: pgmap v652: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:20:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:20:35.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:34 smithi046 ceph-mon[30151]: pgmap v652: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:20:35.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:34 smithi169 ceph-mon[34777]: pgmap v652: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:20:36.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:36 smithi046 ceph-mon[30151]: pgmap v653: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.801+0000 7fa93fc43700 1 -- 172.21.15.46:0/1054400941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938110be0 msgr2=0x7fa938113000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.801+0000 7fa93fc43700 1 --2- 172.21.15.46:0/1054400941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938110be0 0x7fa938113000 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa93000b0a0 tx=0x7fa93000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:20:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.801+0000 7fa93fc43700 1 -- 172.21.15.46:0/1054400941 shutdown_connections 2024-05-09T00:20:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.801+0000 7fa93fc43700 1 --2- 172.21.15.46:0/1054400941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938110be0 0x7fa938113000 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.801+0000 7fa93fc43700 1 --2- 172.21.15.46:0/1054400941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa938076c10 0x7fa938077020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.801+0000 7fa93fc43700 1 --2- 172.21.15.46:0/1054400941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa938077560 0x7fa9381106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.801+0000 7fa93fc43700 1 -- 172.21.15.46:0/1054400941 >> 172.21.15.46:0/1054400941 conn(0x7fa9380ffea0 msgr2=0x7fa9381022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.801+0000 7fa93fc43700 1 -- 172.21.15.46:0/1054400941 shutdown_connections 2024-05-09T00:20:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.801+0000 7fa93fc43700 1 -- 172.21.15.46:0/1054400941 wait complete. 2024-05-09T00:20:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.802+0000 7fa93fc43700 1 Processor -- start 2024-05-09T00:20:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.802+0000 7fa93fc43700 1 -- start start 2024-05-09T00:20:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.802+0000 7fa93fc43700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa938076c10 0x7fa93819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.802+0000 7fa93fc43700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa938077560 0x7fa93819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93fc43700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938110be0 0x7fa9381a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93fc43700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9381166a0 con 0x7fa938110be0 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93fc43700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa938116520 con 0x7fa938077560 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93fc43700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa938116820 con 0x7fa938076c10 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93e1e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938110be0 0x7fa9381a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93e1e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938110be0 0x7fa9381a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54978/0 (socket says 172.21.15.46:54978) 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93e1e0700 1 -- 172.21.15.46:0/2509499385 learned_addr learned my addr 172.21.15.46:0/2509499385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93d9df700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa938076c10 0x7fa93819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93d1de700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa938077560 0x7fa93819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93e1e0700 1 -- 172.21.15.46:0/2509499385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa938076c10 msgr2=0x7fa93819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93e1e0700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa938076c10 0x7fa93819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93e1e0700 1 -- 172.21.15.46:0/2509499385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa938077560 msgr2=0x7fa93819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93e1e0700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa938077560 0x7fa93819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93e1e0700 1 -- 172.21.15.46:0/2509499385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9381a1e90 con 0x7fa938110be0 2024-05-09T00:20:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93d1de700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa938077560 0x7fa93819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:20:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93e1e0700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938110be0 0x7fa9381a17f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa93000b780 tx=0x7fa930004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa92effd700 1 -- 172.21.15.46:0/2509499385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930021070 con 0x7fa938110be0 2024-05-09T00:20:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa92effd700 1 -- 172.21.15.46:0/2509499385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa930007e30 con 0x7fa938110be0 2024-05-09T00:20:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa93fc43700 1 -- 172.21.15.46:0/2509499385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9381a2120 con 0x7fa938110be0 2024-05-09T00:20:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.803+0000 7fa92effd700 1 -- 172.21.15.46:0/2509499385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9300047b0 con 0x7fa938110be0 2024-05-09T00:20:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.804+0000 7fa93fc43700 1 -- 172.21.15.46:0/2509499385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9381a9aa0 con 0x7fa938110be0 2024-05-09T00:20:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.804+0000 7fa93fc43700 1 -- 172.21.15.46:0/2509499385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa938078350 con 0x7fa938110be0 2024-05-09T00:20:36.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.807+0000 7fa92effd700 1 -- 172.21.15.46:0/2509499385 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa930007980 con 0x7fa938110be0 2024-05-09T00:20:36.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.808+0000 7fa92effd700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa9240b2880 0x7fa9240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:36.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.808+0000 7fa93d9df700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa9240b2880 0x7fa9240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:36.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.808+0000 7fa92effd700 1 -- 172.21.15.46:0/2509499385 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fa93000f020 con 0x7fa938110be0 2024-05-09T00:20:36.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.808+0000 7fa93d9df700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa9240b2880 0x7fa9240b4d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa934009c60 tx=0x7fa934009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:36.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:36.812+0000 7fa92effd700 1 -- 172.21.15.46:0/2509499385 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa93002a020 con 0x7fa938110be0 2024-05-09T00:20:36.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:36 smithi142 ceph-mon[34533]: pgmap v653: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.077+0000 7fa93fc43700 1 -- 172.21.15.46:0/2509499385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa93802e5f0 con 0x7fa938110be0 2024-05-09T00:20:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.078+0000 7fa92effd700 1 -- 172.21.15.46:0/2509499385 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7fa930003c00 con 0x7fa938110be0 2024-05-09T00:20:37.078 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:20:37.078 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:20:37.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.080+0000 7fa93fc43700 1 -- 172.21.15.46:0/2509499385 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa9240b2880 msgr2=0x7fa9240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.080+0000 7fa93fc43700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa9240b2880 0x7fa9240b4d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa934009c60 tx=0x7fa934009260 comp rx=0 tx=0).stop 2024-05-09T00:20:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.080+0000 7fa93fc43700 1 -- 172.21.15.46:0/2509499385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938110be0 msgr2=0x7fa9381a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.080+0000 7fa93fc43700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938110be0 0x7fa9381a17f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa93000b780 tx=0x7fa930004d00 comp rx=0 tx=0).stop 2024-05-09T00:20:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.080+0000 7fa93fc43700 1 -- 172.21.15.46:0/2509499385 shutdown_connections 2024-05-09T00:20:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.080+0000 7fa93fc43700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa9240b2880 0x7fa9240b4d90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.080+0000 7fa93fc43700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938110be0 0x7fa9381a17f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.080+0000 7fa93fc43700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa938076c10 0x7fa93819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.080+0000 7fa93fc43700 1 --2- 172.21.15.46:0/2509499385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa938077560 0x7fa93819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.080+0000 7fa93fc43700 1 -- 172.21.15.46:0/2509499385 >> 172.21.15.46:0/2509499385 conn(0x7fa9380ffea0 msgr2=0x7fa9381004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.080+0000 7fa93fc43700 1 -- 172.21.15.46:0/2509499385 shutdown_connections 2024-05-09T00:20:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:37.080+0000 7fa93fc43700 1 -- 172.21.15.46:0/2509499385 wait complete. 2024-05-09T00:20:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:20:37.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:36 smithi169 ceph-mon[34777]: pgmap v653: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:37.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:20:37.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:37 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2509499385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:20:38.006 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:37 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2509499385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:20:38.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:37 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2509499385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:20:38.077 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:20:38.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:38 smithi046 ceph-mon[30151]: pgmap v654: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:20:38.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:38 smithi142 ceph-mon[34533]: pgmap v654: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:20:39.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:38 smithi169 ceph-mon[34777]: pgmap v654: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:20:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.907+0000 7f5748fb7700 1 -- 172.21.15.46:0/2609246377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5744109870 msgr2=0x7f57441111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.907+0000 7f5748fb7700 1 --2- 172.21.15.46:0/2609246377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5744109870 0x7f57441111d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f573c00b0d0 tx=0x7f573c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.908+0000 7f5748fb7700 1 -- 172.21.15.46:0/2609246377 shutdown_connections 2024-05-09T00:20:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.908+0000 7f5748fb7700 1 --2- 172.21.15.46:0/2609246377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5744109870 0x7f57441111d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.908+0000 7f5748fb7700 1 --2- 172.21.15.46:0/2609246377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57441045f0 0x7f5744104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.908+0000 7f5748fb7700 1 --2- 172.21.15.46:0/2609246377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5744108f20 0x7f5744109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.908+0000 7f5748fb7700 1 -- 172.21.15.46:0/2609246377 >> 172.21.15.46:0/2609246377 conn(0x7f5744100040 msgr2=0x7f5744102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.908+0000 7f5748fb7700 1 -- 172.21.15.46:0/2609246377 shutdown_connections 2024-05-09T00:20:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.908+0000 7f5748fb7700 1 -- 172.21.15.46:0/2609246377 wait complete. 2024-05-09T00:20:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f5748fb7700 1 Processor -- start 2024-05-09T00:20:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f5748fb7700 1 -- start start 2024-05-09T00:20:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f5748fb7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57441045f0 0x7f574419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f5748fb7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5744108f20 0x7f574419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f5748fb7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5744109870 0x7f574419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f5748fb7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f57441144a0 con 0x7f5744109870 2024-05-09T00:20:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f5748fb7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5744114320 con 0x7f57441045f0 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f5748fb7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5744114620 con 0x7f5744108f20 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f5742d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5744109870 0x7f574419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f574259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57441045f0 0x7f574419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f5741d9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5744108f20 0x7f574419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f5742d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5744109870 0x7f574419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55010/0 (socket says 172.21.15.46:55010) 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f574259c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57441045f0 0x7f574419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:36444/0 (socket says 172.21.15.46:36444) 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.909+0000 7f5742d9d700 1 -- 172.21.15.46:0/2389058754 learned_addr learned my addr 172.21.15.46:0/2389058754 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.910+0000 7f5742d9d700 1 -- 172.21.15.46:0/2389058754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5744108f20 msgr2=0x7f574419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.910+0000 7f5742d9d700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5744108f20 0x7f574419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.910+0000 7f5742d9d700 1 -- 172.21.15.46:0/2389058754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57441045f0 msgr2=0x7f574419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.910+0000 7f5742d9d700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57441045f0 0x7f574419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.910+0000 7f5742d9d700 1 -- 172.21.15.46:0/2389058754 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574419fcc0 con 0x7f5744109870 2024-05-09T00:20:39.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.910+0000 7f574259c700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57441045f0 0x7f574419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:20:39.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.910+0000 7f5742d9d700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5744109870 0x7f574419f620 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f573c0095c0 tx=0x7f573c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:39.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.910+0000 7f57337fe700 1 -- 172.21.15.46:0/2389058754 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c01f070 con 0x7f5744109870 2024-05-09T00:20:39.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.910+0000 7f5748fb7700 1 -- 172.21.15.46:0/2389058754 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574419ff50 con 0x7f5744109870 2024-05-09T00:20:39.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.910+0000 7f57337fe700 1 -- 172.21.15.46:0/2389058754 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f573c007e20 con 0x7f5744109870 2024-05-09T00:20:39.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.910+0000 7f57337fe700 1 -- 172.21.15.46:0/2389058754 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c01b4a0 con 0x7f5744109870 2024-05-09T00:20:39.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.910+0000 7f5748fb7700 1 -- 172.21.15.46:0/2389058754 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57440759f0 con 0x7f5744109870 2024-05-09T00:20:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.913+0000 7f57337fe700 1 -- 172.21.15.46:0/2389058754 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f573c022070 con 0x7f5744109870 2024-05-09T00:20:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.913+0000 7f57337fe700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f572c0b2880 0x7f572c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.914+0000 7f574259c700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f572c0b2880 0x7f572c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.914+0000 7f57337fe700 1 -- 172.21.15.46:0/2389058754 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f573c00f020 con 0x7f5744109870 2024-05-09T00:20:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.914+0000 7f57579cdb80 1 Processor -- start 2024-05-09T00:20:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.914+0000 7f574259c700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f572c0b2880 0x7f572c0b4d90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f573400ffd0 tx=0x7f573400f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.914+0000 7f57579cdb80 1 -- start start 2024-05-09T00:20:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.914+0000 7f57579cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc28240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.914+0000 7f57579cdb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc28780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.914+0000 7f57579cdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfc22400 0x563dbfc22810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.914+0000 7f57579cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563dbfb450a0 con 0x563dbfb1fd00 2024-05-09T00:20:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.914+0000 7f57579cdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563dbfaea9a0 con 0x563dbfc22400 2024-05-09T00:20:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.914+0000 7f57579cdb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563dbfb41060 con 0x563dbfaa54a0 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f574259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc28240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f574259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc28240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55014/0 (socket says 172.21.15.46:55014) 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f574259c700 1 -- 172.21.15.46:0/3366130856 learned_addr learned my addr 172.21.15.46:0/3366130856 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f5741d9b700 1 --2- 172.21.15.46:0/3366130856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc28780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f5742d9d700 1 --2- 172.21.15.46:0/3366130856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfc22400 0x563dbfc22810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f574259c700 1 -- 172.21.15.46:0/3366130856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 msgr2=0x563dbfc28780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f574259c700 1 --2- 172.21.15.46:0/3366130856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc28780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f574259c700 1 -- 172.21.15.46:0/3366130856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfc22400 msgr2=0x563dbfc22810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f574259c700 1 --2- 172.21.15.46:0/3366130856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfc22400 0x563dbfc22810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f574259c700 1 -- 172.21.15.46:0/3366130856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dbfadb500 con 0x563dbfb1fd00 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f574259c700 1 --2- 172.21.15.46:0/3366130856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc28240 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f573401b850 tx=0x7f5734017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84390fb46b89f933 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f5730ff9700 1 -- 172.21.15.46:0/3366130856 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573402b070 con 0x563dbfb1fd00 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f5730ff9700 1 -- 172.21.15.46:0/3366130856 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5734019b70 con 0x563dbfb1fd00 2024-05-09T00:20:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.915+0000 7f5730ff9700 1 -- 172.21.15.46:0/3366130856 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734026bb0 con 0x563dbfb1fd00 2024-05-09T00:20:39.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.916+0000 7f57579cdb80 1 -- 172.21.15.46:0/3366130856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 msgr2=0x563dbfc28240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:39.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.916+0000 7f57579cdb80 1 --2- 172.21.15.46:0/3366130856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc28240 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f573401b850 tx=0x7f5734017000 comp rx=0 tx=0).stop 2024-05-09T00:20:39.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.916+0000 7f57579cdb80 1 -- 172.21.15.46:0/3366130856 shutdown_connections 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.916+0000 7f57579cdb80 1 --2- 172.21.15.46:0/3366130856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfc22400 0x563dbfc22810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.916+0000 7f57579cdb80 1 --2- 172.21.15.46:0/3366130856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc28240 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.916+0000 7f57579cdb80 1 --2- 172.21.15.46:0/3366130856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc28780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.916+0000 7f57579cdb80 1 -- 172.21.15.46:0/3366130856 >> 172.21.15.46:0/3366130856 conn(0x563dbfaa8f70 msgr2=0x563dbfc1d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.916+0000 7f57579cdb80 1 -- 172.21.15.46:0/3366130856 shutdown_connections 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.917+0000 7f57579cdb80 1 -- 172.21.15.46:0/3366130856 wait complete. 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.917+0000 7f57579cdb80 1 -- start start 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.917+0000 7f57579cdb80 1 Processor -- start 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.917+0000 7f57579cdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfaa54a0 0x563dbfc2bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.917+0000 7f57579cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc2c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.917+0000 7f57579cdb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfc22400 0x563dbfc2ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.917+0000 7f57579cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563dbfc23160 con 0x563dbfb1fd00 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.917+0000 7f57579cdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563dbfadfb20 con 0x563dbfaa54a0 2024-05-09T00:20:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.917+0000 7f57579cdb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563dbfc2f020 con 0x563dbfc22400 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.917+0000 7f57579cdb80 10 client.0 _ll_register_callbacks cb 0x563dbf9b66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.917+0000 7f5741d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc2c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5741d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc2c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55028/0 (socket says 172.21.15.46:55028) 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5741d9b700 1 -- 172.21.15.46:0/998345960 learned_addr learned my addr 172.21.15.46:0/998345960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f574259c700 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfaa54a0 0x563dbfc2bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5742d9d700 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfc22400 0x563dbfc2ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5741d9b700 1 -- 172.21.15.46:0/998345960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfc22400 msgr2=0x563dbfc2ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5741d9b700 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfc22400 0x563dbfc2ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5741d9b700 1 -- 172.21.15.46:0/998345960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfaa54a0 msgr2=0x563dbfc2bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5741d9b700 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfaa54a0 0x563dbfc2bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5741d9b700 1 -- 172.21.15.46:0/998345960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dbf9c0490 con 0x563dbfb1fd00 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5741d9b700 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc2c530 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f573800b990 tx=0x7f573800bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdedaa1824c0d6a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5729ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5729ffb700 1 -- 172.21.15.46:0/998345960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738003c90 con 0x563dbfb1fd00 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5729ffb700 1 -- 172.21.15.46:0/998345960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f5738004830 con 0x563dbfb1fd00 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f57579cdb80 1 -- 172.21.15.46:0/998345960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5734019b70 con 0x563dbfb1fd00 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f5729ffb700 1 -- 172.21.15.46:0/998345960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5738003e30 con 0x563dbfb1fd00 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f57579cdb80 10 client.16872 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.918+0000 7f57579cdb80 1 -- 172.21.15.46:0/998345960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563dbfc39a20 con 0x563dbfb1fd00 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.919+0000 7f5729ffb700 1 -- 172.21.15.46:0/998345960 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5734019b70 con 0x563dbfb1fd00 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.919+0000 7f5729ffb700 1 -- 172.21.15.46:0/998345960 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5738019030 con 0x563dbfb1fd00 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.919+0000 7f57579cdb80 10 client.16872 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.919+0000 7f57579cdb80 10 client.16872 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.919+0000 7f57579cdb80 20 client.16872 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.919+0000 7f57579cdb80 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x563dbfc3c060 0x563dbfc3e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.919+0000 7f57579cdb80 4 client.16872 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:20:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.919+0000 7f57579cdb80 1 -- 172.21.15.46:0/998345960 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dbfb52a80 con 0x563dbfc3c060 2024-05-09T00:20:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.919+0000 7f574259c700 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x563dbfc3c060 0x563dbfc3e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.920+0000 7f574259c700 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x563dbfc3c060 0x563dbfc3e480 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40137a877d55c858 server_cookie=fe166ef96d499763 in_seq=0 out_seq=0 2024-05-09T00:20:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.920+0000 7f5729ffb700 10 client.16872 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:20:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f5729ffb700 1 -- 172.21.15.46:0/998345960 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563dbfb52a80 con 0x563dbfc3c060 2024-05-09T00:20:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f5729ffb700 10 client.16872 handle_command_reply: tid=0 2024-05-09T00:20:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f57579cdb80 1 client.16872 shutdown 2024-05-09T00:20:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f57579cdb80 1 -- 172.21.15.46:0/998345960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 msgr2=0x563dbfc2c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f57579cdb80 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc2c530 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f573800b990 tx=0x7f573800bca0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f57579cdb80 1 -- 172.21.15.46:0/998345960 shutdown_connections 2024-05-09T00:20:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f57579cdb80 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x563dbfc3c060 0x563dbfc3e480 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f57579cdb80 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfc22400 0x563dbfc2ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f57579cdb80 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfaa54a0 0x563dbfc2bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f57579cdb80 1 --2- 172.21.15.46:0/998345960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc2c530 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f57579cdb80 1 -- 172.21.15.46:0/998345960 >> 172.21.15.46:0/998345960 conn(0x563dbfaa8f70 msgr2=0x563dbfc16ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f5729ffb700 0 client.16872 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:20:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f57579cdb80 1 -- 172.21.15.46:0/998345960 shutdown_connections 2024-05-09T00:20:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f57579cdb80 1 -- 172.21.15.46:0/998345960 wait complete. 2024-05-09T00:20:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.921+0000 7f57579cdb80 20 client.16872 trim_cache size 0 max 16384 2024-05-09T00:20:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.936+0000 7f57579cdb80 1 Processor -- start 2024-05-09T00:20:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.936+0000 7f57579cdb80 1 -- start start 2024-05-09T00:20:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f57579cdb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f57579cdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfb1fd00 0x563dbfc17a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f57579cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfc22400 0x563dbfc2af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f57579cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f57441af2c0 con 0x563dbfc22400 2024-05-09T00:20:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f57579cdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f57441acb50 con 0x563dbfb1fd00 2024-05-09T00:20:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f57579cdb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f57441aa9c0 con 0x563dbfaa54a0 2024-05-09T00:20:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f5742d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfc22400 0x563dbfc2af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f5742d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfc22400 0x563dbfc2af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55044/0 (socket says 172.21.15.46:55044) 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f5742d9d700 1 -- 172.21.15.46:0/916897811 learned_addr learned my addr 172.21.15.46:0/916897811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f574259c700 1 --2- 172.21.15.46:0/916897811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f5741d9b700 1 --2- 172.21.15.46:0/916897811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfb1fd00 0x563dbfc17a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f5742d9d700 1 -- 172.21.15.46:0/916897811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 msgr2=0x563dbfc274c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f5742d9d700 1 --2- 172.21.15.46:0/916897811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc274c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f5742d9d700 1 -- 172.21.15.46:0/916897811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfb1fd00 msgr2=0x563dbfc17a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f5742d9d700 1 --2- 172.21.15.46:0/916897811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfb1fd00 0x563dbfc17a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f5742d9d700 1 -- 172.21.15.46:0/916897811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dbfc2e890 con 0x563dbfc22400 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f574259c700 1 --2- 172.21.15.46:0/916897811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc274c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f5741d9b700 1 --2- 172.21.15.46:0/916897811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfb1fd00 0x563dbfc17a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f5742d9d700 1 --2- 172.21.15.46:0/916897811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfc22400 0x563dbfc2af50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f573c0aaa70 tx=0x7f573c00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131b83689a32caf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f572bfff700 1 -- 172.21.15.46:0/916897811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c0aa5f0 con 0x563dbfc22400 2024-05-09T00:20:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f572bfff700 1 -- 172.21.15.46:0/916897811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f573c0b3040 con 0x563dbfc22400 2024-05-09T00:20:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.937+0000 7f572bfff700 1 -- 172.21.15.46:0/916897811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c003e40 con 0x563dbfc22400 2024-05-09T00:20:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.938+0000 7f57579cdb80 1 -- 172.21.15.46:0/916897811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfc22400 msgr2=0x563dbfc2af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.938+0000 7f57579cdb80 1 --2- 172.21.15.46:0/916897811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfc22400 0x563dbfc2af50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f573c0aaa70 tx=0x7f573c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.938+0000 7f57579cdb80 1 -- 172.21.15.46:0/916897811 shutdown_connections 2024-05-09T00:20:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.938+0000 7f57579cdb80 1 --2- 172.21.15.46:0/916897811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfc22400 0x563dbfc2af50 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.938+0000 7f57579cdb80 1 --2- 172.21.15.46:0/916897811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc274c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.938+0000 7f57579cdb80 1 --2- 172.21.15.46:0/916897811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfb1fd00 0x563dbfc17a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.938+0000 7f57579cdb80 1 -- 172.21.15.46:0/916897811 >> 172.21.15.46:0/916897811 conn(0x563dbfaa8f70 msgr2=0x563dbfc2bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.938+0000 7f57579cdb80 1 -- 172.21.15.46:0/916897811 shutdown_connections 2024-05-09T00:20:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.938+0000 7f57579cdb80 1 -- 172.21.15.46:0/916897811 wait complete. 2024-05-09T00:20:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f57579cdb80 1 -- start start 2024-05-09T00:20:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f57579cdb80 1 Processor -- start 2024-05-09T00:20:39.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f57579cdb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc2f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f57579cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc2f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f57579cdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfc22400 0x563dbfca47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f57579cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f57441ab010 con 0x563dbfb1fd00 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f57579cdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f57441ac480 con 0x563dbfc22400 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f57579cdb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f57441ab6e0 con 0x563dbfaa54a0 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f57579cdb80 10 client.0 _ll_register_callbacks cb 0x563dbf9b66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f5741d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc2f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f5741d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc2f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55050/0 (socket says 172.21.15.46:55050) 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f5741d9b700 1 -- 172.21.15.46:0/1781898564 learned_addr learned my addr 172.21.15.46:0/1781898564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f574259c700 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc2f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f5742d9d700 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfc22400 0x563dbfca47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f5741d9b700 1 -- 172.21.15.46:0/1781898564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 msgr2=0x563dbfc2f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f5741d9b700 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc2f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f5741d9b700 1 -- 172.21.15.46:0/1781898564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfc22400 msgr2=0x563dbfca47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f5741d9b700 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfc22400 0x563dbfca47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.939+0000 7f5741d9b700 1 -- 172.21.15.46:0/1781898564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dbfca4ce0 con 0x563dbfb1fd00 2024-05-09T00:20:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.940+0000 7f5741d9b700 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc2f910 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5738004010 tx=0x7f57380053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4e9cf6550ef32c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.940+0000 7f5729ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.940+0000 7f5729ffb700 1 -- 172.21.15.46:0/1781898564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738011c10 con 0x563dbfb1fd00 2024-05-09T00:20:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.940+0000 7f5729ffb700 1 -- 172.21.15.46:0/1781898564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f5738011db0 con 0x563dbfb1fd00 2024-05-09T00:20:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.940+0000 7f57579cdb80 1 -- 172.21.15.46:0/1781898564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f573c0b3040 con 0x563dbfb1fd00 2024-05-09T00:20:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.940+0000 7f5729ffb700 1 -- 172.21.15.46:0/1781898564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5738004510 con 0x563dbfb1fd00 2024-05-09T00:20:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.940+0000 7f57579cdb80 10 client.16878 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.940+0000 7f57579cdb80 1 -- 172.21.15.46:0/1781898564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563dbfca5d40 con 0x563dbfb1fd00 2024-05-09T00:20:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.940+0000 7f5729ffb700 1 -- 172.21.15.46:0/1781898564 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f573c0b3040 con 0x563dbfb1fd00 2024-05-09T00:20:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.940+0000 7f5729ffb700 1 -- 172.21.15.46:0/1781898564 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5738019030 con 0x563dbfb1fd00 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.941+0000 7f57579cdb80 10 client.16878 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.941+0000 7f57579cdb80 10 client.16878 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.941+0000 7f57579cdb80 20 client.16878 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.941+0000 7f57579cdb80 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x563dbfca6450 0x563dbfca8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.941+0000 7f57579cdb80 4 client.16878 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.941+0000 7f57579cdb80 1 -- 172.21.15.46:0/1781898564 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563dbfb4cb20 con 0x563dbfca6450 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.941+0000 7f574259c700 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x563dbfca6450 0x563dbfca8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.941+0000 7f574259c700 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x563dbfca6450 0x563dbfca8870 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cece163ebcf7ce2f server_cookie=abddd89233f60daf in_seq=0 out_seq=0 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.941+0000 7f5729ffb700 10 client.16878 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f5729ffb700 1 -- 172.21.15.46:0/1781898564 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563dbfb4cb20 con 0x563dbfca6450 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f5729ffb700 10 client.16878 handle_command_reply: tid=0 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f57579cdb80 1 client.16878 shutdown 2024-05-09T00:20:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f57579cdb80 1 -- 172.21.15.46:0/1781898564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 msgr2=0x563dbfc2f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f57579cdb80 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc2f910 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5738004010 tx=0x7f57380053c0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f57579cdb80 1 -- 172.21.15.46:0/1781898564 shutdown_connections 2024-05-09T00:20:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f57579cdb80 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x563dbfca6450 0x563dbfca8870 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f57579cdb80 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dbfc22400 0x563dbfca47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f57579cdb80 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dbfaa54a0 0x563dbfc2f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f57579cdb80 1 --2- 172.21.15.46:0/1781898564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dbfb1fd00 0x563dbfc2f910 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f57579cdb80 1 -- 172.21.15.46:0/1781898564 >> 172.21.15.46:0/1781898564 conn(0x563dbfaa8f70 msgr2=0x563dbfc17780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f5729ffb700 0 client.16878 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:20:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f57579cdb80 1 -- 172.21.15.46:0/1781898564 shutdown_connections 2024-05-09T00:20:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f57579cdb80 1 -- 172.21.15.46:0/1781898564 wait complete. 2024-05-09T00:20:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.942+0000 7f57579cdb80 20 client.16878 trim_cache size 0 max 16384 2024-05-09T00:20:39.948 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:20:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.943+0000 7f5748fb7700 1 -- 172.21.15.46:0/2389058754 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f572c0b2880 msgr2=0x7f572c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.943+0000 7f5748fb7700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f572c0b2880 0x7f572c0b4d90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f573400ffd0 tx=0x7f573400f820 comp rx=0 tx=0).stop 2024-05-09T00:20:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.943+0000 7f5748fb7700 1 -- 172.21.15.46:0/2389058754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5744109870 msgr2=0x7f574419f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.943+0000 7f5748fb7700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5744109870 0x7f574419f620 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f573c0095c0 tx=0x7f573c007a20 comp rx=0 tx=0).stop 2024-05-09T00:20:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.943+0000 7f5748fb7700 1 -- 172.21.15.46:0/2389058754 shutdown_connections 2024-05-09T00:20:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.943+0000 7f5748fb7700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f572c0b2880 0x7f572c0b4d90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.943+0000 7f5748fb7700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5744109870 0x7f574419f620 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.943+0000 7f5748fb7700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57441045f0 0x7f574419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.943+0000 7f5748fb7700 1 --2- 172.21.15.46:0/2389058754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5744108f20 0x7f574419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.943+0000 7f5748fb7700 1 -- 172.21.15.46:0/2389058754 >> 172.21.15.46:0/2389058754 conn(0x7f5744100040 msgr2=0x7f5744108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.943+0000 7f5748fb7700 1 -- 172.21.15.46:0/2389058754 shutdown_connections 2024-05-09T00:20:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:39.943+0000 7f5748fb7700 1 -- 172.21.15.46:0/2389058754 wait complete. 2024-05-09T00:20:40.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:20:40.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:40 smithi142 ceph-mon[34533]: pgmap v655: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:41.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:40 smithi046 ceph-mon[30151]: pgmap v655: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:20:41.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:40 smithi169 ceph-mon[34777]: pgmap v655: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:42.772 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:42 smithi046 ceph-mon[30151]: pgmap v656: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.891+0000 7f88bddea700 1 -- 172.21.15.46:0/3453896047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b8108780 msgr2=0x7f88b81100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.891+0000 7f88bddea700 1 --2- 172.21.15.46:0/3453896047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b8108780 0x7f88b81100e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f88b000b0d0 tx=0x7f88b000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.891+0000 7f88bddea700 1 -- 172.21.15.46:0/3453896047 shutdown_connections 2024-05-09T00:20:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.891+0000 7f88bddea700 1 --2- 172.21.15.46:0/3453896047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b8108780 0x7f88b81100e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.891+0000 7f88bddea700 1 --2- 172.21.15.46:0/3453896047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b8103550 0x7f88b8103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.891+0000 7f88bddea700 1 --2- 172.21.15.46:0/3453896047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88b8107e30 0x7f88b8108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.891+0000 7f88bddea700 1 -- 172.21.15.46:0/3453896047 >> 172.21.15.46:0/3453896047 conn(0x7f88b8078e00 msgr2=0x7f88b8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.891+0000 7f88bddea700 1 -- 172.21.15.46:0/3453896047 shutdown_connections 2024-05-09T00:20:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.891+0000 7f88bddea700 1 -- 172.21.15.46:0/3453896047 wait complete. 2024-05-09T00:20:42.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.892+0000 7f88bddea700 1 Processor -- start 2024-05-09T00:20:42.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.892+0000 7f88bddea700 1 -- start start 2024-05-09T00:20:42.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88bddea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b8103550 0x7f88b819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88bddea700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88b8107e30 0x7f88b819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88bddea700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b8108780 0x7f88b819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88bddea700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81133d0 con 0x7f88b8103550 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88bddea700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8113250 con 0x7f88b8107e30 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88bddea700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8113550 con 0x7f88b8108780 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b8103550 0x7f88b819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b8103550 0x7f88b819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55060/0 (socket says 172.21.15.46:55060) 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b77fe700 1 -- 172.21.15.46:0/1810569487 learned_addr learned my addr 172.21.15.46:0/1810569487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88b8107e30 0x7f88b819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b7fff700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b8108780 0x7f88b819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b77fe700 1 -- 172.21.15.46:0/1810569487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b8108780 msgr2=0x7f88b819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b77fe700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b8108780 0x7f88b819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b77fe700 1 -- 172.21.15.46:0/1810569487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88b8107e30 msgr2=0x7f88b819c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:42.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b77fe700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88b8107e30 0x7f88b819c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b77fe700 1 -- 172.21.15.46:0/1810569487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b81a6f10 con 0x7f88b8103550 2024-05-09T00:20:42.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b7fff700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b8108780 0x7f88b819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:20:42.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88b8107e30 0x7f88b819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:20:42.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.893+0000 7f88b77fe700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b8103550 0x7f88b819bc30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f88a800c970 tx=0x7f88a800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:42.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.894+0000 7f88b4ff9700 1 -- 172.21.15.46:0/1810569487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8007aa0 con 0x7f88b8103550 2024-05-09T00:20:42.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.894+0000 7f88b4ff9700 1 -- 172.21.15.46:0/1810569487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a8004ce0 con 0x7f88b8103550 2024-05-09T00:20:42.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.894+0000 7f88bddea700 1 -- 172.21.15.46:0/1810569487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b81a7200 con 0x7f88b8103550 2024-05-09T00:20:42.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.894+0000 7f88b4ff9700 1 -- 172.21.15.46:0/1810569487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a80055c0 con 0x7f88b8103550 2024-05-09T00:20:42.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.894+0000 7f88bddea700 1 -- 172.21.15.46:0/1810569487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b81a76f0 con 0x7f88b8103550 2024-05-09T00:20:42.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.896+0000 7f88b4ff9700 1 -- 172.21.15.46:0/1810569487 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f88a800f450 con 0x7f88b8103550 2024-05-09T00:20:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.897+0000 7f88b4ff9700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f88a00b2880 0x7f88a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.897+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f88a00b2880 0x7f88a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.897+0000 7f88b4ff9700 1 -- 172.21.15.46:0/1810569487 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f88a80e20b0 con 0x7f88b8103550 2024-05-09T00:20:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.897+0000 7f88cc800b80 1 Processor -- start 2024-05-09T00:20:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88cc800b80 1 -- start start 2024-05-09T00:20:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88cc800b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94ea70d00 0x55d94eb77ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88cc800b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 0x55d94eb78030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88cc800b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 0x55d94eb7c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f88a00b2880 0x7f88a00b4d90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f88b810b200 tx=0x7f88ac005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88cc800b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d94ea960a0 con 0x55d94eb7c1e0 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88cc800b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d94ea3b9a0 con 0x55d94ea70d00 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88cc800b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d94ea92060 con 0x55d94e9f64a0 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88b7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 0x55d94eb7c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88b7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 0x55d94eb7c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55070/0 (socket says 172.21.15.46:55070) 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88b7fff700 1 -- 172.21.15.46:0/751323082 learned_addr learned my addr 172.21.15.46:0/751323082 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/751323082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 0x55d94eb78030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88b77fe700 1 --2- 172.21.15.46:0/751323082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94ea70d00 0x55d94eb77ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88b7fff700 1 -- 172.21.15.46:0/751323082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 msgr2=0x55d94eb78030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88b7fff700 1 --2- 172.21.15.46:0/751323082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 0x55d94eb78030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88b7fff700 1 -- 172.21.15.46:0/751323082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94ea70d00 msgr2=0x55d94eb77ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88b7fff700 1 --2- 172.21.15.46:0/751323082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94ea70d00 0x55d94eb77ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88b7fff700 1 -- 172.21.15.46:0/751323082 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d94ea2c500 con 0x55d94eb7c1e0 2024-05-09T00:20:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.898+0000 7f88b77fe700 1 --2- 172.21.15.46:0/751323082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94ea70d00 0x55d94eb77ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:20:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.899+0000 7f88b7fff700 1 --2- 172.21.15.46:0/751323082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 0x55d94eb7c5f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f88b000d000 tx=0x7f88b0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2fe86535c797f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.899+0000 7f889dffb700 1 -- 172.21.15.46:0/751323082 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0020070 con 0x55d94eb7c1e0 2024-05-09T00:20:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.899+0000 7f889dffb700 1 -- 172.21.15.46:0/751323082 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b00092d0 con 0x55d94eb7c1e0 2024-05-09T00:20:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.899+0000 7f889dffb700 1 -- 172.21.15.46:0/751323082 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b001c440 con 0x55d94eb7c1e0 2024-05-09T00:20:42.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.900+0000 7f88cc800b80 1 -- 172.21.15.46:0/751323082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 msgr2=0x55d94eb7c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:42.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.900+0000 7f88cc800b80 1 --2- 172.21.15.46:0/751323082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 0x55d94eb7c5f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f88b000d000 tx=0x7f88b0009b80 comp rx=0 tx=0).stop 2024-05-09T00:20:42.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.900+0000 7f88cc800b80 1 -- 172.21.15.46:0/751323082 shutdown_connections 2024-05-09T00:20:42.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.900+0000 7f88cc800b80 1 --2- 172.21.15.46:0/751323082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 0x55d94eb7c5f0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.900+0000 7f88cc800b80 1 --2- 172.21.15.46:0/751323082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94ea70d00 0x55d94eb77ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.900+0000 7f88cc800b80 1 --2- 172.21.15.46:0/751323082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 0x55d94eb78030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.900+0000 7f88cc800b80 1 -- 172.21.15.46:0/751323082 >> 172.21.15.46:0/751323082 conn(0x55d94e9f9f70 msgr2=0x55d94eb6e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:42.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.900+0000 7f88cc800b80 1 -- 172.21.15.46:0/751323082 shutdown_connections 2024-05-09T00:20:42.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.900+0000 7f88cc800b80 1 -- 172.21.15.46:0/751323082 wait complete. 2024-05-09T00:20:42.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.900+0000 7f88cc800b80 1 -- start start 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.900+0000 7f88cc800b80 1 Processor -- start 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88cc800b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 0x55d94eb813c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88cc800b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94ea70d00 0x55d94eb81900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88cc800b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94eb7c1e0 0x55d94eb81e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88cc800b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d94eb7cea0 con 0x55d94ea70d00 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88cc800b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d94ea30b20 con 0x55d94eb7c1e0 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88cc800b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d94eb7d020 con 0x55d94e9f64a0 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88cc800b80 10 client.0 _ll_register_callbacks cb 0x55d94e9076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88b6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94ea70d00 0x55d94eb81900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88b6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94ea70d00 0x55d94eb81900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55072/0 (socket says 172.21.15.46:55072) 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88b6ffd700 1 -- 172.21.15.46:0/4013004542 learned_addr learned my addr 172.21.15.46:0/4013004542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88b77fe700 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 0x55d94eb813c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88b7fff700 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94eb7c1e0 0x55d94eb81e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88b6ffd700 1 -- 172.21.15.46:0/4013004542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 msgr2=0x55d94eb813c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 0x55d94eb813c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88b6ffd700 1 -- 172.21.15.46:0/4013004542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94eb7c1e0 msgr2=0x55d94eb81e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94eb7c1e0 0x55d94eb81e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88b6ffd700 1 -- 172.21.15.46:0/4013004542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d94e911490 con 0x55d94ea70d00 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.901+0000 7f88b7fff700 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94eb7c1e0 0x55d94eb81e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.902+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94ea70d00 0x55d94eb81900 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00f9d0 tx=0x7f88ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e785d07a6660b288 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.902+0000 7f8896ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.902+0000 7f8896ffd700 1 -- 172.21.15.46:0/4013004542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac01d840 con 0x55d94ea70d00 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.902+0000 7f8896ffd700 1 -- 172.21.15.46:0/4013004542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f88ac01d9e0 con 0x55d94ea70d00 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.902+0000 7f88cc800b80 1 -- 172.21.15.46:0/4013004542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b00092d0 con 0x55d94ea70d00 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.902+0000 7f8896ffd700 1 -- 172.21.15.46:0/4013004542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88ac01dc20 con 0x55d94ea70d00 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.902+0000 7f88cc800b80 10 client.16890 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.902+0000 7f88cc800b80 1 -- 172.21.15.46:0/4013004542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d94eb8ed80 con 0x55d94ea70d00 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.902+0000 7f8896ffd700 1 -- 172.21.15.46:0/4013004542 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b00092d0 con 0x55d94ea70d00 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.902+0000 7f8896ffd700 1 -- 172.21.15.46:0/4013004542 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f88ac01c430 con 0x55d94ea70d00 2024-05-09T00:20:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.903+0000 7f88cc800b80 10 client.16890 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.903+0000 7f88cc800b80 10 client.16890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:20:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.903+0000 7f88cc800b80 20 client.16890 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.903+0000 7f88cc800b80 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55d94eb913c0 0x55d94eb93820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.903+0000 7f88cc800b80 4 client.16890 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:20:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.903+0000 7f88cc800b80 1 -- 172.21.15.46:0/4013004542 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d94eaa3a80 con 0x55d94eb913c0 2024-05-09T00:20:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.903+0000 7f88b77fe700 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55d94eb913c0 0x55d94eb93820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.903+0000 7f8896ffd700 10 client.16890 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:20:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.903+0000 7f88b77fe700 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55d94eb913c0 0x55d94eb93820 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa8f170d862b4e8b server_cookie=60668648545b53f0 in_seq=0 out_seq=0 2024-05-09T00:20:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.904+0000 7f8896ffd700 1 -- 172.21.15.46:0/4013004542 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d94eaa3a80 con 0x55d94eb913c0 2024-05-09T00:20:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.904+0000 7f8896ffd700 10 client.16890 handle_command_reply: tid=0 2024-05-09T00:20:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.904+0000 7f88cc800b80 1 client.16890 shutdown 2024-05-09T00:20:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.904+0000 7f88cc800b80 1 -- 172.21.15.46:0/4013004542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94ea70d00 msgr2=0x55d94eb81900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.904+0000 7f88cc800b80 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94ea70d00 0x55d94eb81900 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00f9d0 tx=0x7f88ac004750 comp rx=0 tx=0).stop 2024-05-09T00:20:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.905+0000 7f88cc800b80 1 -- 172.21.15.46:0/4013004542 shutdown_connections 2024-05-09T00:20:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.905+0000 7f88cc800b80 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55d94eb913c0 0x55d94eb93820 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.905+0000 7f88cc800b80 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94eb7c1e0 0x55d94eb81e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.905+0000 7f8896ffd700 0 client.16890 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:20:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.905+0000 7f88cc800b80 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 0x55d94eb813c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.905+0000 7f88cc800b80 1 --2- 172.21.15.46:0/4013004542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94ea70d00 0x55d94eb81900 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.905+0000 7f88cc800b80 1 -- 172.21.15.46:0/4013004542 >> 172.21.15.46:0/4013004542 conn(0x55d94e9f9f70 msgr2=0x55d94eb67ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.905+0000 7f88cc800b80 1 -- 172.21.15.46:0/4013004542 shutdown_connections 2024-05-09T00:20:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.905+0000 7f88cc800b80 1 -- 172.21.15.46:0/4013004542 wait complete. 2024-05-09T00:20:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.905+0000 7f88cc800b80 20 client.16890 trim_cache size 0 max 16384 2024-05-09T00:20:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.920+0000 7f88cc800b80 1 Processor -- start 2024-05-09T00:20:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.920+0000 7f88cc800b80 1 -- start start 2024-05-09T00:20:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.920+0000 7f88cc800b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 0x55d94eb80700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.920+0000 7f88cc800b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94ea70d00 0x55d94eb81c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.920+0000 7f88cc800b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 0x55d94eb68700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.920+0000 7f88cc800b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81af250 con 0x55d94eb7c1e0 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.920+0000 7f88cc800b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81acae0 con 0x55d94ea70d00 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.920+0000 7f88cc800b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81aa910 con 0x55d94e9f64a0 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.920+0000 7f88b7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 0x55d94eb68700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.920+0000 7f88b7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 0x55d94eb68700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55076/0 (socket says 172.21.15.46:55076) 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.920+0000 7f88b7fff700 1 -- 172.21.15.46:0/3519305870 learned_addr learned my addr 172.21.15.46:0/3519305870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.921+0000 7f88b77fe700 1 --2- 172.21.15.46:0/3519305870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 0x55d94eb80700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.921+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/3519305870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94ea70d00 0x55d94eb81c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.921+0000 7f88b7fff700 1 -- 172.21.15.46:0/3519305870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 msgr2=0x55d94eb80700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.921+0000 7f88b7fff700 1 --2- 172.21.15.46:0/3519305870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 0x55d94eb80700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.921+0000 7f88b7fff700 1 -- 172.21.15.46:0/3519305870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94ea70d00 msgr2=0x55d94eb81c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.921+0000 7f88b7fff700 1 --2- 172.21.15.46:0/3519305870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94ea70d00 0x55d94eb81c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.921+0000 7f88b7fff700 1 -- 172.21.15.46:0/3519305870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d94eb7f3c0 con 0x55d94eb7c1e0 2024-05-09T00:20:42.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.921+0000 7f88b7fff700 1 --2- 172.21.15.46:0/3519305870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 0x55d94eb68700 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f88b0000f80 tx=0x7f88b00060a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16128bb1df0cde7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:42.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.921+0000 7f889d7fa700 1 -- 172.21.15.46:0/3519305870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0020040 con 0x55d94eb7c1e0 2024-05-09T00:20:42.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.921+0000 7f889d7fa700 1 -- 172.21.15.46:0/3519305870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b00075c0 con 0x55d94eb7c1e0 2024-05-09T00:20:42.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.921+0000 7f889d7fa700 1 -- 172.21.15.46:0/3519305870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b00256b0 con 0x55d94eb7c1e0 2024-05-09T00:20:42.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.922+0000 7f88cc800b80 1 -- 172.21.15.46:0/3519305870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 msgr2=0x55d94eb68700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:42.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.922+0000 7f88cc800b80 1 --2- 172.21.15.46:0/3519305870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 0x55d94eb68700 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f88b0000f80 tx=0x7f88b00060a0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.922+0000 7f88cc800b80 1 -- 172.21.15.46:0/3519305870 shutdown_connections 2024-05-09T00:20:42.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.922+0000 7f88cc800b80 1 --2- 172.21.15.46:0/3519305870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94eb7c1e0 0x55d94eb68700 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.922+0000 7f88cc800b80 1 --2- 172.21.15.46:0/3519305870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94e9f64a0 0x55d94eb80700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.922+0000 7f88cc800b80 1 --2- 172.21.15.46:0/3519305870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94ea70d00 0x55d94eb81c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.922+0000 7f88cc800b80 1 -- 172.21.15.46:0/3519305870 >> 172.21.15.46:0/3519305870 conn(0x55d94e9f9f70 msgr2=0x55d94eb67ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:42.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.922+0000 7f88cc800b80 1 -- 172.21.15.46:0/3519305870 shutdown_connections 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.922+0000 7f88cc800b80 1 -- 172.21.15.46:0/3519305870 wait complete. 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.922+0000 7f88cc800b80 1 -- start start 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88cc800b80 1 Processor -- start 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88cc800b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94e9f64a0 0x55d94eb89b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88cc800b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94ea70d00 0x55d94eb84ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88cc800b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94eb7c1e0 0x55d94eb854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88cc800b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81aafa0 con 0x55d94e9f64a0 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88cc800b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81ac410 con 0x55d94eb7c1e0 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88cc800b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81ab670 con 0x55d94ea70d00 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88cc800b80 10 client.0 _ll_register_callbacks cb 0x55d94e9076c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88b77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94e9f64a0 0x55d94eb89b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88b77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94e9f64a0 0x55d94eb89b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55078/0 (socket says 172.21.15.46:55078) 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88b77fe700 1 -- 172.21.15.46:0/1534455597 learned_addr learned my addr 172.21.15.46:0/1534455597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:42.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88b7fff700 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94eb7c1e0 0x55d94eb854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.923+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94ea70d00 0x55d94eb84ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f88b77fe700 1 -- 172.21.15.46:0/1534455597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94ea70d00 msgr2=0x55d94eb84ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f88b77fe700 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94ea70d00 0x55d94eb84ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f88b77fe700 1 -- 172.21.15.46:0/1534455597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94eb7c1e0 msgr2=0x55d94eb854d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f88b77fe700 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94eb7c1e0 0x55d94eb854d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f88b77fe700 1 -- 172.21.15.46:0/1534455597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d94eb85c40 con 0x55d94e9f64a0 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94ea70d00 0x55d94eb84ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f88b77fe700 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94e9f64a0 0x55d94eb89b70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f88a80a57a0 tx=0x7f88a80b0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5111f92321b046df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f8896ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f8896ffd700 1 -- 172.21.15.46:0/1534455597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a80c7070 con 0x55d94e9f64a0 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f8896ffd700 1 -- 172.21.15.46:0/1534455597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f88a8004960 con 0x55d94e9f64a0 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f88cc800b80 1 -- 172.21.15.46:0/1534455597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b00075c0 con 0x55d94e9f64a0 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f8896ffd700 1 -- 172.21.15.46:0/1534455597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88a80a9720 con 0x55d94e9f64a0 2024-05-09T00:20:42.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f88cc800b80 10 client.16896 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f88cc800b80 1 -- 172.21.15.46:0/1534455597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d94ebfbbb0 con 0x55d94e9f64a0 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.924+0000 7f8896ffd700 1 -- 172.21.15.46:0/1534455597 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b00075c0 con 0x55d94e9f64a0 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.925+0000 7f8896ffd700 1 -- 172.21.15.46:0/1534455597 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f88a80910b0 con 0x55d94e9f64a0 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.925+0000 7f88cc800b80 10 client.16896 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.925+0000 7f88cc800b80 10 client.16896 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.925+0000 7f88cc800b80 20 client.16896 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.925+0000 7f88cc800b80 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55d94eb913c0 0x55d94ebfe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.925+0000 7f88cc800b80 4 client.16896 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.925+0000 7f88cc800b80 1 -- 172.21.15.46:0/1534455597 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d94ea9db20 con 0x55d94eb913c0 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.925+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55d94eb913c0 0x55d94ebfe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.925+0000 7f88b6ffd700 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55d94eb913c0 0x55d94ebfe280 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9c98f96b8c0405a server_cookie=d13a9c99791737fa in_seq=0 out_seq=0 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.925+0000 7f8896ffd700 10 client.16896 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f8896ffd700 1 -- 172.21.15.46:0/1534455597 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d94ea9db20 con 0x55d94eb913c0 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f8896ffd700 10 client.16896 handle_command_reply: tid=0 2024-05-09T00:20:42.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f88cc800b80 1 client.16896 shutdown 2024-05-09T00:20:42.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f88cc800b80 1 -- 172.21.15.46:0/1534455597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94e9f64a0 msgr2=0x55d94eb89b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:42.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f88cc800b80 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94e9f64a0 0x55d94eb89b70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f88a80a57a0 tx=0x7f88a80b0040 comp rx=0 tx=0).stop 2024-05-09T00:20:42.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f88cc800b80 1 -- 172.21.15.46:0/1534455597 shutdown_connections 2024-05-09T00:20:42.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f88cc800b80 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55d94eb913c0 0x55d94ebfe280 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f88cc800b80 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d94eb7c1e0 0x55d94eb854d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f88cc800b80 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94e9f64a0 0x55d94eb89b70 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f88cc800b80 1 --2- 172.21.15.46:0/1534455597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d94ea70d00 0x55d94eb84ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f88cc800b80 1 -- 172.21.15.46:0/1534455597 >> 172.21.15.46:0/1534455597 conn(0x55d94e9f9f70 msgr2=0x55d94eb81ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:42.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f8896ffd700 0 client.16896 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:20:42.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f88cc800b80 1 -- 172.21.15.46:0/1534455597 shutdown_connections 2024-05-09T00:20:42.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f88cc800b80 1 -- 172.21.15.46:0/1534455597 wait complete. 2024-05-09T00:20:42.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.926+0000 7f88cc800b80 20 client.16896 trim_cache size 0 max 16384 2024-05-09T00:20:42.931 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:20:42.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.927+0000 7f88bddea700 1 -- 172.21.15.46:0/1810569487 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f88a00b2880 msgr2=0x7f88a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:42.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.927+0000 7f88bddea700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f88a00b2880 0x7f88a00b4d90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f88b810b200 tx=0x7f88ac005db0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.927+0000 7f88bddea700 1 -- 172.21.15.46:0/1810569487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b8103550 msgr2=0x7f88b819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:42.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.927+0000 7f88bddea700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b8103550 0x7f88b819bc30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f88a800c970 tx=0x7f88a800cd30 comp rx=0 tx=0).stop 2024-05-09T00:20:42.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.927+0000 7f88bddea700 1 -- 172.21.15.46:0/1810569487 shutdown_connections 2024-05-09T00:20:42.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.927+0000 7f88bddea700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f88a00b2880 0x7f88a00b4d90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.927+0000 7f88bddea700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b8108780 0x7f88b819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.927+0000 7f88bddea700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b8103550 0x7f88b819bc30 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.927+0000 7f88bddea700 1 --2- 172.21.15.46:0/1810569487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88b8107e30 0x7f88b819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:42.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.927+0000 7f88bddea700 1 -- 172.21.15.46:0/1810569487 >> 172.21.15.46:0/1810569487 conn(0x7f88b8078e00 msgr2=0x7f88b8100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:42.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.927+0000 7f88bddea700 1 -- 172.21.15.46:0/1810569487 shutdown_connections 2024-05-09T00:20:42.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:42.927+0000 7f88bddea700 1 -- 172.21.15.46:0/1810569487 wait complete. 2024-05-09T00:20:42.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:42 smithi142 ceph-mon[34533]: pgmap v656: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:43.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:42 smithi169 ceph-mon[34777]: pgmap v656: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:43.838 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:20:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:20:44.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:44 smithi046 ceph-mon[30151]: pgmap v657: 97 pgs: 97 active+clean; 5.5 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:20:45.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:44 smithi169 ceph-mon[34777]: pgmap v657: 97 pgs: 97 active+clean; 5.5 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:20:45.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:44 smithi142 ceph-mon[34533]: pgmap v657: 97 pgs: 97 active+clean; 5.5 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:20:45.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.919+0000 7f45488d7700 1 -- 172.21.15.46:0/788707152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45441045e0 msgr2=0x7f45441049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.919+0000 7f45488d7700 1 --2- 172.21.15.46:0/788707152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45441045e0 0x7f45441049f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f45340099c0 tx=0x7f4534009cd0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.919+0000 7f45488d7700 1 -- 172.21.15.46:0/788707152 shutdown_connections 2024-05-09T00:20:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.919+0000 7f45488d7700 1 --2- 172.21.15.46:0/788707152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544109860 0x7f45441111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.919+0000 7f45488d7700 1 --2- 172.21.15.46:0/788707152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45441045e0 0x7f45441049f0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.919+0000 7f45488d7700 1 --2- 172.21.15.46:0/788707152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544108f10 0x7f4544109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.919+0000 7f45488d7700 1 -- 172.21.15.46:0/788707152 >> 172.21.15.46:0/788707152 conn(0x7f45440ffed0 msgr2=0x7f4544102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.919+0000 7f45488d7700 1 -- 172.21.15.46:0/788707152 shutdown_connections 2024-05-09T00:20:45.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.919+0000 7f45488d7700 1 -- 172.21.15.46:0/788707152 wait complete. 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.920+0000 7f45488d7700 1 Processor -- start 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.920+0000 7f45488d7700 1 -- start start 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.920+0000 7f45488d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45441045e0 0x7f454419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.920+0000 7f45488d7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544108f10 0x7f454419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.920+0000 7f45488d7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544109860 0x7f454419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.920+0000 7f45488d7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4544114490 con 0x7f45441045e0 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.920+0000 7f45488d7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4544114310 con 0x7f4544108f10 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.920+0000 7f45488d7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4544114610 con 0x7f4544109860 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f454259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45441045e0 0x7f454419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f454259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45441045e0 0x7f454419ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55090/0 (socket says 172.21.15.46:55090) 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f454259c700 1 -- 172.21.15.46:0/3039225788 learned_addr learned my addr 172.21.15.46:0/3039225788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f4542d9d700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544109860 0x7f454419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f4541d9b700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544108f10 0x7f454419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f454259c700 1 -- 172.21.15.46:0/3039225788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544109860 msgr2=0x7f454419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f454259c700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544109860 0x7f454419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f454259c700 1 -- 172.21.15.46:0/3039225788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544108f10 msgr2=0x7f454419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f454259c700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544108f10 0x7f454419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f454259c700 1 -- 172.21.15.46:0/3039225788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454419fd10 con 0x7f45441045e0 2024-05-09T00:20:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f454259c700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45441045e0 0x7f454419ae40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f45340098e0 tx=0x7f45340057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f45337fe700 1 -- 172.21.15.46:0/3039225788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534027070 con 0x7f45441045e0 2024-05-09T00:20:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f45337fe700 1 -- 172.21.15.46:0/3039225788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4534003f20 con 0x7f45441045e0 2024-05-09T00:20:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f45337fe700 1 -- 172.21.15.46:0/3039225788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534022ca0 con 0x7f45441045e0 2024-05-09T00:20:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.921+0000 7f45488d7700 1 -- 172.21.15.46:0/3039225788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454419ffa0 con 0x7f45441045e0 2024-05-09T00:20:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.922+0000 7f45488d7700 1 -- 172.21.15.46:0/3039225788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45441a7920 con 0x7f45441045e0 2024-05-09T00:20:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.924+0000 7f45337fe700 1 -- 172.21.15.46:0/3039225788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f453402a070 con 0x7f45441045e0 2024-05-09T00:20:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.925+0000 7f45337fe700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f452c0b2900 0x7f452c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.925+0000 7f4541d9b700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f452c0b2900 0x7f452c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.925+0000 7f45337fe700 1 -- 172.21.15.46:0/3039225788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f45340e2fe0 con 0x7f45441045e0 2024-05-09T00:20:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f45572edb80 1 Processor -- start 2024-05-09T00:20:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f45572edb80 1 -- start start 2024-05-09T00:20:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f4541d9b700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f452c0b2900 0x7f452c0b4e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4544108e20 tx=0x7f453800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f45572edb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae430024f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f45572edb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae43002a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f45572edb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae42ffcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f45572edb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ae42f1eff0 con 0x55ae42f69160 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f45572edb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ae42ebd7f0 con 0x55ae42e7f4a0 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f45572edb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ae42f1afb0 con 0x55ae42ffc6a0 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f454259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae430024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f454259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae430024f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55102/0 (socket says 172.21.15.46:55102) 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f454259c700 1 -- 172.21.15.46:0/2002106008 learned_addr learned my addr 172.21.15.46:0/2002106008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f4541d9b700 1 --2- 172.21.15.46:0/2002106008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae43002a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.926+0000 7f4542d9d700 1 --2- 172.21.15.46:0/2002106008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae42ffcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.927+0000 7f454259c700 1 -- 172.21.15.46:0/2002106008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 msgr2=0x55ae42ffcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.927+0000 7f454259c700 1 --2- 172.21.15.46:0/2002106008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae42ffcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.927+0000 7f454259c700 1 -- 172.21.15.46:0/2002106008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 msgr2=0x55ae43002a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.927+0000 7f454259c700 1 --2- 172.21.15.46:0/2002106008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae43002a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.927+0000 7f454259c700 1 -- 172.21.15.46:0/2002106008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae42eb5370 con 0x55ae42f69160 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.927+0000 7f454259c700 1 --2- 172.21.15.46:0/2002106008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae430024f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f45340aa370 tx=0x7f4534009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc3aeac12f5401f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.927+0000 7f4530ff9700 1 -- 172.21.15.46:0/2002106008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45340aad70 con 0x55ae42f69160 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.927+0000 7f4530ff9700 1 -- 172.21.15.46:0/2002106008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45340aa670 con 0x55ae42f69160 2024-05-09T00:20:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.927+0000 7f4530ff9700 1 -- 172.21.15.46:0/2002106008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534090e40 con 0x55ae42f69160 2024-05-09T00:20:45.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.928+0000 7f45572edb80 1 -- 172.21.15.46:0/2002106008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 msgr2=0x55ae430024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:45.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.928+0000 7f45572edb80 1 --2- 172.21.15.46:0/2002106008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae430024f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f45340aa370 tx=0x7f4534009cd0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.928+0000 7f45572edb80 1 -- 172.21.15.46:0/2002106008 shutdown_connections 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.928+0000 7f45572edb80 1 --2- 172.21.15.46:0/2002106008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae42ffcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.928+0000 7f45572edb80 1 --2- 172.21.15.46:0/2002106008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae430024f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.928+0000 7f45572edb80 1 --2- 172.21.15.46:0/2002106008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae43002a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.928+0000 7f45572edb80 1 -- 172.21.15.46:0/2002106008 >> 172.21.15.46:0/2002106008 conn(0x55ae42edf6a0 msgr2=0x55ae42ff7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.928+0000 7f45572edb80 1 -- 172.21.15.46:0/2002106008 shutdown_connections 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.928+0000 7f45572edb80 1 -- 172.21.15.46:0/2002106008 wait complete. 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f45572edb80 1 -- start start 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f45572edb80 1 Processor -- start 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f45572edb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae4300bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f45572edb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae4300c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f45572edb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae430060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f45572edb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ae42d9aba0 con 0x55ae42f69160 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f45572edb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ae42ed9150 con 0x55ae42e7f4a0 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f45572edb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ae42ffd360 con 0x55ae42ffc6a0 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f45572edb80 10 client.0 _ll_register_callbacks cb 0x55ae42d906c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f4541d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae4300c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f4541d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae4300c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55114/0 (socket says 172.21.15.46:55114) 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f4541d9b700 1 -- 172.21.15.46:0/1952861378 learned_addr learned my addr 172.21.15.46:0/1952861378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f454259c700 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae4300bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f4542d9d700 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae430060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.929+0000 7f4541d9b700 1 -- 172.21.15.46:0/1952861378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 msgr2=0x55ae430060c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f4541d9b700 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae430060c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f4541d9b700 1 -- 172.21.15.46:0/1952861378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 msgr2=0x55ae4300bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f4541d9b700 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae4300bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f4541d9b700 1 -- 172.21.15.46:0/1952861378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae42d9a490 con 0x55ae42f69160 2024-05-09T00:20:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f4541d9b700 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae4300c200 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f454406c190 tx=0x7f45380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98881219389aafe0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f4529ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f4529ffb700 1 -- 172.21.15.46:0/1952861378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4538028470 con 0x55ae42f69160 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f4529ffb700 1 -- 172.21.15.46:0/1952861378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4538028610 con 0x55ae42f69160 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f45572edb80 1 -- 172.21.15.46:0/1952861378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45340aa670 con 0x55ae42f69160 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f4529ffb700 1 -- 172.21.15.46:0/1952861378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4538028810 con 0x55ae42f69160 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f45572edb80 10 client.16908 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f45572edb80 1 -- 172.21.15.46:0/1952861378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ae43013940 con 0x55ae42f69160 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f4529ffb700 1 -- 172.21.15.46:0/1952861378 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45340aa670 con 0x55ae42f69160 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.930+0000 7f4529ffb700 1 -- 172.21.15.46:0/1952861378 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f453801f990 con 0x55ae42f69160 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.931+0000 7f45572edb80 10 client.16908 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.931+0000 7f45572edb80 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.931+0000 7f45572edb80 20 client.16908 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.931+0000 7f45572edb80 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ae43015f80 0x55ae430183a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.931+0000 7f45572edb80 4 client.16908 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:20:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.931+0000 7f45572edb80 1 -- 172.21.15.46:0/1952861378 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae42f2c9d0 con 0x55ae43015f80 2024-05-09T00:20:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.931+0000 7f454259c700 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ae43015f80 0x55ae430183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.931+0000 7f454259c700 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ae43015f80 0x55ae430183a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99668275cc9c199 server_cookie=98a151242b298a28 in_seq=0 out_seq=0 2024-05-09T00:20:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.931+0000 7f4529ffb700 10 client.16908 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:45.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f4529ffb700 1 -- 172.21.15.46:0/1952861378 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ae42f2c9d0 con 0x55ae43015f80 2024-05-09T00:20:45.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f4529ffb700 10 client.16908 handle_command_reply: tid=0 2024-05-09T00:20:45.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f45572edb80 1 client.16908 shutdown 2024-05-09T00:20:45.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f45572edb80 1 -- 172.21.15.46:0/1952861378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 msgr2=0x55ae4300c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:45.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f45572edb80 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae4300c200 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f454406c190 tx=0x7f45380047c0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f45572edb80 1 -- 172.21.15.46:0/1952861378 shutdown_connections 2024-05-09T00:20:45.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f45572edb80 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ae43015f80 0x55ae430183a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f45572edb80 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae430060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f45572edb80 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae4300bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f45572edb80 1 --2- 172.21.15.46:0/1952861378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae4300c200 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f45572edb80 1 -- 172.21.15.46:0/1952861378 >> 172.21.15.46:0/1952861378 conn(0x55ae42edf6a0 msgr2=0x55ae42ff71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:45.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f4529ffb700 0 client.16908 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:45.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f45572edb80 1 -- 172.21.15.46:0/1952861378 shutdown_connections 2024-05-09T00:20:45.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.932+0000 7f45572edb80 1 -- 172.21.15.46:0/1952861378 wait complete. 2024-05-09T00:20:45.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.933+0000 7f45572edb80 20 client.16908 trim_cache size 0 max 16384 2024-05-09T00:20:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.947+0000 7f45572edb80 1 Processor -- start 2024-05-09T00:20:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f45572edb80 1 -- start start 2024-05-09T00:20:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f45572edb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae43004d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f45572edb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae430007c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f45572edb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae42ffbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f45572edb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45441af2c0 con 0x55ae42f69160 2024-05-09T00:20:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f45572edb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45441acb50 con 0x55ae42e7f4a0 2024-05-09T00:20:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f45572edb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45441aa980 con 0x55ae42ffc6a0 2024-05-09T00:20:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f4541d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae430007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f4541d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae430007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55122/0 (socket says 172.21.15.46:55122) 2024-05-09T00:20:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f4541d9b700 1 -- 172.21.15.46:0/364800244 learned_addr learned my addr 172.21.15.46:0/364800244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f454259c700 1 --2- 172.21.15.46:0/364800244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae43004d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f4542d9d700 1 --2- 172.21.15.46:0/364800244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae42ffbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f4541d9b700 1 -- 172.21.15.46:0/364800244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 msgr2=0x55ae42ffbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f4541d9b700 1 --2- 172.21.15.46:0/364800244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae42ffbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f4541d9b700 1 -- 172.21.15.46:0/364800244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 msgr2=0x55ae43004d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f4541d9b700 1 --2- 172.21.15.46:0/364800244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae43004d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f4541d9b700 1 -- 172.21.15.46:0/364800244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae42ff0f60 con 0x55ae42f69160 2024-05-09T00:20:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f4542d9d700 1 --2- 172.21.15.46:0/364800244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae42ffbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:20:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.948+0000 7f454259c700 1 --2- 172.21.15.46:0/364800244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae43004d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:20:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.949+0000 7f4541d9b700 1 --2- 172.21.15.46:0/364800244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae430007c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f453801ff70 tx=0x7f4538010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b304a83f2500fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.949+0000 7f452bfff700 1 -- 172.21.15.46:0/364800244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4538028670 con 0x55ae42f69160 2024-05-09T00:20:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.949+0000 7f452bfff700 1 -- 172.21.15.46:0/364800244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4538023070 con 0x55ae42f69160 2024-05-09T00:20:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.949+0000 7f452bfff700 1 -- 172.21.15.46:0/364800244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4538005020 con 0x55ae42f69160 2024-05-09T00:20:45.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.949+0000 7f45572edb80 1 -- 172.21.15.46:0/364800244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 msgr2=0x55ae430007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:45.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.949+0000 7f45572edb80 1 --2- 172.21.15.46:0/364800244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae430007c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f453801ff70 tx=0x7f4538010fd0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.949+0000 7f45572edb80 1 -- 172.21.15.46:0/364800244 shutdown_connections 2024-05-09T00:20:45.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.949+0000 7f45572edb80 1 --2- 172.21.15.46:0/364800244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42ffc6a0 0x55ae42ffbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.949+0000 7f45572edb80 1 --2- 172.21.15.46:0/364800244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42e7f4a0 0x55ae43004d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.949+0000 7f45572edb80 1 --2- 172.21.15.46:0/364800244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42f69160 0x55ae430007c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.949+0000 7f45572edb80 1 -- 172.21.15.46:0/364800244 >> 172.21.15.46:0/364800244 conn(0x55ae42edf6a0 msgr2=0x55ae42ffb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:45.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f45572edb80 1 -- 172.21.15.46:0/364800244 shutdown_connections 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f45572edb80 1 -- 172.21.15.46:0/364800244 wait complete. 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f45572edb80 1 -- start start 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f45572edb80 1 Processor -- start 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f45572edb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42e7f4a0 0x55ae430123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f45572edb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42f69160 0x55ae4300d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f45572edb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42ffc6a0 0x55ae4300dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f45572edb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45441ab010 con 0x55ae42e7f4a0 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f45572edb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f45441ac480 con 0x55ae42ffc6a0 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f45572edb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45441ab6e0 con 0x55ae42f69160 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f45572edb80 10 client.0 _ll_register_callbacks cb 0x55ae42d906c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f454259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42e7f4a0 0x55ae430123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f454259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42e7f4a0 0x55ae430123d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55138/0 (socket says 172.21.15.46:55138) 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.950+0000 7f454259c700 1 -- 172.21.15.46:0/2467118166 learned_addr learned my addr 172.21.15.46:0/2467118166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:45.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f4541d9b700 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42f69160 0x55ae4300d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f4542d9d700 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42ffc6a0 0x55ae4300dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f454259c700 1 -- 172.21.15.46:0/2467118166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42f69160 msgr2=0x55ae4300d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f454259c700 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42f69160 0x55ae4300d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f454259c700 1 -- 172.21.15.46:0/2467118166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42ffc6a0 msgr2=0x55ae4300dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f454259c700 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42ffc6a0 0x55ae4300dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f454259c700 1 -- 172.21.15.46:0/2467118166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae4300e1b0 con 0x55ae42e7f4a0 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f4541d9b700 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42f69160 0x55ae4300d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f454259c700 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42e7f4a0 0x55ae430123d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f453400c010 tx=0x7f4534005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a369d2b3e36b7732 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f4529ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f4529ffb700 1 -- 172.21.15.46:0/2467118166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45340c4060 con 0x55ae42e7f4a0 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f4529ffb700 1 -- 172.21.15.46:0/2467118166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4534092070 con 0x55ae42e7f4a0 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f45572edb80 1 -- 172.21.15.46:0/2467118166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4538023070 con 0x55ae42e7f4a0 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f4529ffb700 1 -- 172.21.15.46:0/2467118166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45340cb050 con 0x55ae42e7f4a0 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f45572edb80 10 client.16914 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:45.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.951+0000 7f45572edb80 1 -- 172.21.15.46:0/2467118166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ae43080230 con 0x55ae42e7f4a0 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.952+0000 7f4529ffb700 1 -- 172.21.15.46:0/2467118166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4538023070 con 0x55ae42e7f4a0 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.952+0000 7f4529ffb700 1 -- 172.21.15.46:0/2467118166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f45340aa8d0 con 0x55ae42e7f4a0 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.952+0000 7f45572edb80 10 client.16914 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.952+0000 7f45572edb80 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.952+0000 7f45572edb80 20 client.16914 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.952+0000 7f45572edb80 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ae43080700 0x55ae43082b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.952+0000 7f45572edb80 4 client.16914 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.952+0000 7f45572edb80 1 -- 172.21.15.46:0/2467118166 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae42f26a70 con 0x55ae43080700 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.952+0000 7f4541d9b700 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ae43080700 0x55ae43082b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.952+0000 7f4541d9b700 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ae43080700 0x55ae43082b20 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a7411f5aaf081d server_cookie=fa30ac7cba59a5e7 in_seq=0 out_seq=0 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.952+0000 7f4529ffb700 10 client.16914 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f4529ffb700 1 -- 172.21.15.46:0/2467118166 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae42f26a70 con 0x55ae43080700 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f4529ffb700 10 client.16914 handle_command_reply: tid=0 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f45572edb80 1 client.16914 shutdown 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f45572edb80 1 -- 172.21.15.46:0/2467118166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42e7f4a0 msgr2=0x55ae430123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f45572edb80 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42e7f4a0 0x55ae430123d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f453400c010 tx=0x7f4534005440 comp rx=0 tx=0).stop 2024-05-09T00:20:45.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f45572edb80 1 -- 172.21.15.46:0/2467118166 shutdown_connections 2024-05-09T00:20:45.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f45572edb80 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55ae43080700 0x55ae43082b20 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f45572edb80 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae42ffc6a0 0x55ae4300dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f45572edb80 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae42e7f4a0 0x55ae430123d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f45572edb80 1 --2- 172.21.15.46:0/2467118166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae42f69160 0x55ae4300d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f45572edb80 1 -- 172.21.15.46:0/2467118166 >> 172.21.15.46:0/2467118166 conn(0x55ae42edf6a0 msgr2=0x55ae42ffbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:45.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f4529ffb700 0 client.16914 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:45.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f45572edb80 1 -- 172.21.15.46:0/2467118166 shutdown_connections 2024-05-09T00:20:45.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f45572edb80 1 -- 172.21.15.46:0/2467118166 wait complete. 2024-05-09T00:20:45.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.953+0000 7f45572edb80 20 client.16914 trim_cache size 0 max 16384 2024-05-09T00:20:45.957 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:20:45.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.954+0000 7f45488d7700 1 -- 172.21.15.46:0/3039225788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f452c0b2900 msgr2=0x7f452c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:45.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.954+0000 7f45488d7700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f452c0b2900 0x7f452c0b4e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4544108e20 tx=0x7f453800b410 comp rx=0 tx=0).stop 2024-05-09T00:20:45.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.954+0000 7f45488d7700 1 -- 172.21.15.46:0/3039225788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45441045e0 msgr2=0x7f454419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:45.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.954+0000 7f45488d7700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45441045e0 0x7f454419ae40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f45340098e0 tx=0x7f45340057c0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.954+0000 7f45488d7700 1 -- 172.21.15.46:0/3039225788 shutdown_connections 2024-05-09T00:20:45.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.954+0000 7f45488d7700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f452c0b2900 0x7f452c0b4e10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.954+0000 7f45488d7700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544109860 0x7f454419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.954+0000 7f45488d7700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45441045e0 0x7f454419ae40 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.954+0000 7f45488d7700 1 --2- 172.21.15.46:0/3039225788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544108f10 0x7f454419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:45.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.954+0000 7f45488d7700 1 -- 172.21.15.46:0/3039225788 >> 172.21.15.46:0/3039225788 conn(0x7f45440ffed0 msgr2=0x7f45441022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:45.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.954+0000 7f45488d7700 1 -- 172.21.15.46:0/3039225788 shutdown_connections 2024-05-09T00:20:45.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:45.954+0000 7f45488d7700 1 -- 172.21.15.46:0/3039225788 wait complete. 2024-05-09T00:20:47.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:46 smithi046 ceph-mon[30151]: pgmap v658: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:47.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:46 smithi169 ceph-mon[34777]: pgmap v658: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:47.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:46 smithi142 ceph-mon[34533]: pgmap v658: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:47.693 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:20:47.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:20:47.961 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:20:49.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:48 smithi046 ceph-mon[30151]: pgmap v659: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:20:49.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:48 smithi169 ceph-mon[34777]: pgmap v659: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:20:49.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:48 smithi142 ceph-mon[34533]: pgmap v659: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:20:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.810+0000 7f189b189700 1 -- 172.21.15.46:0/1098939628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894107df0 msgr2=0x7f1894108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.810+0000 7f189b189700 1 --2- 172.21.15.46:0/1098939628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894107df0 0x7f1894108200 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f18800099c0 tx=0x7f1880009cd0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.810+0000 7f189b189700 1 -- 172.21.15.46:0/1098939628 shutdown_connections 2024-05-09T00:20:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.810+0000 7f189b189700 1 --2- 172.21.15.46:0/1098939628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1894108740 0x7f18941100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.810+0000 7f189b189700 1 --2- 172.21.15.46:0/1098939628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18941034c0 0x7f18941038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.810+0000 7f189b189700 1 --2- 172.21.15.46:0/1098939628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894107df0 0x7f1894108200 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.810+0000 7f189b189700 1 -- 172.21.15.46:0/1098939628 >> 172.21.15.46:0/1098939628 conn(0x7f18940fedc0 msgr2=0x7f18941011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.810+0000 7f189b189700 1 -- 172.21.15.46:0/1098939628 shutdown_connections 2024-05-09T00:20:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.810+0000 7f189b189700 1 -- 172.21.15.46:0/1098939628 wait complete. 2024-05-09T00:20:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f189b189700 1 Processor -- start 2024-05-09T00:20:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f189b189700 1 -- start start 2024-05-09T00:20:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f189b189700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18941034c0 0x7f189419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f189b189700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1894107df0 0x7f189419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f189b189700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894108740 0x7f189419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f189b189700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1894113350 con 0x7f1894108740 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f189b189700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18941131d0 con 0x7f1894107df0 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f189b189700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f18941134d0 con 0x7f18941034c0 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f1899726700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894108740 0x7f189419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f1893fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1894107df0 0x7f189419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f1899726700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894108740 0x7f189419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37722/0 (socket says 172.21.15.46:37722) 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f1893fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1894107df0 0x7f189419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:39174/0 (socket says 172.21.15.46:39174) 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.811+0000 7f1899726700 1 -- 172.21.15.46:0/1905638465 learned_addr learned my addr 172.21.15.46:0/1905638465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.812+0000 7f1898f25700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18941034c0 0x7f189419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.812+0000 7f1899726700 1 -- 172.21.15.46:0/1905638465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18941034c0 msgr2=0x7f189419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.812+0000 7f1899726700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18941034c0 0x7f189419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.812+0000 7f1899726700 1 -- 172.21.15.46:0/1905638465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1894107df0 msgr2=0x7f189419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.812+0000 7f1899726700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1894107df0 0x7f189419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.812+0000 7f1899726700 1 -- 172.21.15.46:0/1905638465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189419fc10 con 0x7f1894108740 2024-05-09T00:20:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.812+0000 7f1893fff700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1894107df0 0x7f189419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:20:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.812+0000 7f1899726700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894108740 0x7f189419f620 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f188c00ada0 tx=0x7f188c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.812+0000 7f1891ffb700 1 -- 172.21.15.46:0/1905638465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c017020 con 0x7f1894108740 2024-05-09T00:20:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.812+0000 7f1891ffb700 1 -- 172.21.15.46:0/1905638465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f188c00cd80 con 0x7f1894108740 2024-05-09T00:20:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.812+0000 7f1891ffb700 1 -- 172.21.15.46:0/1905638465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c005260 con 0x7f1894108740 2024-05-09T00:20:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.812+0000 7f189b189700 1 -- 172.21.15.46:0/1905638465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189419ff00 con 0x7f1894108740 2024-05-09T00:20:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.813+0000 7f189b189700 1 -- 172.21.15.46:0/1905638465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18941a03c0 con 0x7f1894108740 2024-05-09T00:20:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.815+0000 7f1891ffb700 1 -- 172.21.15.46:0/1905638465 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f188c00c8d0 con 0x7f1894108740 2024-05-09T00:20:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.816+0000 7f1891ffb700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f18840b2880 0x7f18840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.816+0000 7f1891ffb700 1 -- 172.21.15.46:0/1905638465 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f188c0e1880 con 0x7f1894108740 2024-05-09T00:20:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.816+0000 7f1898f25700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f18840b2880 0x7f18840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.816+0000 7f18a9b9fb80 1 Processor -- start 2024-05-09T00:20:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f1898f25700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f18840b2880 0x7f18840b4d90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f188800ac50 tx=0x7f188800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f18a9b9fb80 1 -- start start 2024-05-09T00:20:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f18a9b9fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51da05d80 0x55c51db0e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f18a9b9fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51db0e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f18a9b9fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db08890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f18a9b9fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c51da2b120 con 0x55c51db08480 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f18a9b9fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c51d9d0a20 con 0x55c51d98b520 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f18a9b9fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c51da270e0 con 0x55c51da05d80 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f1899726700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db08890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f1899726700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db08890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37734/0 (socket says 172.21.15.46:37734) 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f1899726700 1 -- 172.21.15.46:0/2825531348 learned_addr learned my addr 172.21.15.46:0/2825531348 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f1898f25700 1 --2- 172.21.15.46:0/2825531348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51da05d80 0x55c51db0e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f1893fff700 1 --2- 172.21.15.46:0/2825531348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51db0e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f1899726700 1 -- 172.21.15.46:0/2825531348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51da05d80 msgr2=0x55c51db0e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f1899726700 1 --2- 172.21.15.46:0/2825531348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51da05d80 0x55c51db0e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f1899726700 1 -- 172.21.15.46:0/2825531348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 msgr2=0x55c51db0e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f1899726700 1 --2- 172.21.15.46:0/2825531348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51db0e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.817+0000 7f1899726700 1 -- 172.21.15.46:0/2825531348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c51d9c1580 con 0x55c51db08480 2024-05-09T00:20:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.818+0000 7f1893fff700 1 --2- 172.21.15.46:0/2825531348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51db0e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:20:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.818+0000 7f1899726700 1 --2- 172.21.15.46:0/2825531348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db08890 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f188c0a61a0 tx=0x7f188c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd5b0f9b955d8fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.818+0000 7f187effd700 1 -- 172.21.15.46:0/2825531348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c0a8f30 con 0x55c51db08480 2024-05-09T00:20:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.818+0000 7f187effd700 1 -- 172.21.15.46:0/2825531348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f188c0ad2e0 con 0x55c51db08480 2024-05-09T00:20:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.818+0000 7f187effd700 1 -- 172.21.15.46:0/2825531348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c0194e0 con 0x55c51db08480 2024-05-09T00:20:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.819+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2825531348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 msgr2=0x55c51db08890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.819+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2825531348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db08890 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f188c0a61a0 tx=0x7f188c00cbc0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.819+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2825531348 shutdown_connections 2024-05-09T00:20:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.819+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2825531348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db08890 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.819+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2825531348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51da05d80 0x55c51db0e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.819+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2825531348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51db0e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.819+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2825531348 >> 172.21.15.46:0/2825531348 conn(0x55c51d98eff0 msgr2=0x55c51db032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.819+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2825531348 shutdown_connections 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.819+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2825531348 wait complete. 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.819+0000 7f18a9b9fb80 1 -- start start 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.819+0000 7f18a9b9fb80 1 Processor -- start 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f18a9b9fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51db12100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f18a9b9fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51da05d80 0x55c51db12640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f18a9b9fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db12b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f18a9b9fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c51db091e0 con 0x55c51db08480 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f18a9b9fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c51d9c5ba0 con 0x55c51d98b520 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f18a9b9fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c51db194b0 con 0x55c51da05d80 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f18a9b9fb80 10 client.0 _ll_register_callbacks cb 0x55c51d89c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f1893fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51da05d80 0x55c51db12640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f1893fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51da05d80 0x55c51db12640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:37310/0 (socket says 172.21.15.46:37310) 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f1893fff700 1 -- 172.21.15.46:0/2766966759 learned_addr learned my addr 172.21.15.46:0/2766966759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f1899726700 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db12b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f1898f25700 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51db12100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f1893fff700 1 -- 172.21.15.46:0/2766966759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 msgr2=0x55c51db12100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f1893fff700 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51db12100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f1893fff700 1 -- 172.21.15.46:0/2766966759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 msgr2=0x55c51db12b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f1893fff700 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db12b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.820+0000 7f1893fff700 1 -- 172.21.15.46:0/2766966759 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c51d8a6510 con 0x55c51da05d80 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.821+0000 7f1893fff700 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51da05d80 0x55c51db12640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1880009990 tx=0x7f18800037c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cf0b4de4a23ec38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.821+0000 7f1873fff700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.821+0000 7f1873fff700 1 -- 172.21.15.46:0/2766966759 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18800050b0 con 0x55c51da05d80 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.821+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2766966759 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f188c0ad2e0 con 0x55c51da05d80 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.821+0000 7f1873fff700 1 -- 172.21.15.46:0/2766966759 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1880005250 con 0x55c51da05d80 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.821+0000 7f1873fff700 1 -- 172.21.15.46:0/2766966759 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1880005450 con 0x55c51da05d80 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.821+0000 7f18a9b9fb80 10 client.26431 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.821+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2766966759 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c51db23f40 con 0x55c51da05d80 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.821+0000 7f1873fff700 1 -- 172.21.15.46:0/2766966759 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f188c0ad2e0 con 0x55c51da05d80 2024-05-09T00:20:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.822+0000 7f1873fff700 1 -- 172.21.15.46:0/2766966759 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f188002b400 con 0x55c51da05d80 2024-05-09T00:20:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.822+0000 7f18a9b9fb80 10 client.26431 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.822+0000 7f18a9b9fb80 10 client.26431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:20:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.822+0000 7f18a9b9fb80 20 client.26431 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.822+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c51db26620 0x55c51db28a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.822+0000 7f18a9b9fb80 4 client.26431 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:20:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.822+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2766966759 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c51da38b00 con 0x55c51db26620 2024-05-09T00:20:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.822+0000 7f1898f25700 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c51db26620 0x55c51db28a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.822+0000 7f1873fff700 10 client.26431 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:49.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.822+0000 7f1898f25700 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c51db26620 0x55c51db28a40 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7efa744cde27967 server_cookie=457eed6f8a31fe9d in_seq=0 out_seq=0 2024-05-09T00:20:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.823+0000 7f1873fff700 1 -- 172.21.15.46:0/2766966759 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c51da38b00 con 0x55c51db26620 2024-05-09T00:20:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.823+0000 7f1873fff700 10 client.26431 handle_command_reply: tid=0 2024-05-09T00:20:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.823+0000 7f18a9b9fb80 1 client.26431 shutdown 2024-05-09T00:20:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.823+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2766966759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51da05d80 msgr2=0x55c51db12640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.823+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51da05d80 0x55c51db12640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1880009990 tx=0x7f18800037c0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.823+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2766966759 shutdown_connections 2024-05-09T00:20:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.823+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c51db26620 0x55c51db28a40 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.823+0000 7f1873fff700 0 client.26431 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.823+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db12b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.823+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51db12100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.823+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2766966759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51da05d80 0x55c51db12640 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.824+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2766966759 >> 172.21.15.46:0/2766966759 conn(0x55c51d98eff0 msgr2=0x55c51dafd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.824+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2766966759 shutdown_connections 2024-05-09T00:20:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.824+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2766966759 wait complete. 2024-05-09T00:20:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.824+0000 7f18a9b9fb80 20 client.26431 trim_cache size 0 max 16384 2024-05-09T00:20:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.844+0000 7f18a9b9fb80 1 Processor -- start 2024-05-09T00:20:49.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.844+0000 7f18a9b9fb80 1 -- start start 2024-05-09T00:20:49.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f18a9b9fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51dafdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f18a9b9fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51da05d80 0x55c51db0c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f18a9b9fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51db08480 0x55c51db222f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f18a9b9fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f18941af230 con 0x55c51da05d80 2024-05-09T00:20:49.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f18a9b9fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18941acac0 con 0x55c51d98b520 2024-05-09T00:20:49.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f18a9b9fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f18941aa910 con 0x55c51db08480 2024-05-09T00:20:49.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f1893fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51da05d80 0x55c51db0c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f1893fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51da05d80 0x55c51db0c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37750/0 (socket says 172.21.15.46:37750) 2024-05-09T00:20:49.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f1893fff700 1 -- 172.21.15.46:0/2941790592 learned_addr learned my addr 172.21.15.46:0/2941790592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:49.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f1898f25700 1 --2- 172.21.15.46:0/2941790592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51dafdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f1899726700 1 --2- 172.21.15.46:0/2941790592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51db08480 0x55c51db222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f1893fff700 1 -- 172.21.15.46:0/2941790592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51db08480 msgr2=0x55c51db222f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:49.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f1893fff700 1 --2- 172.21.15.46:0/2941790592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51db08480 0x55c51db222f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f1893fff700 1 -- 172.21.15.46:0/2941790592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 msgr2=0x55c51dafdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:49.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f1893fff700 1 --2- 172.21.15.46:0/2941790592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51dafdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f1893fff700 1 -- 172.21.15.46:0/2941790592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c51db131c0 con 0x55c51da05d80 2024-05-09T00:20:49.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.845+0000 7f1893fff700 1 --2- 172.21.15.46:0/2941790592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51da05d80 0x55c51db0c430 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1880006010 tx=0x7f1880003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2545aa28ec36d462 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:49.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.846+0000 7f187e7fc700 1 -- 172.21.15.46:0/2941790592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188003c070 con 0x55c51da05d80 2024-05-09T00:20:49.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.846+0000 7f187e7fc700 1 -- 172.21.15.46:0/2941790592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1880027070 con 0x55c51da05d80 2024-05-09T00:20:49.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.846+0000 7f187e7fc700 1 -- 172.21.15.46:0/2941790592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188002b660 con 0x55c51da05d80 2024-05-09T00:20:49.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.846+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2941790592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51da05d80 msgr2=0x55c51db0c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.846+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2941790592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51da05d80 0x55c51db0c430 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1880006010 tx=0x7f1880003f80 comp rx=0 tx=0).stop 2024-05-09T00:20:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.846+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2941790592 shutdown_connections 2024-05-09T00:20:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.846+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2941790592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51db08480 0x55c51db222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.846+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2941790592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51d98b520 0x55c51dafdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.846+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/2941790592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51da05d80 0x55c51db0c430 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.846+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2941790592 >> 172.21.15.46:0/2941790592 conn(0x55c51d98eff0 msgr2=0x55c51db0bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.847+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2941790592 shutdown_connections 2024-05-09T00:20:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.847+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/2941790592 wait complete. 2024-05-09T00:20:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.847+0000 7f18a9b9fb80 1 -- start start 2024-05-09T00:20:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.847+0000 7f18a9b9fb80 1 Processor -- start 2024-05-09T00:20:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.847+0000 7f18a9b9fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51d98b520 0x55c51db1e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f18a9b9fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51da05d80 0x55c51db1ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f18a9b9fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db1ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f18a9b9fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f18941aaf80 con 0x55c51db08480 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f18a9b9fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18941ac3f0 con 0x55c51da05d80 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f18a9b9fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f18941ab650 con 0x55c51d98b520 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f18a9b9fb80 10 client.0 _ll_register_callbacks cb 0x55c51d89c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1899726700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db1ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1899726700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db1ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37756/0 (socket says 172.21.15.46:37756) 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1899726700 1 -- 172.21.15.46:0/1116698005 learned_addr learned my addr 172.21.15.46:0/1116698005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1898f25700 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51d98b520 0x55c51db1e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1893fff700 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51da05d80 0x55c51db1ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1899726700 1 -- 172.21.15.46:0/1116698005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51d98b520 msgr2=0x55c51db1e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1899726700 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51d98b520 0x55c51db1e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1899726700 1 -- 172.21.15.46:0/1116698005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51da05d80 msgr2=0x55c51db1ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1899726700 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51da05d80 0x55c51db1ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1899726700 1 -- 172.21.15.46:0/1116698005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c51db1d0d0 con 0x55c51db08480 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1893fff700 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51da05d80 0x55c51db1ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1899726700 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db1ca20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f188c0a8ec0 tx=0x7f188c0194e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ad7415dc8e4b6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1873fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1873fff700 1 -- 172.21.15.46:0/1116698005 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c0ad2e0 con 0x55c51db08480 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f1873fff700 1 -- 172.21.15.46:0/1116698005 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f188c0a93a0 con 0x55c51db08480 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.848+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/1116698005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1880027070 con 0x55c51db08480 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.849+0000 7f1873fff700 1 -- 172.21.15.46:0/1116698005 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f188c0a8f30 con 0x55c51db08480 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.849+0000 7f18a9b9fb80 10 client.16932 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.849+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/1116698005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c51db91860 con 0x55c51db08480 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.849+0000 7f1873fff700 1 -- 172.21.15.46:0/1116698005 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1880027070 con 0x55c51db08480 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.849+0000 7f1873fff700 1 -- 172.21.15.46:0/1116698005 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f188c0bf020 con 0x55c51db08480 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.849+0000 7f18a9b9fb80 10 client.16932 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.849+0000 7f18a9b9fb80 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:20:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.849+0000 7f18a9b9fb80 20 client.16932 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.849+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c51db26620 0x55c51db93b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.849+0000 7f18a9b9fb80 4 client.16932 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:20:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.849+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/1116698005 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c51da32ba0 con 0x55c51db26620 2024-05-09T00:20:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.849+0000 7f1898f25700 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c51db26620 0x55c51db93b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.850+0000 7f1873fff700 10 client.16932 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.850+0000 7f1898f25700 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c51db26620 0x55c51db93b50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8d6cfa15a9842 server_cookie=30c4f29961136232 in_seq=0 out_seq=0 2024-05-09T00:20:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f1873fff700 1 -- 172.21.15.46:0/1116698005 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c51da32ba0 con 0x55c51db26620 2024-05-09T00:20:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f1873fff700 10 client.16932 handle_command_reply: tid=0 2024-05-09T00:20:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f18a9b9fb80 1 client.16932 shutdown 2024-05-09T00:20:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/1116698005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 msgr2=0x55c51db1ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db1ca20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f188c0a8ec0 tx=0x7f188c0194e0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/1116698005 shutdown_connections 2024-05-09T00:20:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55c51db26620 0x55c51db93b50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c51db08480 0x55c51db1ca20 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c51d98b520 0x55c51db1e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f18a9b9fb80 1 --2- 172.21.15.46:0/1116698005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c51da05d80 0x55c51db1ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/1116698005 >> 172.21.15.46:0/1116698005 conn(0x55c51d98eff0 msgr2=0x55c51dafd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:49.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f1873fff700 0 client.16932 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:49.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/1116698005 shutdown_connections 2024-05-09T00:20:49.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f18a9b9fb80 1 -- 172.21.15.46:0/1116698005 wait complete. 2024-05-09T00:20:49.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.851+0000 7f18a9b9fb80 20 client.16932 trim_cache size 0 max 16384 2024-05-09T00:20:49.855 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:20:49.855 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:20:49.855 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "de63bcca-269d-45e5-84ac-f4271908c256", 2024-05-09T00:20:49.855 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:20:49.856 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:20:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.852+0000 7f189b189700 1 -- 172.21.15.46:0/1905638465 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f18840b2880 msgr2=0x7f18840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.852+0000 7f189b189700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f18840b2880 0x7f18840b4d90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f188800ac50 tx=0x7f188800a380 comp rx=0 tx=0).stop 2024-05-09T00:20:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.852+0000 7f189b189700 1 -- 172.21.15.46:0/1905638465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894108740 msgr2=0x7f189419f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.852+0000 7f189b189700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894108740 0x7f189419f620 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f188c00ada0 tx=0x7f188c00c330 comp rx=0 tx=0).stop 2024-05-09T00:20:49.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.852+0000 7f189b189700 1 -- 172.21.15.46:0/1905638465 shutdown_connections 2024-05-09T00:20:49.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.852+0000 7f189b189700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f18840b2880 0x7f18840b4d90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.852+0000 7f189b189700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894108740 0x7f189419f620 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.852+0000 7f189b189700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18941034c0 0x7f189419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.852+0000 7f189b189700 1 --2- 172.21.15.46:0/1905638465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1894107df0 0x7f189419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:49.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.852+0000 7f189b189700 1 -- 172.21.15.46:0/1905638465 >> 172.21.15.46:0/1905638465 conn(0x7f18940fedc0 msgr2=0x7f189406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:49.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.852+0000 7f189b189700 1 -- 172.21.15.46:0/1905638465 shutdown_connections 2024-05-09T00:20:49.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:49.852+0000 7f189b189700 1 -- 172.21.15.46:0/1905638465 wait complete. 2024-05-09T00:20:50.710 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:50 smithi046 ceph-mon[30151]: pgmap v660: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:50.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:50 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:20:50.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:50 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:20:50.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:50 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:20:50.711 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:50 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:20:50.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:20:50.971 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:20:51.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:50 smithi169 ceph-mon[34777]: pgmap v660: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:51.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:50 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:20:51.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:50 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:20:51.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:50 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:20:51.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:50 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:20:51.192 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:50 smithi142 ceph-mon[34533]: pgmap v660: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:50 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:20:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:50 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:20:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:50 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:20:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:50 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:20:52.767 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:52 smithi046 ceph-mon[30151]: pgmap v661: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:52.767 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:52 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:20:52.768 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:52 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:20:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.767+0000 7ff6ea74e700 1 -- 172.21.15.46:0/4206186883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6e4105480 msgr2=0x7ff6e4105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.767+0000 7ff6ea74e700 1 --2- 172.21.15.46:0/4206186883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6e4105480 0x7ff6e4105890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40098f0 tx=0x7ff6d4009c00 comp rx=0 tx=0).stop 2024-05-09T00:20:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.767+0000 7ff6ea74e700 1 -- 172.21.15.46:0/4206186883 shutdown_connections 2024-05-09T00:20:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.767+0000 7ff6ea74e700 1 --2- 172.21.15.46:0/4206186883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e410e8f0 0x7ff6e4110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.767+0000 7ff6ea74e700 1 --2- 172.21.15.46:0/4206186883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6e4105480 0x7ff6e4105890 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.767+0000 7ff6ea74e700 1 --2- 172.21.15.46:0/4206186883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e4105dd0 0x7ff6e410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.767+0000 7ff6ea74e700 1 -- 172.21.15.46:0/4206186883 >> 172.21.15.46:0/4206186883 conn(0x7ff6e40fef30 msgr2=0x7ff6e4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.767+0000 7ff6ea74e700 1 -- 172.21.15.46:0/4206186883 shutdown_connections 2024-05-09T00:20:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.768+0000 7ff6ea74e700 1 -- 172.21.15.46:0/4206186883 wait complete. 2024-05-09T00:20:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.768+0000 7ff6ea74e700 1 Processor -- start 2024-05-09T00:20:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.769+0000 7ff6ea74e700 1 -- start start 2024-05-09T00:20:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.769+0000 7ff6ea74e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e4105480 0x7ff6e41a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.769+0000 7ff6ea74e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e4105dd0 0x7ff6e41a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.769+0000 7ff6ea74e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6e410e8f0 0x7ff6e41ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.769+0000 7ff6ea74e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e4114440 con 0x7ff6e4105dd0 2024-05-09T00:20:52.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6ea74e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41142c0 con 0x7ff6e4105480 2024-05-09T00:20:52.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e8ceb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6e410e8f0 0x7ff6e41ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6ea74e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41145c0 con 0x7ff6e410e8f0 2024-05-09T00:20:52.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e4105480 0x7ff6e41a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e8ceb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6e410e8f0 0x7ff6e41ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:37354/0 (socket says 172.21.15.46:37354) 2024-05-09T00:20:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e3fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e4105480 0x7ff6e41a2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:39240/0 (socket says 172.21.15.46:39240) 2024-05-09T00:20:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e8ceb700 1 -- 172.21.15.46:0/3977252199 learned_addr learned my addr 172.21.15.46:0/3977252199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e4105dd0 0x7ff6e41a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e8ceb700 1 -- 172.21.15.46:0/3977252199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e4105480 msgr2=0x7ff6e41a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e8ceb700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e4105480 0x7ff6e41a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e8ceb700 1 -- 172.21.15.46:0/3977252199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e4105dd0 msgr2=0x7ff6e41a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e8ceb700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e4105dd0 0x7ff6e41a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e8ceb700 1 -- 172.21.15.46:0/3977252199 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e41ad6e0 con 0x7ff6e410e8f0 2024-05-09T00:20:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e4105dd0 0x7ff6e41a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:20:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e3fff700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e4105480 0x7ff6e41a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:20:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e8ceb700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6e410e8f0 0x7ff6e41ad1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00e3c0 tx=0x7ff6dc00e6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.770+0000 7ff6e17fa700 1 -- 172.21.15.46:0/3977252199 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0092d0 con 0x7ff6e410e8f0 2024-05-09T00:20:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.771+0000 7ff6e17fa700 1 -- 172.21.15.46:0/3977252199 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6dc00ee70 con 0x7ff6e410e8f0 2024-05-09T00:20:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.771+0000 7ff6ea74e700 1 -- 172.21.15.46:0/3977252199 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e41ad910 con 0x7ff6e410e8f0 2024-05-09T00:20:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.771+0000 7ff6e17fa700 1 -- 172.21.15.46:0/3977252199 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc004030 con 0x7ff6e410e8f0 2024-05-09T00:20:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.771+0000 7ff6ea74e700 1 -- 172.21.15.46:0/3977252199 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e41ade20 con 0x7ff6e410e8f0 2024-05-09T00:20:52.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.775+0000 7ff6e17fa700 1 -- 172.21.15.46:0/3977252199 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff6dc012050 con 0x7ff6e410e8f0 2024-05-09T00:20:52.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.775+0000 7ff6f9164b80 1 Processor -- start 2024-05-09T00:20:52.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.775+0000 7ff6f9164b80 1 -- start start 2024-05-09T00:20:52.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6f9164b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dae27d00 0x5626daf30240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6f9164b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626dadad4a0 0x5626daf30780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6f9164b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626daf2a400 0x5626daf2a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6f9164b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5626dae4d0a0 con 0x5626dadad4a0 2024-05-09T00:20:52.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6f9164b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5626dadf29a0 con 0x5626dae27d00 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6f9164b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5626dae49060 con 0x5626daf2a400 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6e37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dae27d00 0x5626daf30240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6e37fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dae27d00 0x5626daf30240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:39246/0 (socket says 172.21.15.46:39246) 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6e37fe700 1 -- 172.21.15.46:0/3233594818 learned_addr learned my addr 172.21.15.46:0/3233594818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/3233594818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626daf2a400 0x5626daf2a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6e17fa700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff6cc0b2900 0x7ff6cc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6e3fff700 1 --2- 172.21.15.46:0/3233594818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626dadad4a0 0x5626daf30780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6e8ceb700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff6cc0b2900 0x7ff6cc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6e37fe700 1 -- 172.21.15.46:0/3233594818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626daf2a400 msgr2=0x5626daf2a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/3233594818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626daf2a400 0x5626daf2a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6e37fe700 1 -- 172.21.15.46:0/3233594818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626dadad4a0 msgr2=0x5626daf30780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/3233594818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626dadad4a0 0x5626daf30780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.776+0000 7ff6e37fe700 1 -- 172.21.15.46:0/3233594818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626dade3500 con 0x5626dae27d00 2024-05-09T00:20:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.777+0000 7ff6e17fa700 1 -- 172.21.15.46:0/3977252199 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff6dc0e1e00 con 0x7ff6e410e8f0 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.777+0000 7ff6e8ceb700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff6cc0b2900 0x7ff6cc0b4e10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0cee50 tx=0x7ff6dc0cbe70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.777+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/3233594818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dae27d00 0x5626daf30240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80128c0 tx=0x7ff6d80097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=648834f6c27eb94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.777+0000 7ff6ca7fc700 1 -- 172.21.15.46:0/3233594818 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d800f450 con 0x5626dae27d00 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.777+0000 7ff6ca7fc700 1 -- 172.21.15.46:0/3233594818 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d800faa0 con 0x5626dae27d00 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.777+0000 7ff6ca7fc700 1 -- 172.21.15.46:0/3233594818 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d801e930 con 0x5626dae27d00 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.778+0000 7ff6f9164b80 1 -- 172.21.15.46:0/3233594818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dae27d00 msgr2=0x5626daf30240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.778+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/3233594818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dae27d00 0x5626daf30240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80128c0 tx=0x7ff6d80097f0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.778+0000 7ff6f9164b80 1 -- 172.21.15.46:0/3233594818 shutdown_connections 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.778+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/3233594818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626daf2a400 0x5626daf2a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.778+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/3233594818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dae27d00 0x5626daf30240 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.778+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/3233594818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626dadad4a0 0x5626daf30780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.778+0000 7ff6f9164b80 1 -- 172.21.15.46:0/3233594818 >> 172.21.15.46:0/3233594818 conn(0x5626dadb0f70 msgr2=0x5626daf25220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.779+0000 7ff6f9164b80 1 -- 172.21.15.46:0/3233594818 shutdown_connections 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.779+0000 7ff6f9164b80 1 -- 172.21.15.46:0/3233594818 wait complete. 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.779+0000 7ff6f9164b80 1 -- start start 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.779+0000 7ff6f9164b80 1 Processor -- start 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6f9164b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dadad4a0 0x5626daf33f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6f9164b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dae27d00 0x5626daf34480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6f9164b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626daf2a400 0x5626daf349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6f9164b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5626daf2b160 con 0x5626daf2a400 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6f9164b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5626dade7b20 con 0x5626dadad4a0 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6f9164b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5626daf37020 con 0x5626dae27d00 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6f9164b80 10 client.0 _ll_register_callbacks cb 0x5626dacbe6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6e37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626daf2a400 0x5626daf349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6e37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626daf2a400 0x5626daf349c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37798/0 (socket says 172.21.15.46:37798) 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6e37fe700 1 -- 172.21.15.46:0/29726702 learned_addr learned my addr 172.21.15.46:0/29726702 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6e3fff700 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dae27d00 0x5626daf34480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dadad4a0 0x5626daf33f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6e3fff700 1 -- 172.21.15.46:0/29726702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dadad4a0 msgr2=0x5626daf33f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6e3fff700 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dadad4a0 0x5626daf33f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.780+0000 7ff6e3fff700 1 -- 172.21.15.46:0/29726702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626daf2a400 msgr2=0x5626daf349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.781+0000 7ff6e3fff700 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626daf2a400 0x5626daf349c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.781+0000 7ff6e3fff700 1 -- 172.21.15.46:0/29726702 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626dacc8490 con 0x5626dae27d00 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.781+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626daf2a400 0x5626daf349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.781+0000 7ff6e3fff700 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dae27d00 0x5626daf34480 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40098c0 tx=0x7ff6d4005960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90972e9da89071e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.781+0000 7ff6c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6c37fe700 1 -- 172.21.15.46:0/29726702 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4027020 con 0x5626dae27d00 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6c37fe700 1 -- 172.21.15.46:0/29726702 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff6d4005590 con 0x5626dae27d00 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6f9164b80 1 -- 172.21.15.46:0/29726702 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d800faa0 con 0x5626dae27d00 2024-05-09T00:20:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6c37fe700 1 -- 172.21.15.46:0/29726702 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6d4003680 con 0x5626dae27d00 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6f9164b80 10 client.26449 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6f9164b80 1 -- 172.21.15.46:0/29726702 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626daf41960 con 0x5626dae27d00 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6c37fe700 1 -- 172.21.15.46:0/29726702 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d800faa0 con 0x5626dae27d00 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6c37fe700 1 -- 172.21.15.46:0/29726702 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff6d400b9f0 con 0x5626dae27d00 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6f9164b80 10 client.26449 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6f9164b80 10 client.26449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6f9164b80 20 client.26449 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5626daf43f40 0x5626daf46360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6f9164b80 4 client.26449 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.782+0000 7ff6f9164b80 1 -- 172.21.15.46:0/29726702 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626dae5aa80 con 0x5626daf43f40 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.783+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5626daf43f40 0x5626daf46360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.783+0000 7ff6c37fe700 10 client.26449 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.783+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5626daf43f40 0x5626daf46360 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb8e95576de099b server_cookie=2b3ca45787a0f771 in_seq=0 out_seq=0 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.784+0000 7ff6c37fe700 1 -- 172.21.15.46:0/29726702 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5626dae5aa80 con 0x5626daf43f40 2024-05-09T00:20:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.784+0000 7ff6c37fe700 10 client.26449 handle_command_reply: tid=0 2024-05-09T00:20:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.784+0000 7ff6f9164b80 1 client.26449 shutdown 2024-05-09T00:20:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.784+0000 7ff6f9164b80 1 -- 172.21.15.46:0/29726702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dae27d00 msgr2=0x5626daf34480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.784+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dae27d00 0x5626daf34480 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40098c0 tx=0x7ff6d4005960 comp rx=0 tx=0).stop 2024-05-09T00:20:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.784+0000 7ff6f9164b80 1 -- 172.21.15.46:0/29726702 shutdown_connections 2024-05-09T00:20:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.784+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5626daf43f40 0x5626daf46360 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.784+0000 7ff6c37fe700 0 client.26449 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.784+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626daf2a400 0x5626daf349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.784+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dadad4a0 0x5626daf33f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.784+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/29726702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dae27d00 0x5626daf34480 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.784+0000 7ff6f9164b80 1 -- 172.21.15.46:0/29726702 >> 172.21.15.46:0/29726702 conn(0x5626dadb0f70 msgr2=0x5626daf1eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.785+0000 7ff6f9164b80 1 -- 172.21.15.46:0/29726702 shutdown_connections 2024-05-09T00:20:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.785+0000 7ff6f9164b80 1 -- 172.21.15.46:0/29726702 wait complete. 2024-05-09T00:20:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.785+0000 7ff6f9164b80 20 client.26449 trim_cache size 0 max 16384 2024-05-09T00:20:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.806+0000 7ff6f9164b80 1 Processor -- start 2024-05-09T00:20:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.806+0000 7ff6f9164b80 1 -- start start 2024-05-09T00:20:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.806+0000 7ff6f9164b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dadad4a0 0x5626daf32bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.806+0000 7ff6f9164b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626dae27d00 0x5626daf2e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.806+0000 7ff6f9164b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626daf2a400 0x5626daf34800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.806+0000 7ff6f9164b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41b7a50 con 0x5626dae27d00 2024-05-09T00:20:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.806+0000 7ff6f9164b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41b52e0 con 0x5626daf2a400 2024-05-09T00:20:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.806+0000 7ff6f9164b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41b3130 con 0x5626dadad4a0 2024-05-09T00:20:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.806+0000 7ff6e3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626dae27d00 0x5626daf2e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.806+0000 7ff6e3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626dae27d00 0x5626daf2e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37812/0 (socket says 172.21.15.46:37812) 2024-05-09T00:20:52.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.806+0000 7ff6e3fff700 1 -- 172.21.15.46:0/1570444271 learned_addr learned my addr 172.21.15.46:0/1570444271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:52.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.806+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/1570444271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dadad4a0 0x5626daf32bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.807+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/1570444271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626daf2a400 0x5626daf34800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.807+0000 7ff6e3fff700 1 -- 172.21.15.46:0/1570444271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dadad4a0 msgr2=0x5626daf32bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:52.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.807+0000 7ff6e3fff700 1 --2- 172.21.15.46:0/1570444271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dadad4a0 0x5626daf32bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.807+0000 7ff6e3fff700 1 -- 172.21.15.46:0/1570444271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626daf2a400 msgr2=0x5626daf34800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:52.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.807+0000 7ff6e3fff700 1 --2- 172.21.15.46:0/1570444271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626daf2a400 0x5626daf34800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.807+0000 7ff6e3fff700 1 -- 172.21.15.46:0/1570444271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626daf367e0 con 0x5626dae27d00 2024-05-09T00:20:52.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.807+0000 7ff6e3fff700 1 --2- 172.21.15.46:0/1570444271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626dae27d00 0x5626daf2e6d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4000c00 tx=0x7ff6d4005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d139d255132e6b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:52.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.807+0000 7ff6c9ffb700 1 -- 172.21.15.46:0/1570444271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4027040 con 0x5626dae27d00 2024-05-09T00:20:52.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.807+0000 7ff6c9ffb700 1 -- 172.21.15.46:0/1570444271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d4022960 con 0x5626dae27d00 2024-05-09T00:20:52.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.807+0000 7ff6c9ffb700 1 -- 172.21.15.46:0/1570444271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4003a80 con 0x5626dae27d00 2024-05-09T00:20:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.808+0000 7ff6f9164b80 1 -- 172.21.15.46:0/1570444271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626dae27d00 msgr2=0x5626daf2e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.808+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/1570444271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626dae27d00 0x5626daf2e6d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4000c00 tx=0x7ff6d4005520 comp rx=0 tx=0).stop 2024-05-09T00:20:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.808+0000 7ff6f9164b80 1 -- 172.21.15.46:0/1570444271 shutdown_connections 2024-05-09T00:20:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.808+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/1570444271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626daf2a400 0x5626daf34800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.808+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/1570444271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dadad4a0 0x5626daf32bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.808+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/1570444271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626dae27d00 0x5626daf2e6d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.808+0000 7ff6f9164b80 1 -- 172.21.15.46:0/1570444271 >> 172.21.15.46:0/1570444271 conn(0x5626dadb0f70 msgr2=0x5626daf1eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.808+0000 7ff6f9164b80 1 -- 172.21.15.46:0/1570444271 shutdown_connections 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.808+0000 7ff6f9164b80 1 -- 172.21.15.46:0/1570444271 wait complete. 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.809+0000 7ff6f9164b80 1 -- start start 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.809+0000 7ff6f9164b80 1 Processor -- start 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.809+0000 7ff6f9164b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dadad4a0 0x5626daf3c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.809+0000 7ff6f9164b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dae27d00 0x5626daf4b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.809+0000 7ff6f9164b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626daf2a400 0x5626daf4b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.809+0000 7ff6f9164b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41b37a0 con 0x5626daf2a400 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.809+0000 7ff6f9164b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41b4c10 con 0x5626dae27d00 2024-05-09T00:20:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.809+0000 7ff6f9164b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41b3e70 con 0x5626dadad4a0 2024-05-09T00:20:52.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6f9164b80 10 client.0 _ll_register_callbacks cb 0x5626dacbe6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:52.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6e37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626daf2a400 0x5626daf4b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6e37fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dadad4a0 0x5626daf3c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6e37fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dadad4a0 0x5626daf3c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:37406/0 (socket says 172.21.15.46:37406) 2024-05-09T00:20:52.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6e37fe700 1 -- 172.21.15.46:0/94960108 learned_addr learned my addr 172.21.15.46:0/94960108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:52.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6e3fff700 1 --2- 172.21.15.46:0/94960108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dae27d00 0x5626daf4b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6e37fe700 1 -- 172.21.15.46:0/94960108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dae27d00 msgr2=0x5626daf4b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:52.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/94960108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dae27d00 0x5626daf4b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6e37fe700 1 -- 172.21.15.46:0/94960108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626daf2a400 msgr2=0x5626daf4b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:52.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/94960108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626daf2a400 0x5626daf4b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6e37fe700 1 -- 172.21.15.46:0/94960108 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626daf4be90 con 0x5626dadad4a0 2024-05-09T00:20:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6e37fe700 1 --2- 172.21.15.46:0/94960108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dadad4a0 0x5626daf3c840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6d801e4a0 tx=0x7ff6d80097f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14086a7f98ee5d8e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.810+0000 7ff6c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:20:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6c37fe700 1 -- 172.21.15.46:0/94960108 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8005040 con 0x5626dadad4a0 2024-05-09T00:20:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6c37fe700 1 -- 172.21.15.46:0/94960108 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff6d802dc70 con 0x5626dadad4a0 2024-05-09T00:20:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6f9164b80 1 -- 172.21.15.46:0/94960108 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d4022960 con 0x5626dadad4a0 2024-05-09T00:20:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6c37fe700 1 -- 172.21.15.46:0/94960108 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6d80051e0 con 0x5626dadad4a0 2024-05-09T00:20:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6f9164b80 10 client.26455 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6f9164b80 1 -- 172.21.15.46:0/94960108 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626dafae670 con 0x5626dadad4a0 2024-05-09T00:20:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6c37fe700 1 -- 172.21.15.46:0/94960108 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d4022960 con 0x5626dadad4a0 2024-05-09T00:20:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6c37fe700 1 -- 172.21.15.46:0/94960108 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff6d802d8f0 con 0x5626dadad4a0 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6f9164b80 10 client.26455 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6f9164b80 10 client.26455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6f9164b80 20 client.26455 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/94960108 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5626dafaeb20 0x5626dafb0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6f9164b80 4 client.26455 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.811+0000 7ff6f9164b80 1 -- 172.21.15.46:0/94960108 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5626dae54b20 con 0x5626dafaeb20 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.812+0000 7ff6e3fff700 1 --2- 172.21.15.46:0/94960108 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5626dafaeb20 0x5626dafb0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.812+0000 7ff6c37fe700 10 client.26455 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.812+0000 7ff6e3fff700 1 --2- 172.21.15.46:0/94960108 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5626dafaeb20 0x5626dafb0f40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f469686c96f4bd1 server_cookie=b0c1cb574315504b in_seq=0 out_seq=0 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.812+0000 7ff6c37fe700 1 -- 172.21.15.46:0/94960108 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5626dae54b20 con 0x5626dafaeb20 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.812+0000 7ff6c37fe700 10 client.26455 handle_command_reply: tid=0 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.812+0000 7ff6f9164b80 1 client.26455 shutdown 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.813+0000 7ff6f9164b80 1 -- 172.21.15.46:0/94960108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dadad4a0 msgr2=0x5626daf3c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.813+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/94960108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dadad4a0 0x5626daf3c840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6d801e4a0 tx=0x7ff6d80097f0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.813+0000 7ff6f9164b80 1 -- 172.21.15.46:0/94960108 shutdown_connections 2024-05-09T00:20:52.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.813+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/94960108 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5626dafaeb20 0x5626dafb0f40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.813+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/94960108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626daf2a400 0x5626daf4b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.813+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/94960108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626dadad4a0 0x5626daf3c840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.813+0000 7ff6c37fe700 0 client.26455 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:20:52.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.813+0000 7ff6f9164b80 1 --2- 172.21.15.46:0/94960108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5626dae27d00 0x5626daf4b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.813+0000 7ff6f9164b80 1 -- 172.21.15.46:0/94960108 >> 172.21.15.46:0/94960108 conn(0x5626dadb0f70 msgr2=0x5626daf2e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:52.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.813+0000 7ff6f9164b80 1 -- 172.21.15.46:0/94960108 shutdown_connections 2024-05-09T00:20:52.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.813+0000 7ff6f9164b80 1 -- 172.21.15.46:0/94960108 wait complete. 2024-05-09T00:20:52.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.813+0000 7ff6f9164b80 20 client.26455 trim_cache size 0 max 16384 2024-05-09T00:20:52.819 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:20:52.819 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:20:52.819 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:20:52.819 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:20:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.814+0000 7ff6ea74e700 1 -- 172.21.15.46:0/3977252199 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff6cc0b2900 msgr2=0x7ff6cc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.814+0000 7ff6ea74e700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff6cc0b2900 0x7ff6cc0b4e10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0cee50 tx=0x7ff6dc0cbe70 comp rx=0 tx=0).stop 2024-05-09T00:20:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.814+0000 7ff6ea74e700 1 -- 172.21.15.46:0/3977252199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6e410e8f0 msgr2=0x7ff6e41ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.814+0000 7ff6ea74e700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6e410e8f0 0x7ff6e41ad1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00e3c0 tx=0x7ff6dc00e6d0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.814+0000 7ff6ea74e700 1 -- 172.21.15.46:0/3977252199 shutdown_connections 2024-05-09T00:20:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.814+0000 7ff6ea74e700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff6cc0b2900 0x7ff6cc0b4e10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.814+0000 7ff6ea74e700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6e410e8f0 0x7ff6e41ad1a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.814+0000 7ff6ea74e700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6e4105480 0x7ff6e41a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.814+0000 7ff6ea74e700 1 --2- 172.21.15.46:0/3977252199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6e4105dd0 0x7ff6e41a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.814+0000 7ff6ea74e700 1 -- 172.21.15.46:0/3977252199 >> 172.21.15.46:0/3977252199 conn(0x7ff6e40fef30 msgr2=0x7ff6e40ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.814+0000 7ff6ea74e700 1 -- 172.21.15.46:0/3977252199 shutdown_connections 2024-05-09T00:20:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:52.814+0000 7ff6ea74e700 1 -- 172.21.15.46:0/3977252199 wait complete. 2024-05-09T00:20:52.925 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:52 smithi169 ceph-mon[34777]: pgmap v661: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:52.926 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:52 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:20:52.926 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:52 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:20:52.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:52 smithi142 ceph-mon[34533]: pgmap v661: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:20:52.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:52 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:20:52.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:52 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:20:53.558 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:20:53.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:20:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:20:54.735 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:54 smithi046 ceph-mon[30151]: pgmap v662: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:20:54.736 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:54.736 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:54.736 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:54.736 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:54 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:54.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:54 smithi169 ceph-mon[34777]: pgmap v662: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:20:54.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:54.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:54.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:54.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:54 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:54.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:54 smithi142 ceph-mon[34533]: pgmap v662: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:20:54.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:54.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:54.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:54.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:54 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.053+0000 7f65b3e20700 1 -- 172.21.15.46:0/1103502811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac107df0 msgr2=0x7f65ac108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.053+0000 7f65b3e20700 1 --2- 172.21.15.46:0/1103502811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac107df0 0x7f65ac108200 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f65a000b0d0 tx=0x7f65a000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:20:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.053+0000 7f65b3e20700 1 -- 172.21.15.46:0/1103502811 shutdown_connections 2024-05-09T00:20:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.053+0000 7f65b3e20700 1 --2- 172.21.15.46:0/1103502811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65ac108740 0x7f65ac1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.053+0000 7f65b3e20700 1 --2- 172.21.15.46:0/1103502811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65ac1034c0 0x7f65ac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.053+0000 7f65b3e20700 1 --2- 172.21.15.46:0/1103502811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac107df0 0x7f65ac108200 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.053+0000 7f65b3e20700 1 -- 172.21.15.46:0/1103502811 >> 172.21.15.46:0/1103502811 conn(0x7f65ac0fedc0 msgr2=0x7f65ac1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.054+0000 7f65b3e20700 1 -- 172.21.15.46:0/1103502811 shutdown_connections 2024-05-09T00:20:56.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.054+0000 7f65b3e20700 1 -- 172.21.15.46:0/1103502811 wait complete. 2024-05-09T00:20:56.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.054+0000 7f65b3e20700 1 Processor -- start 2024-05-09T00:20:56.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.054+0000 7f65b3e20700 1 -- start start 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b3e20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac1034c0 0x7f65ac19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b3e20700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65ac107df0 0x7f65ac19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b3e20700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65ac108740 0x7f65ac19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b3e20700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac113350 con 0x7f65ac1034c0 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b3e20700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1131d0 con 0x7f65ac108740 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b3e20700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1134d0 con 0x7f65ac107df0 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b1bbc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac1034c0 0x7f65ac19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b1bbc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac1034c0 0x7f65ac19ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37832/0 (socket says 172.21.15.46:37832) 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b1bbc700 1 -- 172.21.15.46:0/3301904741 learned_addr learned my addr 172.21.15.46:0/3301904741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b13bb700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65ac107df0 0x7f65ac19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b1bbc700 1 -- 172.21.15.46:0/3301904741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65ac107df0 msgr2=0x7f65ac19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b1bbc700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65ac107df0 0x7f65ac19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b1bbc700 1 -- 172.21.15.46:0/3301904741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65ac108740 msgr2=0x7f65ac19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b1bbc700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65ac108740 0x7f65ac19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b1bbc700 1 -- 172.21.15.46:0/3301904741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac19fd60 con 0x7f65ac1034c0 2024-05-09T00:20:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.055+0000 7f65b1bbc700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac1034c0 0x7f65ac19ae90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f659c00e9a0 tx=0x7f659c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.056+0000 7f65aaffd700 1 -- 172.21.15.46:0/3301904741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c00cbd0 con 0x7f65ac1034c0 2024-05-09T00:20:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.056+0000 7f65aaffd700 1 -- 172.21.15.46:0/3301904741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f659c00cd70 con 0x7f65ac1034c0 2024-05-09T00:20:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.056+0000 7f65b3e20700 1 -- 172.21.15.46:0/3301904741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ac1a0050 con 0x7f65ac1034c0 2024-05-09T00:20:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.056+0000 7f65b3e20700 1 -- 172.21.15.46:0/3301904741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ac1a7970 con 0x7f65ac1034c0 2024-05-09T00:20:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.056+0000 7f65aaffd700 1 -- 172.21.15.46:0/3301904741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c018680 con 0x7f65ac1034c0 2024-05-09T00:20:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.057+0000 7f65b3e20700 1 -- 172.21.15.46:0/3301904741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6598005360 con 0x7f65ac1034c0 2024-05-09T00:20:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.058+0000 7f65aaffd700 1 -- 172.21.15.46:0/3301904741 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f659c018820 con 0x7f65ac1034c0 2024-05-09T00:20:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.059+0000 7f65aaffd700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f65940b2880 0x7f65940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.059+0000 7f65b13bb700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f65940b2880 0x7f65940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.059+0000 7f65aaffd700 1 -- 172.21.15.46:0/3301904741 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f659c0e28a0 con 0x7f65ac1034c0 2024-05-09T00:20:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.060+0000 7f65b13bb700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f65940b2880 0x7f65940b4d90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f65ac109b40 tx=0x7f65a000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:56.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.063+0000 7f65aaffd700 1 -- 172.21.15.46:0/3301904741 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f659c0aa5c0 con 0x7f65ac1034c0 2024-05-09T00:20:56.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.337+0000 7f65b3e20700 1 -- 172.21.15.46:0/3301904741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6598005650 con 0x7f65ac1034c0 2024-05-09T00:20:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.338+0000 7f65aaffd700 1 -- 172.21.15.46:0/3301904741 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f659c0a9ec0 con 0x7f65ac1034c0 2024-05-09T00:20:56.338 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:20:56.338 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:20:56.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.341+0000 7f65b3e20700 1 -- 172.21.15.46:0/3301904741 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f65940b2880 msgr2=0x7f65940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:56.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.341+0000 7f65b3e20700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f65940b2880 0x7f65940b4d90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f65ac109b40 tx=0x7f65a000b040 comp rx=0 tx=0).stop 2024-05-09T00:20:56.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.341+0000 7f65b3e20700 1 -- 172.21.15.46:0/3301904741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac1034c0 msgr2=0x7f65ac19ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.341+0000 7f65b3e20700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac1034c0 0x7f65ac19ae90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f659c00e9a0 tx=0x7f659c00ecb0 comp rx=0 tx=0).stop 2024-05-09T00:20:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.341+0000 7f65b3e20700 1 -- 172.21.15.46:0/3301904741 shutdown_connections 2024-05-09T00:20:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.341+0000 7f65b3e20700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f65940b2880 0x7f65940b4d90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.341+0000 7f65b3e20700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65ac108740 0x7f65ac19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.341+0000 7f65b3e20700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac1034c0 0x7f65ac19ae90 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.341+0000 7f65b3e20700 1 --2- 172.21.15.46:0/3301904741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65ac107df0 0x7f65ac19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.341+0000 7f65b3e20700 1 -- 172.21.15.46:0/3301904741 >> 172.21.15.46:0/3301904741 conn(0x7f65ac0fedc0 msgr2=0x7f65ac06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.341+0000 7f65b3e20700 1 -- 172.21.15.46:0/3301904741 shutdown_connections 2024-05-09T00:20:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:56.341+0000 7f65b3e20700 1 -- 172.21.15.46:0/3301904741 wait complete. 2024-05-09T00:20:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:20:56.715 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:56 smithi046 ceph-mon[30151]: pgmap v663: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:56.715 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:56.715 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:56 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:56.715 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:56 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3301904741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:20:56.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:56 smithi169 ceph-mon[34777]: pgmap v663: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:56.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:56.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:56 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:56.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:56 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3301904741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:20:56.936 INFO:tasks.workunit.client.0.smithi046.stdout:375/600: open fd = 3 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:376/600: open fd = 3 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:377/600: open fd = 3 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:378/600: open fd = 3 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.937 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:379/600: open fd = 3 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:380/600: open fd = 3 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:381/600: open fd = 3 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:382/600: open fd = 3 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.938 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:383/600: open fd = 3 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:384/600: open fd = 3 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:385/600: open fd = 3 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:386/600: open fd = 3 2024-05-09T00:20:56.939 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:387/600: open fd = 3 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:388/600: open fd = 3 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:389/600: open fd = 3 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.940 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.941 INFO:tasks.workunit.client.0.smithi046.stdout:390/600: open fd = 3 2024-05-09T00:20:56.941 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.941 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.941 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.941 INFO:tasks.workunit.client.0.smithi046.stdout:391/600: open fd = 3 2024-05-09T00:20:56.941 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.941 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.941 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.941 INFO:tasks.workunit.client.0.smithi046.stdout:392/600: open fd = 3 2024-05-09T00:20:56.941 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.941 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:393/600: open fd = 3 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:394/600: open fd = 3 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:395/600: open fd = 3 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.942 INFO:tasks.workunit.client.0.smithi046.stdout:396/600: open fd = 3 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:397/600: open fd = 3 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:398/600: open fd = 3 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:399/600: open fd = 3 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:400/600: open fd = 3 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:401/600: open fd = 3 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:402/600: open fd = 3 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.944 INFO:tasks.workunit.client.0.smithi046.stdout:403/600: open fd = 3 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:404/600: open fd = 3 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:405/600: open fd = 3 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:406/600: open fd = 3 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.945 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:407/600: open fd = 3 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:408/600: open fd = 3 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:409/600: open fd = 3 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:410/600: open fd = 3 2024-05-09T00:20:56.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:411/600: open fd = 3 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:412/600: open fd = 3 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:413/600: open fd = 3 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.947 INFO:tasks.workunit.client.0.smithi046.stdout:414/600: open fd = 3 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:415/600: open fd = 3 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:416/600: open fd = 3 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:417/600: open fd = 3 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.948 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.949 INFO:tasks.workunit.client.0.smithi046.stdout:418/600: open fd = 3 2024-05-09T00:20:56.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.949 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.949 INFO:tasks.workunit.client.0.smithi046.stdout:419/600: open fd = 3 2024-05-09T00:20:56.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.949 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.949 INFO:tasks.workunit.client.0.smithi046.stdout:420/600: open fd = 3 2024-05-09T00:20:56.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.949 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:421/600: open fd = 3 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:422/600: open fd = 3 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:423/600: open fd = 3 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:424/600: open fd = 3 2024-05-09T00:20:56.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:425/600: open fd = 3 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:426/600: open fd = 3 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:427/600: open fd = 3 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:428/600: open fd = 3 2024-05-09T00:20:56.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:429/600: open fd = 3 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:430/600: open fd = 3 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:431/600: open fd = 3 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.952 INFO:tasks.workunit.client.0.smithi046.stdout:432/600: open fd = 3 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:433/600: open fd = 3 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:434/600: open fd = 3 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:435/600: open fd = 3 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.953 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.954 INFO:tasks.workunit.client.0.smithi046.stdout:436/600: open fd = 3 2024-05-09T00:20:56.954 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.954 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:20:56.954 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:20:56.954 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:56 smithi142 ceph-mon[34533]: pgmap v663: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:20:56.954 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:56.954 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:56 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:56.954 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:56 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3301904741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:20:57.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:20:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:20:57.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:20:57.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:20:57.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:57.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:20:57.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:20:57.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:57.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:20:57.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:20:57.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:20:58.778 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:20:58 smithi046 ceph-mon[30151]: pgmap v664: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:20:58.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:20:58 smithi169 ceph-mon[34777]: pgmap v664: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:20:58.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:20:58 smithi142 ceph-mon[34533]: pgmap v664: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:20:59.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.022+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2752673428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdacc108740 msgr2=0x7fdacc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:59.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.022+0000 7fdad3bcd700 1 --2- 172.21.15.46:0/2752673428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdacc108740 0x7fdacc1100a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fdac400b0d0 tx=0x7fdac400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.022+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2752673428 shutdown_connections 2024-05-09T00:20:59.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.022+0000 7fdad3bcd700 1 --2- 172.21.15.46:0/2752673428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdacc108740 0x7fdacc1100a0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.022+0000 7fdad3bcd700 1 --2- 172.21.15.46:0/2752673428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdacc1034c0 0x7fdacc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.022+0000 7fdad3bcd700 1 --2- 172.21.15.46:0/2752673428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdacc107df0 0x7fdacc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.022+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2752673428 >> 172.21.15.46:0/2752673428 conn(0x7fdacc0fedc0 msgr2=0x7fdacc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.022+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2752673428 shutdown_connections 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.022+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2752673428 wait complete. 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.023+0000 7fdad3bcd700 1 Processor -- start 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.023+0000 7fdad3bcd700 1 -- start start 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.023+0000 7fdad3bcd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdacc1034c0 0x7fdacc19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.023+0000 7fdad3bcd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdacc107df0 0x7fdacc19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.023+0000 7fdad3bcd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdacc108740 0x7fdacc19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.023+0000 7fdad3bcd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc113370 con 0x7fdacc108740 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.023+0000 7fdad3bcd700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1131f0 con 0x7fdacc1034c0 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.023+0000 7fdad3bcd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1134f0 con 0x7fdacc107df0 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.023+0000 7fdad216a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdacc108740 0x7fdacc19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.023+0000 7fdad1168700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdacc107df0 0x7fdacc19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.023+0000 7fdad1969700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdacc1034c0 0x7fdacc19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.023+0000 7fdad216a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdacc108740 0x7fdacc19f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34974/0 (socket says 172.21.15.46:34974) 2024-05-09T00:20:59.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdad216a700 1 -- 172.21.15.46:0/2517729906 learned_addr learned my addr 172.21.15.46:0/2517729906 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:59.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdad1168700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdacc107df0 0x7fdacc19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:49772/0 (socket says 172.21.15.46:49772) 2024-05-09T00:20:59.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdad1969700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdacc1034c0 0x7fdacc19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:36430/0 (socket says 172.21.15.46:36430) 2024-05-09T00:20:59.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdad216a700 1 -- 172.21.15.46:0/2517729906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdacc107df0 msgr2=0x7fdacc19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:59.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdad216a700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdacc107df0 0x7fdacc19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdad216a700 1 -- 172.21.15.46:0/2517729906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdacc1034c0 msgr2=0x7fdacc19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:59.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdad216a700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdacc1034c0 0x7fdacc19ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdad216a700 1 -- 172.21.15.46:0/2517729906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdacc19fc60 con 0x7fdacc108740 2024-05-09T00:20:59.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdad216a700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdacc108740 0x7fdacc19f5c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fdac40095c0 tx=0x7fdac4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:59.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdac2ffd700 1 -- 172.21.15.46:0/2517729906 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4003c00 con 0x7fdacc108740 2024-05-09T00:20:59.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdac2ffd700 1 -- 172.21.15.46:0/2517729906 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac4007e20 con 0x7fdacc108740 2024-05-09T00:20:59.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2517729906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdacc19fef0 con 0x7fdacc108740 2024-05-09T00:20:59.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdac2ffd700 1 -- 172.21.15.46:0/2517729906 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40054b0 con 0x7fdacc108740 2024-05-09T00:20:59.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.024+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2517729906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdacc1a0370 con 0x7fdacc108740 2024-05-09T00:20:59.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.027+0000 7fdac2ffd700 1 -- 172.21.15.46:0/2517729906 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdac402b020 con 0x7fdacc108740 2024-05-09T00:20:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.028+0000 7fdac2ffd700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdab80b2880 0x7fdab80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.028+0000 7fdad1969700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdab80b2880 0x7fdab80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.028+0000 7fdac2ffd700 1 -- 172.21.15.46:0/2517729906 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fdac400f020 con 0x7fdacc108740 2024-05-09T00:20:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.028+0000 7fdae25e3b80 1 Processor -- start 2024-05-09T00:20:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.028+0000 7fdad1969700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdab80b2880 0x7fdab80b4d90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdac8005950 tx=0x7fdac800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.028+0000 7fdae25e3b80 1 -- start start 2024-05-09T00:20:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdae25e3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561529fb7d00 0x56152a0c0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdae25e3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 0x56152a0c0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdae25e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56152a0ba400 0x56152a0ba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdae25e3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561529fdd0a0 con 0x561529f3d4a0 2024-05-09T00:20:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdae25e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561529f829a0 con 0x56152a0ba400 2024-05-09T00:20:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdae25e3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561529fd9060 con 0x561529fb7d00 2024-05-09T00:20:59.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdad1168700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 0x56152a0c0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdad1168700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 0x56152a0c0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34980/0 (socket says 172.21.15.46:34980) 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdad1168700 1 -- 172.21.15.46:0/864102137 learned_addr learned my addr 172.21.15.46:0/864102137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdad216a700 1 --2- 172.21.15.46:0/864102137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56152a0ba400 0x56152a0ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdad1168700 1 -- 172.21.15.46:0/864102137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561529fb7d00 msgr2=0x56152a0c0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdad1168700 1 --2- 172.21.15.46:0/864102137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561529fb7d00 0x56152a0c0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdad1168700 1 -- 172.21.15.46:0/864102137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56152a0ba400 msgr2=0x56152a0ba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdad1168700 1 --2- 172.21.15.46:0/864102137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56152a0ba400 0x56152a0ba810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdad1168700 1 -- 172.21.15.46:0/864102137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561529f73500 con 0x561529f3d4a0 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdad1168700 1 --2- 172.21.15.46:0/864102137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 0x56152a0c0780 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00bc70 tx=0x7fdabc00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d8b78f22939258 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.029+0000 7fdab7fff700 1 -- 172.21.15.46:0/864102137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc010c90 con 0x561529f3d4a0 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.030+0000 7fdab7fff700 1 -- 172.21.15.46:0/864102137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdabc010e30 con 0x561529f3d4a0 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.030+0000 7fdab7fff700 1 -- 172.21.15.46:0/864102137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc005280 con 0x561529f3d4a0 2024-05-09T00:20:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.030+0000 7fdae25e3b80 1 -- 172.21.15.46:0/864102137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 msgr2=0x56152a0c0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.030+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/864102137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 0x56152a0c0780 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00bc70 tx=0x7fdabc00bf80 comp rx=0 tx=0).stop 2024-05-09T00:20:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.030+0000 7fdae25e3b80 1 -- 172.21.15.46:0/864102137 shutdown_connections 2024-05-09T00:20:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.030+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/864102137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56152a0ba400 0x56152a0ba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.030+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/864102137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561529fb7d00 0x56152a0c0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.031+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/864102137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 0x56152a0c0780 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.031+0000 7fdae25e3b80 1 -- 172.21.15.46:0/864102137 >> 172.21.15.46:0/864102137 conn(0x561529f40f70 msgr2=0x56152a0b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.031+0000 7fdae25e3b80 1 -- 172.21.15.46:0/864102137 shutdown_connections 2024-05-09T00:20:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.031+0000 7fdae25e3b80 1 -- 172.21.15.46:0/864102137 wait complete. 2024-05-09T00:20:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.031+0000 7fdae25e3b80 1 -- start start 2024-05-09T00:20:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.031+0000 7fdae25e3b80 1 Processor -- start 2024-05-09T00:20:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdae25e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529f3d4a0 0x56152a0c4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdae25e3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561529fb7d00 0x56152a0c4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdae25e3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152a0ba400 0x56152a0c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdae25e3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56152a0bb160 con 0x56152a0ba400 2024-05-09T00:20:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdae25e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561529f77b20 con 0x561529f3d4a0 2024-05-09T00:20:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdae25e3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56152a0c7020 con 0x561529fb7d00 2024-05-09T00:20:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdae25e3b80 10 client.0 _ll_register_callbacks cb 0x561529e4e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdad216a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152a0ba400 0x56152a0c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdad216a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152a0ba400 0x56152a0c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34984/0 (socket says 172.21.15.46:34984) 2024-05-09T00:20:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdad216a700 1 -- 172.21.15.46:0/2370112882 learned_addr learned my addr 172.21.15.46:0/2370112882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdad1168700 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561529fb7d00 0x56152a0c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdad1969700 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529f3d4a0 0x56152a0c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdad216a700 1 -- 172.21.15.46:0/2370112882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561529fb7d00 msgr2=0x56152a0c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdad216a700 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561529fb7d00 0x56152a0c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdad216a700 1 -- 172.21.15.46:0/2370112882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529f3d4a0 msgr2=0x56152a0c4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdad216a700 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529f3d4a0 0x56152a0c4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.032+0000 7fdad216a700 1 -- 172.21.15.46:0/2370112882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561529e58490 con 0x56152a0ba400 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdad216a700 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152a0ba400 0x56152a0c4a80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fdac40e4a30 tx=0x7fdac40aeb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680cd38da3f84260 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdab57fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdab57fa700 1 -- 172.21.15.46:0/2370112882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40add50 con 0x56152a0ba400 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdab57fa700 1 -- 172.21.15.46:0/2370112882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fdac4094b50 con 0x56152a0ba400 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdae25e3b80 1 -- 172.21.15.46:0/2370112882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdabc010e30 con 0x56152a0ba400 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdab57fa700 1 -- 172.21.15.46:0/2370112882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdac40ae0f0 con 0x56152a0ba400 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdae25e3b80 10 client.16968 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdae25e3b80 1 -- 172.21.15.46:0/2370112882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56152a0d19e0 con 0x56152a0ba400 2024-05-09T00:20:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdab57fa700 1 -- 172.21.15.46:0/2370112882 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdabc010e30 con 0x56152a0ba400 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdab57fa700 1 -- 172.21.15.46:0/2370112882 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdac4011070 con 0x56152a0ba400 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdae25e3b80 10 client.16968 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdae25e3b80 10 client.16968 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.033+0000 7fdae25e3b80 20 client.16968 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.034+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x56152a0d3f00 0x56152a0d6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.034+0000 7fdae25e3b80 4 client.16968 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.034+0000 7fdae25e3b80 1 -- 172.21.15.46:0/2370112882 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561529feaa80 con 0x56152a0d3f00 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.034+0000 7fdad1969700 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x56152a0d3f00 0x56152a0d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.034+0000 7fdab57fa700 10 client.16968 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.034+0000 7fdad1969700 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x56152a0d3f00 0x56152a0d6370 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19f10619d5cb6dc7 server_cookie=51a860d5c43ada38 in_seq=0 out_seq=0 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.035+0000 7fdab57fa700 1 -- 172.21.15.46:0/2370112882 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561529feaa80 con 0x56152a0d3f00 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.035+0000 7fdab57fa700 10 client.16968 handle_command_reply: tid=0 2024-05-09T00:20:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.035+0000 7fdae25e3b80 1 client.16968 shutdown 2024-05-09T00:20:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.035+0000 7fdae25e3b80 1 -- 172.21.15.46:0/2370112882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152a0ba400 msgr2=0x56152a0c4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.035+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152a0ba400 0x56152a0c4a80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fdac40e4a30 tx=0x7fdac40aeb20 comp rx=0 tx=0).stop 2024-05-09T00:20:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.036+0000 7fdae25e3b80 1 -- 172.21.15.46:0/2370112882 shutdown_connections 2024-05-09T00:20:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.036+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x56152a0d3f00 0x56152a0d6370 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.036+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56152a0ba400 0x56152a0c4a80 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.036+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529f3d4a0 0x56152a0c4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.036+0000 7fdab57fa700 0 client.16968 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:20:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.036+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/2370112882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561529fb7d00 0x56152a0c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.036+0000 7fdae25e3b80 1 -- 172.21.15.46:0/2370112882 >> 172.21.15.46:0/2370112882 conn(0x561529f40f70 msgr2=0x56152a0aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.036+0000 7fdae25e3b80 1 -- 172.21.15.46:0/2370112882 shutdown_connections 2024-05-09T00:20:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.036+0000 7fdae25e3b80 1 -- 172.21.15.46:0/2370112882 wait complete. 2024-05-09T00:20:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.036+0000 7fdae25e3b80 20 client.16968 trim_cache size 0 max 16384 2024-05-09T00:20:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.051+0000 7fdae25e3b80 1 Processor -- start 2024-05-09T00:20:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.051+0000 7fdae25e3b80 1 -- start start 2024-05-09T00:20:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.051+0000 7fdae25e3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 0x56152a0bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.051+0000 7fdae25e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529fb7d00 0x56152a0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.051+0000 7fdae25e3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56152a0ba400 0x56152a0d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.051+0000 7fdae25e3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1af240 con 0x561529f3d4a0 2024-05-09T00:20:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.051+0000 7fdae25e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1acad0 con 0x561529fb7d00 2024-05-09T00:20:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.051+0000 7fdae25e3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1aa900 con 0x56152a0ba400 2024-05-09T00:20:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.051+0000 7fdad1969700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 0x56152a0bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.051+0000 7fdad1969700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 0x56152a0bf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34992/0 (socket says 172.21.15.46:34992) 2024-05-09T00:20:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdad1969700 1 -- 172.21.15.46:0/1091198589 learned_addr learned my addr 172.21.15.46:0/1091198589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdad1168700 1 --2- 172.21.15.46:0/1091198589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529fb7d00 0x56152a0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdad216a700 1 --2- 172.21.15.46:0/1091198589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56152a0ba400 0x56152a0d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdad1969700 1 -- 172.21.15.46:0/1091198589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56152a0ba400 msgr2=0x56152a0d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdad1969700 1 --2- 172.21.15.46:0/1091198589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56152a0ba400 0x56152a0d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdad1969700 1 -- 172.21.15.46:0/1091198589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529fb7d00 msgr2=0x56152a0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdad1969700 1 --2- 172.21.15.46:0/1091198589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529fb7d00 0x56152a0cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdad1969700 1 -- 172.21.15.46:0/1091198589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56152a0c3dd0 con 0x561529f3d4a0 2024-05-09T00:20:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdad1168700 1 --2- 172.21.15.46:0/1091198589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529fb7d00 0x56152a0cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:20:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdad1969700 1 --2- 172.21.15.46:0/1091198589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 0x56152a0bf870 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdac8005d90 tx=0x7fdac8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64abd81d0944fee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdab77fe700 1 -- 172.21.15.46:0/1091198589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac802b070 con 0x561529f3d4a0 2024-05-09T00:20:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdab77fe700 1 -- 172.21.15.46:0/1091198589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac8011b80 con 0x561529f3d4a0 2024-05-09T00:20:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.052+0000 7fdab77fe700 1 -- 172.21.15.46:0/1091198589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac801e730 con 0x561529f3d4a0 2024-05-09T00:20:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.053+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1091198589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 msgr2=0x56152a0bf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.053+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/1091198589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 0x56152a0bf870 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdac8005d90 tx=0x7fdac8013000 comp rx=0 tx=0).stop 2024-05-09T00:20:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.053+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1091198589 shutdown_connections 2024-05-09T00:20:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.053+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/1091198589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56152a0ba400 0x56152a0d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.053+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/1091198589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529f3d4a0 0x56152a0bf870 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.053+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/1091198589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529fb7d00 0x56152a0cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.053+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1091198589 >> 172.21.15.46:0/1091198589 conn(0x561529f40f70 msgr2=0x56152a0bf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.053+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1091198589 shutdown_connections 2024-05-09T00:20:59.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.053+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1091198589 wait complete. 2024-05-09T00:20:59.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdae25e3b80 1 -- start start 2024-05-09T00:20:59.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdae25e3b80 1 Processor -- start 2024-05-09T00:20:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdae25e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529f3d4a0 0x56152a0cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdae25e3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529fb7d00 0x56152a0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdae25e3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56152a0ba400 0x56152a0dad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdae25e3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1aaf90 con 0x561529fb7d00 2024-05-09T00:20:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdae25e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1ac400 con 0x561529f3d4a0 2024-05-09T00:20:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdae25e3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1ab660 con 0x56152a0ba400 2024-05-09T00:20:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdae25e3b80 10 client.0 _ll_register_callbacks cb 0x561529e4e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:20:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdad1168700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529fb7d00 0x56152a0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdad1168700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529fb7d00 0x56152a0cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35008/0 (socket says 172.21.15.46:35008) 2024-05-09T00:20:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdad1168700 1 -- 172.21.15.46:0/1855549191 learned_addr learned my addr 172.21.15.46:0/1855549191 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdad216a700 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56152a0ba400 0x56152a0dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.054+0000 7fdad1969700 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529f3d4a0 0x56152a0cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdad1168700 1 -- 172.21.15.46:0/1855549191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56152a0ba400 msgr2=0x56152a0dad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdad1168700 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56152a0ba400 0x56152a0dad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdad1168700 1 -- 172.21.15.46:0/1855549191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529f3d4a0 msgr2=0x56152a0cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdad1168700 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529f3d4a0 0x56152a0cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdad1168700 1 -- 172.21.15.46:0/1855549191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56152a0db4b0 con 0x561529fb7d00 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdad1969700 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529f3d4a0 0x56152a0cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdad1168700 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529fb7d00 0x56152a0cc7a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x56152a0b6c90 tx=0x7fdabc010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6355ce9b9fc05db server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdab57fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdab57fa700 1 -- 172.21.15.46:0/1855549191 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc02b060 con 0x561529fb7d00 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1855549191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac8011b80 con 0x561529fb7d00 2024-05-09T00:20:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdab57fa700 1 -- 172.21.15.46:0/1855549191 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fdabc003bc0 con 0x561529fb7d00 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdab57fa700 1 -- 172.21.15.46:0/1855549191 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdabc003dc0 con 0x561529fb7d00 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdae25e3b80 10 client.16974 fetch_fsmap learned FSMap version 26 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1855549191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56152a13e210 con 0x561529fb7d00 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.055+0000 7fdab57fa700 1 -- 172.21.15.46:0/1855549191 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac8011b80 con 0x561529fb7d00 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.056+0000 7fdab57fa700 1 -- 172.21.15.46:0/1855549191 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdabc030070 con 0x561529fb7d00 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.056+0000 7fdae25e3b80 10 client.16974 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.056+0000 7fdae25e3b80 10 client.16974 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.056+0000 7fdae25e3b80 20 client.16974 populate_metadata read hostname 'smithi046' 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.056+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x56152a13e640 0x56152a140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.056+0000 7fdae25e3b80 4 client.16974 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.056+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1855549191 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561529fe4b20 con 0x56152a13e640 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.056+0000 7fdad1969700 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x56152a13e640 0x56152a140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.056+0000 7fdad1969700 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x56152a13e640 0x56152a140a60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a57534674db491ae server_cookie=10e6acb8ed10eb77 in_seq=0 out_seq=0 2024-05-09T00:20:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.056+0000 7fdab57fa700 10 client.16974 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdab57fa700 1 -- 172.21.15.46:0/1855549191 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561529fe4b20 con 0x56152a13e640 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdab57fa700 10 client.16974 handle_command_reply: tid=0 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdae25e3b80 1 client.16974 shutdown 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1855549191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529fb7d00 msgr2=0x56152a0cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529fb7d00 0x56152a0cc7a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x56152a0b6c90 tx=0x7fdabc010fa0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1855549191 shutdown_connections 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x56152a13e640 0x56152a140a60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56152a0ba400 0x56152a0dad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561529f3d4a0 0x56152a0cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdab57fa700 0 client.16974 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdae25e3b80 1 --2- 172.21.15.46:0/1855549191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561529fb7d00 0x56152a0cc7a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1855549191 >> 172.21.15.46:0/1855549191 conn(0x561529f40f70 msgr2=0x56152a0aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1855549191 shutdown_connections 2024-05-09T00:20:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdae25e3b80 1 -- 172.21.15.46:0/1855549191 wait complete. 2024-05-09T00:20:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.057+0000 7fdae25e3b80 20 client.16974 trim_cache size 0 max 16384 2024-05-09T00:20:59.063 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:20:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.058+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2517729906 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdab80b2880 msgr2=0x7fdab80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.058+0000 7fdad3bcd700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdab80b2880 0x7fdab80b4d90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdac8005950 tx=0x7fdac800b410 comp rx=0 tx=0).stop 2024-05-09T00:20:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.058+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2517729906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdacc108740 msgr2=0x7fdacc19f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:20:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.058+0000 7fdad3bcd700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdacc108740 0x7fdacc19f5c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fdac40095c0 tx=0x7fdac4007a20 comp rx=0 tx=0).stop 2024-05-09T00:20:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.058+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2517729906 shutdown_connections 2024-05-09T00:20:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.058+0000 7fdad3bcd700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdab80b2880 0x7fdab80b4d90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.058+0000 7fdad3bcd700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdacc108740 0x7fdacc19f5c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.058+0000 7fdad3bcd700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdacc1034c0 0x7fdacc19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.058+0000 7fdad3bcd700 1 --2- 172.21.15.46:0/2517729906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdacc107df0 0x7fdacc19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:20:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.058+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2517729906 >> 172.21.15.46:0/2517729906 conn(0x7fdacc0fedc0 msgr2=0x7fdacc06b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:20:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.058+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2517729906 shutdown_connections 2024-05-09T00:20:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:20:59.058+0000 7fdad3bcd700 1 -- 172.21.15.46:0/2517729906 wait complete. 2024-05-09T00:20:59.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:21:00.009 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:21:00.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:00 smithi046 ceph-mon[30151]: pgmap v665: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:21:00.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:00 smithi169 ceph-mon[34777]: pgmap v665: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:21:00.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:00 smithi142 ceph-mon[34533]: pgmap v665: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:21:01.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.793+0000 7f0f3e869700 1 -- 172.21.15.46:0/3910049950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f381034e0 msgr2=0x7f0f381038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:01.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.793+0000 7f0f3e869700 1 --2- 172.21.15.46:0/3910049950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f381034e0 0x7f0f381038f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0f280099c0 tx=0x7f0f28009cd0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.793+0000 7f0f3e869700 1 -- 172.21.15.46:0/3910049950 shutdown_connections 2024-05-09T00:21:01.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.793+0000 7f0f3e869700 1 --2- 172.21.15.46:0/3910049950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f38108760 0x7f0f381100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.793+0000 7f0f3e869700 1 --2- 172.21.15.46:0/3910049950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f381034e0 0x7f0f381038f0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.793+0000 7f0f3e869700 1 --2- 172.21.15.46:0/3910049950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f38107e10 0x7f0f38108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.793+0000 7f0f3e869700 1 -- 172.21.15.46:0/3910049950 >> 172.21.15.46:0/3910049950 conn(0x7f0f380fef30 msgr2=0x7f0f38101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:01.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.793+0000 7f0f3e869700 1 -- 172.21.15.46:0/3910049950 shutdown_connections 2024-05-09T00:21:01.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.793+0000 7f0f3e869700 1 -- 172.21.15.46:0/3910049950 wait complete. 2024-05-09T00:21:01.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.793+0000 7f0f3e869700 1 Processor -- start 2024-05-09T00:21:01.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.793+0000 7f0f3e869700 1 -- start start 2024-05-09T00:21:01.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f3e869700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f381034e0 0x7f0f38110aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f3e869700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f38107e10 0x7f0f38110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f3e869700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f38108760 0x7f0f38111520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f3e869700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f381144e0 con 0x7f0f38107e10 2024-05-09T00:21:01.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f3e869700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f38114360 con 0x7f0f381034e0 2024-05-09T00:21:01.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f3e869700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f38114660 con 0x7f0f38108760 2024-05-09T00:21:01.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f38107e10 0x7f0f38110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f38107e10 0x7f0f38110fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35034/0 (socket says 172.21.15.46:35034) 2024-05-09T00:21:01.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f2ffff700 1 -- 172.21.15.46:0/431987159 learned_addr learned my addr 172.21.15.46:0/431987159 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:01.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f3ce06700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f38108760 0x7f0f38111520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f37fff700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f381034e0 0x7f0f38110aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f2ffff700 1 -- 172.21.15.46:0/431987159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f38108760 msgr2=0x7f0f38111520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:01.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f38108760 0x7f0f38111520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f2ffff700 1 -- 172.21.15.46:0/431987159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f381034e0 msgr2=0x7f0f38110aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:01.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f381034e0 0x7f0f38110aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f2ffff700 1 -- 172.21.15.46:0/431987159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f381a91b0 con 0x7f0f38107e10 2024-05-09T00:21:01.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f3ce06700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f38108760 0x7f0f38111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:01.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.794+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f38107e10 0x7f0f38110fe0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000e970 tx=0x7f0f2000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:01.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.795+0000 7f0f35ffb700 1 -- 172.21.15.46:0/431987159 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2000cbd0 con 0x7f0f38107e10 2024-05-09T00:21:01.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.795+0000 7f0f35ffb700 1 -- 172.21.15.46:0/431987159 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f2000cd70 con 0x7f0f38107e10 2024-05-09T00:21:01.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.795+0000 7f0f3e869700 1 -- 172.21.15.46:0/431987159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f381a94a0 con 0x7f0f38107e10 2024-05-09T00:21:01.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.795+0000 7f0f35ffb700 1 -- 172.21.15.46:0/431987159 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20010430 con 0x7f0f38107e10 2024-05-09T00:21:01.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.795+0000 7f0f3e869700 1 -- 172.21.15.46:0/431987159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f381a99e0 con 0x7f0f38107e10 2024-05-09T00:21:01.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.797+0000 7f0f35ffb700 1 -- 172.21.15.46:0/431987159 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0f20004d70 con 0x7f0f38107e10 2024-05-09T00:21:01.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.798+0000 7f0f35ffb700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0f240b2880 0x7f0f240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.798+0000 7f0f37fff700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0f240b2880 0x7f0f240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.798+0000 7f0f35ffb700 1 -- 172.21.15.46:0/431987159 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f0f200e2560 con 0x7f0f38107e10 2024-05-09T00:21:01.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.798+0000 7f0f4d27fb80 1 Processor -- start 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.798+0000 7f0f4d27fb80 1 -- start start 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f37fff700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0f240b2880 0x7f0f240b4d90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0f28003760 tx=0x7f0f2801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f4d27fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd56783240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f4d27fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd566004a0 0x55bd56783780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f4d27fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd5677d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f4d27fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bd566a00a0 con 0x55bd5667ad00 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f4d27fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bd566459a0 con 0x55bd5677d400 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f4d27fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bd5669c060 con 0x55bd566004a0 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd56783240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd56783240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35036/0 (socket says 172.21.15.46:35036) 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f37fff700 1 -- 172.21.15.46:0/155585294 learned_addr learned my addr 172.21.15.46:0/155585294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/155585294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd566004a0 0x55bd56783780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f3ce06700 1 --2- 172.21.15.46:0/155585294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd5677d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f37fff700 1 -- 172.21.15.46:0/155585294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd566004a0 msgr2=0x55bd56783780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f37fff700 1 --2- 172.21.15.46:0/155585294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd566004a0 0x55bd56783780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f37fff700 1 -- 172.21.15.46:0/155585294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 msgr2=0x55bd5677d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f37fff700 1 --2- 172.21.15.46:0/155585294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd5677d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f37fff700 1 -- 172.21.15.46:0/155585294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd56636500 con 0x55bd5667ad00 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.799+0000 7f0f3ce06700 1 --2- 172.21.15.46:0/155585294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd5677d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.800+0000 7f0f37fff700 1 --2- 172.21.15.46:0/155585294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd56783240 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0f280099c0 tx=0x7f0f2801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da20fb27ee14125 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.800+0000 7f0f2e7fc700 1 -- 172.21.15.46:0/155585294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2800b790 con 0x55bd5667ad00 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.800+0000 7f0f2e7fc700 1 -- 172.21.15.46:0/155585294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f2800bde0 con 0x55bd5667ad00 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.800+0000 7f0f2e7fc700 1 -- 172.21.15.46:0/155585294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28022da0 con 0x55bd5667ad00 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.801+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/155585294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 msgr2=0x55bd56783240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.801+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/155585294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd56783240 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0f280099c0 tx=0x7f0f2801e000 comp rx=0 tx=0).stop 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.801+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/155585294 shutdown_connections 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.801+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/155585294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd5677d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.801+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/155585294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd56783240 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.801+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/155585294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd566004a0 0x55bd56783780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.801+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/155585294 >> 172.21.15.46:0/155585294 conn(0x55bd56603f70 msgr2=0x55bd56778220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.801+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/155585294 shutdown_connections 2024-05-09T00:21:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.801+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/155585294 wait complete. 2024-05-09T00:21:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.801+0000 7f0f4d27fb80 1 -- start start 2024-05-09T00:21:01.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.801+0000 7f0f4d27fb80 1 Processor -- start 2024-05-09T00:21:01.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f4d27fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd566004a0 0x55bd56787080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f4d27fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd5667ad00 0x55bd567875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f4d27fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd56787b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f4d27fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bd5677e0a0 con 0x55bd566004a0 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f4d27fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bd5663ab20 con 0x55bd5677d400 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f4d27fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bd5677e220 con 0x55bd5667ad00 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f4d27fb80 10 client.0 _ll_register_callbacks cb 0x55bd565116c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd566004a0 0x55bd56787080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd566004a0 0x55bd56787080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35050/0 (socket says 172.21.15.46:35050) 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f37fff700 1 -- 172.21.15.46:0/1338321388 learned_addr learned my addr 172.21.15.46:0/1338321388 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f3ce06700 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd56787b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd5667ad00 0x55bd567875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f37fff700 1 -- 172.21.15.46:0/1338321388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd5667ad00 msgr2=0x55bd567875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f37fff700 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd5667ad00 0x55bd567875c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f37fff700 1 -- 172.21.15.46:0/1338321388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 msgr2=0x55bd56787b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f37fff700 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd56787b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f37fff700 1 -- 172.21.15.46:0/1338321388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd5651b490 con 0x55bd566004a0 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f3ce06700 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd56787b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd5667ad00 0x55bd567875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f37fff700 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd566004a0 0x55bd56787080 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0f28005850 tx=0x7f0f28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a9bb854c05e83f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.802+0000 7f0f177fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f177fe700 1 -- 172.21.15.46:0/1338321388 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2800b930 con 0x55bd566004a0 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f177fe700 1 -- 172.21.15.46:0/1338321388 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f0f2800bad0 con 0x55bd566004a0 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/1338321388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f2800bde0 con 0x55bd566004a0 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f177fe700 1 -- 172.21.15.46:0/1338321388 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f28022d70 con 0x55bd566004a0 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f4d27fb80 10 client.16986 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/1338321388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd5677e0a0 con 0x55bd566004a0 2024-05-09T00:21:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f177fe700 1 -- 172.21.15.46:0/1338321388 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f2800bde0 con 0x55bd566004a0 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f177fe700 1 -- 172.21.15.46:0/1338321388 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0f2800f460 con 0x55bd566004a0 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f4d27fb80 10 client.16986 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f4d27fb80 10 client.16986 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f4d27fb80 20 client.16986 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55bd56796bd0 0x55bd56798ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f4d27fb80 4 client.16986 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.803+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/1338321388 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd566ada80 con 0x55bd56796bd0 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.804+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55bd56796bd0 0x55bd56798ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.804+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55bd56796bd0 0x55bd56798ff0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e16007df49cfeef server_cookie=ff4ef22032bc76ea in_seq=0 out_seq=0 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.804+0000 7f0f177fe700 10 client.16986 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f177fe700 1 -- 172.21.15.46:0/1338321388 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bd566ada80 con 0x55bd56796bd0 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f177fe700 10 client.16986 handle_command_reply: tid=0 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f4d27fb80 1 client.16986 shutdown 2024-05-09T00:21:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/1338321388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd566004a0 msgr2=0x55bd56787080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd566004a0 0x55bd56787080 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0f28005850 tx=0x7f0f28000bc0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/1338321388 shutdown_connections 2024-05-09T00:21:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55bd56796bd0 0x55bd56798ff0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd56787b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd566004a0 0x55bd56787080 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/1338321388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd5667ad00 0x55bd567875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/1338321388 >> 172.21.15.46:0/1338321388 conn(0x55bd56603f70 msgr2=0x55bd56771ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f177fe700 0 client.16986 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:21:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.805+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/1338321388 shutdown_connections 2024-05-09T00:21:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.806+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/1338321388 wait complete. 2024-05-09T00:21:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.806+0000 7f0f4d27fb80 20 client.16986 trim_cache size 0 max 16384 2024-05-09T00:21:01.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.821+0000 7f0f4d27fb80 1 Processor -- start 2024-05-09T00:21:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.821+0000 7f0f4d27fb80 1 -- start start 2024-05-09T00:21:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.821+0000 7f0f4d27fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd566004a0 0x55bd56778390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.821+0000 7f0f4d27fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd567894d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.821+0000 7f0f4d27fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd5677d400 0x55bd56793100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.821+0000 7f0f4d27fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f381b36f0 con 0x55bd5667ad00 2024-05-09T00:21:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.821+0000 7f0f4d27fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f381b0f80 con 0x55bd566004a0 2024-05-09T00:21:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.821+0000 7f0f4d27fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f381aedb0 con 0x55bd5677d400 2024-05-09T00:21:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.821+0000 7f0f2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd567894d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.821+0000 7f0f2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd567894d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35066/0 (socket says 172.21.15.46:35066) 2024-05-09T00:21:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.822+0000 7f0f2ffff700 1 -- 172.21.15.46:0/673081751 learned_addr learned my addr 172.21.15.46:0/673081751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:01.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.822+0000 7f0f37fff700 1 --2- 172.21.15.46:0/673081751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd566004a0 0x55bd56778390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.822+0000 7f0f3ce06700 1 --2- 172.21.15.46:0/673081751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd5677d400 0x55bd56793100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.822+0000 7f0f2ffff700 1 -- 172.21.15.46:0/673081751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd5677d400 msgr2=0x55bd56793100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.822+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/673081751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd5677d400 0x55bd56793100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.822+0000 7f0f2ffff700 1 -- 172.21.15.46:0/673081751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd566004a0 msgr2=0x55bd56778390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.822+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/673081751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd566004a0 0x55bd56778390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.822+0000 7f0f2ffff700 1 -- 172.21.15.46:0/673081751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd567749d0 con 0x55bd5667ad00 2024-05-09T00:21:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.822+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/673081751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd567894d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0f200e0020 tx=0x7f0f200dfef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2988632eccb6d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.822+0000 7f0f2dffb700 1 -- 172.21.15.46:0/673081751 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f200a9650 con 0x55bd5667ad00 2024-05-09T00:21:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.822+0000 7f0f2dffb700 1 -- 172.21.15.46:0/673081751 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f200aa240 con 0x55bd5667ad00 2024-05-09T00:21:01.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.822+0000 7f0f2dffb700 1 -- 172.21.15.46:0/673081751 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f200a9da0 con 0x55bd5667ad00 2024-05-09T00:21:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.823+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/673081751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 msgr2=0x55bd567894d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.823+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/673081751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd567894d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0f200e0020 tx=0x7f0f200dfef0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.823+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/673081751 shutdown_connections 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.823+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/673081751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd5677d400 0x55bd56793100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.823+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/673081751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd566004a0 0x55bd56778390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.823+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/673081751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd567894d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.823+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/673081751 >> 172.21.15.46:0/673081751 conn(0x55bd56603f70 msgr2=0x55bd56771ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.823+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/673081751 shutdown_connections 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.823+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/673081751 wait complete. 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.823+0000 7f0f4d27fb80 1 -- start start 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f4d27fb80 1 Processor -- start 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f4d27fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd566004a0 0x55bd56799220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f4d27fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd56799780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f4d27fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd567f90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f4d27fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f381af440 con 0x55bd5667ad00 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f4d27fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f381b08b0 con 0x55bd5677d400 2024-05-09T00:21:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f4d27fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f381afb10 con 0x55bd566004a0 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f4d27fb80 10 client.0 _ll_register_callbacks cb 0x55bd565116c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd56799780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd56799780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35074/0 (socket says 172.21.15.46:35074) 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f2ffff700 1 -- 172.21.15.46:0/3232141010 learned_addr learned my addr 172.21.15.46:0/3232141010 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f37fff700 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd566004a0 0x55bd56799220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.824+0000 7f0f3ce06700 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd567f90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f2ffff700 1 -- 172.21.15.46:0/3232141010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd566004a0 msgr2=0x55bd56799220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd566004a0 0x55bd56799220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f2ffff700 1 -- 172.21.15.46:0/3232141010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 msgr2=0x55bd567f90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd567f90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f2ffff700 1 -- 172.21.15.46:0/3232141010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd567f9720 con 0x55bd5667ad00 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f3ce06700 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd567f90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f2ffff700 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd56799780 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0f200a9840 tx=0x7f0f200a9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d240ed6980450527 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f177fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f177fe700 1 -- 172.21.15.46:0/3232141010 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f200a9b40 con 0x55bd5667ad00 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f177fe700 1 -- 172.21.15.46:0/3232141010 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f0f200a6130 con 0x55bd5667ad00 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/3232141010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f200aa240 con 0x55bd5667ad00 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f177fe700 1 -- 172.21.15.46:0/3232141010 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f200adef0 con 0x55bd5667ad00 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f4d27fb80 10 client.16992 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/3232141010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd56800aa0 con 0x55bd5667ad00 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.825+0000 7f0f177fe700 1 -- 172.21.15.46:0/3232141010 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f200aa240 con 0x55bd5667ad00 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.826+0000 7f0f177fe700 1 -- 172.21.15.46:0/3232141010 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0f28003ce0 con 0x55bd5667ad00 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.826+0000 7f0f4d27fb80 10 client.16992 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.826+0000 7f0f4d27fb80 10 client.16992 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.826+0000 7f0f4d27fb80 20 client.16992 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.826+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55bd56801160 0x55bd56803580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.826+0000 7f0f4d27fb80 4 client.16992 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.826+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/3232141010 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd566a7b20 con 0x55bd56801160 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.826+0000 7f0f37fff700 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55bd56801160 0x55bd56803580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.826+0000 7f0f37fff700 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55bd56801160 0x55bd56803580 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c29a57433b53230 server_cookie=ceb05b6e7dc75df1 in_seq=0 out_seq=0 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.826+0000 7f0f177fe700 10 client.16992 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f177fe700 1 -- 172.21.15.46:0/3232141010 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd566a7b20 con 0x55bd56801160 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f177fe700 10 client.16992 handle_command_reply: tid=0 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f4d27fb80 1 client.16992 shutdown 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/3232141010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 msgr2=0x55bd56799780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd56799780 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0f200a9840 tx=0x7f0f200a9a40 comp rx=0 tx=0).stop 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/3232141010 shutdown_connections 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55bd56801160 0x55bd56803580 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd5677d400 0x55bd567f90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd566004a0 0x55bd56799220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f4d27fb80 1 --2- 172.21.15.46:0/3232141010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd5667ad00 0x55bd56799780 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/3232141010 >> 172.21.15.46:0/3232141010 conn(0x55bd56603f70 msgr2=0x55bd56780cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f177fe700 0 client.16992 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:21:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/3232141010 shutdown_connections 2024-05-09T00:21:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f4d27fb80 1 -- 172.21.15.46:0/3232141010 wait complete. 2024-05-09T00:21:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.827+0000 7f0f4d27fb80 20 client.16992 trim_cache size 0 max 16384 2024-05-09T00:21:01.832 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:21:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.828+0000 7f0f3e869700 1 -- 172.21.15.46:0/431987159 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0f240b2880 msgr2=0x7f0f240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.828+0000 7f0f3e869700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0f240b2880 0x7f0f240b4d90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0f28003760 tx=0x7f0f2801a040 comp rx=0 tx=0).stop 2024-05-09T00:21:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.828+0000 7f0f3e869700 1 -- 172.21.15.46:0/431987159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f38107e10 msgr2=0x7f0f38110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.828+0000 7f0f3e869700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f38107e10 0x7f0f38110fe0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000e970 tx=0x7f0f2000ed30 comp rx=0 tx=0).stop 2024-05-09T00:21:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.828+0000 7f0f3e869700 1 -- 172.21.15.46:0/431987159 shutdown_connections 2024-05-09T00:21:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.828+0000 7f0f3e869700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0f240b2880 0x7f0f240b4d90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.828+0000 7f0f3e869700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f38108760 0x7f0f38111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.828+0000 7f0f3e869700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f381034e0 0x7f0f38110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.828+0000 7f0f3e869700 1 --2- 172.21.15.46:0/431987159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f38107e10 0x7f0f38110fe0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.828+0000 7f0f3e869700 1 -- 172.21.15.46:0/431987159 >> 172.21.15.46:0/431987159 conn(0x7f0f380fef30 msgr2=0x7f0f381071b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.828+0000 7f0f3e869700 1 -- 172.21.15.46:0/431987159 shutdown_connections 2024-05-09T00:21:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:01.828+0000 7f0f3e869700 1 -- 172.21.15.46:0/431987159 wait complete. 2024-05-09T00:21:02.546 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:21:02.796 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:21:02.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:02 smithi046 ceph-mon[30151]: pgmap v666: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:21:02.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:02 smithi169 ceph-mon[34777]: pgmap v666: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:21:02.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:02 smithi142 ceph-mon[34533]: pgmap v666: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:21:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.519+0000 7f9fee346700 1 -- 172.21.15.46:0/49260663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe81092d0 msgr2=0x7f9fe81096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.519+0000 7f9fee346700 1 --2- 172.21.15.46:0/49260663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe81092d0 0x7f9fe81096e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009990 tx=0x7f9fd8009ca0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.519+0000 7f9fee346700 1 -- 172.21.15.46:0/49260663 shutdown_connections 2024-05-09T00:21:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.519+0000 7f9fee346700 1 --2- 172.21.15.46:0/49260663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fe8104330 0x7f9fe8110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.519+0000 7f9fee346700 1 --2- 172.21.15.46:0/49260663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe81092d0 0x7f9fe81096e0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.519+0000 7f9fee346700 1 --2- 172.21.15.46:0/49260663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe81039e0 0x7f9fe8103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.519+0000 7f9fee346700 1 -- 172.21.15.46:0/49260663 >> 172.21.15.46:0/49260663 conn(0x7f9fe80fe4e0 msgr2=0x7f9fe8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.519+0000 7f9fee346700 1 -- 172.21.15.46:0/49260663 shutdown_connections 2024-05-09T00:21:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.519+0000 7f9fee346700 1 -- 172.21.15.46:0/49260663 wait complete. 2024-05-09T00:21:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fee346700 1 Processor -- start 2024-05-09T00:21:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fee346700 1 -- start start 2024-05-09T00:21:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fee346700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe81039e0 0x7f9fe819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fee346700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8104330 0x7f9fe819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fee346700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fe81092d0 0x7f9fe81a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fee346700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8113370 con 0x7f9fe8104330 2024-05-09T00:21:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fee346700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe81131f0 con 0x7f9fe81092d0 2024-05-09T00:21:04.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fee346700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe81134f0 con 0x7f9fe81039e0 2024-05-09T00:21:04.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fdffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8104330 0x7f9fe819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fec8e3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fe81092d0 0x7f9fe81a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fec8e3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fe81092d0 0x7f9fe81a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:36542/0 (socket says 172.21.15.46:36542) 2024-05-09T00:21:04.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fdffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8104330 0x7f9fe819d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35096/0 (socket says 172.21.15.46:35096) 2024-05-09T00:21:04.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fec8e3700 1 -- 172.21.15.46:0/3348316650 learned_addr learned my addr 172.21.15.46:0/3348316650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:04.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.520+0000 7f9fe7fff700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe81039e0 0x7f9fe819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.521+0000 7f9fec8e3700 1 -- 172.21.15.46:0/3348316650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe81039e0 msgr2=0x7f9fe819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:04.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.521+0000 7f9fec8e3700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe81039e0 0x7f9fe819d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.521+0000 7f9fec8e3700 1 -- 172.21.15.46:0/3348316650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8104330 msgr2=0x7f9fe819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:04.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.521+0000 7f9fec8e3700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8104330 0x7f9fe819d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.521+0000 7f9fec8e3700 1 -- 172.21.15.46:0/3348316650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe81a1ed0 con 0x7f9fe81092d0 2024-05-09T00:21:04.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.521+0000 7f9fdffff700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8104330 0x7f9fe819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:21:04.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.521+0000 7f9fec8e3700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fe81092d0 0x7f9fe81a1830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000ef30 tx=0x7f9fe000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:04.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.521+0000 7f9fe5ffb700 1 -- 172.21.15.46:0/3348316650 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00092e0 con 0x7f9fe81092d0 2024-05-09T00:21:04.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.521+0000 7f9fee346700 1 -- 172.21.15.46:0/3348316650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe81a2160 con 0x7f9fe81092d0 2024-05-09T00:21:04.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.521+0000 7f9fe5ffb700 1 -- 172.21.15.46:0/3348316650 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe0004bb0 con 0x7f9fe81092d0 2024-05-09T00:21:04.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.521+0000 7f9fe5ffb700 1 -- 172.21.15.46:0/3348316650 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0004020 con 0x7f9fe81092d0 2024-05-09T00:21:04.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.521+0000 7f9fee346700 1 -- 172.21.15.46:0/3348316650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe81a9ae0 con 0x7f9fe81092d0 2024-05-09T00:21:04.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.524+0000 7f9fe5ffb700 1 -- 172.21.15.46:0/3348316650 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9fe001b030 con 0x7f9fe81092d0 2024-05-09T00:21:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.525+0000 7f9fe5ffb700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9fd40b2880 0x7f9fd40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.525+0000 7f9fe7fff700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9fd40b2880 0x7f9fd40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.525+0000 7f9fe5ffb700 1 -- 172.21.15.46:0/3348316650 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9fe00e1cf0 con 0x7f9fe81092d0 2024-05-09T00:21:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.525+0000 7f9ffcd5cb80 1 Processor -- start 2024-05-09T00:21:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.525+0000 7f9ffcd5cb80 1 -- start start 2024-05-09T00:21:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.525+0000 7f9fe7fff700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9fd40b2880 0x7f9fd40b4d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800b2e0 tx=0x7f9fd8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.525+0000 7f9ffcd5cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 0x563fc083cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.525+0000 7f9ffcd5cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc06b14a0 0x563fc083d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.525+0000 7f9ffcd5cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 0x563fc08372c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.525+0000 7f9ffcd5cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563fc0750ff0 con 0x563fc0836eb0 2024-05-09T00:21:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.525+0000 7f9ffcd5cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563fc06ef7f0 con 0x563fc079b160 2024-05-09T00:21:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.525+0000 7f9ffcd5cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563fc074cfb0 con 0x563fc06b14a0 2024-05-09T00:21:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.526+0000 7f9fe7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 0x563fc083cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.526+0000 7f9fec8e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 0x563fc08372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.526+0000 7f9fe7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 0x563fc083cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:36546/0 (socket says 172.21.15.46:36546) 2024-05-09T00:21:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.526+0000 7f9fe7fff700 1 -- 172.21.15.46:0/4099520565 learned_addr learned my addr 172.21.15.46:0/4099520565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.526+0000 7f9fdffff700 1 --2- 172.21.15.46:0/4099520565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc06b14a0 0x563fc083d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.526+0000 7f9fec8e3700 1 -- 172.21.15.46:0/4099520565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc06b14a0 msgr2=0x563fc083d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:04.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.526+0000 7f9fec8e3700 1 --2- 172.21.15.46:0/4099520565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc06b14a0 0x563fc083d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.526+0000 7f9fec8e3700 1 -- 172.21.15.46:0/4099520565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 msgr2=0x563fc083cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:04.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.526+0000 7f9fec8e3700 1 --2- 172.21.15.46:0/4099520565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 0x563fc083cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.526+0000 7f9fec8e3700 1 -- 172.21.15.46:0/4099520565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc06e7370 con 0x563fc0836eb0 2024-05-09T00:21:04.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.526+0000 7f9fe7fff700 1 --2- 172.21.15.46:0/4099520565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 0x563fc083cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:21:04.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.526+0000 7f9fec8e3700 1 --2- 172.21.15.46:0/4099520565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 0x563fc08372c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00df7b0 tx=0x7f9fe00a8dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17804b6d71af2fe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:04.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.528+0000 7f9fde7fc700 1 -- 172.21.15.46:0/4099520565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00a9360 con 0x563fc0836eb0 2024-05-09T00:21:04.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.528+0000 7f9fde7fc700 1 -- 172.21.15.46:0/4099520565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe00ad710 con 0x563fc0836eb0 2024-05-09T00:21:04.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.528+0000 7f9fde7fc700 1 -- 172.21.15.46:0/4099520565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00a4f30 con 0x563fc0836eb0 2024-05-09T00:21:04.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.529+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/4099520565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 msgr2=0x563fc08372c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:04.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.529+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/4099520565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 0x563fc08372c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00df7b0 tx=0x7f9fe00a8dc0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.529+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/4099520565 shutdown_connections 2024-05-09T00:21:04.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.529+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/4099520565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 0x563fc08372c0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.529+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/4099520565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 0x563fc083cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.529+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/4099520565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc06b14a0 0x563fc083d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.529+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/4099520565 >> 172.21.15.46:0/4099520565 conn(0x563fc07116a0 msgr2=0x563fc0829400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.529+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/4099520565 shutdown_connections 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.529+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/4099520565 wait complete. 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.529+0000 7f9ffcd5cb80 1 -- start start 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9ffcd5cb80 1 Processor -- start 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9ffcd5cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc06b14a0 0x563fc0840ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9ffcd5cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc079b160 0x563fc0840ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9ffcd5cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc0836eb0 0x563fc0841530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9ffcd5cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563fc05ccba0 con 0x563fc079b160 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9ffcd5cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563fc070b150 con 0x563fc06b14a0 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9ffcd5cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563fc0837cc0 con 0x563fc0836eb0 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9ffcd5cb80 10 client.0 _ll_register_callbacks cb 0x563fc05c26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9fdffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc079b160 0x563fc0840ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9fdffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc079b160 0x563fc0840ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35100/0 (socket says 172.21.15.46:35100) 2024-05-09T00:21:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9fe7fff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc06b14a0 0x563fc0840ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9fdffff700 1 -- 172.21.15.46:0/2418769993 learned_addr learned my addr 172.21.15.46:0/2418769993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.530+0000 7f9fec8e3700 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc0836eb0 0x563fc0841530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9fe7fff700 1 -- 172.21.15.46:0/2418769993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc0836eb0 msgr2=0x563fc0841530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9fe7fff700 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc0836eb0 0x563fc0841530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9fe7fff700 1 -- 172.21.15.46:0/2418769993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc079b160 msgr2=0x563fc0840ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9fe7fff700 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc079b160 0x563fc0840ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9fe7fff700 1 -- 172.21.15.46:0/2418769993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc05cc490 con 0x563fc06b14a0 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9fdffff700 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc079b160 0x563fc0840ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9fe7fff700 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc06b14a0 0x563fc0840ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009720 tx=0x7f9fd801a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ac8f0065105aa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9fc77fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9fc77fe700 1 -- 172.21.15.46:0/2418769993 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd800f460 con 0x563fc06b14a0 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2418769993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fe00ad710 con 0x563fc06b14a0 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9fc77fe700 1 -- 172.21.15.46:0/2418769993 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9fd800f600 con 0x563fc06b14a0 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9fc77fe700 1 -- 172.21.15.46:0/2418769993 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9fd800f800 con 0x563fc06b14a0 2024-05-09T00:21:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9ffcd5cb80 10 client.26423 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.531+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2418769993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563fc084e390 con 0x563fc06b14a0 2024-05-09T00:21:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.532+0000 7f9fc77fe700 1 -- 172.21.15.46:0/2418769993 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe00ad710 con 0x563fc06b14a0 2024-05-09T00:21:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.532+0000 7f9fc77fe700 1 -- 172.21.15.46:0/2418769993 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9fd8038030 con 0x563fc06b14a0 2024-05-09T00:21:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.532+0000 7f9ffcd5cb80 10 client.26423 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.532+0000 7f9ffcd5cb80 10 client.26423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:21:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.532+0000 7f9ffcd5cb80 20 client.26423 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.532+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563fc08509d0 0x563fc0852e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.532+0000 7f9ffcd5cb80 4 client.26423 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:21:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.532+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2418769993 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fc075e9d0 con 0x563fc08509d0 2024-05-09T00:21:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.532+0000 7f9fdffff700 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563fc08509d0 0x563fc0852e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.533+0000 7f9fdffff700 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563fc08509d0 0x563fc0852e30 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34ca5a0ece946ec9 server_cookie=bcde56d05eb498b1 in_seq=0 out_seq=0 2024-05-09T00:21:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.533+0000 7f9fc77fe700 10 client.26423 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.533+0000 7f9fc77fe700 1 -- 172.21.15.46:0/2418769993 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563fc075e9d0 con 0x563fc08509d0 2024-05-09T00:21:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.533+0000 7f9fc77fe700 10 client.26423 handle_command_reply: tid=0 2024-05-09T00:21:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.533+0000 7f9ffcd5cb80 1 client.26423 shutdown 2024-05-09T00:21:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.534+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2418769993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc06b14a0 msgr2=0x563fc0840ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.534+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc06b14a0 0x563fc0840ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009720 tx=0x7f9fd801a050 comp rx=0 tx=0).stop 2024-05-09T00:21:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.534+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2418769993 shutdown_connections 2024-05-09T00:21:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.534+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563fc08509d0 0x563fc0852e30 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.534+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc0836eb0 0x563fc0841530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.534+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc06b14a0 0x563fc0840ab0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.534+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/2418769993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc079b160 0x563fc0840ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.534+0000 7f9fc77fe700 0 client.26423 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.534+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2418769993 >> 172.21.15.46:0/2418769993 conn(0x563fc07116a0 msgr2=0x563fc08291c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.534+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2418769993 shutdown_connections 2024-05-09T00:21:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.534+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2418769993 wait complete. 2024-05-09T00:21:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.534+0000 7f9ffcd5cb80 20 client.26423 trim_cache size 0 max 16384 2024-05-09T00:21:04.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.549+0000 7f9ffcd5cb80 1 Processor -- start 2024-05-09T00:21:04.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.549+0000 7f9ffcd5cb80 1 -- start start 2024-05-09T00:21:04.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.549+0000 7f9ffcd5cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc06b14a0 0x563fc0842f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.549+0000 7f9ffcd5cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 0x563fc082e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.549+0000 7f9ffcd5cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 0x563fc083c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.549+0000 7f9ffcd5cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe81af2e0 con 0x563fc0836eb0 2024-05-09T00:21:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.549+0000 7f9ffcd5cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe81acb70 con 0x563fc079b160 2024-05-09T00:21:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.549+0000 7f9ffcd5cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe810dea0 con 0x563fc06b14a0 2024-05-09T00:21:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.549+0000 7f9fec8e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 0x563fc083c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.549+0000 7f9fdffff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 0x563fc082e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.549+0000 7f9fec8e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 0x563fc083c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35114/0 (socket says 172.21.15.46:35114) 2024-05-09T00:21:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.549+0000 7f9fec8e3700 1 -- 172.21.15.46:0/3408498962 learned_addr learned my addr 172.21.15.46:0/3408498962 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.550+0000 7f9fe7fff700 1 --2- 172.21.15.46:0/3408498962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc06b14a0 0x563fc0842f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.550+0000 7f9fdffff700 1 -- 172.21.15.46:0/3408498962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc06b14a0 msgr2=0x563fc0842f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.550+0000 7f9fdffff700 1 --2- 172.21.15.46:0/3408498962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc06b14a0 0x563fc0842f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.550+0000 7f9fdffff700 1 -- 172.21.15.46:0/3408498962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 msgr2=0x563fc083c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.550+0000 7f9fdffff700 1 --2- 172.21.15.46:0/3408498962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 0x563fc083c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.550+0000 7f9fdffff700 1 -- 172.21.15.46:0/3408498962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc083f020 con 0x563fc079b160 2024-05-09T00:21:04.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.550+0000 7f9fec8e3700 1 --2- 172.21.15.46:0/3408498962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 0x563fc083c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:21:04.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.550+0000 7f9fdffff700 1 --2- 172.21.15.46:0/3408498962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 0x563fc082e0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x563fc0853560 tx=0x7f9fd000b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76b713e7801cf96f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:04.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.550+0000 7f9fddffb700 1 -- 172.21.15.46:0/3408498962 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0004220 con 0x563fc079b160 2024-05-09T00:21:04.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.550+0000 7f9fddffb700 1 -- 172.21.15.46:0/3408498962 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fd0024030 con 0x563fc079b160 2024-05-09T00:21:04.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.550+0000 7f9fddffb700 1 -- 172.21.15.46:0/3408498962 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0005520 con 0x563fc079b160 2024-05-09T00:21:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.551+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/3408498962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 msgr2=0x563fc082e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.551+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/3408498962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 0x563fc082e0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x563fc0853560 tx=0x7f9fd000b410 comp rx=0 tx=0).stop 2024-05-09T00:21:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.551+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/3408498962 shutdown_connections 2024-05-09T00:21:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.551+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/3408498962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc0836eb0 0x563fc083c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.551+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/3408498962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc06b14a0 0x563fc0842f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.551+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/3408498962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc079b160 0x563fc082e0c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.551+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/3408498962 >> 172.21.15.46:0/3408498962 conn(0x563fc07116a0 msgr2=0x563fc083f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.551+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/3408498962 shutdown_connections 2024-05-09T00:21:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.551+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/3408498962 wait complete. 2024-05-09T00:21:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.551+0000 7f9ffcd5cb80 1 -- start start 2024-05-09T00:21:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9ffcd5cb80 1 Processor -- start 2024-05-09T00:21:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9ffcd5cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc06b14a0 0x563fc0848b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9ffcd5cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc079b160 0x563fc0849080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9ffcd5cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc0836eb0 0x563fc08575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9ffcd5cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe81ab030 con 0x563fc06b14a0 2024-05-09T00:21:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9ffcd5cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe81ac4a0 con 0x563fc0836eb0 2024-05-09T00:21:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9ffcd5cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe81ab700 con 0x563fc079b160 2024-05-09T00:21:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9ffcd5cb80 10 client.0 _ll_register_callbacks cb 0x563fc05c26c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:04.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9fe7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc06b14a0 0x563fc0848b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9fe7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc06b14a0 0x563fc0848b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35122/0 (socket says 172.21.15.46:35122) 2024-05-09T00:21:04.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9fe7fff700 1 -- 172.21.15.46:0/2253497369 learned_addr learned my addr 172.21.15.46:0/2253497369 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:04.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9fec8e3700 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc0836eb0 0x563fc08575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.552+0000 7f9fdffff700 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc079b160 0x563fc0849080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9fec8e3700 1 -- 172.21.15.46:0/2253497369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc079b160 msgr2=0x563fc0849080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:04.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9fec8e3700 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc079b160 0x563fc0849080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9fec8e3700 1 -- 172.21.15.46:0/2253497369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc06b14a0 msgr2=0x563fc0848b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:04.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9fec8e3700 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc06b14a0 0x563fc0848b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9fec8e3700 1 -- 172.21.15.46:0/2253497369 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc0858ea0 con 0x563fc0836eb0 2024-05-09T00:21:04.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9fe7fff700 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc06b14a0 0x563fc0848b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:21:04.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9fec8e3700 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc0836eb0 0x563fc08575e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00a9060 tx=0x7f9fe0004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b01b0118b9833b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:04.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9fc77fe700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:21:04.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9fc77fe700 1 -- 172.21.15.46:0/2253497369 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00c3060 con 0x563fc0836eb0 2024-05-09T00:21:04.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9fc77fe700 1 -- 172.21.15.46:0/2253497369 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9fe00a87d0 con 0x563fc0836eb0 2024-05-09T00:21:04.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2253497369 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fd0024030 con 0x563fc0836eb0 2024-05-09T00:21:04.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9fc77fe700 1 -- 172.21.15.46:0/2253497369 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9fe00a9360 con 0x563fc0836eb0 2024-05-09T00:21:04.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9ffcd5cb80 10 client.26429 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:04.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2253497369 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563fc08ba580 con 0x563fc0836eb0 2024-05-09T00:21:04.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.553+0000 7f9fc77fe700 1 -- 172.21.15.46:0/2253497369 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fd0024030 con 0x563fc0836eb0 2024-05-09T00:21:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.554+0000 7f9fc77fe700 1 -- 172.21.15.46:0/2253497369 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9fe00a8030 con 0x563fc0836eb0 2024-05-09T00:21:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.554+0000 7f9ffcd5cb80 10 client.26429 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.554+0000 7f9ffcd5cb80 10 client.26429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:21:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.554+0000 7f9ffcd5cb80 20 client.26429 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.554+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563fc08ba8c0 0x563fc08bcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.554+0000 7f9ffcd5cb80 4 client.26429 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:21:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.554+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2253497369 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fc0758a70 con 0x563fc08ba8c0 2024-05-09T00:21:04.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.554+0000 7f9fe7fff700 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563fc08ba8c0 0x563fc08bcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:04.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.554+0000 7f9fc77fe700 10 client.26429 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:04.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.554+0000 7f9fe7fff700 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563fc08ba8c0 0x563fc08bcce0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38de7042c3bf7a00 server_cookie=8b6ba62763269879 in_seq=0 out_seq=0 2024-05-09T00:21:04.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9fc77fe700 1 -- 172.21.15.46:0/2253497369 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fc0758a70 con 0x563fc08ba8c0 2024-05-09T00:21:04.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9fc77fe700 10 client.26429 handle_command_reply: tid=0 2024-05-09T00:21:04.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9ffcd5cb80 1 client.26429 shutdown 2024-05-09T00:21:04.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2253497369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc0836eb0 msgr2=0x563fc08575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:04.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc0836eb0 0x563fc08575e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00a9060 tx=0x7f9fe0004f90 comp rx=0 tx=0).stop 2024-05-09T00:21:04.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2253497369 shutdown_connections 2024-05-09T00:21:04.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563fc08ba8c0 0x563fc08bcce0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fc0836eb0 0x563fc08575e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc06b14a0 0x563fc0848b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9ffcd5cb80 1 --2- 172.21.15.46:0/2253497369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563fc079b160 0x563fc0849080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9fc77fe700 0 client.26429 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2253497369 >> 172.21.15.46:0/2253497369 conn(0x563fc07116a0 msgr2=0x563fc082de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2253497369 shutdown_connections 2024-05-09T00:21:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9ffcd5cb80 1 -- 172.21.15.46:0/2253497369 wait complete. 2024-05-09T00:21:04.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9ffcd5cb80 20 client.26429 trim_cache size 0 max 16384 2024-05-09T00:21:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:21:04.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9fee346700 1 -- 172.21.15.46:0/3348316650 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9fd40b2880 msgr2=0x7f9fd40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:04.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9fee346700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9fd40b2880 0x7f9fd40b4d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800b2e0 tx=0x7f9fd8005960 comp rx=0 tx=0).stop 2024-05-09T00:21:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9fee346700 1 -- 172.21.15.46:0/3348316650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fe81092d0 msgr2=0x7f9fe81a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.555+0000 7f9fee346700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fe81092d0 0x7f9fe81a1830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000ef30 tx=0x7f9fe000c3a0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.556+0000 7f9fee346700 1 -- 172.21.15.46:0/3348316650 shutdown_connections 2024-05-09T00:21:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.556+0000 7f9fee346700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9fd40b2880 0x7f9fd40b4d90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.556+0000 7f9fee346700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fe81092d0 0x7f9fe81a1830 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.556+0000 7f9fee346700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fe81039e0 0x7f9fe819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.556+0000 7f9fee346700 1 --2- 172.21.15.46:0/3348316650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe8104330 0x7f9fe819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.556+0000 7f9fee346700 1 -- 172.21.15.46:0/3348316650 >> 172.21.15.46:0/3348316650 conn(0x7f9fe80fe4e0 msgr2=0x7f9fe81028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.556+0000 7f9fee346700 1 -- 172.21.15.46:0/3348316650 shutdown_connections 2024-05-09T00:21:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:04.556+0000 7f9fee346700 1 -- 172.21.15.46:0/3348316650 wait complete. 2024-05-09T00:21:04.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:04 smithi046 ceph-mon[30151]: pgmap v667: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:04.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:04 smithi169 ceph-mon[34777]: pgmap v667: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:04.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:04 smithi142 ceph-mon[34533]: pgmap v667: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:06.259 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:21:06.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:21:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:21:06.562 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:06 smithi046 ceph-mon[30151]: pgmap v668: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:21:06.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:06 smithi169 ceph-mon[34777]: pgmap v668: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:21:06.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:06 smithi142 ceph-mon[34533]: pgmap v668: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:21:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.335+0000 7ffa0241d700 1 -- 172.21.15.46:0/1570353574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9fc1034e0 msgr2=0x7ff9fc1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.335+0000 7ffa0241d700 1 --2- 172.21.15.46:0/1570353574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9fc1034e0 0x7ff9fc1038f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0099c0 tx=0x7ff9ec009cd0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.335+0000 7ffa0241d700 1 -- 172.21.15.46:0/1570353574 shutdown_connections 2024-05-09T00:21:08.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.335+0000 7ffa0241d700 1 --2- 172.21.15.46:0/1570353574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9fc108760 0x7ff9fc1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.335+0000 7ffa0241d700 1 --2- 172.21.15.46:0/1570353574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9fc1034e0 0x7ff9fc1038f0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.335+0000 7ffa0241d700 1 --2- 172.21.15.46:0/1570353574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc107e10 0x7ff9fc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.335+0000 7ffa0241d700 1 -- 172.21.15.46:0/1570353574 >> 172.21.15.46:0/1570353574 conn(0x7ff9fc0fef30 msgr2=0x7ff9fc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.335+0000 7ffa0241d700 1 -- 172.21.15.46:0/1570353574 shutdown_connections 2024-05-09T00:21:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.335+0000 7ffa0241d700 1 -- 172.21.15.46:0/1570353574 wait complete. 2024-05-09T00:21:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.336+0000 7ffa0241d700 1 Processor -- start 2024-05-09T00:21:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.336+0000 7ffa0241d700 1 -- start start 2024-05-09T00:21:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.336+0000 7ffa0241d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9fc1034e0 0x7ff9fc19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.336+0000 7ffa0241d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc107e10 0x7ff9fc19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.336+0000 7ffa0241d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9fc108760 0x7ff9fc1a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa0241d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc113390 con 0x7ff9fc108760 2024-05-09T00:21:08.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa0241d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc113210 con 0x7ff9fc107e10 2024-05-09T00:21:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa0241d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc113510 con 0x7ff9fc1034e0 2024-05-09T00:21:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa009ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9fc108760 0x7ff9fc1a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa009ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9fc108760 0x7ff9fc1a6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33656/0 (socket says 172.21.15.46:33656) 2024-05-09T00:21:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa009ba700 1 -- 172.21.15.46:0/900918278 learned_addr learned my addr 172.21.15.46:0/900918278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ff9f3fff700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc107e10 0x7ff9fc19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ff9fbfff700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9fc1034e0 0x7ff9fc19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa009ba700 1 -- 172.21.15.46:0/900918278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9fc1034e0 msgr2=0x7ff9fc19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa009ba700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9fc1034e0 0x7ff9fc19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa009ba700 1 -- 172.21.15.46:0/900918278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc107e10 msgr2=0x7ff9fc19c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa009ba700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc107e10 0x7ff9fc19c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa009ba700 1 -- 172.21.15.46:0/900918278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9fc19c9b0 con 0x7ff9fc108760 2024-05-09T00:21:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ff9fbfff700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9fc1034e0 0x7ff9fc19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa009ba700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9fc108760 0x7ff9fc1a6cd0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4007ee0 tx=0x7ff9f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ff9f9ffb700 1 -- 172.21.15.46:0/900918278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400dbd0 con 0x7ff9fc108760 2024-05-09T00:21:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ff9f9ffb700 1 -- 172.21.15.46:0/900918278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f400dd70 con 0x7ff9fc108760 2024-05-09T00:21:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.337+0000 7ffa0241d700 1 -- 172.21.15.46:0/900918278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9fc1a7270 con 0x7ff9fc108760 2024-05-09T00:21:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.338+0000 7ff9f9ffb700 1 -- 172.21.15.46:0/900918278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4005990 con 0x7ff9fc108760 2024-05-09T00:21:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.338+0000 7ffa0241d700 1 -- 172.21.15.46:0/900918278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9fc1a77b0 con 0x7ff9fc108760 2024-05-09T00:21:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.341+0000 7ff9f9ffb700 1 -- 172.21.15.46:0/900918278 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff9f401a020 con 0x7ff9fc108760 2024-05-09T00:21:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.342+0000 7ff9f9ffb700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9e80b2900 0x7ff9e80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.342+0000 7ff9f9ffb700 1 -- 172.21.15.46:0/900918278 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff9f40e1480 con 0x7ff9fc108760 2024-05-09T00:21:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.342+0000 7ff9fbfff700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9e80b2900 0x7ff9e80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ffa10e33b80 1 Processor -- start 2024-05-09T00:21:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ff9fbfff700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9e80b2900 0x7ff9e80b4e10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec000c00 tx=0x7ff9ec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ffa10e33b80 1 -- start start 2024-05-09T00:21:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ffa10e33b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae66bf1e0 0x563ae6758570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ffa10e33b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae65d5520 0x563ae6758ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ffa10e33b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae6752720 0x563ae6752b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ffa10e33b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563ae6675070 con 0x563ae65d5520 2024-05-09T00:21:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ffa10e33b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ae6613870 con 0x563ae66bf1e0 2024-05-09T00:21:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ffa10e33b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563ae6671030 con 0x563ae6752720 2024-05-09T00:21:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae65d5520 0x563ae6758ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae65d5520 0x563ae6758ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33662/0 (socket says 172.21.15.46:33662) 2024-05-09T00:21:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ff9f3fff700 1 -- 172.21.15.46:0/1901193985 learned_addr learned my addr 172.21.15.46:0/1901193985 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ff9fbfff700 1 --2- 172.21.15.46:0/1901193985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae66bf1e0 0x563ae6758570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ffa009ba700 1 --2- 172.21.15.46:0/1901193985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae6752720 0x563ae6752b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ff9f3fff700 1 -- 172.21.15.46:0/1901193985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae6752720 msgr2=0x563ae6752b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ff9f3fff700 1 --2- 172.21.15.46:0/1901193985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae6752720 0x563ae6752b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ff9f3fff700 1 -- 172.21.15.46:0/1901193985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae66bf1e0 msgr2=0x563ae6758570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.343+0000 7ff9f3fff700 1 --2- 172.21.15.46:0/1901193985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae66bf1e0 0x563ae6758570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.344+0000 7ff9f3fff700 1 -- 172.21.15.46:0/1901193985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae660b3f0 con 0x563ae65d5520 2024-05-09T00:21:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.344+0000 7ff9fbfff700 1 --2- 172.21.15.46:0/1901193985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae66bf1e0 0x563ae6758570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.344+0000 7ff9f3fff700 1 --2- 172.21.15.46:0/1901193985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae65d5520 0x563ae6758ab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400ace0 tx=0x7ff9e400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e72c73b69447c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.344+0000 7ff9f27fc700 1 -- 172.21.15.46:0/1901193985 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e400fe50 con 0x563ae65d5520 2024-05-09T00:21:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.344+0000 7ff9f27fc700 1 -- 172.21.15.46:0/1901193985 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e40044d0 con 0x563ae65d5520 2024-05-09T00:21:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.344+0000 7ff9f27fc700 1 -- 172.21.15.46:0/1901193985 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e40186a0 con 0x563ae65d5520 2024-05-09T00:21:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.345+0000 7ffa10e33b80 1 -- 172.21.15.46:0/1901193985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae65d5520 msgr2=0x563ae6758ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.345+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/1901193985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae65d5520 0x563ae6758ab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400ace0 tx=0x7ff9e400f3c0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.345+0000 7ffa10e33b80 1 -- 172.21.15.46:0/1901193985 shutdown_connections 2024-05-09T00:21:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.345+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/1901193985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae6752720 0x563ae6752b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.345+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/1901193985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae66bf1e0 0x563ae6758570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.345+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/1901193985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae65d5520 0x563ae6758ab0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.345+0000 7ffa10e33b80 1 -- 172.21.15.46:0/1901193985 >> 172.21.15.46:0/1901193985 conn(0x563ae6635720 msgr2=0x563ae674d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:08.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.345+0000 7ffa10e33b80 1 -- 172.21.15.46:0/1901193985 shutdown_connections 2024-05-09T00:21:08.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.345+0000 7ffa10e33b80 1 -- 172.21.15.46:0/1901193985 wait complete. 2024-05-09T00:21:08.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.345+0000 7ffa10e33b80 1 -- start start 2024-05-09T00:21:08.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa10e33b80 1 Processor -- start 2024-05-09T00:21:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa10e33b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae65d5520 0x563ae675c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa10e33b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae66bf1e0 0x563ae675c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa10e33b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae6752720 0x563ae675cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa10e33b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563ae64f0c20 con 0x563ae6752720 2024-05-09T00:21:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa10e33b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ae662f1d0 con 0x563ae66bf1e0 2024-05-09T00:21:08.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa10e33b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563ae675f250 con 0x563ae65d5520 2024-05-09T00:21:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa10e33b80 10 client.0 _ll_register_callbacks cb 0x563ae64e6740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa009ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae6752720 0x563ae675cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa009ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae6752720 0x563ae675cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33666/0 (socket says 172.21.15.46:33666) 2024-05-09T00:21:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa009ba700 1 -- 172.21.15.46:0/3797427514 learned_addr learned my addr 172.21.15.46:0/3797427514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ff9fbfff700 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae65d5520 0x563ae675c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ff9f3fff700 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae66bf1e0 0x563ae675c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa009ba700 1 -- 172.21.15.46:0/3797427514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae65d5520 msgr2=0x563ae675c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:08.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa009ba700 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae65d5520 0x563ae675c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa009ba700 1 -- 172.21.15.46:0/3797427514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae66bf1e0 msgr2=0x563ae675c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:08.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.346+0000 7ffa009ba700 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae66bf1e0 0x563ae675c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.347+0000 7ffa009ba700 1 -- 172.21.15.46:0/3797427514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae64f0510 con 0x563ae6752720 2024-05-09T00:21:08.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.347+0000 7ffa009ba700 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae6752720 0x563ae675cd10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40a89f0 tx=0x7ff9f40a41f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0914b0ea39f0f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.347+0000 7ff9db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.347+0000 7ff9db7fe700 1 -- 172.21.15.46:0/3797427514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40a8570 con 0x563ae6752720 2024-05-09T00:21:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.347+0000 7ff9db7fe700 1 -- 172.21.15.46:0/3797427514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff9f40a8cc0 con 0x563ae6752720 2024-05-09T00:21:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.347+0000 7ffa10e33b80 1 -- 172.21.15.46:0/3797427514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e40044d0 con 0x563ae6752720 2024-05-09T00:21:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.347+0000 7ff9db7fe700 1 -- 172.21.15.46:0/3797427514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9f40a9160 con 0x563ae6752720 2024-05-09T00:21:08.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.347+0000 7ffa10e33b80 10 client.17022 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.347+0000 7ffa10e33b80 1 -- 172.21.15.46:0/3797427514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ae6769d10 con 0x563ae6752720 2024-05-09T00:21:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.347+0000 7ff9db7fe700 1 -- 172.21.15.46:0/3797427514 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e40044d0 con 0x563ae6752720 2024-05-09T00:21:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.347+0000 7ff9db7fe700 1 -- 172.21.15.46:0/3797427514 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff9f40b1430 con 0x563ae6752720 2024-05-09T00:21:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.348+0000 7ffa10e33b80 10 client.17022 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.348+0000 7ffa10e33b80 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:21:08.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.348+0000 7ffa10e33b80 20 client.17022 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.348+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ae676c350 0x563ae676e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.348+0000 7ffa10e33b80 4 client.17022 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:21:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.348+0000 7ffa10e33b80 1 -- 172.21.15.46:0/3797427514 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ae6682a50 con 0x563ae676c350 2024-05-09T00:21:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.348+0000 7ff9fbfff700 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ae676c350 0x563ae676e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.348+0000 7ff9fbfff700 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ae676c350 0x563ae676e770 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513a7b66680e790c server_cookie=53fa8b3dfbae56a9 in_seq=0 out_seq=0 2024-05-09T00:21:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.348+0000 7ff9db7fe700 10 client.17022 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.349+0000 7ff9db7fe700 1 -- 172.21.15.46:0/3797427514 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ae6682a50 con 0x563ae676c350 2024-05-09T00:21:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.349+0000 7ff9db7fe700 10 client.17022 handle_command_reply: tid=0 2024-05-09T00:21:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.349+0000 7ffa10e33b80 1 client.17022 shutdown 2024-05-09T00:21:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.349+0000 7ffa10e33b80 1 -- 172.21.15.46:0/3797427514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae6752720 msgr2=0x563ae675cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.349+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae6752720 0x563ae675cd10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40a89f0 tx=0x7ff9f40a41f0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.349+0000 7ffa10e33b80 1 -- 172.21.15.46:0/3797427514 shutdown_connections 2024-05-09T00:21:08.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.349+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ae676c350 0x563ae676e770 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.349+0000 7ff9db7fe700 0 client.17022 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:08.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.349+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae6752720 0x563ae675cd10 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.349+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae65d5520 0x563ae675c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.349+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/3797427514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae66bf1e0 0x563ae675c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.350+0000 7ffa10e33b80 1 -- 172.21.15.46:0/3797427514 >> 172.21.15.46:0/3797427514 conn(0x563ae6635720 msgr2=0x563ae674d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.350+0000 7ffa10e33b80 1 -- 172.21.15.46:0/3797427514 shutdown_connections 2024-05-09T00:21:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.350+0000 7ffa10e33b80 1 -- 172.21.15.46:0/3797427514 wait complete. 2024-05-09T00:21:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.350+0000 7ffa10e33b80 20 client.17022 trim_cache size 0 max 16384 2024-05-09T00:21:08.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.370+0000 7ffa10e33b80 1 Processor -- start 2024-05-09T00:21:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.370+0000 7ffa10e33b80 1 -- start start 2024-05-09T00:21:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ffa10e33b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae65d5520 0x563ae6752140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ffa10e33b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 0x563ae6773e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ffa10e33b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae6752720 0x563ae6774380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ffa10e33b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1b14e0 con 0x563ae66bf1e0 2024-05-09T00:21:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ffa10e33b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1aed70 con 0x563ae65d5520 2024-05-09T00:21:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ffa10e33b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1acba0 con 0x563ae6752720 2024-05-09T00:21:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 0x563ae6773e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 0x563ae6773e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33672/0 (socket says 172.21.15.46:33672) 2024-05-09T00:21:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ff9f3fff700 1 -- 172.21.15.46:0/345723798 learned_addr learned my addr 172.21.15.46:0/345723798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ff9fbfff700 1 --2- 172.21.15.46:0/345723798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae65d5520 0x563ae6752140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ffa009ba700 1 --2- 172.21.15.46:0/345723798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae6752720 0x563ae6774380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ff9f3fff700 1 -- 172.21.15.46:0/345723798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae6752720 msgr2=0x563ae6774380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ff9f3fff700 1 --2- 172.21.15.46:0/345723798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae6752720 0x563ae6774380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ff9f3fff700 1 -- 172.21.15.46:0/345723798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae65d5520 msgr2=0x563ae6752140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ff9f3fff700 1 --2- 172.21.15.46:0/345723798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae65d5520 0x563ae6752140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ff9f3fff700 1 -- 172.21.15.46:0/345723798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae674ed80 con 0x563ae66bf1e0 2024-05-09T00:21:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.371+0000 7ff9f3fff700 1 --2- 172.21.15.46:0/345723798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 0x563ae6773e40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4000c00 tx=0x7ff9e4003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e74df433c1afc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.372+0000 7ff9f1ffb700 1 -- 172.21.15.46:0/345723798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e400fea0 con 0x563ae66bf1e0 2024-05-09T00:21:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.372+0000 7ff9f1ffb700 1 -- 172.21.15.46:0/345723798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e4003e40 con 0x563ae66bf1e0 2024-05-09T00:21:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.372+0000 7ff9f1ffb700 1 -- 172.21.15.46:0/345723798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4004af0 con 0x563ae66bf1e0 2024-05-09T00:21:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.372+0000 7ffa10e33b80 1 -- 172.21.15.46:0/345723798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 msgr2=0x563ae6773e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.372+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/345723798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 0x563ae6773e40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4000c00 tx=0x7ff9e4003dd0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.372+0000 7ffa10e33b80 1 -- 172.21.15.46:0/345723798 shutdown_connections 2024-05-09T00:21:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.372+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/345723798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae6752720 0x563ae6774380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.372+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/345723798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae65d5520 0x563ae6752140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.372+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/345723798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 0x563ae6773e40 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.372+0000 7ffa10e33b80 1 -- 172.21.15.46:0/345723798 >> 172.21.15.46:0/345723798 conn(0x563ae6635720 msgr2=0x563ae675c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.373+0000 7ffa10e33b80 1 -- 172.21.15.46:0/345723798 shutdown_connections 2024-05-09T00:21:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.373+0000 7ffa10e33b80 1 -- 172.21.15.46:0/345723798 wait complete. 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.373+0000 7ffa10e33b80 1 -- start start 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.373+0000 7ffa10e33b80 1 Processor -- start 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.373+0000 7ffa10e33b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae65d5520 0x563ae6761680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.373+0000 7ffa10e33b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 0x563ae6761bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ffa10e33b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae6752720 0x563ae67d6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ffa10e33b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1ad230 con 0x563ae66bf1e0 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ffa10e33b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1ae6a0 con 0x563ae6752720 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ffa10e33b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc1ad900 con 0x563ae65d5520 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ffa10e33b80 10 client.0 _ll_register_callbacks cb 0x563ae64e6740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 0x563ae6761bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 0x563ae6761bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33676/0 (socket says 172.21.15.46:33676) 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9fbfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae65d5520 0x563ae6761680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9f3fff700 1 -- 172.21.15.46:0/4247363362 learned_addr learned my addr 172.21.15.46:0/4247363362 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9f3fff700 1 -- 172.21.15.46:0/4247363362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae65d5520 msgr2=0x563ae6761680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9f3fff700 1 --2- 172.21.15.46:0/4247363362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae65d5520 0x563ae6761680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9f3fff700 1 -- 172.21.15.46:0/4247363362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae6752720 msgr2=0x563ae67d6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9f3fff700 1 --2- 172.21.15.46:0/4247363362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae6752720 0x563ae67d6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9f3fff700 1 -- 172.21.15.46:0/4247363362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae676ec10 con 0x563ae66bf1e0 2024-05-09T00:21:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9f3fff700 1 --2- 172.21.15.46:0/4247363362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 0x563ae6761bc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400a3c0 tx=0x7ff9e4003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b61f7d50c3ad31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9db7fe700 1 -- 172.21.15.46:0/4247363362 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4033030 con 0x563ae66bf1e0 2024-05-09T00:21:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ff9db7fe700 1 -- 172.21.15.46:0/4247363362 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff9e4004980 con 0x563ae66bf1e0 2024-05-09T00:21:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.374+0000 7ffa10e33b80 1 -- 172.21.15.46:0/4247363362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e4003e40 con 0x563ae66bf1e0 2024-05-09T00:21:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.375+0000 7ff9db7fe700 1 -- 172.21.15.46:0/4247363362 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9e400fea0 con 0x563ae66bf1e0 2024-05-09T00:21:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.375+0000 7ffa10e33b80 10 client.17028 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.375+0000 7ffa10e33b80 1 -- 172.21.15.46:0/4247363362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ae67d7fa0 con 0x563ae66bf1e0 2024-05-09T00:21:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.375+0000 7ff9db7fe700 1 -- 172.21.15.46:0/4247363362 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e4003e40 con 0x563ae66bf1e0 2024-05-09T00:21:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.375+0000 7ff9db7fe700 1 -- 172.21.15.46:0/4247363362 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff9e4005610 con 0x563ae66bf1e0 2024-05-09T00:21:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.375+0000 7ffa10e33b80 10 client.17028 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.375+0000 7ffa10e33b80 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:21:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.375+0000 7ffa10e33b80 20 client.17028 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.375+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/4247363362 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ae676c350 0x563ae67da4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.375+0000 7ffa10e33b80 4 client.17028 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:21:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.375+0000 7ffa10e33b80 1 -- 172.21.15.46:0/4247363362 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563ae667caf0 con 0x563ae676c350 2024-05-09T00:21:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.375+0000 7ff9fbfff700 1 --2- 172.21.15.46:0/4247363362 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ae676c350 0x563ae67da4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.376+0000 7ff9fbfff700 1 --2- 172.21.15.46:0/4247363362 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ae676c350 0x563ae67da4c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cfc420cf6ba2b54 server_cookie=f23734a043e8292 in_seq=0 out_seq=0 2024-05-09T00:21:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.376+0000 7ff9db7fe700 10 client.17028 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.377+0000 7ff9db7fe700 1 -- 172.21.15.46:0/4247363362 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563ae667caf0 con 0x563ae676c350 2024-05-09T00:21:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.377+0000 7ff9db7fe700 10 client.17028 handle_command_reply: tid=0 2024-05-09T00:21:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.377+0000 7ffa10e33b80 1 client.17028 shutdown 2024-05-09T00:21:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.377+0000 7ffa10e33b80 1 -- 172.21.15.46:0/4247363362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 msgr2=0x563ae6761bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.377+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/4247363362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 0x563ae6761bc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400a3c0 tx=0x7ff9e4003960 comp rx=0 tx=0).stop 2024-05-09T00:21:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.377+0000 7ffa10e33b80 1 -- 172.21.15.46:0/4247363362 shutdown_connections 2024-05-09T00:21:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.377+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/4247363362 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563ae676c350 0x563ae67da4c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.377+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/4247363362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ae6752720 0x563ae67d6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.377+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/4247363362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae65d5520 0x563ae6761680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.377+0000 7ffa10e33b80 1 --2- 172.21.15.46:0/4247363362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563ae66bf1e0 0x563ae6761bc0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.377+0000 7ffa10e33b80 1 -- 172.21.15.46:0/4247363362 >> 172.21.15.46:0/4247363362 conn(0x563ae6635720 msgr2=0x563ae6755e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.377+0000 7ff9db7fe700 0 client.17028 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa10e33b80 1 -- 172.21.15.46:0/4247363362 shutdown_connections 2024-05-09T00:21:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa10e33b80 1 -- 172.21.15.46:0/4247363362 wait complete. 2024-05-09T00:21:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa10e33b80 20 client.17028 trim_cache size 0 max 16384 2024-05-09T00:21:08.387 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:21:08.387 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:21:08.387 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "d8dd8f56-820d-4c47-ad34-140dfe325d29", 2024-05-09T00:21:08.387 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:21:08.387 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:21:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa0241d700 1 -- 172.21.15.46:0/900918278 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9e80b2900 msgr2=0x7ff9e80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa0241d700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9e80b2900 0x7ff9e80b4e10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec000c00 tx=0x7ff9ec01a040 comp rx=0 tx=0).stop 2024-05-09T00:21:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa0241d700 1 -- 172.21.15.46:0/900918278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9fc108760 msgr2=0x7ff9fc1a6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa0241d700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9fc108760 0x7ff9fc1a6cd0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4007ee0 tx=0x7ff9f400d3b0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa0241d700 1 -- 172.21.15.46:0/900918278 shutdown_connections 2024-05-09T00:21:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa0241d700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9e80b2900 0x7ff9e80b4e10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa0241d700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9fc108760 0x7ff9fc1a6cd0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa0241d700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9fc1034e0 0x7ff9fc19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa0241d700 1 --2- 172.21.15.46:0/900918278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9fc107e10 0x7ff9fc19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:08.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa0241d700 1 -- 172.21.15.46:0/900918278 >> 172.21.15.46:0/900918278 conn(0x7ff9fc0fef30 msgr2=0x7ff9fc1096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:08.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa0241d700 1 -- 172.21.15.46:0/900918278 shutdown_connections 2024-05-09T00:21:08.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:08.378+0000 7ffa0241d700 1 -- 172.21.15.46:0/900918278 wait complete. 2024-05-09T00:21:08.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:08 smithi046 ceph-mon[30151]: pgmap v669: 97 pgs: 97 active+clean; 5.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:21:08.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:08 smithi169 ceph-mon[34777]: pgmap v669: 97 pgs: 97 active+clean; 5.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:21:08.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:08 smithi142 ceph-mon[34533]: pgmap v669: 97 pgs: 97 active+clean; 5.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:21:09.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:21:09.346 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:21:09.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:09 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:21:09.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:09 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:21:09.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:09 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:21:09.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:09 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:21:09.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:09 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:21:09.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:09 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:21:09.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:09 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:21:09.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:09 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:21:09.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:09 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:21:09.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:09 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:21:09.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:09 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:21:09.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:09 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:21:10.782 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:10 smithi046 ceph-mon[30151]: pgmap v670: 97 pgs: 97 active+clean; 5.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:10.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:10 smithi169 ceph-mon[34777]: pgmap v670: 97 pgs: 97 active+clean; 5.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:10.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:10 smithi142 ceph-mon[34533]: pgmap v670: 97 pgs: 97 active+clean; 5.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:11.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.025+0000 7f76474a2700 1 -- 172.21.15.46:0/2680298709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7640109860 msgr2=0x7f76401111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:11.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.025+0000 7f76474a2700 1 --2- 172.21.15.46:0/2680298709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7640109860 0x7f76401111c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f76380099c0 tx=0x7f7638009cd0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.025+0000 7f76474a2700 1 -- 172.21.15.46:0/2680298709 shutdown_connections 2024-05-09T00:21:11.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.025+0000 7f76474a2700 1 --2- 172.21.15.46:0/2680298709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7640109860 0x7f76401111c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.025+0000 7f76474a2700 1 --2- 172.21.15.46:0/2680298709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76401045e0 0x7f76401049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.025+0000 7f76474a2700 1 --2- 172.21.15.46:0/2680298709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7640108f10 0x7f7640109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.025+0000 7f76474a2700 1 -- 172.21.15.46:0/2680298709 >> 172.21.15.46:0/2680298709 conn(0x7f76400ffe70 msgr2=0x7f76401022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.025+0000 7f76474a2700 1 -- 172.21.15.46:0/2680298709 shutdown_connections 2024-05-09T00:21:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.025+0000 7f76474a2700 1 -- 172.21.15.46:0/2680298709 wait complete. 2024-05-09T00:21:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.026+0000 7f76474a2700 1 Processor -- start 2024-05-09T00:21:11.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.026+0000 7f76474a2700 1 -- start start 2024-05-09T00:21:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.026+0000 7f76474a2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76401045e0 0x7f764019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.026+0000 7f76474a2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7640108f10 0x7f764019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.026+0000 7f76474a2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7640109860 0x7f764019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.026+0000 7f76474a2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76401144b0 con 0x7f7640108f10 2024-05-09T00:21:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.026+0000 7f76474a2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7640114330 con 0x7f76401045e0 2024-05-09T00:21:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.026+0000 7f76474a2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7640114630 con 0x7f7640109860 2024-05-09T00:21:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f7644a3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7640108f10 0x7f764019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f7644a3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7640108f10 0x7f764019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33704/0 (socket says 172.21.15.46:33704) 2024-05-09T00:21:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f7644a3d700 1 -- 172.21.15.46:0/555761839 learned_addr learned my addr 172.21.15.46:0/555761839 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f764523e700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76401045e0 0x7f764019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f7645a3f700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7640109860 0x7f764019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f7644a3d700 1 -- 172.21.15.46:0/555761839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7640109860 msgr2=0x7f764019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f7644a3d700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7640109860 0x7f764019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f7644a3d700 1 -- 172.21.15.46:0/555761839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76401045e0 msgr2=0x7f764019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f7644a3d700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76401045e0 0x7f764019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f7644a3d700 1 -- 172.21.15.46:0/555761839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764019fc90 con 0x7f7640108f10 2024-05-09T00:21:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f7644a3d700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7640108f10 0x7f764019b360 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f763000c970 tx=0x7f763000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f76367fc700 1 -- 172.21.15.46:0/555761839 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763000ce60 con 0x7f7640108f10 2024-05-09T00:21:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f76367fc700 1 -- 172.21.15.46:0/555761839 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7630004830 con 0x7f7640108f10 2024-05-09T00:21:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f76474a2700 1 -- 172.21.15.46:0/555761839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764019ff80 con 0x7f7640108f10 2024-05-09T00:21:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f76367fc700 1 -- 172.21.15.46:0/555761839 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630005530 con 0x7f7640108f10 2024-05-09T00:21:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.027+0000 7f76474a2700 1 -- 172.21.15.46:0/555761839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76401a78a0 con 0x7f7640108f10 2024-05-09T00:21:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.030+0000 7f76367fc700 1 -- 172.21.15.46:0/555761839 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f763000f450 con 0x7f7640108f10 2024-05-09T00:21:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.031+0000 7f76367fc700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f762c0b2880 0x7f762c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.031+0000 7f764523e700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f762c0b2880 0x7f762c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.031+0000 7f76367fc700 1 -- 172.21.15.46:0/555761839 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f76300e1db0 con 0x7f7640108f10 2024-05-09T00:21:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.031+0000 7f7655eb8b80 1 Processor -- start 2024-05-09T00:21:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.031+0000 7f7655eb8b80 1 -- start start 2024-05-09T00:21:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f7655eb8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 0x56303cdcc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f7655eb8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cc494a0 0x56303cdcc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f7655eb8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cdc6400 0x56303cdc6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f7655eb8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56303cce90a0 con 0x56303ccc3d00 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f7655eb8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56303cc8e9a0 con 0x56303cc494a0 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f7655eb8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56303cce5060 con 0x56303cdc6400 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f764523e700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f762c0b2880 0x7f762c0b4d90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f763c009550 tx=0x7f763c0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f764523e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 0x56303cdcc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f764523e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 0x56303cdcc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33706/0 (socket says 172.21.15.46:33706) 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f764523e700 1 -- 172.21.15.46:0/2763079949 learned_addr learned my addr 172.21.15.46:0/2763079949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f7645a3f700 1 --2- 172.21.15.46:0/2763079949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cdc6400 0x56303cdc6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f7644a3d700 1 --2- 172.21.15.46:0/2763079949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cc494a0 0x56303cdcc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f764523e700 1 -- 172.21.15.46:0/2763079949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cdc6400 msgr2=0x56303cdc6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f764523e700 1 --2- 172.21.15.46:0/2763079949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cdc6400 0x56303cdc6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f764523e700 1 -- 172.21.15.46:0/2763079949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cc494a0 msgr2=0x56303cdcc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f764523e700 1 --2- 172.21.15.46:0/2763079949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cc494a0 0x56303cdcc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f764523e700 1 -- 172.21.15.46:0/2763079949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303cc7f500 con 0x56303ccc3d00 2024-05-09T00:21:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f7644a3d700 1 --2- 172.21.15.46:0/2763079949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cc494a0 0x56303cdcc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:21:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.032+0000 7f764523e700 1 --2- 172.21.15.46:0/2763079949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 0x56303cdcc240 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f763c004700 tx=0x7f763c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb863d348622efee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.033+0000 7f762b7fe700 1 -- 172.21.15.46:0/2763079949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c026070 con 0x56303ccc3d00 2024-05-09T00:21:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.033+0000 7f762b7fe700 1 -- 172.21.15.46:0/2763079949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763c01d040 con 0x56303ccc3d00 2024-05-09T00:21:11.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.033+0000 7f762b7fe700 1 -- 172.21.15.46:0/2763079949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c01c050 con 0x56303ccc3d00 2024-05-09T00:21:11.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.033+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2763079949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 msgr2=0x56303cdcc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:11.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.033+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/2763079949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 0x56303cdcc240 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f763c004700 tx=0x7f763c013000 comp rx=0 tx=0).stop 2024-05-09T00:21:11.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.034+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2763079949 shutdown_connections 2024-05-09T00:21:11.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.034+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/2763079949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cdc6400 0x56303cdc6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.034+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/2763079949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 0x56303cdcc240 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.034+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/2763079949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cc494a0 0x56303cdcc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.034+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2763079949 >> 172.21.15.46:0/2763079949 conn(0x56303cc4cf70 msgr2=0x56303cdc1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.034+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2763079949 shutdown_connections 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.034+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2763079949 wait complete. 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.034+0000 7f7655eb8b80 1 -- start start 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.034+0000 7f7655eb8b80 1 Processor -- start 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f7655eb8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cc494a0 0x56303cdd0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f7655eb8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303ccc3d00 0x56303cdd0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f7655eb8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cdc6400 0x56303cdd0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f7655eb8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56303cdc7160 con 0x56303cc494a0 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f7655eb8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56303cc83b20 con 0x56303cdc6400 2024-05-09T00:21:11.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f7655eb8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56303cdd3020 con 0x56303ccc3d00 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f7655eb8b80 10 client.0 _ll_register_callbacks cb 0x56303cb5a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f764523e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cc494a0 0x56303cdd0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f764523e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cc494a0 0x56303cdd0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33722/0 (socket says 172.21.15.46:33722) 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f764523e700 1 -- 172.21.15.46:0/1820991009 learned_addr learned my addr 172.21.15.46:0/1820991009 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f7644a3d700 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303ccc3d00 0x56303cdd0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f7645a3f700 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cdc6400 0x56303cdd0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f764523e700 1 -- 172.21.15.46:0/1820991009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303ccc3d00 msgr2=0x56303cdd0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f764523e700 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303ccc3d00 0x56303cdd0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f764523e700 1 -- 172.21.15.46:0/1820991009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cdc6400 msgr2=0x56303cdd0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f764523e700 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cdc6400 0x56303cdd0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f764523e700 1 -- 172.21.15.46:0/1820991009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303cb64490 con 0x56303cc494a0 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.035+0000 7f7645a3f700 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cdc6400 0x56303cdd0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.036+0000 7f764523e700 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cc494a0 0x56303cdd0040 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f763c009620 tx=0x7f763c004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1a3bd28a35468 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:11.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.036+0000 7f7628ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.036+0000 7f7628ff9700 1 -- 172.21.15.46:0/1820991009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c02e030 con 0x56303cc494a0 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.036+0000 7f7628ff9700 1 -- 172.21.15.46:0/1820991009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f763c021e00 con 0x56303cc494a0 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.036+0000 7f7655eb8b80 1 -- 172.21.15.46:0/1820991009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763c01d040 con 0x56303cc494a0 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.036+0000 7f7628ff9700 1 -- 172.21.15.46:0/1820991009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f763c01c020 con 0x56303cc494a0 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.036+0000 7f7655eb8b80 10 client.17040 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.036+0000 7f7655eb8b80 1 -- 172.21.15.46:0/1820991009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56303cddd980 con 0x56303cc494a0 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.036+0000 7f7628ff9700 1 -- 172.21.15.46:0/1820991009 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763c01d040 con 0x56303cc494a0 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.036+0000 7f7628ff9700 1 -- 172.21.15.46:0/1820991009 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f763c029070 con 0x56303cc494a0 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.037+0000 7f7655eb8b80 10 client.17040 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.037+0000 7f7655eb8b80 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.037+0000 7f7655eb8b80 20 client.17040 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.037+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56303cddff00 0x56303cde2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.037+0000 7f7655eb8b80 4 client.17040 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.037+0000 7f7655eb8b80 1 -- 172.21.15.46:0/1820991009 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56303ccf6a80 con 0x56303cddff00 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.037+0000 7f7644a3d700 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56303cddff00 0x56303cde2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.037+0000 7f7644a3d700 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56303cddff00 0x56303cde2320 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69eeab5240aa309b server_cookie=4c3497bbf0da651c in_seq=0 out_seq=0 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.037+0000 7f7628ff9700 10 client.17040 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.038+0000 7f7628ff9700 1 -- 172.21.15.46:0/1820991009 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56303ccf6a80 con 0x56303cddff00 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.038+0000 7f7628ff9700 10 client.17040 handle_command_reply: tid=0 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.038+0000 7f7655eb8b80 1 client.17040 shutdown 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.038+0000 7f7655eb8b80 1 -- 172.21.15.46:0/1820991009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cc494a0 msgr2=0x56303cdd0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.038+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cc494a0 0x56303cdd0040 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f763c009620 tx=0x7f763c004db0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.038+0000 7f7655eb8b80 1 -- 172.21.15.46:0/1820991009 shutdown_connections 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.038+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56303cddff00 0x56303cde2320 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.038+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cdc6400 0x56303cdd0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.038+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cc494a0 0x56303cdd0040 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.038+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/1820991009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303ccc3d00 0x56303cdd0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.038+0000 7f7655eb8b80 1 -- 172.21.15.46:0/1820991009 >> 172.21.15.46:0/1820991009 conn(0x56303cc4cf70 msgr2=0x56303cdbaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:11.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.038+0000 7f7628ff9700 0 client.17040 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:11.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.039+0000 7f7655eb8b80 1 -- 172.21.15.46:0/1820991009 shutdown_connections 2024-05-09T00:21:11.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.039+0000 7f7655eb8b80 1 -- 172.21.15.46:0/1820991009 wait complete. 2024-05-09T00:21:11.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.039+0000 7f7655eb8b80 20 client.17040 trim_cache size 0 max 16384 2024-05-09T00:21:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f7655eb8b80 1 Processor -- start 2024-05-09T00:21:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f7655eb8b80 1 -- start start 2024-05-09T00:21:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f7655eb8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cc494a0 0x56303cdd0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f7655eb8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303ccc3d00 0x56303cddc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f7655eb8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cdc6400 0x56303cdcef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f7655eb8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76401b1570 con 0x56303cdc6400 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f7655eb8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f76401aee00 con 0x56303ccc3d00 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f7655eb8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f76401acc30 con 0x56303cc494a0 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f7645a3f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cdc6400 0x56303cdcef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f7645a3f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cdc6400 0x56303cdcef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33736/0 (socket says 172.21.15.46:33736) 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f7645a3f700 1 -- 172.21.15.46:0/3727223783 learned_addr learned my addr 172.21.15.46:0/3727223783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f764523e700 1 --2- 172.21.15.46:0/3727223783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cc494a0 0x56303cdd0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.060+0000 7f7644a3d700 1 --2- 172.21.15.46:0/3727223783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303ccc3d00 0x56303cddc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.061+0000 7f7645a3f700 1 -- 172.21.15.46:0/3727223783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cc494a0 msgr2=0x56303cdd0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.061+0000 7f7645a3f700 1 --2- 172.21.15.46:0/3727223783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cc494a0 0x56303cdd0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.061+0000 7f7645a3f700 1 -- 172.21.15.46:0/3727223783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303ccc3d00 msgr2=0x56303cddc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:11.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.061+0000 7f7645a3f700 1 --2- 172.21.15.46:0/3727223783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303ccc3d00 0x56303cddc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.061+0000 7f7645a3f700 1 -- 172.21.15.46:0/3727223783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303cdbd9d0 con 0x56303cdc6400 2024-05-09T00:21:11.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.061+0000 7f7644a3d700 1 --2- 172.21.15.46:0/3727223783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303ccc3d00 0x56303cddc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:21:11.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.061+0000 7f7645a3f700 1 --2- 172.21.15.46:0/3727223783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cdc6400 0x56303cdcef10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f76380099c0 tx=0x7f7638009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9f2f516f3d225e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:11.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.061+0000 7f762affd700 1 -- 172.21.15.46:0/3727223783 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638027070 con 0x56303cdc6400 2024-05-09T00:21:11.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.061+0000 7f762affd700 1 -- 172.21.15.46:0/3727223783 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76380050c0 con 0x56303cdc6400 2024-05-09T00:21:11.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.061+0000 7f762affd700 1 -- 172.21.15.46:0/3727223783 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763800bb10 con 0x56303cdc6400 2024-05-09T00:21:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.062+0000 7f7655eb8b80 1 -- 172.21.15.46:0/3727223783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cdc6400 msgr2=0x56303cdcef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.062+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/3727223783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cdc6400 0x56303cdcef10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f76380099c0 tx=0x7f7638009510 comp rx=0 tx=0).stop 2024-05-09T00:21:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.062+0000 7f7655eb8b80 1 -- 172.21.15.46:0/3727223783 shutdown_connections 2024-05-09T00:21:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.062+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/3727223783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303cdc6400 0x56303cdcef10 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.062+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/3727223783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cc494a0 0x56303cdd0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.062+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/3727223783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303ccc3d00 0x56303cddc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.062+0000 7f7655eb8b80 1 -- 172.21.15.46:0/3727223783 >> 172.21.15.46:0/3727223783 conn(0x56303cc4cf70 msgr2=0x56303cdce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.062+0000 7f7655eb8b80 1 -- 172.21.15.46:0/3727223783 shutdown_connections 2024-05-09T00:21:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.062+0000 7f7655eb8b80 1 -- 172.21.15.46:0/3727223783 wait complete. 2024-05-09T00:21:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.062+0000 7f7655eb8b80 1 -- start start 2024-05-09T00:21:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f7655eb8b80 1 Processor -- start 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f7655eb8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cc494a0 0x56303ce42a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f7655eb8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 0x56303ce42fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f7655eb8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cdc6400 0x56303cde4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f7655eb8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76401ad2c0 con 0x56303ccc3d00 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f7655eb8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f76401ae730 con 0x56303cc494a0 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f7655eb8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f76401ad990 con 0x56303cdc6400 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f7655eb8b80 10 client.0 _ll_register_callbacks cb 0x56303cb5a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f7644a3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 0x56303ce42fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f7644a3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 0x56303ce42fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33752/0 (socket says 172.21.15.46:33752) 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f7644a3d700 1 -- 172.21.15.46:0/2914179240 learned_addr learned my addr 172.21.15.46:0/2914179240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f764523e700 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cc494a0 0x56303ce42a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.063+0000 7f7645a3f700 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cdc6400 0x56303cde4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7644a3d700 1 -- 172.21.15.46:0/2914179240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cdc6400 msgr2=0x56303cde4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7644a3d700 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cdc6400 0x56303cde4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7644a3d700 1 -- 172.21.15.46:0/2914179240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cc494a0 msgr2=0x56303ce42a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7644a3d700 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cc494a0 0x56303ce42a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7644a3d700 1 -- 172.21.15.46:0/2914179240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303cde4cb0 con 0x56303ccc3d00 2024-05-09T00:21:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7644a3d700 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 0x56303ce42fc0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f76300a90d0 tx=0x7f76300a5340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9988b0b9d3f4accc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7628ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7628ff9700 1 -- 172.21.15.46:0/2914179240 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763006d070 con 0x56303ccc3d00 2024-05-09T00:21:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7628ff9700 1 -- 172.21.15.46:0/2914179240 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f76300a5660 con 0x56303ccc3d00 2024-05-09T00:21:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2914179240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76380050c0 con 0x56303ccc3d00 2024-05-09T00:21:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7628ff9700 1 -- 172.21.15.46:0/2914179240 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f763001d040 con 0x56303ccc3d00 2024-05-09T00:21:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7655eb8b80 10 client.17046 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:11.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2914179240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56303ce4a0d0 con 0x56303ccc3d00 2024-05-09T00:21:11.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7628ff9700 1 -- 172.21.15.46:0/2914179240 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76380050c0 con 0x56303ccc3d00 2024-05-09T00:21:11.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.064+0000 7f7628ff9700 1 -- 172.21.15.46:0/2914179240 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f76300ca290 con 0x56303ccc3d00 2024-05-09T00:21:11.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.065+0000 7f7655eb8b80 10 client.17046 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:11.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.065+0000 7f7655eb8b80 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:21:11.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.065+0000 7f7655eb8b80 20 client.17046 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:11.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.065+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56303ce4a790 0x56303ce4cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:11.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.065+0000 7f7655eb8b80 4 client.17046 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:21:11.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.065+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2914179240 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56303ccf0b20 con 0x56303ce4a790 2024-05-09T00:21:11.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.065+0000 7f764523e700 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56303ce4a790 0x56303ce4cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:11.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.065+0000 7f7628ff9700 10 client.17046 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:11.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.065+0000 7f764523e700 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56303ce4a790 0x56303ce4cbb0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b32476fc71705f6f server_cookie=e0ef13eedc52874d in_seq=0 out_seq=0 2024-05-09T00:21:11.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7628ff9700 1 -- 172.21.15.46:0/2914179240 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56303ccf0b20 con 0x56303ce4a790 2024-05-09T00:21:11.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7628ff9700 10 client.17046 handle_command_reply: tid=0 2024-05-09T00:21:11.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7655eb8b80 1 client.17046 shutdown 2024-05-09T00:21:11.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2914179240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 msgr2=0x56303ce42fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:11.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 0x56303ce42fc0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f76300a90d0 tx=0x7f76300a5340 comp rx=0 tx=0).stop 2024-05-09T00:21:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2914179240 shutdown_connections 2024-05-09T00:21:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x56303ce4a790 0x56303ce4cbb0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56303cdc6400 0x56303cde4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56303cc494a0 0x56303ce42a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7628ff9700 0 client.17046 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7655eb8b80 1 --2- 172.21.15.46:0/2914179240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56303ccc3d00 0x56303ce42fc0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2914179240 >> 172.21.15.46:0/2914179240 conn(0x56303cc4cf70 msgr2=0x56303cdc9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2914179240 shutdown_connections 2024-05-09T00:21:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7655eb8b80 1 -- 172.21.15.46:0/2914179240 wait complete. 2024-05-09T00:21:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.066+0000 7f7655eb8b80 20 client.17046 trim_cache size 0 max 16384 2024-05-09T00:21:11.074 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:21:11.075 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:21:11.075 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T00:21:11.075 INFO:teuthology.orchestra.run.smithi046.stdout: "d8dd8f56-820d-4c47-ad34-140dfe325d29": { 2024-05-09T00:21:11.075 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T00:21:11.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "d8dd8f56-820d-4c47-ad34-140dfe325d29", 2024-05-09T00:21:11.075 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T00:21:11.075 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:21:11.075 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:21:11.075 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:21:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.067+0000 7f76474a2700 1 -- 172.21.15.46:0/555761839 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f762c0b2880 msgr2=0x7f762c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.067+0000 7f76474a2700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f762c0b2880 0x7f762c0b4d90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f763c009550 tx=0x7f763c0091b0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.067+0000 7f76474a2700 1 -- 172.21.15.46:0/555761839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7640108f10 msgr2=0x7f764019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.067+0000 7f76474a2700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7640108f10 0x7f764019b360 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f763000c970 tx=0x7f763000cc80 comp rx=0 tx=0).stop 2024-05-09T00:21:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.067+0000 7f76474a2700 1 -- 172.21.15.46:0/555761839 shutdown_connections 2024-05-09T00:21:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.067+0000 7f76474a2700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f762c0b2880 0x7f762c0b4d90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.067+0000 7f76474a2700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7640109860 0x7f764019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.067+0000 7f76474a2700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76401045e0 0x7f764019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.067+0000 7f76474a2700 1 --2- 172.21.15.46:0/555761839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7640108f10 0x7f764019b360 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.067+0000 7f76474a2700 1 -- 172.21.15.46:0/555761839 >> 172.21.15.46:0/555761839 conn(0x7f76400ffe70 msgr2=0x7f7640108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.067+0000 7f76474a2700 1 -- 172.21.15.46:0/555761839 shutdown_connections 2024-05-09T00:21:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:11.067+0000 7f76474a2700 1 -- 172.21.15.46:0/555761839 wait complete. 2024-05-09T00:21:11.747 INFO:tasks.cephfs.filesystem:scrub status for tag:d8dd8f56-820d-4c47-ad34-140dfe325d29 - {'path': '/', 'tag': 'd8dd8f56-820d-4c47-ad34-140dfe325d29', 'options': 'recursive,force'} 2024-05-09T00:21:12.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:12 smithi046 ceph-mon[30151]: pgmap v671: 97 pgs: 97 active+clean; 5.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:21:12.813 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:12 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:21:12.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:12 smithi169 ceph-mon[34777]: pgmap v671: 97 pgs: 97 active+clean; 5.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:21:12.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:12 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:21:12.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:12 smithi142 ceph-mon[34533]: pgmap v671: 97 pgs: 97 active+clean; 5.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:21:12.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:12 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:21:14.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:14 smithi046 ceph-mon[30151]: pgmap v672: 97 pgs: 97 active+clean; 5.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:14.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:14 smithi169 ceph-mon[34777]: pgmap v672: 97 pgs: 97 active+clean; 5.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:14.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:14 smithi142 ceph-mon[34533]: pgmap v672: 97 pgs: 97 active+clean; 5.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:16.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:16 smithi046 ceph-mon[30151]: pgmap v673: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:16.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:16 smithi169 ceph-mon[34777]: pgmap v673: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:16.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:16 smithi142 ceph-mon[34533]: pgmap v673: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:18.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:18 smithi046 ceph-mon[30151]: pgmap v674: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:18.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:18 smithi169 ceph-mon[34777]: pgmap v674: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:18.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:18 smithi142 ceph-mon[34533]: pgmap v674: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:20.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:20 smithi046 ceph-mon[30151]: pgmap v675: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:21:20.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:20 smithi169 ceph-mon[34777]: pgmap v675: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:21:20.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:20 smithi142 ceph-mon[34533]: pgmap v675: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:21:22.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:22 smithi046 ceph-mon[30151]: pgmap v676: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:21:22.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:22 smithi169 ceph-mon[34777]: pgmap v676: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:21:22.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:22 smithi142 ceph-mon[34533]: pgmap v676: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:21:24.813 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:24 smithi046 ceph-mon[30151]: pgmap v677: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:21:24.858 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:24 smithi169 ceph-mon[34777]: pgmap v677: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:21:24.944 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:24 smithi142 ceph-mon[34533]: pgmap v677: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:21:26.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:26 smithi046 ceph-mon[30151]: pgmap v678: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:21:26.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:26 smithi169 ceph-mon[34777]: pgmap v678: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:21:26.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:26 smithi142 ceph-mon[34533]: pgmap v678: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:21:28.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:28 smithi169 ceph-mon[34777]: pgmap v679: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:21:28.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:28 smithi142 ceph-mon[34533]: pgmap v679: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:21:29.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:28 smithi046 ceph-mon[30151]: pgmap v679: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:21:30.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:30 smithi169 ceph-mon[34777]: pgmap v680: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:30.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:30 smithi142 ceph-mon[34533]: pgmap v680: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:31.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:30 smithi046 ceph-mon[30151]: pgmap v680: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:32.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:32 smithi169 ceph-mon[34777]: pgmap v681: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:21:32.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:32 smithi142 ceph-mon[34533]: pgmap v681: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:21:33.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:32 smithi046 ceph-mon[30151]: pgmap v681: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:21:34.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:34 smithi169 ceph-mon[34777]: pgmap v682: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:34.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:34 smithi142 ceph-mon[34533]: pgmap v682: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:35.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:34 smithi046 ceph-mon[30151]: pgmap v682: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:36.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:36 smithi169 ceph-mon[34777]: pgmap v683: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:36.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:36 smithi142 ceph-mon[34533]: pgmap v683: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:37.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:36 smithi046 ceph-mon[30151]: pgmap v683: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:38.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:38 smithi169 ceph-mon[34777]: pgmap v684: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:21:38.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:38 smithi142 ceph-mon[34533]: pgmap v684: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:21:39.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:38 smithi046 ceph-mon[30151]: pgmap v684: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:21:40.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:40 smithi169 ceph-mon[34777]: pgmap v685: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:40.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:40 smithi142 ceph-mon[34533]: pgmap v685: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:41.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:40 smithi046 ceph-mon[30151]: pgmap v685: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:41.748 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:21:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:21:42.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:42 smithi046 ceph-mon[30151]: pgmap v686: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:21:42.858 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:42 smithi169 ceph-mon[34777]: pgmap v686: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:21:42.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:42 smithi142 ceph-mon[34533]: pgmap v686: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:21:43.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.910+0000 7fe1e064e700 1 -- 172.21.15.46:0/3423593079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1d8107df0 msgr2=0x7fe1d8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:43.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.910+0000 7fe1e064e700 1 --2- 172.21.15.46:0/3423593079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1d8107df0 0x7fe1d8108200 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80099c0 tx=0x7fe1c8009cd0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.911+0000 7fe1e064e700 1 -- 172.21.15.46:0/3423593079 shutdown_connections 2024-05-09T00:21:43.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.911+0000 7fe1e064e700 1 --2- 172.21.15.46:0/3423593079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1d8108740 0x7fe1d81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.911+0000 7fe1e064e700 1 --2- 172.21.15.46:0/3423593079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d81034c0 0x7fe1d81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.911+0000 7fe1e064e700 1 --2- 172.21.15.46:0/3423593079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1d8107df0 0x7fe1d8108200 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.911+0000 7fe1e064e700 1 -- 172.21.15.46:0/3423593079 >> 172.21.15.46:0/3423593079 conn(0x7fe1d80fedf0 msgr2=0x7fe1d8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:43.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.911+0000 7fe1e064e700 1 -- 172.21.15.46:0/3423593079 shutdown_connections 2024-05-09T00:21:43.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.911+0000 7fe1e064e700 1 -- 172.21.15.46:0/3423593079 wait complete. 2024-05-09T00:21:43.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1e064e700 1 Processor -- start 2024-05-09T00:21:43.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1e064e700 1 -- start start 2024-05-09T00:21:43.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1e064e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1d81034c0 0x7fe1d819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1e064e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d8107df0 0x7fe1d819ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1e064e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1d8108740 0x7fe1d8199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1e064e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d8113350 con 0x7fe1d81034c0 2024-05-09T00:21:43.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1e064e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81131d0 con 0x7fe1d8108740 2024-05-09T00:21:43.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1e064e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81134d0 con 0x7fe1d8107df0 2024-05-09T00:21:43.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1de3ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1d81034c0 0x7fe1d819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1de3ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1d81034c0 0x7fe1d819f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45906/0 (socket says 172.21.15.46:45906) 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1de3ea700 1 -- 172.21.15.46:0/2027238180 learned_addr learned my addr 172.21.15.46:0/2027238180 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1ddbe9700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d8107df0 0x7fe1d819ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1debeb700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1d8108740 0x7fe1d8199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1de3ea700 1 -- 172.21.15.46:0/2027238180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d8107df0 msgr2=0x7fe1d819ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.912+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d8107df0 0x7fe1d819ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.913+0000 7fe1de3ea700 1 -- 172.21.15.46:0/2027238180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1d8108740 msgr2=0x7fe1d8199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.913+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1d8108740 0x7fe1d8199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.913+0000 7fe1de3ea700 1 -- 172.21.15.46:0/2027238180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d819a280 con 0x7fe1d81034c0 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.913+0000 7fe1debeb700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1d8108740 0x7fe1d8199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.913+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1d81034c0 0x7fe1d819f9c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400d980 tx=0x7fe1d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.913+0000 7fe1cf7fe700 1 -- 172.21.15.46:0/2027238180 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4016070 con 0x7fe1d81034c0 2024-05-09T00:21:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.913+0000 7fe1e064e700 1 -- 172.21.15.46:0/2027238180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d819a570 con 0x7fe1d81034c0 2024-05-09T00:21:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.913+0000 7fe1cf7fe700 1 -- 172.21.15.46:0/2027238180 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d4004830 con 0x7fe1d81034c0 2024-05-09T00:21:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.913+0000 7fe1cf7fe700 1 -- 172.21.15.46:0/2027238180 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d40054c0 con 0x7fe1d81034c0 2024-05-09T00:21:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.913+0000 7fe1e064e700 1 -- 172.21.15.46:0/2027238180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d80759f0 con 0x7fe1d81034c0 2024-05-09T00:21:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.916+0000 7fe1cf7fe700 1 -- 172.21.15.46:0/2027238180 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe1d4004220 con 0x7fe1d81034c0 2024-05-09T00:21:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.917+0000 7fe1cf7fe700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe1c40b2880 0x7fe1c40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.917+0000 7fe1ef064b80 1 Processor -- start 2024-05-09T00:21:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.917+0000 7fe1ddbe9700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe1c40b2880 0x7fe1c40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.917+0000 7fe1cf7fe700 1 -- 172.21.15.46:0/2027238180 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe1d40e20e0 con 0x7fe1d81034c0 2024-05-09T00:21:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.917+0000 7fe1ef064b80 1 -- start start 2024-05-09T00:21:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1ef064b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 0x5625249e5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1ef064b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248624a0 0x5625249e5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1ef064b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249df810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1ef064b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5625249020a0 con 0x5625249df400 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1ef064b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5625248a79a0 con 0x5625248624a0 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1ef064b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5625248fe060 con 0x5625248dcd00 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1ddbe9700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe1c40b2880 0x7fe1c40b4d90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8006010 tx=0x7fe1c801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1debeb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1debeb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249df810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45910/0 (socket says 172.21.15.46:45910) 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1debeb700 1 -- 172.21.15.46:0/83425643 learned_addr learned my addr 172.21.15.46:0/83425643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/83425643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 0x5625249e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1ddbe9700 1 --2- 172.21.15.46:0/83425643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248624a0 0x5625249e5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1debeb700 1 -- 172.21.15.46:0/83425643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 msgr2=0x5625249e5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1debeb700 1 --2- 172.21.15.46:0/83425643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 0x5625249e5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1debeb700 1 -- 172.21.15.46:0/83425643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248624a0 msgr2=0x5625249e5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1debeb700 1 --2- 172.21.15.46:0/83425643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248624a0 0x5625249e5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1debeb700 1 -- 172.21.15.46:0/83425643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562524898500 con 0x5625249df400 2024-05-09T00:21:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1debeb700 1 --2- 172.21.15.46:0/83425643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249df810 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009dd0 tx=0x7fe1d00077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6eee0a7be29a8a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.918+0000 7fe1ccff9700 1 -- 172.21.15.46:0/83425643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0004bb0 con 0x5625249df400 2024-05-09T00:21:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.919+0000 7fe1ccff9700 1 -- 172.21.15.46:0/83425643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d0004d50 con 0x5625249df400 2024-05-09T00:21:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.919+0000 7fe1ccff9700 1 -- 172.21.15.46:0/83425643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0014670 con 0x5625249df400 2024-05-09T00:21:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.919+0000 7fe1ef064b80 1 -- 172.21.15.46:0/83425643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 msgr2=0x5625249df810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.919+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/83425643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249df810 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009dd0 tx=0x7fe1d00077e0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.919+0000 7fe1ef064b80 1 -- 172.21.15.46:0/83425643 shutdown_connections 2024-05-09T00:21:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.919+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/83425643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249df810 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.919+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/83425643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 0x5625249e5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.920+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/83425643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248624a0 0x5625249e5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.920+0000 7fe1ef064b80 1 -- 172.21.15.46:0/83425643 >> 172.21.15.46:0/83425643 conn(0x562524865f70 msgr2=0x5625249da220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.920+0000 7fe1ef064b80 1 -- 172.21.15.46:0/83425643 shutdown_connections 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.920+0000 7fe1ef064b80 1 -- 172.21.15.46:0/83425643 wait complete. 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.920+0000 7fe1ef064b80 1 -- start start 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.920+0000 7fe1ef064b80 1 Processor -- start 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.920+0000 7fe1ef064b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248624a0 0x5625249e9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.920+0000 7fe1ef064b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248dcd00 0x5625249e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.920+0000 7fe1ef064b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.920+0000 7fe1ef064b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5625249e0160 con 0x5625249df400 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.920+0000 7fe1ef064b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56252489cb20 con 0x5625248dcd00 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.920+0000 7fe1ef064b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5625249ec020 con 0x5625248624a0 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1ef064b80 10 client.0 _ll_register_callbacks cb 0x5625247736c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1debeb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1debeb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249e9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45914/0 (socket says 172.21.15.46:45914) 2024-05-09T00:21:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1debeb700 1 -- 172.21.15.46:0/2844794977 learned_addr learned my addr 172.21.15.46:0/2844794977 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1ddbe9700 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248dcd00 0x5625249e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248624a0 0x5625249e9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1debeb700 1 -- 172.21.15.46:0/2844794977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248624a0 msgr2=0x5625249e9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1debeb700 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248624a0 0x5625249e9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1debeb700 1 -- 172.21.15.46:0/2844794977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248dcd00 msgr2=0x5625249e95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1debeb700 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248dcd00 0x5625249e95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1debeb700 1 -- 172.21.15.46:0/2844794977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252477d490 con 0x5625249df400 2024-05-09T00:21:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1debeb700 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249e9b00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0005670 tx=0x7fe1d0006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf68b945eb30d6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1c1ffb700 1 -- 172.21.15.46:0/2844794977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0022070 con 0x5625249df400 2024-05-09T00:21:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1c1ffb700 1 -- 172.21.15.46:0/2844794977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe1d0004b50 con 0x5625249df400 2024-05-09T00:21:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.921+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2844794977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1d0004d50 con 0x5625249df400 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.922+0000 7fe1c1ffb700 1 -- 172.21.15.46:0/2844794977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe1d002e050 con 0x5625249df400 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.922+0000 7fe1ef064b80 10 client.17058 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.922+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2844794977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625249f6aa0 con 0x5625249df400 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.922+0000 7fe1c1ffb700 1 -- 172.21.15.46:0/2844794977 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d0004d50 con 0x5625249df400 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.922+0000 7fe1c1ffb700 1 -- 172.21.15.46:0/2844794977 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe1d00145c0 con 0x5625249df400 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.922+0000 7fe1ef064b80 10 client.17058 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.922+0000 7fe1ef064b80 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.922+0000 7fe1ef064b80 20 client.17058 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.922+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5625249f8f70 0x5625249fb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.922+0000 7fe1ef064b80 4 client.17058 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.922+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2844794977 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56252490fa80 con 0x5625249f8f70 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.922+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5625249f8f70 0x5625249fb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.923+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5625249f8f70 0x5625249fb390 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9681af00de0c7f37 server_cookie=fa35ee7aeca3b358 in_seq=0 out_seq=0 2024-05-09T00:21:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.923+0000 7fe1c1ffb700 10 client.17058 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1c1ffb700 1 -- 172.21.15.46:0/2844794977 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56252490fa80 con 0x5625249f8f70 2024-05-09T00:21:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1c1ffb700 10 client.17058 handle_command_reply: tid=0 2024-05-09T00:21:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1ef064b80 1 client.17058 shutdown 2024-05-09T00:21:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2844794977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 msgr2=0x5625249e9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249e9b00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0005670 tx=0x7fe1d0006000 comp rx=0 tx=0).stop 2024-05-09T00:21:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2844794977 shutdown_connections 2024-05-09T00:21:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5625249f8f70 0x5625249fb390 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249e9b00 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248624a0 0x5625249e9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1c1ffb700 0 client.17058 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/2844794977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248dcd00 0x5625249e95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2844794977 >> 172.21.15.46:0/2844794977 conn(0x562524865f70 msgr2=0x5625249d3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2844794977 shutdown_connections 2024-05-09T00:21:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2844794977 wait complete. 2024-05-09T00:21:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.924+0000 7fe1ef064b80 20 client.17058 trim_cache size 0 max 16384 2024-05-09T00:21:43.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1ef064b80 1 Processor -- start 2024-05-09T00:21:43.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1ef064b80 1 -- start start 2024-05-09T00:21:43.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1ef064b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625248624a0 0x5625249d4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1ef064b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 0x5625249e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1ef064b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625249df400 0x5625249f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1ef064b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81b1580 con 0x5625248624a0 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1ef064b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81aee10 con 0x5625249df400 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1ef064b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81acc60 con 0x5625248dcd00 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1de3ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625248624a0 0x5625249d4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1de3ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625248624a0 0x5625249d4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45930/0 (socket says 172.21.15.46:45930) 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1de3ea700 1 -- 172.21.15.46:0/3428291490 learned_addr learned my addr 172.21.15.46:0/3428291490 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1ddbe9700 1 --2- 172.21.15.46:0/3428291490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 0x5625249e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1debeb700 1 --2- 172.21.15.46:0/3428291490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625249df400 0x5625249f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1de3ea700 1 -- 172.21.15.46:0/3428291490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 msgr2=0x5625249e7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/3428291490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 0x5625249e7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1de3ea700 1 -- 172.21.15.46:0/3428291490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625249df400 msgr2=0x5625249f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:43.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/3428291490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625249df400 0x5625249f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1de3ea700 1 -- 172.21.15.46:0/3428291490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625249ea4e0 con 0x5625248624a0 2024-05-09T00:21:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.945+0000 7fe1debeb700 1 --2- 172.21.15.46:0/3428291490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625249df400 0x5625249f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.946+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/3428291490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625248624a0 0x5625249d4a20 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40a5740 tx=0x7fe1d4091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4f68e291a348f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.946+0000 7fe1c3fff700 1 -- 172.21.15.46:0/3428291490 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d40b2520 con 0x5625248624a0 2024-05-09T00:21:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.946+0000 7fe1c3fff700 1 -- 172.21.15.46:0/3428291490 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d406b030 con 0x5625248624a0 2024-05-09T00:21:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.946+0000 7fe1c3fff700 1 -- 172.21.15.46:0/3428291490 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4004ce0 con 0x5625248624a0 2024-05-09T00:21:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.946+0000 7fe1ef064b80 1 -- 172.21.15.46:0/3428291490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625248624a0 msgr2=0x5625249d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.946+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/3428291490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625248624a0 0x5625249d4a20 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40a5740 tx=0x7fe1d4091040 comp rx=0 tx=0).stop 2024-05-09T00:21:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.946+0000 7fe1ef064b80 1 -- 172.21.15.46:0/3428291490 shutdown_connections 2024-05-09T00:21:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.946+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/3428291490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625249df400 0x5625249f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.946+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/3428291490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625248624a0 0x5625249d4a20 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.946+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/3428291490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 0x5625249e7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.946+0000 7fe1ef064b80 1 -- 172.21.15.46:0/3428291490 >> 172.21.15.46:0/3428291490 conn(0x562524865f70 msgr2=0x5625249de3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:43.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.947+0000 7fe1ef064b80 1 -- 172.21.15.46:0/3428291490 shutdown_connections 2024-05-09T00:21:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.947+0000 7fe1ef064b80 1 -- 172.21.15.46:0/3428291490 wait complete. 2024-05-09T00:21:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.947+0000 7fe1ef064b80 1 -- start start 2024-05-09T00:21:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.947+0000 7fe1ef064b80 1 Processor -- start 2024-05-09T00:21:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.947+0000 7fe1ef064b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248624a0 0x5625249fc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.947+0000 7fe1ef064b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 0x5625249fc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.947+0000 7fe1ef064b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.947+0000 7fe1ef064b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81ad2d0 con 0x5625249df400 2024-05-09T00:21:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.947+0000 7fe1ef064b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81ae740 con 0x5625248624a0 2024-05-09T00:21:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.947+0000 7fe1ef064b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81ad9a0 con 0x5625248dcd00 2024-05-09T00:21:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.947+0000 7fe1ef064b80 10 client.0 _ll_register_callbacks cb 0x5625247736c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.947+0000 7fe1debeb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1debeb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45942/0 (socket says 172.21.15.46:45942) 2024-05-09T00:21:43.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1debeb700 1 -- 172.21.15.46:0/2939325056 learned_addr learned my addr 172.21.15.46:0/2939325056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:43.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248624a0 0x5625249fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1ddbe9700 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 0x5625249fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1debeb700 1 -- 172.21.15.46:0/2939325056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 msgr2=0x5625249fc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:43.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1debeb700 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 0x5625249fc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1debeb700 1 -- 172.21.15.46:0/2939325056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248624a0 msgr2=0x5625249fc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:43.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1debeb700 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248624a0 0x5625249fc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1debeb700 1 -- 172.21.15.46:0/2939325056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562524a66ac0 con 0x5625249df400 2024-05-09T00:21:43.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1debeb700 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249fcc80 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0002230 tx=0x7fe1d00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca95a93125025ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:43.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:43.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1c1ffb700 1 -- 172.21.15.46:0/2939325056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d002b070 con 0x5625249df400 2024-05-09T00:21:43.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2939325056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1d406b030 con 0x5625249df400 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1c1ffb700 1 -- 172.21.15.46:0/2939325056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe1d00143e0 con 0x5625249df400 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1c1ffb700 1 -- 172.21.15.46:0/2939325056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe1d003a050 con 0x5625249df400 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1ef064b80 10 client.17064 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.948+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2939325056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562524a67d30 con 0x5625249df400 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.949+0000 7fe1c1ffb700 1 -- 172.21.15.46:0/2939325056 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d406b030 con 0x5625249df400 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.949+0000 7fe1c1ffb700 1 -- 172.21.15.46:0/2939325056 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe1d0033c90 con 0x5625249df400 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.949+0000 7fe1ef064b80 10 client.17064 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.949+0000 7fe1ef064b80 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.949+0000 7fe1ef064b80 20 client.17064 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.949+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562524a68380 0x562524a6a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.949+0000 7fe1ef064b80 4 client.17064 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.949+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2939325056 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562524909b20 con 0x562524a68380 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.949+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562524a68380 0x562524a6a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:43.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.949+0000 7fe1c1ffb700 10 client.17064 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.949+0000 7fe1de3ea700 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562524a68380 0x562524a6a7a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2685491456ed0697 server_cookie=7ee07ec60dc60345 in_seq=0 out_seq=0 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1c1ffb700 1 -- 172.21.15.46:0/2939325056 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562524909b20 con 0x562524a68380 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1c1ffb700 10 client.17064 handle_command_reply: tid=0 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1ef064b80 1 client.17064 shutdown 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2939325056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 msgr2=0x5625249fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249fcc80 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0002230 tx=0x7fe1d00058d0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2939325056 shutdown_connections 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x562524a68380 0x562524a6a7a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5625249df400 0x5625249fcc80 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625248624a0 0x5625249fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1ef064b80 1 --2- 172.21.15.46:0/2939325056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5625248dcd00 0x5625249fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2939325056 >> 172.21.15.46:0/2939325056 conn(0x562524865f70 msgr2=0x5625249d4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:43.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1c1ffb700 0 client.17064 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:43.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2939325056 shutdown_connections 2024-05-09T00:21:43.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1ef064b80 1 -- 172.21.15.46:0/2939325056 wait complete. 2024-05-09T00:21:43.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.950+0000 7fe1ef064b80 20 client.17064 trim_cache size 0 max 16384 2024-05-09T00:21:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:21:43.957 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:21:43.957 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:21:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:21:43.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.951+0000 7fe1e064e700 1 -- 172.21.15.46:0/2027238180 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe1c40b2880 msgr2=0x7fe1c40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:43.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.951+0000 7fe1e064e700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe1c40b2880 0x7fe1c40b4d90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8006010 tx=0x7fe1c801a040 comp rx=0 tx=0).stop 2024-05-09T00:21:43.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.951+0000 7fe1e064e700 1 -- 172.21.15.46:0/2027238180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1d81034c0 msgr2=0x7fe1d819f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:43.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.951+0000 7fe1e064e700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1d81034c0 0x7fe1d819f9c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400d980 tx=0x7fe1d400dc90 comp rx=0 tx=0).stop 2024-05-09T00:21:43.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.951+0000 7fe1e064e700 1 -- 172.21.15.46:0/2027238180 shutdown_connections 2024-05-09T00:21:43.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.951+0000 7fe1e064e700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe1c40b2880 0x7fe1c40b4d90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.951+0000 7fe1e064e700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1d8108740 0x7fe1d8199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.951+0000 7fe1e064e700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1d81034c0 0x7fe1d819f9c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.951+0000 7fe1e064e700 1 --2- 172.21.15.46:0/2027238180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d8107df0 0x7fe1d819ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:43.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.951+0000 7fe1e064e700 1 -- 172.21.15.46:0/2027238180 >> 172.21.15.46:0/2027238180 conn(0x7fe1d80fedf0 msgr2=0x7fe1d81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:43.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.951+0000 7fe1e064e700 1 -- 172.21.15.46:0/2027238180 shutdown_connections 2024-05-09T00:21:43.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:43.951+0000 7fe1e064e700 1 -- 172.21.15.46:0/2027238180 wait complete. 2024-05-09T00:21:44.712 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:44 smithi046 ceph-mon[30151]: pgmap v687: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:44.741 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:21:44.741 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:21:44.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:44 smithi169 ceph-mon[34777]: pgmap v687: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:44.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:44 smithi142 ceph-mon[34533]: pgmap v687: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:21:46.751 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:46 smithi046 ceph-mon[30151]: pgmap v688: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:46.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:46 smithi169 ceph-mon[34777]: pgmap v688: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:46.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:46 smithi142 ceph-mon[34533]: pgmap v688: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.019+0000 7f5fb9c25700 1 -- 172.21.15.46:0/3458370754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb41045e0 msgr2=0x7f5fb41049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.019+0000 7f5fb9c25700 1 --2- 172.21.15.46:0/3458370754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb41045e0 0x7f5fb41049f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40099c0 tx=0x7f5fa4009cd0 comp rx=0 tx=0).stop 2024-05-09T00:21:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.019+0000 7f5fb9c25700 1 -- 172.21.15.46:0/3458370754 shutdown_connections 2024-05-09T00:21:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.019+0000 7f5fb9c25700 1 --2- 172.21.15.46:0/3458370754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fb4109860 0x7f5fb41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.019+0000 7f5fb9c25700 1 --2- 172.21.15.46:0/3458370754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb41045e0 0x7f5fb41049f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.019+0000 7f5fb9c25700 1 --2- 172.21.15.46:0/3458370754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb4108f10 0x7f5fb4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.019+0000 7f5fb9c25700 1 -- 172.21.15.46:0/3458370754 >> 172.21.15.46:0/3458370754 conn(0x7f5fb40ffef0 msgr2=0x7f5fb4102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.019+0000 7f5fb9c25700 1 -- 172.21.15.46:0/3458370754 shutdown_connections 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.019+0000 7f5fb9c25700 1 -- 172.21.15.46:0/3458370754 wait complete. 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.020+0000 7f5fb9c25700 1 Processor -- start 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.020+0000 7f5fb9c25700 1 -- start start 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.020+0000 7f5fb9c25700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fb41045e0 0x7f5fb4076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.020+0000 7f5fb9c25700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4108f10 0x7f5fb4077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.020+0000 7f5fb9c25700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb4109860 0x7f5fb4077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.020+0000 7f5fb9c25700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41144e0 con 0x7f5fb4108f10 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.020+0000 7f5fb9c25700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb4114360 con 0x7f5fb4109860 2024-05-09T00:21:47.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.020+0000 7f5fb9c25700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb4114660 con 0x7f5fb41045e0 2024-05-09T00:21:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.020+0000 7f5fb2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4108f10 0x7f5fb4077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.020+0000 7f5fb2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4108f10 0x7f5fb4077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58508/0 (socket says 172.21.15.46:58508) 2024-05-09T00:21:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.020+0000 7f5fb2ffd700 1 -- 172.21.15.46:0/2496264692 learned_addr learned my addr 172.21.15.46:0/2496264692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb3fff700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb4109860 0x7f5fb4077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb37fe700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fb41045e0 0x7f5fb4076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb2ffd700 1 -- 172.21.15.46:0/2496264692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fb41045e0 msgr2=0x7f5fb4076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb2ffd700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fb41045e0 0x7f5fb4076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb2ffd700 1 -- 172.21.15.46:0/2496264692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb4109860 msgr2=0x7f5fb4077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb2ffd700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb4109860 0x7f5fb4077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb2ffd700 1 -- 172.21.15.46:0/2496264692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb41a91a0 con 0x7f5fb4108f10 2024-05-09T00:21:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb37fe700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fb41045e0 0x7f5fb4076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb2ffd700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4108f10 0x7f5fb4077110 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800c8f0 tx=0x7f5fa800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb0ff9700 1 -- 172.21.15.46:0/2496264692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8007940 con 0x7f5fb4108f10 2024-05-09T00:21:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb9c25700 1 -- 172.21.15.46:0/2496264692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb41a9490 con 0x7f5fb4108f10 2024-05-09T00:21:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb0ff9700 1 -- 172.21.15.46:0/2496264692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa800f450 con 0x7f5fb4108f10 2024-05-09T00:21:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb0ff9700 1 -- 172.21.15.46:0/2496264692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8018660 con 0x7f5fb4108f10 2024-05-09T00:21:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.021+0000 7f5fb9c25700 1 -- 172.21.15.46:0/2496264692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb41a99d0 con 0x7f5fb4108f10 2024-05-09T00:21:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.023+0000 7f5fb9c25700 1 -- 172.21.15.46:0/2496264692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb40512c0 con 0x7f5fb4108f10 2024-05-09T00:21:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.025+0000 7f5fb0ff9700 1 -- 172.21.15.46:0/2496264692 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5fa800f890 con 0x7f5fb4108f10 2024-05-09T00:21:47.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.026+0000 7f5fb0ff9700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5f9c0b2880 0x7f5f9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:47.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.026+0000 7f5fb37fe700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5f9c0b2880 0x7f5f9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:47.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.026+0000 7f5fb0ff9700 1 -- 172.21.15.46:0/2496264692 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f5fa80e21a0 con 0x7f5fb4108f10 2024-05-09T00:21:47.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.027+0000 7f5fb37fe700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5f9c0b2880 0x7f5f9c0b4d90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009750 tx=0x7f5fa401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:47.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.030+0000 7f5fb0ff9700 1 -- 172.21.15.46:0/2496264692 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5fa8014030 con 0x7f5fb4108f10 2024-05-09T00:21:47.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.295+0000 7f5fb9c25700 1 -- 172.21.15.46:0/2496264692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fb4078100 con 0x7f5fb4108f10 2024-05-09T00:21:47.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.296+0000 7f5fb0ff9700 1 -- 172.21.15.46:0/2496264692 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f5fa80a9f50 con 0x7f5fb4108f10 2024-05-09T00:21:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:21:47.297 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:21:47.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.298+0000 7f5fb9c25700 1 -- 172.21.15.46:0/2496264692 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5f9c0b2880 msgr2=0x7f5f9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:47.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.298+0000 7f5fb9c25700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5f9c0b2880 0x7f5f9c0b4d90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009750 tx=0x7f5fa401a040 comp rx=0 tx=0).stop 2024-05-09T00:21:47.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.298+0000 7f5fb9c25700 1 -- 172.21.15.46:0/2496264692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4108f10 msgr2=0x7f5fb4077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:47.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.298+0000 7f5fb9c25700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4108f10 0x7f5fb4077110 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800c8f0 tx=0x7f5fa800cc00 comp rx=0 tx=0).stop 2024-05-09T00:21:47.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.298+0000 7f5fb9c25700 1 -- 172.21.15.46:0/2496264692 shutdown_connections 2024-05-09T00:21:47.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.298+0000 7f5fb9c25700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f5f9c0b2880 0x7f5f9c0b4d90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:47.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.298+0000 7f5fb9c25700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5fb4109860 0x7f5fb4077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:47.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.298+0000 7f5fb9c25700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fb41045e0 0x7f5fb4076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:47.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.298+0000 7f5fb9c25700 1 --2- 172.21.15.46:0/2496264692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4108f10 0x7f5fb4077110 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:47.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.298+0000 7f5fb9c25700 1 -- 172.21.15.46:0/2496264692 >> 172.21.15.46:0/2496264692 conn(0x7f5fb40ffef0 msgr2=0x7f5fb410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:47.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.299+0000 7f5fb9c25700 1 -- 172.21.15.46:0/2496264692 shutdown_connections 2024-05-09T00:21:47.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:47.299+0000 7f5fb9c25700 1 -- 172.21.15.46:0/2496264692 wait complete. 2024-05-09T00:21:47.300 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:21:47.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:47 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2496264692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:21:47.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:47 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2496264692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:21:48.006 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:47 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2496264692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:21:48.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:21:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:21:48.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:48 smithi046 ceph-mon[30151]: pgmap v689: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:21:48.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:48 smithi169 ceph-mon[34777]: pgmap v689: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:21:48.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:48 smithi142 ceph-mon[34533]: pgmap v689: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-05-09T00:21:50.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.114+0000 7fdc782b1700 1 -- 172.21.15.46:0/3161829660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70108f00 msgr2=0x7fdc70109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:50.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.114+0000 7fdc782b1700 1 --2- 172.21.15.46:0/3161829660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70108f00 0x7fdc70109310 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fdc600099c0 tx=0x7fdc60009cd0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.114+0000 7fdc782b1700 1 -- 172.21.15.46:0/3161829660 shutdown_connections 2024-05-09T00:21:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.114+0000 7fdc782b1700 1 --2- 172.21.15.46:0/3161829660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc70109850 0x7fdc701111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.114+0000 7fdc782b1700 1 --2- 172.21.15.46:0/3161829660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc701045d0 0x7fdc701049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.114+0000 7fdc782b1700 1 --2- 172.21.15.46:0/3161829660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70108f00 0x7fdc70109310 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.114+0000 7fdc782b1700 1 -- 172.21.15.46:0/3161829660 >> 172.21.15.46:0/3161829660 conn(0x7fdc700ffea0 msgr2=0x7fdc701022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.114+0000 7fdc782b1700 1 -- 172.21.15.46:0/3161829660 shutdown_connections 2024-05-09T00:21:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.114+0000 7fdc782b1700 1 -- 172.21.15.46:0/3161829660 wait complete. 2024-05-09T00:21:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.115+0000 7fdc782b1700 1 Processor -- start 2024-05-09T00:21:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.115+0000 7fdc782b1700 1 -- start start 2024-05-09T00:21:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.115+0000 7fdc782b1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc701045d0 0x7fdc7019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.115+0000 7fdc782b1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc70108f00 0x7fdc7019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.115+0000 7fdc782b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70109850 0x7fdc7019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.115+0000 7fdc782b1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70114480 con 0x7fdc70109850 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.115+0000 7fdc782b1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70114300 con 0x7fdc701045d0 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.115+0000 7fdc782b1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70114600 con 0x7fdc70108f00 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc7684e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70109850 0x7fdc7019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc7684e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70109850 0x7fdc7019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58526/0 (socket says 172.21.15.46:58526) 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc7684e700 1 -- 172.21.15.46:0/3609651817 learned_addr learned my addr 172.21.15.46:0/3609651817 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc7604d700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc701045d0 0x7fdc7019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc7584c700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc70108f00 0x7fdc7019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc7684e700 1 -- 172.21.15.46:0/3609651817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc70108f00 msgr2=0x7fdc7019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc7684e700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc70108f00 0x7fdc7019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc7684e700 1 -- 172.21.15.46:0/3609651817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc701045d0 msgr2=0x7fdc7019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc7684e700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc701045d0 0x7fdc7019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc7684e700 1 -- 172.21.15.46:0/3609651817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7019fca0 con 0x7fdc70109850 2024-05-09T00:21:50.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc7684e700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70109850 0x7fdc7019f600 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fdc68007ee0 tx=0x7fdc6800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:50.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc677fe700 1 -- 172.21.15.46:0/3609651817 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68004bb0 con 0x7fdc70109850 2024-05-09T00:21:50.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc677fe700 1 -- 172.21.15.46:0/3609651817 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc68004d50 con 0x7fdc70109850 2024-05-09T00:21:50.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc782b1700 1 -- 172.21.15.46:0/3609651817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7019ff90 con 0x7fdc70109850 2024-05-09T00:21:50.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.116+0000 7fdc677fe700 1 -- 172.21.15.46:0/3609651817 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68005920 con 0x7fdc70109850 2024-05-09T00:21:50.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.117+0000 7fdc782b1700 1 -- 172.21.15.46:0/3609651817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc701a78b0 con 0x7fdc70109850 2024-05-09T00:21:50.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.119+0000 7fdc677fe700 1 -- 172.21.15.46:0/3609651817 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdc6801a020 con 0x7fdc70109850 2024-05-09T00:21:50.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.120+0000 7fdc677fe700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdc5c0b2900 0x7fdc5c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.120+0000 7fdc7604d700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdc5c0b2900 0x7fdc5c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.120+0000 7fdc677fe700 1 -- 172.21.15.46:0/3609651817 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fdc680e1800 con 0x7fdc70109850 2024-05-09T00:21:50.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.120+0000 7fdc86cc7b80 1 Processor -- start 2024-05-09T00:21:50.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc7604d700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdc5c0b2900 0x7fdc5c0b4e10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c004630 tx=0x7fdc6c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:50.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc86cc7b80 1 -- start start 2024-05-09T00:21:50.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc86cc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 0x55823fc6e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc86cc7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823faeb4a0 0x55823fc6e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc86cc7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823fc68400 0x55823fc68810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc86cc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55823fb8b0a0 con 0x55823fb65d00 2024-05-09T00:21:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc86cc7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55823fb309a0 con 0x55823fc68400 2024-05-09T00:21:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc86cc7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55823fb87060 con 0x55823faeb4a0 2024-05-09T00:21:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc7604d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 0x55823fc6e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc7604d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 0x55823fc6e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58534/0 (socket says 172.21.15.46:58534) 2024-05-09T00:21:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc7604d700 1 -- 172.21.15.46:0/4139852102 learned_addr learned my addr 172.21.15.46:0/4139852102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc7684e700 1 --2- 172.21.15.46:0/4139852102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823fc68400 0x55823fc68810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc7584c700 1 --2- 172.21.15.46:0/4139852102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823faeb4a0 0x55823fc6e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc7604d700 1 -- 172.21.15.46:0/4139852102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823faeb4a0 msgr2=0x55823fc6e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc7604d700 1 --2- 172.21.15.46:0/4139852102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823faeb4a0 0x55823fc6e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc7604d700 1 -- 172.21.15.46:0/4139852102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823fc68400 msgr2=0x55823fc68810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:50.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc7604d700 1 --2- 172.21.15.46:0/4139852102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823fc68400 0x55823fc68810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.121+0000 7fdc7604d700 1 -- 172.21.15.46:0/4139852102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55823fb21500 con 0x55823fb65d00 2024-05-09T00:21:50.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.122+0000 7fdc7604d700 1 --2- 172.21.15.46:0/4139852102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 0x55823fc6e240 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c003b10 tx=0x7fdc6c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f734106f199bfb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:50.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.122+0000 7fdc64d47700 1 -- 172.21.15.46:0/4139852102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0118d0 con 0x55823fb65d00 2024-05-09T00:21:50.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.122+0000 7fdc64d47700 1 -- 172.21.15.46:0/4139852102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc6c01e420 con 0x55823fb65d00 2024-05-09T00:21:50.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.122+0000 7fdc64d47700 1 -- 172.21.15.46:0/4139852102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c01c5b0 con 0x55823fb65d00 2024-05-09T00:21:50.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.123+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/4139852102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 msgr2=0x55823fc6e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:50.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.123+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/4139852102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 0x55823fc6e240 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c003b10 tx=0x7fdc6c013000 comp rx=0 tx=0).stop 2024-05-09T00:21:50.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.123+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/4139852102 shutdown_connections 2024-05-09T00:21:50.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.123+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/4139852102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823fc68400 0x55823fc68810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.123+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/4139852102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 0x55823fc6e240 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.123+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/4139852102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823faeb4a0 0x55823fc6e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.123+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/4139852102 >> 172.21.15.46:0/4139852102 conn(0x55823faeef70 msgr2=0x55823fc63220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:50.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.123+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/4139852102 shutdown_connections 2024-05-09T00:21:50.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.123+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/4139852102 wait complete. 2024-05-09T00:21:50.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.123+0000 7fdc86cc7b80 1 -- start start 2024-05-09T00:21:50.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.123+0000 7fdc86cc7b80 1 Processor -- start 2024-05-09T00:21:50.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc86cc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823faeb4a0 0x55823fc71ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc86cc7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823fb65d00 0x55823fc72530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc86cc7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fc68400 0x55823fc72a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc86cc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55823fc69160 con 0x55823faeb4a0 2024-05-09T00:21:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc86cc7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55823fb25b20 con 0x55823fb65d00 2024-05-09T00:21:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc86cc7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55823fc75020 con 0x55823fc68400 2024-05-09T00:21:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc86cc7b80 10 client.0 _ll_register_callbacks cb 0x55823f9fc6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc7604d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823faeb4a0 0x55823fc71ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc7604d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823faeb4a0 0x55823fc71ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58548/0 (socket says 172.21.15.46:58548) 2024-05-09T00:21:50.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc7604d700 1 -- 172.21.15.46:0/80349674 learned_addr learned my addr 172.21.15.46:0/80349674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc7584c700 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823fb65d00 0x55823fc72530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc7684e700 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fc68400 0x55823fc72a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc7604d700 1 -- 172.21.15.46:0/80349674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fc68400 msgr2=0x55823fc72a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc7604d700 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fc68400 0x55823fc72a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc7604d700 1 -- 172.21.15.46:0/80349674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823fb65d00 msgr2=0x55823fc72530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc7604d700 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823fb65d00 0x55823fc72530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc7604d700 1 -- 172.21.15.46:0/80349674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823fa06490 con 0x55823faeb4a0 2024-05-09T00:21:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc7604d700 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823faeb4a0 0x55823fc71ff0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c01ede0 tx=0x7fdc6c01ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315a605783712e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.124+0000 7fdc59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc59ffb700 1 -- 172.21.15.46:0/80349674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c035060 con 0x55823faeb4a0 2024-05-09T00:21:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc59ffb700 1 -- 172.21.15.46:0/80349674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fdc6c01e8e0 con 0x55823faeb4a0 2024-05-09T00:21:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/80349674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc6c01e420 con 0x55823faeb4a0 2024-05-09T00:21:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc59ffb700 1 -- 172.21.15.46:0/80349674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc6c0118d0 con 0x55823faeb4a0 2024-05-09T00:21:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc86cc7b80 10 client.17082 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/80349674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55823fc7f970 con 0x55823faeb4a0 2024-05-09T00:21:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc59ffb700 1 -- 172.21.15.46:0/80349674 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc6c01e420 con 0x55823faeb4a0 2024-05-09T00:21:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc59ffb700 1 -- 172.21.15.46:0/80349674 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdc6c03a070 con 0x55823faeb4a0 2024-05-09T00:21:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc86cc7b80 10 client.17082 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc86cc7b80 10 client.17082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:21:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc86cc7b80 20 client.17082 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55823fc81f50 0x55823fc84370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc86cc7b80 4 client.17082 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:21:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.125+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/80349674 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55823fb98a80 con 0x55823fc81f50 2024-05-09T00:21:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.126+0000 7fdc7584c700 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55823fc81f50 0x55823fc84370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.126+0000 7fdc7584c700 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55823fc81f50 0x55823fc84370 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b7348adad84cc94 server_cookie=e4f4cd5ef3deb773 in_seq=0 out_seq=0 2024-05-09T00:21:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.126+0000 7fdc59ffb700 10 client.17082 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:21:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc59ffb700 1 -- 172.21.15.46:0/80349674 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55823fb98a80 con 0x55823fc81f50 2024-05-09T00:21:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc59ffb700 10 client.17082 handle_command_reply: tid=0 2024-05-09T00:21:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc86cc7b80 1 client.17082 shutdown 2024-05-09T00:21:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/80349674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823faeb4a0 msgr2=0x55823fc71ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823faeb4a0 0x55823fc71ff0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c01ede0 tx=0x7fdc6c01ebb0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/80349674 shutdown_connections 2024-05-09T00:21:50.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55823fc81f50 0x55823fc84370 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fc68400 0x55823fc72a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823faeb4a0 0x55823fc71ff0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/80349674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823fb65d00 0x55823fc72530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/80349674 >> 172.21.15.46:0/80349674 conn(0x55823faeef70 msgr2=0x55823fc5cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:50.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc59ffb700 0 client.17082 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:21:50.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/80349674 shutdown_connections 2024-05-09T00:21:50.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/80349674 wait complete. 2024-05-09T00:21:50.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.127+0000 7fdc86cc7b80 20 client.17082 trim_cache size 0 max 16384 2024-05-09T00:21:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.142+0000 7fdc86cc7b80 1 Processor -- start 2024-05-09T00:21:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.142+0000 7fdc86cc7b80 1 -- start start 2024-05-09T00:21:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc86cc7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823faeb4a0 0x55823fc6d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc86cc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 0x55823fc63330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc86cc7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fc68400 0x55823fc7e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc86cc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701b14a0 con 0x55823fb65d00 2024-05-09T00:21:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc86cc7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701aed30 con 0x55823faeb4a0 2024-05-09T00:21:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc86cc7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701acba0 con 0x55823fc68400 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc7584c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 0x55823fc63330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc7584c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 0x55823fc63330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58552/0 (socket says 172.21.15.46:58552) 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc7584c700 1 -- 172.21.15.46:0/1048330006 learned_addr learned my addr 172.21.15.46:0/1048330006 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc7604d700 1 --2- 172.21.15.46:0/1048330006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823faeb4a0 0x55823fc6d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc7584c700 1 -- 172.21.15.46:0/1048330006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fc68400 msgr2=0x55823fc7e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc7684e700 1 --2- 172.21.15.46:0/1048330006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fc68400 0x55823fc7e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc7584c700 1 --2- 172.21.15.46:0/1048330006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fc68400 0x55823fc7e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc7584c700 1 -- 172.21.15.46:0/1048330006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823faeb4a0 msgr2=0x55823fc6d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc7584c700 1 --2- 172.21.15.46:0/1048330006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823faeb4a0 0x55823fc6d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc7584c700 1 -- 172.21.15.46:0/1048330006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55823fc74840 con 0x55823fb65d00 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc7604d700 1 --2- 172.21.15.46:0/1048330006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823faeb4a0 0x55823fc6d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.143+0000 7fdc7584c700 1 --2- 172.21.15.46:0/1048330006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 0x55823fc63330 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009990 tx=0x7fdc60005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8314359ebb49ed50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.144+0000 7fdc5bfff700 1 -- 172.21.15.46:0/1048330006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60004ca0 con 0x55823fb65d00 2024-05-09T00:21:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.144+0000 7fdc5bfff700 1 -- 172.21.15.46:0/1048330006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc60004e40 con 0x55823fb65d00 2024-05-09T00:21:50.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.144+0000 7fdc5bfff700 1 -- 172.21.15.46:0/1048330006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000be60 con 0x55823fb65d00 2024-05-09T00:21:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.144+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/1048330006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 msgr2=0x55823fc63330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.144+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/1048330006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 0x55823fc63330 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009990 tx=0x7fdc60005040 comp rx=0 tx=0).stop 2024-05-09T00:21:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.144+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/1048330006 shutdown_connections 2024-05-09T00:21:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.144+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/1048330006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fc68400 0x55823fc7e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.144+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/1048330006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823faeb4a0 0x55823fc6d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.144+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/1048330006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fb65d00 0x55823fc63330 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.144+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/1048330006 >> 172.21.15.46:0/1048330006 conn(0x55823faeef70 msgr2=0x55823fc5cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.145+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/1048330006 shutdown_connections 2024-05-09T00:21:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.145+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/1048330006 wait complete. 2024-05-09T00:21:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.145+0000 7fdc86cc7b80 1 -- start start 2024-05-09T00:21:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.145+0000 7fdc86cc7b80 1 Processor -- start 2024-05-09T00:21:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.145+0000 7fdc86cc7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823faeb4a0 0x55823fc84290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.145+0000 7fdc86cc7b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fb65d00 0x55823fc847d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.145+0000 7fdc86cc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fc68400 0x55823fcea490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.145+0000 7fdc86cc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ad1f0 con 0x55823fc68400 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.145+0000 7fdc86cc7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ae660 con 0x55823faeb4a0 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.145+0000 7fdc86cc7b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ad8c0 con 0x55823fb65d00 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.145+0000 7fdc86cc7b80 10 client.0 _ll_register_callbacks cb 0x55823f9fc6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.145+0000 7fdc7684e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fc68400 0x55823fcea490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc7684e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fc68400 0x55823fcea490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58564/0 (socket says 172.21.15.46:58564) 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc7604d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823faeb4a0 0x55823fc84290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc7684e700 1 -- 172.21.15.46:0/846748281 learned_addr learned my addr 172.21.15.46:0/846748281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc7584c700 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fb65d00 0x55823fc847d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc7684e700 1 -- 172.21.15.46:0/846748281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fb65d00 msgr2=0x55823fc847d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc7684e700 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fb65d00 0x55823fc847d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc7684e700 1 -- 172.21.15.46:0/846748281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823faeb4a0 msgr2=0x55823fc84290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc7684e700 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823faeb4a0 0x55823fc84290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc7684e700 1 -- 172.21.15.46:0/846748281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823fcea9d0 con 0x55823fc68400 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc7604d700 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823faeb4a0 0x55823fc84290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc7684e700 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fc68400 0x55823fcea490 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fdc680a4e10 tx=0x7fdc68005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1f62331a532d53 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc59ffb700 1 -- 172.21.15.46:0/846748281 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc680a88f0 con 0x55823fc68400 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc59ffb700 1 -- 172.21.15.46:0/846748281 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fdc680b19c0 con 0x55823fc68400 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/846748281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc60004e40 con 0x55823fc68400 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc59ffb700 1 -- 172.21.15.46:0/846748281 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc680a94e0 con 0x55823fc68400 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc86cc7b80 10 client.17088 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.146+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/846748281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55823fcebb20 con 0x55823fc68400 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.147+0000 7fdc59ffb700 1 -- 172.21.15.46:0/846748281 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc60004e40 con 0x55823fc68400 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.147+0000 7fdc59ffb700 1 -- 172.21.15.46:0/846748281 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdc680a4750 con 0x55823fc68400 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.147+0000 7fdc86cc7b80 10 client.17088 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.147+0000 7fdc86cc7b80 10 client.17088 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.147+0000 7fdc86cc7b80 20 client.17088 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.147+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55823fcec1d0 0x55823fcee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.147+0000 7fdc86cc7b80 4 client.17088 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.147+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/846748281 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55823fb92b20 con 0x55823fcec1d0 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.147+0000 7fdc7604d700 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55823fcec1d0 0x55823fcee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc59ffb700 10 client.17088 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc7604d700 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55823fcec1d0 0x55823fcee5f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=acd51bf982ed8118 server_cookie=7d5945646a52455f in_seq=0 out_seq=0 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc59ffb700 1 -- 172.21.15.46:0/846748281 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55823fb92b20 con 0x55823fcec1d0 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc59ffb700 10 client.17088 handle_command_reply: tid=0 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc86cc7b80 1 client.17088 shutdown 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/846748281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fc68400 msgr2=0x55823fcea490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fc68400 0x55823fcea490 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fdc680a4e10 tx=0x7fdc68005240 comp rx=0 tx=0).stop 2024-05-09T00:21:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/846748281 shutdown_connections 2024-05-09T00:21:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55823fcec1d0 0x55823fcee5f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55823fc68400 0x55823fcea490 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc59ffb700 0 client.17088 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:21:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55823faeb4a0 0x55823fc84290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc86cc7b80 1 --2- 172.21.15.46:0/846748281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55823fb65d00 0x55823fc847d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/846748281 >> 172.21.15.46:0/846748281 conn(0x55823faeef70 msgr2=0x55823fc6c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/846748281 shutdown_connections 2024-05-09T00:21:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc86cc7b80 1 -- 172.21.15.46:0/846748281 wait complete. 2024-05-09T00:21:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.148+0000 7fdc86cc7b80 20 client.17088 trim_cache size 0 max 16384 2024-05-09T00:21:50.154 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:21:50.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.149+0000 7fdc782b1700 1 -- 172.21.15.46:0/3609651817 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdc5c0b2900 msgr2=0x7fdc5c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:50.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.149+0000 7fdc782b1700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdc5c0b2900 0x7fdc5c0b4e10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c004630 tx=0x7fdc6c00b3c0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.149+0000 7fdc782b1700 1 -- 172.21.15.46:0/3609651817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70109850 msgr2=0x7fdc7019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:50.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.149+0000 7fdc782b1700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70109850 0x7fdc7019f600 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fdc68007ee0 tx=0x7fdc6800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.149+0000 7fdc782b1700 1 -- 172.21.15.46:0/3609651817 shutdown_connections 2024-05-09T00:21:50.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.149+0000 7fdc782b1700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fdc5c0b2900 0x7fdc5c0b4e10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.149+0000 7fdc782b1700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70109850 0x7fdc7019f600 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.149+0000 7fdc782b1700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc701045d0 0x7fdc7019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.149+0000 7fdc782b1700 1 --2- 172.21.15.46:0/3609651817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc70108f00 0x7fdc7019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:50.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.149+0000 7fdc782b1700 1 -- 172.21.15.46:0/3609651817 >> 172.21.15.46:0/3609651817 conn(0x7fdc700ffea0 msgr2=0x7fdc7010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:50.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.149+0000 7fdc782b1700 1 -- 172.21.15.46:0/3609651817 shutdown_connections 2024-05-09T00:21:50.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:50.149+0000 7fdc782b1700 1 -- 172.21.15.46:0/3609651817 wait complete. 2024-05-09T00:21:50.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:50 smithi046 ceph-mon[30151]: pgmap v690: 97 pgs: 97 active+clean; 6.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:50.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:50 smithi169 ceph-mon[34777]: pgmap v690: 97 pgs: 97 active+clean; 6.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:50.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:50 smithi142 ceph-mon[34533]: pgmap v690: 97 pgs: 97 active+clean; 6.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:50.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:21:51.218 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:21:52.791 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:52 smithi046 ceph-mon[30151]: pgmap v691: 97 pgs: 97 active+clean; 6.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:21:52.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:52 smithi169 ceph-mon[34777]: pgmap v691: 97 pgs: 97 active+clean; 6.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:21:52.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:52 smithi142 ceph-mon[34533]: pgmap v691: 97 pgs: 97 active+clean; 6.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:21:53.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.057+0000 7f267bbc6700 1 -- 172.21.15.46:0/2771198870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2674108740 msgr2=0x7f26741100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.057+0000 7f267bbc6700 1 --2- 172.21.15.46:0/2771198870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2674108740 0x7f26741100a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f266c00b0d0 tx=0x7f266c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.057+0000 7f267bbc6700 1 -- 172.21.15.46:0/2771198870 shutdown_connections 2024-05-09T00:21:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.057+0000 7f267bbc6700 1 --2- 172.21.15.46:0/2771198870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2674108740 0x7f26741100a0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.057+0000 7f267bbc6700 1 --2- 172.21.15.46:0/2771198870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26741034c0 0x7f26741038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.057+0000 7f267bbc6700 1 --2- 172.21.15.46:0/2771198870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2674107df0 0x7f2674108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.057+0000 7f267bbc6700 1 -- 172.21.15.46:0/2771198870 >> 172.21.15.46:0/2771198870 conn(0x7f26740fedf0 msgr2=0x7f2674101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.057+0000 7f267bbc6700 1 -- 172.21.15.46:0/2771198870 shutdown_connections 2024-05-09T00:21:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.057+0000 7f267bbc6700 1 -- 172.21.15.46:0/2771198870 wait complete. 2024-05-09T00:21:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.058+0000 7f267bbc6700 1 Processor -- start 2024-05-09T00:21:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.058+0000 7f267bbc6700 1 -- start start 2024-05-09T00:21:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267bbc6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26741034c0 0x7f267419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267bbc6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2674107df0 0x7f267419c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267bbc6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2674108740 0x7f267419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267bbc6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2674113370 con 0x7f2674108740 2024-05-09T00:21:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267bbc6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26741131f0 con 0x7f26741034c0 2024-05-09T00:21:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267bbc6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26741134f0 con 0x7f2674107df0 2024-05-09T00:21:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267a163700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2674108740 0x7f267419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267a163700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2674108740 0x7f267419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58590/0 (socket says 172.21.15.46:58590) 2024-05-09T00:21:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267a163700 1 -- 172.21.15.46:0/1023939053 learned_addr learned my addr 172.21.15.46:0/1023939053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f2679161700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2674107df0 0x7f267419c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f2679962700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26741034c0 0x7f267419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267a163700 1 -- 172.21.15.46:0/1023939053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2674107df0 msgr2=0x7f267419c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267a163700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2674107df0 0x7f267419c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267a163700 1 -- 172.21.15.46:0/1023939053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26741034c0 msgr2=0x7f267419bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267a163700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26741034c0 0x7f267419bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.059+0000 7f267a163700 1 -- 172.21.15.46:0/1023939053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26741a6f10 con 0x7f2674108740 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.060+0000 7f267a163700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2674108740 0x7f267419c6f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f266c0095c0 tx=0x7f266c007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.060+0000 7f266affd700 1 -- 172.21.15.46:0/1023939053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c01f070 con 0x7f2674108740 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.060+0000 7f266affd700 1 -- 172.21.15.46:0/1023939053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c003c00 con 0x7f2674108740 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.060+0000 7f266affd700 1 -- 172.21.15.46:0/1023939053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c01b6a0 con 0x7f2674108740 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.060+0000 7f267bbc6700 1 -- 172.21.15.46:0/1023939053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26741a71a0 con 0x7f2674108740 2024-05-09T00:21:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.060+0000 7f267bbc6700 1 -- 172.21.15.46:0/1023939053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26741a75a0 con 0x7f2674108740 2024-05-09T00:21:53.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.063+0000 7f266affd700 1 -- 172.21.15.46:0/1023939053 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f266c022070 con 0x7f2674108740 2024-05-09T00:21:53.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.063+0000 7f268a5dcb80 1 Processor -- start 2024-05-09T00:21:53.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f268a5dcb80 1 -- start start 2024-05-09T00:21:53.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f266affd700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f26600b2900 0x7f26600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f2679161700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f26600b2900 0x7f26600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f266affd700 1 -- 172.21.15.46:0/1023939053 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f266c00f020 con 0x7f2674108740 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f268a5dcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edcf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f268a5dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 0x562d7edcfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f268a5dcb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7edc96a0 0x562d7edc9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f268a5dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d7ecebff0 con 0x562d7ec4c4a0 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f268a5dcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562d7ec8a7f0 con 0x562d7ed36160 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f268a5dcb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562d7ece7fb0 con 0x562d7edc96a0 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f2679161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 0x562d7edcfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f2679161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 0x562d7edcfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58604/0 (socket says 172.21.15.46:58604) 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f267a163700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7edc96a0 0x562d7edc9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f2679161700 1 -- 172.21.15.46:0/1222801152 learned_addr learned my addr 172.21.15.46:0/1222801152 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.064+0000 7f2679962700 1 --2- 172.21.15.46:0/1222801152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edcf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.065+0000 7f2679161700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f26600b2900 0x7f26600b4e10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2664012500 tx=0x7f266400a730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:53.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.065+0000 7f2679161700 1 -- 172.21.15.46:0/1222801152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7edc96a0 msgr2=0x562d7edc9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:53.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.065+0000 7f2679161700 1 --2- 172.21.15.46:0/1222801152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7edc96a0 0x562d7edc9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.065+0000 7f2679161700 1 -- 172.21.15.46:0/1222801152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 msgr2=0x562d7edcf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:53.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.065+0000 7f2679161700 1 --2- 172.21.15.46:0/1222801152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edcf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.065+0000 7f2679161700 1 -- 172.21.15.46:0/1222801152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d7ec82370 con 0x562d7ec4c4a0 2024-05-09T00:21:53.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.065+0000 7f2679161700 1 --2- 172.21.15.46:0/1222801152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 0x562d7edcfa30 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2664010620 tx=0x7f266401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c0ead69ff3b872 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:53.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.065+0000 7f265ffff700 1 -- 172.21.15.46:0/1222801152 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664022020 con 0x562d7ec4c4a0 2024-05-09T00:21:53.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.065+0000 7f265ffff700 1 -- 172.21.15.46:0/1222801152 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2664004ce0 con 0x562d7ec4c4a0 2024-05-09T00:21:53.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.065+0000 7f265ffff700 1 -- 172.21.15.46:0/1222801152 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266401e440 con 0x562d7ec4c4a0 2024-05-09T00:21:53.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.066+0000 7f268a5dcb80 1 -- 172.21.15.46:0/1222801152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 msgr2=0x562d7edcfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:53.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.066+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/1222801152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 0x562d7edcfa30 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2664010620 tx=0x7f266401c000 comp rx=0 tx=0).stop 2024-05-09T00:21:53.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.066+0000 7f268a5dcb80 1 -- 172.21.15.46:0/1222801152 shutdown_connections 2024-05-09T00:21:53.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.066+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/1222801152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7edc96a0 0x562d7edc9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.066+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/1222801152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edcf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.066+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/1222801152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 0x562d7edcfa30 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.066+0000 7f268a5dcb80 1 -- 172.21.15.46:0/1222801152 >> 172.21.15.46:0/1222801152 conn(0x562d7ecac6a0 msgr2=0x562d7edc4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.066+0000 7f268a5dcb80 1 -- 172.21.15.46:0/1222801152 shutdown_connections 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.066+0000 7f268a5dcb80 1 -- 172.21.15.46:0/1222801152 wait complete. 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.067+0000 7f268a5dcb80 1 -- start start 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.067+0000 7f268a5dcb80 1 Processor -- start 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.067+0000 7f268a5dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 0x562d7edd3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.067+0000 7f268a5dcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edd3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.067+0000 7f268a5dcb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7edc96a0 0x562d7edd3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.067+0000 7f268a5dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d7eb67ba0 con 0x562d7ec4c4a0 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.067+0000 7f268a5dcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562d7eca6150 con 0x562d7ed36160 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.067+0000 7f268a5dcb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562d7edd61d0 con 0x562d7edc96a0 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.067+0000 7f268a5dcb80 10 client.0 _ll_register_callbacks cb 0x562d7eb5d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.067+0000 7f2679962700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 0x562d7edd3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.067+0000 7f2679962700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 0x562d7edd3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58618/0 (socket says 172.21.15.46:58618) 2024-05-09T00:21:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.067+0000 7f2679962700 1 -- 172.21.15.46:0/4088349029 learned_addr learned my addr 172.21.15.46:0/4088349029 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f2679161700 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edd3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f267a163700 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7edc96a0 0x562d7edd3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f2679962700 1 -- 172.21.15.46:0/4088349029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7edc96a0 msgr2=0x562d7edd3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f2679962700 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7edc96a0 0x562d7edd3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f2679962700 1 -- 172.21.15.46:0/4088349029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 msgr2=0x562d7edd3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f2679962700 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edd3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f2679962700 1 -- 172.21.15.46:0/4088349029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d7eb67490 con 0x562d7ec4c4a0 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f2679161700 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edd3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f2679962700 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 0x562d7edd3200 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2670012500 tx=0x7f26700128c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b60a8ab605a036 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f265d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f265d7fa700 1 -- 172.21.15.46:0/4088349029 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670004220 con 0x562d7ec4c4a0 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f265d7fa700 1 -- 172.21.15.46:0/4088349029 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f26700043c0 con 0x562d7ec4c4a0 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4088349029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2664004ce0 con 0x562d7ec4c4a0 2024-05-09T00:21:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f265d7fa700 1 -- 172.21.15.46:0/4088349029 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2670003680 con 0x562d7ec4c4a0 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f268a5dcb80 10 client.17100 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.068+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4088349029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d7ede0bb0 con 0x562d7ec4c4a0 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.069+0000 7f265d7fa700 1 -- 172.21.15.46:0/4088349029 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2664004ce0 con 0x562d7ec4c4a0 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.069+0000 7f265d7fa700 1 -- 172.21.15.46:0/4088349029 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2670019030 con 0x562d7ec4c4a0 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.069+0000 7f268a5dcb80 10 client.17100 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.069+0000 7f268a5dcb80 10 client.17100 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.069+0000 7f268a5dcb80 20 client.17100 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.069+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d7ede3240 0x562d7ede56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.069+0000 7f268a5dcb80 4 client.17100 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.069+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4088349029 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d7ecf99d0 con 0x562d7ede3240 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.069+0000 7f2679161700 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d7ede3240 0x562d7ede56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.070+0000 7f265d7fa700 10 client.17100 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:21:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.070+0000 7f2679161700 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d7ede3240 0x562d7ede56e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88e85ec37babc6fd server_cookie=7cac7ab9ccaa6fe5 in_seq=0 out_seq=0 2024-05-09T00:21:53.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.070+0000 7f265d7fa700 1 -- 172.21.15.46:0/4088349029 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562d7ecf99d0 con 0x562d7ede3240 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.070+0000 7f265d7fa700 10 client.17100 handle_command_reply: tid=0 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f268a5dcb80 1 client.17100 shutdown 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4088349029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 msgr2=0x562d7edd3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 0x562d7edd3200 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2670012500 tx=0x7f26700128c0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4088349029 shutdown_connections 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d7ede3240 0x562d7ede56e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7edc96a0 0x562d7edd3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ec4c4a0 0x562d7edd3200 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/4088349029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edd3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f265d7fa700 0 client.17100 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4088349029 >> 172.21.15.46:0/4088349029 conn(0x562d7ecac6a0 msgr2=0x562d7edc41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4088349029 shutdown_connections 2024-05-09T00:21:53.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4088349029 wait complete. 2024-05-09T00:21:53.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.071+0000 7f268a5dcb80 20 client.17100 trim_cache size 0 max 16384 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.086+0000 7f268a5dcb80 1 Processor -- start 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.086+0000 7f268a5dcb80 1 -- start start 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.086+0000 7f268a5dcb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7ec4c4a0 0x562d7edd0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.086+0000 7f268a5dcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edc90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.086+0000 7f268a5dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7edc96a0 0x562d7edd3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.086+0000 7f268a5dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26741b1490 con 0x562d7edc96a0 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.086+0000 7f268a5dcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26741aed20 con 0x562d7ed36160 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.086+0000 7f268a5dcb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26741acb50 con 0x562d7ec4c4a0 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f267a163700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7edc96a0 0x562d7edd3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f267a163700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7edc96a0 0x562d7edd3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58624/0 (socket says 172.21.15.46:58624) 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f267a163700 1 -- 172.21.15.46:0/4228860049 learned_addr learned my addr 172.21.15.46:0/4228860049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f2679962700 1 --2- 172.21.15.46:0/4228860049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7ec4c4a0 0x562d7edd0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f267a163700 1 -- 172.21.15.46:0/4228860049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7ec4c4a0 msgr2=0x562d7edd0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f267a163700 1 --2- 172.21.15.46:0/4228860049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7ec4c4a0 0x562d7edd0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f267a163700 1 -- 172.21.15.46:0/4228860049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 msgr2=0x562d7edc90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f2679161700 1 --2- 172.21.15.46:0/4228860049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edc90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f267a163700 1 --2- 172.21.15.46:0/4228860049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edc90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f267a163700 1 -- 172.21.15.46:0/4228860049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d7edbdf60 con 0x562d7edc96a0 2024-05-09T00:21:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f267a163700 1 --2- 172.21.15.46:0/4228860049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7edc96a0 0x562d7edd3ac0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f266c009830 tx=0x7f266c005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105df1744db6bb9e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f265f7fe700 1 -- 172.21.15.46:0/4228860049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c0c4070 con 0x562d7edc96a0 2024-05-09T00:21:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f265f7fe700 1 -- 172.21.15.46:0/4228860049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c0b1040 con 0x562d7edc96a0 2024-05-09T00:21:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.087+0000 7f265f7fe700 1 -- 172.21.15.46:0/4228860049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c0cbe60 con 0x562d7edc96a0 2024-05-09T00:21:53.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.088+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4228860049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7edc96a0 msgr2=0x562d7edd3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:53.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.088+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/4228860049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7edc96a0 0x562d7edd3ac0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f266c009830 tx=0x7f266c005640 comp rx=0 tx=0).stop 2024-05-09T00:21:53.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.088+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4228860049 shutdown_connections 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.088+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/4228860049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7edc96a0 0x562d7edd3ac0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.088+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/4228860049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7ec4c4a0 0x562d7edd0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.088+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/4228860049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7ed36160 0x562d7edc90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.088+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4228860049 >> 172.21.15.46:0/4228860049 conn(0x562d7ecac6a0 msgr2=0x562d7edcd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.088+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4228860049 shutdown_connections 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.088+0000 7f268a5dcb80 1 -- 172.21.15.46:0/4228860049 wait complete. 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.088+0000 7f268a5dcb80 1 -- start start 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f268a5dcb80 1 Processor -- start 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f268a5dcb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7ec4c4a0 0x562d7eddb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f268a5dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ed36160 0x562d7edea2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f268a5dcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7edc96a0 0x562d7edea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f268a5dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26741ad1e0 con 0x562d7ed36160 2024-05-09T00:21:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f268a5dcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26741ae650 con 0x562d7edc96a0 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f268a5dcb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26741ad8b0 con 0x562d7ec4c4a0 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f268a5dcb80 10 client.0 _ll_register_callbacks cb 0x562d7eb5d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f2679161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ed36160 0x562d7edea2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f2679161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ed36160 0x562d7edea2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58632/0 (socket says 172.21.15.46:58632) 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f2679161700 1 -- 172.21.15.46:0/3968508885 learned_addr learned my addr 172.21.15.46:0/3968508885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f2679962700 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7ec4c4a0 0x562d7eddb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.089+0000 7f267a163700 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7edc96a0 0x562d7edea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f2679161700 1 -- 172.21.15.46:0/3968508885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7ec4c4a0 msgr2=0x562d7eddb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f2679161700 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7ec4c4a0 0x562d7eddb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f2679161700 1 -- 172.21.15.46:0/3968508885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7edc96a0 msgr2=0x562d7edea7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f2679161700 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7edc96a0 0x562d7edea7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f2679161700 1 -- 172.21.15.46:0/3968508885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d7edead30 con 0x562d7ed36160 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f2679962700 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7ec4c4a0 0x562d7eddb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:21:53.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f2679161700 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ed36160 0x562d7edea2b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2664010620 tx=0x7f2664005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e3ab12a25db413 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f265d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f265d7fa700 1 -- 172.21.15.46:0/3968508885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664022040 con 0x562d7ed36160 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f265d7fa700 1 -- 172.21.15.46:0/3968508885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f26640056e0 con 0x562d7ed36160 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f268a5dcb80 1 -- 172.21.15.46:0/3968508885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f266c0b1040 con 0x562d7ed36160 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f265d7fa700 1 -- 172.21.15.46:0/3968508885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f266400a2b0 con 0x562d7ed36160 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f268a5dcb80 10 client.17106 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f268a5dcb80 1 -- 172.21.15.46:0/3968508885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d7ee4d720 con 0x562d7ed36160 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.090+0000 7f265d7fa700 1 -- 172.21.15.46:0/3968508885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c0b1040 con 0x562d7ed36160 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.091+0000 7f265d7fa700 1 -- 172.21.15.46:0/3968508885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2664027040 con 0x562d7ed36160 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.091+0000 7f268a5dcb80 10 client.17106 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.091+0000 7f268a5dcb80 10 client.17106 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.091+0000 7f268a5dcb80 20 client.17106 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.091+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d7ee4daf0 0x562d7ee4ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.091+0000 7f268a5dcb80 4 client.17106 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:21:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.091+0000 7f268a5dcb80 1 -- 172.21.15.46:0/3968508885 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d7edced70 con 0x562d7ee4daf0 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.091+0000 7f2679962700 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d7ee4daf0 0x562d7ee4ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.091+0000 7f265d7fa700 10 client.17106 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.091+0000 7f2679962700 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d7ee4daf0 0x562d7ee4ff10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1da7cbadefbb4a5e server_cookie=9b9e72c4908a2578 in_seq=0 out_seq=0 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f265d7fa700 1 -- 172.21.15.46:0/3968508885 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d7edced70 con 0x562d7ee4daf0 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f265d7fa700 10 client.17106 handle_command_reply: tid=0 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f268a5dcb80 1 client.17106 shutdown 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f268a5dcb80 1 -- 172.21.15.46:0/3968508885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ed36160 msgr2=0x562d7edea2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ed36160 0x562d7edea2b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2664010620 tx=0x7f2664005f70 comp rx=0 tx=0).stop 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f268a5dcb80 1 -- 172.21.15.46:0/3968508885 shutdown_connections 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x562d7ee4daf0 0x562d7ee4ff10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562d7edc96a0 0x562d7edea7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d7ec4c4a0 0x562d7eddb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f268a5dcb80 1 --2- 172.21.15.46:0/3968508885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7ed36160 0x562d7edea2b0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f265d7fa700 0 client.17106 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:21:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f268a5dcb80 1 -- 172.21.15.46:0/3968508885 >> 172.21.15.46:0/3968508885 conn(0x562d7ecac6a0 msgr2=0x562d7edd19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f268a5dcb80 1 -- 172.21.15.46:0/3968508885 shutdown_connections 2024-05-09T00:21:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f268a5dcb80 1 -- 172.21.15.46:0/3968508885 wait complete. 2024-05-09T00:21:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.092+0000 7f268a5dcb80 20 client.17106 trim_cache size 0 max 16384 2024-05-09T00:21:53.098 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:21:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.093+0000 7f267bbc6700 1 -- 172.21.15.46:0/1023939053 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f26600b2900 msgr2=0x7f26600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.093+0000 7f267bbc6700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f26600b2900 0x7f26600b4e10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2664012500 tx=0x7f266400a730 comp rx=0 tx=0).stop 2024-05-09T00:21:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.093+0000 7f267bbc6700 1 -- 172.21.15.46:0/1023939053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2674108740 msgr2=0x7f267419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.093+0000 7f267bbc6700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2674108740 0x7f267419c6f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f266c0095c0 tx=0x7f266c007ad0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.093+0000 7f267bbc6700 1 -- 172.21.15.46:0/1023939053 shutdown_connections 2024-05-09T00:21:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.093+0000 7f267bbc6700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f26600b2900 0x7f26600b4e10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.093+0000 7f267bbc6700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2674108740 0x7f267419c6f0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.093+0000 7f267bbc6700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26741034c0 0x7f267419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.093+0000 7f267bbc6700 1 --2- 172.21.15.46:0/1023939053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2674107df0 0x7f267419c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.093+0000 7f267bbc6700 1 -- 172.21.15.46:0/1023939053 >> 172.21.15.46:0/1023939053 conn(0x7f26740fedf0 msgr2=0x7f2674107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:53.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.093+0000 7f267bbc6700 1 -- 172.21.15.46:0/1023939053 shutdown_connections 2024-05-09T00:21:53.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:53.093+0000 7f267bbc6700 1 -- 172.21.15.46:0/1023939053 wait complete. 2024-05-09T00:21:53.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:21:54.065 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:21:54.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:54 smithi046 ceph-mon[30151]: pgmap v692: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:54.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:54 smithi142 ceph-mon[34533]: pgmap v692: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:55.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:54 smithi169 ceph-mon[34777]: pgmap v692: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:21:55.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.891+0000 7f7aa797d700 1 -- 172.21.15.46:0/1239431124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa0108760 msgr2=0x7f7aa01100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:55.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.891+0000 7f7aa797d700 1 --2- 172.21.15.46:0/1239431124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa0108760 0x7f7aa01100c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800b0d0 tx=0x7f7a9800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.891+0000 7f7aa797d700 1 -- 172.21.15.46:0/1239431124 shutdown_connections 2024-05-09T00:21:55.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.891+0000 7f7aa797d700 1 --2- 172.21.15.46:0/1239431124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa0108760 0x7f7aa01100c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.891+0000 7f7aa797d700 1 --2- 172.21.15.46:0/1239431124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7aa01034e0 0x7f7aa01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.891+0000 7f7aa797d700 1 --2- 172.21.15.46:0/1239431124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aa0107e10 0x7f7aa0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.891+0000 7f7aa797d700 1 -- 172.21.15.46:0/1239431124 >> 172.21.15.46:0/1239431124 conn(0x7f7aa00fef30 msgr2=0x7f7aa0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:55.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.891+0000 7f7aa797d700 1 -- 172.21.15.46:0/1239431124 shutdown_connections 2024-05-09T00:21:55.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.891+0000 7f7aa797d700 1 -- 172.21.15.46:0/1239431124 wait complete. 2024-05-09T00:21:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa797d700 1 Processor -- start 2024-05-09T00:21:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa797d700 1 -- start start 2024-05-09T00:21:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa797d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aa01034e0 0x7f7aa019d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa797d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7aa0107e10 0x7f7aa019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa797d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa0108760 0x7f7aa01a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa797d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa0113400 con 0x7f7aa0108760 2024-05-09T00:21:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa797d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa0113280 con 0x7f7aa0107e10 2024-05-09T00:21:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa797d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa0113580 con 0x7f7aa01034e0 2024-05-09T00:21:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa5f1a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa0108760 0x7f7aa01a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa5f1a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa0108760 0x7f7aa01a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58656/0 (socket says 172.21.15.46:58656) 2024-05-09T00:21:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa5f1a700 1 -- 172.21.15.46:0/639602326 learned_addr learned my addr 172.21.15.46:0/639602326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa5719700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aa01034e0 0x7f7aa019d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.892+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7aa0107e10 0x7f7aa019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.893+0000 7f7aa5f1a700 1 -- 172.21.15.46:0/639602326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aa01034e0 msgr2=0x7f7aa019d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.893+0000 7f7aa5f1a700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aa01034e0 0x7f7aa019d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.893+0000 7f7aa5f1a700 1 -- 172.21.15.46:0/639602326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7aa0107e10 msgr2=0x7f7aa019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.893+0000 7f7aa5f1a700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7aa0107e10 0x7f7aa019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.893+0000 7f7aa5f1a700 1 -- 172.21.15.46:0/639602326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa01a1fa0 con 0x7f7aa0108760 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.893+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7aa0107e10 0x7f7aa019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.893+0000 7f7aa5f1a700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa0108760 0x7f7aa01a1900 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7a980095c0 tx=0x7f7a98007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.893+0000 7f7a967fc700 1 -- 172.21.15.46:0/639602326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9801f070 con 0x7f7aa0108760 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.893+0000 7f7aa797d700 1 -- 172.21.15.46:0/639602326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa01a2230 con 0x7f7aa0108760 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.893+0000 7f7a967fc700 1 -- 172.21.15.46:0/639602326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a98003c00 con 0x7f7aa0108760 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.893+0000 7f7aa797d700 1 -- 172.21.15.46:0/639602326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa01a9bb0 con 0x7f7aa0108760 2024-05-09T00:21:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.893+0000 7f7a967fc700 1 -- 172.21.15.46:0/639602326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9801b6a0 con 0x7f7aa0108760 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.896+0000 7f7a967fc700 1 -- 172.21.15.46:0/639602326 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7a98022070 con 0x7f7aa0108760 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.897+0000 7f7a967fc700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7a8c0b2880 0x7f7a8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.897+0000 7f7aa5719700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7a8c0b2880 0x7f7a8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.897+0000 7f7a967fc700 1 -- 172.21.15.46:0/639602326 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7a9800f020 con 0x7f7aa0108760 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.897+0000 7f7ab6393b80 1 Processor -- start 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.897+0000 7f7ab6393b80 1 -- start start 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.897+0000 7f7aa5719700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7a8c0b2880 0x7f7a8c0b4d90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c007be0 tx=0x7f7a9c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.897+0000 7f7ab6393b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d638c5d00 0x558d639ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.897+0000 7f7ab6393b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 0x558d639ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.897+0000 7f7ab6393b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d639c8400 0x558d639c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.897+0000 7f7ab6393b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558d638eb0a0 con 0x558d6384b4a0 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.897+0000 7f7ab6393b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558d638909a0 con 0x558d638c5d00 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.897+0000 7f7ab6393b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558d638e7060 con 0x558d639c8400 2024-05-09T00:21:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7aa4f18700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 0x558d639ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7aa4f18700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 0x558d639ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58668/0 (socket says 172.21.15.46:58668) 2024-05-09T00:21:55.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7aa4f18700 1 -- 172.21.15.46:0/1361703316 learned_addr learned my addr 172.21.15.46:0/1361703316 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:55.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7aa5719700 1 --2- 172.21.15.46:0/1361703316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d638c5d00 0x558d639ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7aa5f1a700 1 --2- 172.21.15.46:0/1361703316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d639c8400 0x558d639c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7aa4f18700 1 -- 172.21.15.46:0/1361703316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d639c8400 msgr2=0x558d639c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:55.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/1361703316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d639c8400 0x558d639c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7aa4f18700 1 -- 172.21.15.46:0/1361703316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d638c5d00 msgr2=0x558d639ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:55.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/1361703316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d638c5d00 0x558d639ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7aa4f18700 1 -- 172.21.15.46:0/1361703316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d63881500 con 0x558d6384b4a0 2024-05-09T00:21:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/1361703316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 0x558d639ce780 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7a90007860 tx=0x7f7a90007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d621743784276aad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7a8b7fe700 1 -- 172.21.15.46:0/1361703316 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900125f0 con 0x558d6384b4a0 2024-05-09T00:21:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7a8b7fe700 1 -- 172.21.15.46:0/1361703316 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a90018070 con 0x558d6384b4a0 2024-05-09T00:21:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.898+0000 7f7a8b7fe700 1 -- 172.21.15.46:0/1361703316 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90013ae0 con 0x558d6384b4a0 2024-05-09T00:21:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.899+0000 7f7ab6393b80 1 -- 172.21.15.46:0/1361703316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 msgr2=0x558d639ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.899+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/1361703316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 0x558d639ce780 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7a90007860 tx=0x7f7a90007b70 comp rx=0 tx=0).stop 2024-05-09T00:21:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.899+0000 7f7ab6393b80 1 -- 172.21.15.46:0/1361703316 shutdown_connections 2024-05-09T00:21:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.899+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/1361703316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d639c8400 0x558d639c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.899+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/1361703316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d638c5d00 0x558d639ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.899+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/1361703316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 0x558d639ce780 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.899+0000 7f7ab6393b80 1 -- 172.21.15.46:0/1361703316 >> 172.21.15.46:0/1361703316 conn(0x558d6384ef70 msgr2=0x558d639c3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.899+0000 7f7ab6393b80 1 -- 172.21.15.46:0/1361703316 shutdown_connections 2024-05-09T00:21:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.900+0000 7f7ab6393b80 1 -- 172.21.15.46:0/1361703316 wait complete. 2024-05-09T00:21:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.900+0000 7f7ab6393b80 1 -- start start 2024-05-09T00:21:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.900+0000 7f7ab6393b80 1 Processor -- start 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.900+0000 7f7ab6393b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d6384b4a0 0x558d639d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.900+0000 7f7ab6393b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d638c5d00 0x558d639d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.900+0000 7f7ab6393b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639c8400 0x558d639d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7ab6393b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558d639c9160 con 0x558d638c5d00 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7ab6393b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558d63885b20 con 0x558d639c8400 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7ab6393b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558d639d5020 con 0x558d6384b4a0 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7ab6393b80 10 client.0 _ll_register_callbacks cb 0x558d6375c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7aa4f18700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d638c5d00 0x558d639d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7aa4f18700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d638c5d00 0x558d639d2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58674/0 (socket says 172.21.15.46:58674) 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7aa4f18700 1 -- 172.21.15.46:0/663603591 learned_addr learned my addr 172.21.15.46:0/663603591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7aa5719700 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d6384b4a0 0x558d639d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7aa5f1a700 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639c8400 0x558d639d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7aa4f18700 1 -- 172.21.15.46:0/663603591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d6384b4a0 msgr2=0x558d639d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d6384b4a0 0x558d639d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7aa4f18700 1 -- 172.21.15.46:0/663603591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639c8400 msgr2=0x558d639d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639c8400 0x558d639d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7aa4f18700 1 -- 172.21.15.46:0/663603591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d63766490 con 0x558d638c5d00 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7aa5719700 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d6384b4a0 0x558d639d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d638c5d00 0x558d639d2500 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7a900073d0 tx=0x7f7a90013ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc48644c984385a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7a88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7a88ff9700 1 -- 172.21.15.46:0/663603591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900125f0 con 0x558d638c5d00 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7ab6393b80 1 -- 172.21.15.46:0/663603591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a90018070 con 0x558d638c5d00 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.901+0000 7f7a88ff9700 1 -- 172.21.15.46:0/663603591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7a90012790 con 0x558d638c5d00 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.902+0000 7f7a88ff9700 1 -- 172.21.15.46:0/663603591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7a90012990 con 0x558d638c5d00 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.902+0000 7f7ab6393b80 10 client.17118 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.902+0000 7f7ab6393b80 1 -- 172.21.15.46:0/663603591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558d639df990 con 0x558d638c5d00 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.902+0000 7f7a88ff9700 1 -- 172.21.15.46:0/663603591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a90018070 con 0x558d638c5d00 2024-05-09T00:21:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.902+0000 7f7a88ff9700 1 -- 172.21.15.46:0/663603591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7a900279f0 con 0x558d638c5d00 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.902+0000 7f7ab6393b80 10 client.17118 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.902+0000 7f7ab6393b80 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.902+0000 7f7ab6393b80 20 client.17118 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.902+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x558d639e1fd0 0x558d639e43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.902+0000 7f7ab6393b80 4 client.17118 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.902+0000 7f7ab6393b80 1 -- 172.21.15.46:0/663603591 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d638f8a80 con 0x558d639e1fd0 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.903+0000 7f7aa5719700 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x558d639e1fd0 0x558d639e43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.903+0000 7f7aa5719700 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x558d639e1fd0 0x558d639e43f0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f661b7f7b966e48d server_cookie=c307f3bc720900e2 in_seq=0 out_seq=0 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.903+0000 7f7a88ff9700 10 client.17118 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7a88ff9700 1 -- 172.21.15.46:0/663603591 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d638f8a80 con 0x558d639e1fd0 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7a88ff9700 10 client.17118 handle_command_reply: tid=0 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7ab6393b80 1 client.17118 shutdown 2024-05-09T00:21:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7ab6393b80 1 -- 172.21.15.46:0/663603591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d638c5d00 msgr2=0x558d639d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d638c5d00 0x558d639d2500 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7a900073d0 tx=0x7f7a90013ae0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7ab6393b80 1 -- 172.21.15.46:0/663603591 shutdown_connections 2024-05-09T00:21:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x558d639e1fd0 0x558d639e43f0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639c8400 0x558d639d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d6384b4a0 0x558d639d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7a88ff9700 0 client.17118 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/663603591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d638c5d00 0x558d639d2500 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7ab6393b80 1 -- 172.21.15.46:0/663603591 >> 172.21.15.46:0/663603591 conn(0x558d6384ef70 msgr2=0x558d639bcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7ab6393b80 1 -- 172.21.15.46:0/663603591 shutdown_connections 2024-05-09T00:21:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7ab6393b80 1 -- 172.21.15.46:0/663603591 wait complete. 2024-05-09T00:21:55.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.904+0000 7f7ab6393b80 20 client.17118 trim_cache size 0 max 16384 2024-05-09T00:21:55.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.919+0000 7f7ab6393b80 1 Processor -- start 2024-05-09T00:21:55.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7ab6393b80 1 -- start start 2024-05-09T00:21:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7ab6393b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d6384b4a0 0x558d639cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7ab6393b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d639c8400 0x558d639dde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7ab6393b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639de3a0 0x558d639e9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7ab6393b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01b36f0 con 0x558d639c8400 2024-05-09T00:21:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7ab6393b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01b0f80 con 0x558d639de3a0 2024-05-09T00:21:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7ab6393b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01aedf0 con 0x558d6384b4a0 2024-05-09T00:21:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7aa4f18700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d639c8400 0x558d639dde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7aa4f18700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d639c8400 0x558d639dde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58684/0 (socket says 172.21.15.46:58684) 2024-05-09T00:21:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7aa4f18700 1 -- 172.21.15.46:0/3687791256 learned_addr learned my addr 172.21.15.46:0/3687791256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7aa5719700 1 --2- 172.21.15.46:0/3687791256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d6384b4a0 0x558d639cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7aa4f18700 1 -- 172.21.15.46:0/3687791256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d6384b4a0 msgr2=0x558d639cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/3687791256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d6384b4a0 0x558d639cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7aa4f18700 1 -- 172.21.15.46:0/3687791256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639de3a0 msgr2=0x558d639e9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/3687791256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639de3a0 0x558d639e9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7aa4f18700 1 -- 172.21.15.46:0/3687791256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d639c8fb0 con 0x558d639c8400 2024-05-09T00:21:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.920+0000 7f7aa5719700 1 --2- 172.21.15.46:0/3687791256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d6384b4a0 0x558d639cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:21:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.921+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/3687791256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d639c8400 0x558d639dde60 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7a90027e10 tx=0x7f7a90009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9f8d2c1f5d05d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.921+0000 7f7a8affd700 1 -- 172.21.15.46:0/3687791256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900125f0 con 0x558d639c8400 2024-05-09T00:21:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.921+0000 7f7a8affd700 1 -- 172.21.15.46:0/3687791256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a90036070 con 0x558d639c8400 2024-05-09T00:21:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.921+0000 7f7a8affd700 1 -- 172.21.15.46:0/3687791256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90032ba0 con 0x558d639c8400 2024-05-09T00:21:55.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.922+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3687791256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d639c8400 msgr2=0x558d639dde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:55.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.922+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/3687791256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d639c8400 0x558d639dde60 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7a90027e10 tx=0x7f7a90009000 comp rx=0 tx=0).stop 2024-05-09T00:21:55.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.922+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3687791256 shutdown_connections 2024-05-09T00:21:55.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.922+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/3687791256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639de3a0 0x558d639e9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.922+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/3687791256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d6384b4a0 0x558d639cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.922+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/3687791256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d639c8400 0x558d639dde60 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.922+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3687791256 >> 172.21.15.46:0/3687791256 conn(0x558d6384ef70 msgr2=0x558d639cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:55.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.922+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3687791256 shutdown_connections 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.922+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3687791256 wait complete. 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7ab6393b80 1 -- start start 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7ab6393b80 1 Processor -- start 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7ab6393b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 0x558d639e9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7ab6393b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d639c8400 0x558d639e9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7ab6393b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639ea110 0x558d63a4b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7ab6393b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01af440 con 0x558d6384b4a0 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7ab6393b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01b08b0 con 0x558d639ea110 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7ab6393b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01afb10 con 0x558d639c8400 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7ab6393b80 10 client.0 _ll_register_callbacks cb 0x558d6375c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7aa5719700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 0x558d639e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7aa5719700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 0x558d639e9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58686/0 (socket says 172.21.15.46:58686) 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7aa5719700 1 -- 172.21.15.46:0/3959587119 learned_addr learned my addr 172.21.15.46:0/3959587119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:55.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7aa5f1a700 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639ea110 0x558d63a4b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.923+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d639c8400 0x558d639e9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7aa5719700 1 -- 172.21.15.46:0/3959587119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d639c8400 msgr2=0x558d639e9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7aa5719700 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d639c8400 0x558d639e9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7aa5719700 1 -- 172.21.15.46:0/3959587119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639ea110 msgr2=0x558d63a4b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7aa5719700 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639ea110 0x558d63a4b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7aa5719700 1 -- 172.21.15.46:0/3959587119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d639ceaf0 con 0x558d6384b4a0 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7aa5719700 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 0x558d639e9690 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x558d639e49f0 tx=0x7f7a9c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e92efe8a748896 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7a88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7a88ff9700 1 -- 172.21.15.46:0/3959587119 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c003af0 con 0x558d6384b4a0 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7a88ff9700 1 -- 172.21.15.46:0/3959587119 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7a9c003c90 con 0x558d6384b4a0 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3959587119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a90036070 con 0x558d6384b4a0 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7a88ff9700 1 -- 172.21.15.46:0/3959587119 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7a9c0281d0 con 0x558d6384b4a0 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7ab6393b80 10 client.17124 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.924+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3959587119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558d639bf9d0 con 0x558d6384b4a0 2024-05-09T00:21:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.925+0000 7f7a88ff9700 1 -- 172.21.15.46:0/3959587119 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a90036070 con 0x558d6384b4a0 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.925+0000 7f7a88ff9700 1 -- 172.21.15.46:0/3959587119 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7a9c028ce0 con 0x558d6384b4a0 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.925+0000 7f7ab6393b80 10 client.17124 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.925+0000 7f7ab6393b80 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.925+0000 7f7ab6393b80 20 client.17124 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.925+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x558d63a4d110 0x558d63a4f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.925+0000 7f7ab6393b80 4 client.17124 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.925+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3959587119 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d638f2b20 con 0x558d63a4d110 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.925+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x558d63a4d110 0x558d63a4f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7a88ff9700 10 client.17124 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7aa4f18700 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x558d63a4d110 0x558d63a4f530 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5022033291791ce4 server_cookie=5bd81e23a3e68737 in_seq=0 out_seq=0 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7a88ff9700 1 -- 172.21.15.46:0/3959587119 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d638f2b20 con 0x558d63a4d110 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7a88ff9700 10 client.17124 handle_command_reply: tid=0 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7ab6393b80 1 client.17124 shutdown 2024-05-09T00:21:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3959587119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 msgr2=0x558d639e9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 0x558d639e9690 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x558d639e49f0 tx=0x7f7a9c0047c0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3959587119 shutdown_connections 2024-05-09T00:21:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x558d63a4d110 0x558d63a4f530 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d639ea110 0x558d63a4b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7a88ff9700 0 client.17124 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d6384b4a0 0x558d639e9690 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7ab6393b80 1 --2- 172.21.15.46:0/3959587119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558d639c8400 0x558d639e9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3959587119 >> 172.21.15.46:0/3959587119 conn(0x558d6384ef70 msgr2=0x558d639bcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3959587119 shutdown_connections 2024-05-09T00:21:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7ab6393b80 1 -- 172.21.15.46:0/3959587119 wait complete. 2024-05-09T00:21:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.926+0000 7f7ab6393b80 20 client.17124 trim_cache size 0 max 16384 2024-05-09T00:21:55.939 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:21:55.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.927+0000 7f7aa797d700 1 -- 172.21.15.46:0/639602326 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7a8c0b2880 msgr2=0x7f7a8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:55.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.927+0000 7f7aa797d700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7a8c0b2880 0x7f7a8c0b4d90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c007be0 tx=0x7f7a9c0073d0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.927+0000 7f7aa797d700 1 -- 172.21.15.46:0/639602326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa0108760 msgr2=0x7f7aa01a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:55.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.927+0000 7f7aa797d700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa0108760 0x7f7aa01a1900 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7a980095c0 tx=0x7f7a98007a20 comp rx=0 tx=0).stop 2024-05-09T00:21:55.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.927+0000 7f7aa797d700 1 -- 172.21.15.46:0/639602326 shutdown_connections 2024-05-09T00:21:55.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.927+0000 7f7aa797d700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7a8c0b2880 0x7f7a8c0b4d90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.927+0000 7f7aa797d700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa0108760 0x7f7aa01a1900 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.927+0000 7f7aa797d700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aa01034e0 0x7f7aa019d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.927+0000 7f7aa797d700 1 --2- 172.21.15.46:0/639602326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7aa0107e10 0x7f7aa019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:55.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.927+0000 7f7aa797d700 1 -- 172.21.15.46:0/639602326 >> 172.21.15.46:0/639602326 conn(0x7f7aa00fef30 msgr2=0x7f7aa01097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:55.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.927+0000 7f7aa797d700 1 -- 172.21.15.46:0/639602326 shutdown_connections 2024-05-09T00:21:55.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:55.927+0000 7f7aa797d700 1 -- 172.21.15.46:0/639602326 wait complete. 2024-05-09T00:21:56.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:56 smithi142 ceph-mon[34533]: pgmap v693: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:57.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:56 smithi046 ceph-mon[30151]: pgmap v693: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:57.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:56 smithi169 ceph-mon[34777]: pgmap v693: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:21:57.619 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:21:57.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:21:57.876 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:21:57.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:57 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:21:58.000 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:57 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:21:58.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:57 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:21:58.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:58 smithi142 ceph-mon[34533]: pgmap v694: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:21:58.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:58 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:21:58.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:58 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:21:58.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:58 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:21:58.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:21:58 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:21:59.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:58 smithi046 ceph-mon[30151]: pgmap v694: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:21:59.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:21:59.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:21:59.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:21:59.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:21:58 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:21:59.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:58 smithi169 ceph-mon[34777]: pgmap v694: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:21:59.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:58 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:21:59.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:58 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:21:59.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:58 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:21:59.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:21:58 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:21:59.440 INFO:tasks.workunit.client.0.smithi046.stdout:437/600: open fd = 3 2024-05-09T00:21:59.440 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.440 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.440 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.440 INFO:tasks.workunit.client.0.smithi046.stdout:438/600: open fd = 3 2024-05-09T00:21:59.440 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.440 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.440 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.440 INFO:tasks.workunit.client.0.smithi046.stdout:439/600: open fd = 3 2024-05-09T00:21:59.440 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:440/600: open fd = 3 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:441/600: open fd = 3 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:442/600: open fd = 3 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.441 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:443/600: open fd = 3 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:444/600: open fd = 3 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:445/600: open fd = 3 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.442 INFO:tasks.workunit.client.0.smithi046.stdout:446/600: open fd = 3 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:447/600: open fd = 3 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:448/600: open fd = 3 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:449/600: open fd = 3 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.443 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:450/600: open fd = 3 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:451/600: open fd = 3 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:452/600: open fd = 3 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:453/600: open fd = 3 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.444 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:454/600: open fd = 3 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:455/600: open fd = 3 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:456/600: open fd = 3 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.445 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:457/600: open fd = 3 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:458/600: open fd = 3 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:459/600: open fd = 3 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:460/600: open fd = 3 2024-05-09T00:21:59.446 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:461/600: open fd = 3 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:462/600: open fd = 3 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:463/600: open fd = 3 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.447 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:464/600: open fd = 3 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:465/600: open fd = 3 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:466/600: open fd = 3 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:467/600: open fd = 3 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.448 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:468/600: open fd = 3 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:469/600: open fd = 3 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:470/600: open fd = 3 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.449 INFO:tasks.workunit.client.0.smithi046.stdout:471/600: open fd = 3 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:472/600: open fd = 3 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:473/600: open fd = 3 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:474/600: open fd = 3 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.450 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:475/600: open fd = 3 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:476/600: open fd = 3 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:477/600: open fd = 3 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.451 INFO:tasks.workunit.client.0.smithi046.stdout:478/600: open fd = 3 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:479/600: open fd = 3 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:480/600: open fd = 3 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:481/600: open fd = 3 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.452 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:482/600: open fd = 3 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:483/600: open fd = 3 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:484/600: open fd = 3 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:485/600: open fd = 3 2024-05-09T00:21:59.453 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:486/600: open fd = 3 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:487/600: open fd = 3 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:488/600: open fd = 3 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.454 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:489/600: open fd = 3 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:490/600: open fd = 3 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:491/600: open fd = 3 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:492/600: open fd = 3 2024-05-09T00:21:59.455 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:493/600: open fd = 3 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:494/600: open fd = 3 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:495/600: open fd = 3 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.456 INFO:tasks.workunit.client.0.smithi046.stdout:496/600: open fd = 3 2024-05-09T00:21:59.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.457 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.457 INFO:tasks.workunit.client.0.smithi046.stdout:497/600: open fd = 3 2024-05-09T00:21:59.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.457 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.457 INFO:tasks.workunit.client.0.smithi046.stdout:498/600: open fd = 3 2024-05-09T00:21:59.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.457 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:21:59.457 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:21:59.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.731+0000 7f36905a9700 1 -- 172.21.15.46:0/322785000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3688107df0 msgr2=0x7f3688108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:59.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.731+0000 7f36905a9700 1 --2- 172.21.15.46:0/322785000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3688107df0 0x7f3688108200 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f367c0099c0 tx=0x7f367c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.731+0000 7f36905a9700 1 -- 172.21.15.46:0/322785000 shutdown_connections 2024-05-09T00:21:59.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.731+0000 7f36905a9700 1 --2- 172.21.15.46:0/322785000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3688108740 0x7f36881100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.731+0000 7f36905a9700 1 --2- 172.21.15.46:0/322785000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36881034c0 0x7f36881038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.731+0000 7f36905a9700 1 --2- 172.21.15.46:0/322785000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3688107df0 0x7f3688108200 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.731+0000 7f36905a9700 1 -- 172.21.15.46:0/322785000 >> 172.21.15.46:0/322785000 conn(0x7f36880fedc0 msgr2=0x7f36881011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:59.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.731+0000 7f36905a9700 1 -- 172.21.15.46:0/322785000 shutdown_connections 2024-05-09T00:21:59.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.731+0000 7f36905a9700 1 -- 172.21.15.46:0/322785000 wait complete. 2024-05-09T00:21:59.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.732+0000 7f36905a9700 1 Processor -- start 2024-05-09T00:21:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.733+0000 7f36905a9700 1 -- start start 2024-05-09T00:21:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.733+0000 7f36905a9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36881034c0 0x7f368819bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.733+0000 7f36905a9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3688107df0 0x7f368819c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.733+0000 7f36905a9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3688108740 0x7f368819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.733+0000 7f36905a9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36881133a0 con 0x7f3688107df0 2024-05-09T00:21:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.733+0000 7f36905a9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3688113220 con 0x7f3688108740 2024-05-09T00:21:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.733+0000 7f36905a9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3688113520 con 0x7f36881034c0 2024-05-09T00:21:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.733+0000 7f368db44700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3688107df0 0x7f368819c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.733+0000 7f368db44700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3688107df0 0x7f368819c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46164/0 (socket says 172.21.15.46:46164) 2024-05-09T00:21:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.733+0000 7f368db44700 1 -- 172.21.15.46:0/2891048344 learned_addr learned my addr 172.21.15.46:0/2891048344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.733+0000 7f368e345700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36881034c0 0x7f368819bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.733+0000 7f368eb46700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3688108740 0x7f368819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.734+0000 7f368db44700 1 -- 172.21.15.46:0/2891048344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36881034c0 msgr2=0x7f368819bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.734+0000 7f368db44700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36881034c0 0x7f368819bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.734+0000 7f368db44700 1 -- 172.21.15.46:0/2891048344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3688108740 msgr2=0x7f368819c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.734+0000 7f368db44700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3688108740 0x7f368819c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.734+0000 7f368db44700 1 -- 172.21.15.46:0/2891048344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36881a7010 con 0x7f3688107df0 2024-05-09T00:21:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.734+0000 7f368db44700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3688107df0 0x7f368819c200 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f367c00b580 tx=0x7f367c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.734+0000 7f367b7fe700 1 -- 172.21.15.46:0/2891048344 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c009e60 con 0x7f3688107df0 2024-05-09T00:21:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.734+0000 7f367b7fe700 1 -- 172.21.15.46:0/2891048344 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f367c003c70 con 0x7f3688107df0 2024-05-09T00:21:59.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.734+0000 7f36905a9700 1 -- 172.21.15.46:0/2891048344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36881a72a0 con 0x7f3688107df0 2024-05-09T00:21:59.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.734+0000 7f367b7fe700 1 -- 172.21.15.46:0/2891048344 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c004910 con 0x7f3688107df0 2024-05-09T00:21:59.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.734+0000 7f36905a9700 1 -- 172.21.15.46:0/2891048344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36881a7690 con 0x7f3688107df0 2024-05-09T00:21:59.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.738+0000 7f367b7fe700 1 -- 172.21.15.46:0/2891048344 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f367c0037c0 con 0x7f3688107df0 2024-05-09T00:21:59.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.739+0000 7f367b7fe700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f36740b29d0 0x7f36740b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.739+0000 7f369efbfb80 1 Processor -- start 2024-05-09T00:21:59.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.739+0000 7f367b7fe700 1 -- 172.21.15.46:0/2891048344 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f367c0e2970 con 0x7f3688107df0 2024-05-09T00:21:59.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.739+0000 7f368e345700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f36740b29d0 0x7f36740b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.739+0000 7f369efbfb80 1 -- start start 2024-05-09T00:21:59.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.739+0000 7f369efbfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 0x5580d43af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.739+0000 7f369efbfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d4228520 0x5580d43afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f369efbfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43a9b30 0x5580d43a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f369efbfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5580d42c8070 con 0x5580d43121e0 2024-05-09T00:21:59.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f369efbfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5580d4266870 con 0x5580d43a9b30 2024-05-09T00:21:59.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f369efbfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5580d42c4030 con 0x5580d4228520 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f368e345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 0x5580d43af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f368e345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 0x5580d43af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46168/0 (socket says 172.21.15.46:46168) 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f368e345700 1 -- 172.21.15.46:0/1697162167 learned_addr learned my addr 172.21.15.46:0/1697162167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f368db44700 1 --2- 172.21.15.46:0/1697162167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d4228520 0x5580d43afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f368eb46700 1 --2- 172.21.15.46:0/1697162167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43a9b30 0x5580d43a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f368e345700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f36740b29d0 0x7f36740b4ee0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3684009c60 tx=0x7f3684009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f368e345700 1 -- 172.21.15.46:0/1697162167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d4228520 msgr2=0x5580d43afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f368e345700 1 --2- 172.21.15.46:0/1697162167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d4228520 0x5580d43afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f368e345700 1 -- 172.21.15.46:0/1697162167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43a9b30 msgr2=0x5580d43a9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f368e345700 1 --2- 172.21.15.46:0/1697162167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43a9b30 0x5580d43a9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f368e345700 1 -- 172.21.15.46:0/1697162167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580d425e3f0 con 0x5580d43121e0 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.740+0000 7f368e345700 1 --2- 172.21.15.46:0/1697162167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 0x5580d43af8f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3684009fd0 tx=0x7f368401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27631e071b04eeb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.741+0000 7f3678ff9700 1 -- 172.21.15.46:0/1697162167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684004a30 con 0x5580d43121e0 2024-05-09T00:21:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.741+0000 7f3678ff9700 1 -- 172.21.15.46:0/1697162167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36840054d0 con 0x5580d43121e0 2024-05-09T00:21:59.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.741+0000 7f3678ff9700 1 -- 172.21.15.46:0/1697162167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368400e910 con 0x5580d43121e0 2024-05-09T00:21:59.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.742+0000 7f369efbfb80 1 -- 172.21.15.46:0/1697162167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 msgr2=0x5580d43af8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:59.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.742+0000 7f369efbfb80 1 --2- 172.21.15.46:0/1697162167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 0x5580d43af8f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3684009fd0 tx=0x7f368401b000 comp rx=0 tx=0).stop 2024-05-09T00:21:59.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.742+0000 7f369efbfb80 1 -- 172.21.15.46:0/1697162167 shutdown_connections 2024-05-09T00:21:59.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.742+0000 7f369efbfb80 1 --2- 172.21.15.46:0/1697162167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43a9b30 0x5580d43a9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.742+0000 7f369efbfb80 1 --2- 172.21.15.46:0/1697162167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 0x5580d43af8f0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.742+0000 7f369efbfb80 1 --2- 172.21.15.46:0/1697162167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d4228520 0x5580d43afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.742+0000 7f369efbfb80 1 -- 172.21.15.46:0/1697162167 >> 172.21.15.46:0/1697162167 conn(0x5580d4288720 msgr2=0x5580d43a0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:59.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.742+0000 7f369efbfb80 1 -- 172.21.15.46:0/1697162167 shutdown_connections 2024-05-09T00:21:59.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.742+0000 7f369efbfb80 1 -- 172.21.15.46:0/1697162167 wait complete. 2024-05-09T00:21:59.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.742+0000 7f369efbfb80 1 -- start start 2024-05-09T00:21:59.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.742+0000 7f369efbfb80 1 Processor -- start 2024-05-09T00:21:59.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f369efbfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d4228520 0x5580d43b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f369efbfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 0x5580d43b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f369efbfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f369efbfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5580d4143c20 con 0x5580d43121e0 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f369efbfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5580d42821d0 con 0x5580d4228520 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f369efbfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5580d43aa940 con 0x5580d43a9b30 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f369efbfb80 10 client.0 _ll_register_callbacks cb 0x5580d4139740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f368db44700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 0x5580d43b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f368db44700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 0x5580d43b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46178/0 (socket says 172.21.15.46:46178) 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f368db44700 1 -- 172.21.15.46:0/628355814 learned_addr learned my addr 172.21.15.46:0/628355814 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f368eb46700 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f368e345700 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d4228520 0x5580d43b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f368db44700 1 -- 172.21.15.46:0/628355814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 msgr2=0x5580d43b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f368db44700 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f368db44700 1 -- 172.21.15.46:0/628355814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d4228520 msgr2=0x5580d43b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f368db44700 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d4228520 0x5580d43b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f368db44700 1 -- 172.21.15.46:0/628355814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580d4143510 con 0x5580d43121e0 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f368e345700 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d4228520 0x5580d43b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.743+0000 7f368eb46700 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.744+0000 7f368db44700 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 0x5580d43b3b40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f367c0e0470 tx=0x7f367c0aa560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6986614844d6c09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.744+0000 7f3671ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.744+0000 7f3671ffb700 1 -- 172.21.15.46:0/628355814 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c003e20 con 0x5580d43121e0 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.744+0000 7f3671ffb700 1 -- 172.21.15.46:0/628355814 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f367c0a9280 con 0x5580d43121e0 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.744+0000 7f369efbfb80 1 -- 172.21.15.46:0/628355814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36840054d0 con 0x5580d43121e0 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.744+0000 7f3671ffb700 1 -- 172.21.15.46:0/628355814 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f367c0aa020 con 0x5580d43121e0 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.744+0000 7f369efbfb80 10 client.17136 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.744+0000 7f369efbfb80 1 -- 172.21.15.46:0/628355814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580d43c0f10 con 0x5580d43121e0 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.744+0000 7f3671ffb700 1 -- 172.21.15.46:0/628355814 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36840054d0 con 0x5580d43121e0 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.744+0000 7f3671ffb700 1 -- 172.21.15.46:0/628355814 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f367c0a8cf0 con 0x5580d43121e0 2024-05-09T00:21:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.744+0000 7f369efbfb80 10 client.17136 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.744+0000 7f369efbfb80 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:21:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.745+0000 7f369efbfb80 20 client.17136 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.745+0000 7f369efbfb80 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580d43c34d0 0x5580d43c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.745+0000 7f369efbfb80 4 client.17136 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:21:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.745+0000 7f369efbfb80 1 -- 172.21.15.46:0/628355814 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580d42d5a50 con 0x5580d43c34d0 2024-05-09T00:21:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.745+0000 7f368e345700 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580d43c34d0 0x5580d43c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.745+0000 7f368e345700 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580d43c34d0 0x5580d43c5980 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0acb02cb01518e6 server_cookie=6102f7d10fad412b in_seq=0 out_seq=0 2024-05-09T00:21:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.745+0000 7f3671ffb700 10 client.17136 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.746+0000 7f3671ffb700 1 -- 172.21.15.46:0/628355814 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5580d42d5a50 con 0x5580d43c34d0 2024-05-09T00:21:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.746+0000 7f3671ffb700 10 client.17136 handle_command_reply: tid=0 2024-05-09T00:21:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.746+0000 7f369efbfb80 1 client.17136 shutdown 2024-05-09T00:21:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.746+0000 7f369efbfb80 1 -- 172.21.15.46:0/628355814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 msgr2=0x5580d43b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.746+0000 7f369efbfb80 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 0x5580d43b3b40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f367c0e0470 tx=0x7f367c0aa560 comp rx=0 tx=0).stop 2024-05-09T00:21:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.746+0000 7f369efbfb80 1 -- 172.21.15.46:0/628355814 shutdown_connections 2024-05-09T00:21:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.746+0000 7f369efbfb80 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580d43c34d0 0x5580d43c5980 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.746+0000 7f369efbfb80 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.746+0000 7f369efbfb80 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d4228520 0x5580d43b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.746+0000 7f369efbfb80 1 --2- 172.21.15.46:0/628355814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d43121e0 0x5580d43b3b40 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.746+0000 7f369efbfb80 1 -- 172.21.15.46:0/628355814 >> 172.21.15.46:0/628355814 conn(0x5580d4288720 msgr2=0x5580d43a0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.747+0000 7f3671ffb700 0 client.17136 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.747+0000 7f369efbfb80 1 -- 172.21.15.46:0/628355814 shutdown_connections 2024-05-09T00:21:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.747+0000 7f369efbfb80 1 -- 172.21.15.46:0/628355814 wait complete. 2024-05-09T00:21:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.747+0000 7f369efbfb80 20 client.17136 trim_cache size 0 max 16384 2024-05-09T00:21:59.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.780+0000 7f369efbfb80 1 Processor -- start 2024-05-09T00:21:59.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.780+0000 7f369efbfb80 1 -- start start 2024-05-09T00:21:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f369efbfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 0x5580d43add60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f369efbfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43121e0 0x5580d43aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f369efbfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43bf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f369efbfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36881af350 con 0x5580d4228520 2024-05-09T00:21:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f369efbfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f36881acbe0 con 0x5580d43121e0 2024-05-09T00:21:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f369efbfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f36881aaa10 con 0x5580d43a9b30 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f368e345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 0x5580d43add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f368e345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 0x5580d43add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46182/0 (socket says 172.21.15.46:46182) 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f368e345700 1 -- 172.21.15.46:0/3248210244 learned_addr learned my addr 172.21.15.46:0/3248210244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f368eb46700 1 --2- 172.21.15.46:0/3248210244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43bf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f368db44700 1 --2- 172.21.15.46:0/3248210244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43121e0 0x5580d43aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f368e345700 1 -- 172.21.15.46:0/3248210244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 msgr2=0x5580d43bf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f368e345700 1 --2- 172.21.15.46:0/3248210244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43bf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f368e345700 1 -- 172.21.15.46:0/3248210244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43121e0 msgr2=0x5580d43aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f368e345700 1 --2- 172.21.15.46:0/3248210244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43121e0 0x5580d43aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f368e345700 1 -- 172.21.15.46:0/3248210244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580d4399fe0 con 0x5580d4228520 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.781+0000 7f368db44700 1 --2- 172.21.15.46:0/3248210244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43121e0 0x5580d43aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.782+0000 7f368e345700 1 --2- 172.21.15.46:0/3248210244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 0x5580d43add60 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f368401cae0 tx=0x7f36840057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ffaeebba6d6bab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.782+0000 7f3673fff700 1 -- 172.21.15.46:0/3248210244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684032070 con 0x5580d4228520 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.782+0000 7f3673fff700 1 -- 172.21.15.46:0/3248210244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36840044d0 con 0x5580d4228520 2024-05-09T00:21:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.782+0000 7f3673fff700 1 -- 172.21.15.46:0/3248210244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36840038e0 con 0x5580d4228520 2024-05-09T00:21:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.783+0000 7f369efbfb80 1 -- 172.21.15.46:0/3248210244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 msgr2=0x5580d43add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.783+0000 7f369efbfb80 1 --2- 172.21.15.46:0/3248210244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 0x5580d43add60 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f368401cae0 tx=0x7f36840057c0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.783+0000 7f369efbfb80 1 -- 172.21.15.46:0/3248210244 shutdown_connections 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.783+0000 7f369efbfb80 1 --2- 172.21.15.46:0/3248210244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43bf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.783+0000 7f369efbfb80 1 --2- 172.21.15.46:0/3248210244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 0x5580d43add60 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.783+0000 7f369efbfb80 1 --2- 172.21.15.46:0/3248210244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43121e0 0x5580d43aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.783+0000 7f369efbfb80 1 -- 172.21.15.46:0/3248210244 >> 172.21.15.46:0/3248210244 conn(0x5580d4288720 msgr2=0x5580d43b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.783+0000 7f369efbfb80 1 -- 172.21.15.46:0/3248210244 shutdown_connections 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.783+0000 7f369efbfb80 1 -- 172.21.15.46:0/3248210244 wait complete. 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.783+0000 7f369efbfb80 1 -- start start 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.783+0000 7f369efbfb80 1 Processor -- start 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f369efbfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 0x5580d43bb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f369efbfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43121e0 0x5580d43bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f369efbfb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43b8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f369efbfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36881ab0a0 con 0x5580d4228520 2024-05-09T00:21:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f369efbfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f36881ac510 con 0x5580d43121e0 2024-05-09T00:21:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f369efbfb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f36881ab770 con 0x5580d43a9b30 2024-05-09T00:21:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f369efbfb80 10 client.0 _ll_register_callbacks cb 0x5580d4139740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:21:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368e345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 0x5580d43bb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368e345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 0x5580d43bb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46196/0 (socket says 172.21.15.46:46196) 2024-05-09T00:21:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368e345700 1 -- 172.21.15.46:0/3604775929 learned_addr learned my addr 172.21.15.46:0/3604775929 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:21:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368eb46700 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368db44700 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43121e0 0x5580d43bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368e345700 1 -- 172.21.15.46:0/3604775929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 msgr2=0x5580d43b8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368e345700 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43b8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368e345700 1 -- 172.21.15.46:0/3604775929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43121e0 msgr2=0x5580d43bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:21:59.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368e345700 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43121e0 0x5580d43bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368e345700 1 -- 172.21.15.46:0/3604775929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580d43b8ef0 con 0x5580d4228520 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368db44700 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43121e0 0x5580d43bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368eb46700 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f368e345700 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 0x5580d43bb7a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f368400e520 tx=0x7f3684000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68982d281b4405d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.784+0000 7f3671ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f3671ffb700 1 -- 172.21.15.46:0/3604775929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684032050 con 0x5580d4228520 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f3671ffb700 1 -- 172.21.15.46:0/3604775929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f368400eb60 con 0x5580d4228520 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f369efbfb80 1 -- 172.21.15.46:0/3604775929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36840044d0 con 0x5580d4228520 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f3671ffb700 1 -- 172.21.15.46:0/3604775929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f368400eda0 con 0x5580d4228520 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f369efbfb80 10 client.17142 fetch_fsmap learned FSMap version 26 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f369efbfb80 1 -- 172.21.15.46:0/3604775929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580d442e690 con 0x5580d4228520 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f3671ffb700 1 -- 172.21.15.46:0/3604775929 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36840044d0 con 0x5580d4228520 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f3671ffb700 1 -- 172.21.15.46:0/3604775929 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3684023070 con 0x5580d4228520 2024-05-09T00:21:59.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f369efbfb80 10 client.17142 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:21:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f369efbfb80 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:21:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f369efbfb80 20 client.17142 populate_metadata read hostname 'smithi046' 2024-05-09T00:21:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f369efbfb80 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580d442ecb0 0x5580d44310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:21:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f369efbfb80 4 client.17142 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:21:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.785+0000 7f369efbfb80 1 -- 172.21.15.46:0/3604775929 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5580d42cfaf0 con 0x5580d442ecb0 2024-05-09T00:21:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.786+0000 7f368db44700 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580d442ecb0 0x5580d44310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:21:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.786+0000 7f368db44700 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580d442ecb0 0x5580d44310d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba2869d6fdda694a server_cookie=fb841b5dce34d74d in_seq=0 out_seq=0 2024-05-09T00:21:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.786+0000 7f3671ffb700 10 client.17142 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:59.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.787+0000 7f3671ffb700 1 -- 172.21.15.46:0/3604775929 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5580d42cfaf0 con 0x5580d442ecb0 2024-05-09T00:21:59.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.787+0000 7f3671ffb700 10 client.17142 handle_command_reply: tid=0 2024-05-09T00:21:59.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.787+0000 7f369efbfb80 1 client.17142 shutdown 2024-05-09T00:21:59.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.787+0000 7f369efbfb80 1 -- 172.21.15.46:0/3604775929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 msgr2=0x5580d43bb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:59.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.787+0000 7f369efbfb80 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 0x5580d43bb7a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f368400e520 tx=0x7f3684000bc0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.788+0000 7f369efbfb80 1 -- 172.21.15.46:0/3604775929 shutdown_connections 2024-05-09T00:21:59.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.788+0000 7f369efbfb80 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x5580d442ecb0 0x5580d44310d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.788+0000 7f369efbfb80 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5580d43a9b30 0x5580d43b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.788+0000 7f369efbfb80 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580d4228520 0x5580d43bb7a0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.788+0000 7f369efbfb80 1 --2- 172.21.15.46:0/3604775929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5580d43121e0 0x5580d43bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.788+0000 7f369efbfb80 1 -- 172.21.15.46:0/3604775929 >> 172.21.15.46:0/3604775929 conn(0x5580d4288720 msgr2=0x5580d43adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:59.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.788+0000 7f3671ffb700 0 client.17142 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:21:59.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.788+0000 7f369efbfb80 1 -- 172.21.15.46:0/3604775929 shutdown_connections 2024-05-09T00:21:59.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.788+0000 7f369efbfb80 1 -- 172.21.15.46:0/3604775929 wait complete. 2024-05-09T00:21:59.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.788+0000 7f369efbfb80 20 client.17142 trim_cache size 0 max 16384 2024-05-09T00:21:59.792 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:21:59.792 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:21:59.792 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "23ca872c-6fd7-47d4-9cec-153b7dda6e97", 2024-05-09T00:21:59.793 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:21:59.793 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:21:59.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.788+0000 7f36905a9700 1 -- 172.21.15.46:0/2891048344 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f36740b29d0 msgr2=0x7f36740b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:59.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.788+0000 7f36905a9700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f36740b29d0 0x7f36740b4ee0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3684009c60 tx=0x7f3684009260 comp rx=0 tx=0).stop 2024-05-09T00:21:59.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.789+0000 7f36905a9700 1 -- 172.21.15.46:0/2891048344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3688107df0 msgr2=0x7f368819c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:21:59.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.789+0000 7f36905a9700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3688107df0 0x7f368819c200 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f367c00b580 tx=0x7f367c004270 comp rx=0 tx=0).stop 2024-05-09T00:21:59.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.789+0000 7f36905a9700 1 -- 172.21.15.46:0/2891048344 shutdown_connections 2024-05-09T00:21:59.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.789+0000 7f36905a9700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f36740b29d0 0x7f36740b4ee0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.789+0000 7f36905a9700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3688108740 0x7f368819c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.789+0000 7f36905a9700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36881034c0 0x7f368819bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.789+0000 7f36905a9700 1 --2- 172.21.15.46:0/2891048344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3688107df0 0x7f368819c200 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:21:59.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.789+0000 7f36905a9700 1 -- 172.21.15.46:0/2891048344 >> 172.21.15.46:0/2891048344 conn(0x7f36880fedc0 msgr2=0x7f36881096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:21:59.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.789+0000 7f36905a9700 1 -- 172.21.15.46:0/2891048344 shutdown_connections 2024-05-09T00:21:59.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:21:59.789+0000 7f36905a9700 1 -- 172.21.15.46:0/2891048344 wait complete. 2024-05-09T00:22:00.563 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:22:00.563 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:22:00.563 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:00 smithi046 ceph-mon[30151]: pgmap v695: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:00.563 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:00 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:22:00.563 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:00 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:22:00.563 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:00 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:22:00.563 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:00 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:22:00.564 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:22:00.564 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:22:00.564 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:22:00.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:22:00.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:22:00.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:00 smithi169 ceph-mon[34777]: pgmap v695: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:00.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:00 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:22:00.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:00 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:22:00.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:00 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:22:00.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:00 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:22:00.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:22:00.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:22:00.607 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:22:00.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:22:00.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:22:00.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:22:00.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:00 smithi142 ceph-mon[34533]: pgmap v695: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:00.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:00 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:22:00.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:00 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:22:00.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:00 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:22:00.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:00 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:22:00.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:22:00.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:22:00.694 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:22:00.926 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:22:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.805+0000 7f85dc3aa700 1 -- 172.21.15.46:0/2972739490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4109870 msgr2=0x7f85d41111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.805+0000 7f85dc3aa700 1 --2- 172.21.15.46:0/2972739490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4109870 0x7f85d41111d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00b0d0 tx=0x7f85cc00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.805+0000 7f85dc3aa700 1 -- 172.21.15.46:0/2972739490 shutdown_connections 2024-05-09T00:22:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.805+0000 7f85dc3aa700 1 --2- 172.21.15.46:0/2972739490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4109870 0x7f85d41111d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.805+0000 7f85dc3aa700 1 --2- 172.21.15.46:0/2972739490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85d41045f0 0x7f85d4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.805+0000 7f85dc3aa700 1 --2- 172.21.15.46:0/2972739490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85d4108f20 0x7f85d4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.805+0000 7f85dc3aa700 1 -- 172.21.15.46:0/2972739490 >> 172.21.15.46:0/2972739490 conn(0x7f85d4100040 msgr2=0x7f85d4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:02.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:02 smithi046 ceph-mon[30151]: pgmap v696: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:22:02.807 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:02 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:22:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.806+0000 7f85dc3aa700 1 -- 172.21.15.46:0/2972739490 shutdown_connections 2024-05-09T00:22:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.806+0000 7f85dc3aa700 1 -- 172.21.15.46:0/2972739490 wait complete. 2024-05-09T00:22:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.806+0000 7f85dc3aa700 1 Processor -- start 2024-05-09T00:22:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.806+0000 7f85dc3aa700 1 -- start start 2024-05-09T00:22:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85dc3aa700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85d41045f0 0x7f85d419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85dc3aa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85d4108f20 0x7f85d419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85dc3aa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4109870 0x7f85d419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85dc3aa700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85d41144c0 con 0x7f85d4109870 2024-05-09T00:22:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85dc3aa700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85d4114340 con 0x7f85d4108f20 2024-05-09T00:22:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85dc3aa700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f85d4114640 con 0x7f85d41045f0 2024-05-09T00:22:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85da947700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4109870 0x7f85d419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85da146700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85d41045f0 0x7f85d419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85d9945700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85d4108f20 0x7f85d419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85da146700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85d41045f0 0x7f85d419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:42848/0 (socket says 172.21.15.46:42848) 2024-05-09T00:22:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85da146700 1 -- 172.21.15.46:0/1261207559 learned_addr learned my addr 172.21.15.46:0/1261207559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85d9945700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85d4108f20 0x7f85d419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:57774/0 (socket says 172.21.15.46:57774) 2024-05-09T00:22:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85da947700 1 -- 172.21.15.46:0/1261207559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85d41045f0 msgr2=0x7f85d419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85da947700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85d41045f0 0x7f85d419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85da947700 1 -- 172.21.15.46:0/1261207559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85d4108f20 msgr2=0x7f85d419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85da947700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85d4108f20 0x7f85d419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85da947700 1 -- 172.21.15.46:0/1261207559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d419fc30 con 0x7f85d4109870 2024-05-09T00:22:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.807+0000 7f85da146700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85d41045f0 0x7f85d419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:22:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.808+0000 7f85da947700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4109870 0x7f85d419f640 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009fd0 tx=0x7f85cc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.808+0000 7f85cb7fe700 1 -- 172.21.15.46:0/1261207559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc021020 con 0x7f85d4109870 2024-05-09T00:22:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.808+0000 7f85cb7fe700 1 -- 172.21.15.46:0/1261207559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85cc007e40 con 0x7f85d4109870 2024-05-09T00:22:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.808+0000 7f85dc3aa700 1 -- 172.21.15.46:0/1261207559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d419fec0 con 0x7f85d4109870 2024-05-09T00:22:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.808+0000 7f85cb7fe700 1 -- 172.21.15.46:0/1261207559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc005050 con 0x7f85d4109870 2024-05-09T00:22:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.808+0000 7f85dc3aa700 1 -- 172.21.15.46:0/1261207559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d41a0450 con 0x7f85d4109870 2024-05-09T00:22:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.811+0000 7f85cb7fe700 1 -- 172.21.15.46:0/1261207559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f85cc024020 con 0x7f85d4109870 2024-05-09T00:22:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.811+0000 7f85cb7fe700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f85c00b2900 0x7f85c00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85da146700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f85c00b2900 0x7f85c00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85cb7fe700 1 -- 172.21.15.46:0/1261207559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f85cc00f020 con 0x7f85d4109870 2024-05-09T00:22:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85eadc0b80 1 Processor -- start 2024-05-09T00:22:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85eadc0b80 1 -- start start 2024-05-09T00:22:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85da146700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f85c00b2900 0x7f85c00b4e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f85d000f4d0 tx=0x7f85d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85eadc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6e02d00 0x55a6f6f0b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85eadc0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6d884a0 0x55a6f6f0b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85eadc0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6f05400 0x55a6f6f05810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85eadc0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a6f6e280a0 con 0x55a6f6f05400 2024-05-09T00:22:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85eadc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a6f6dcd9a0 con 0x55a6f6e02d00 2024-05-09T00:22:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85eadc0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a6f6e24060 con 0x55a6f6d884a0 2024-05-09T00:22:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85da947700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6f05400 0x55a6f6f05810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85da947700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6f05400 0x55a6f6f05810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46214/0 (socket says 172.21.15.46:46214) 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.812+0000 7f85da947700 1 -- 172.21.15.46:0/78943533 learned_addr learned my addr 172.21.15.46:0/78943533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.813+0000 7f85da146700 1 --2- 172.21.15.46:0/78943533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6e02d00 0x55a6f6f0b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.813+0000 7f85d9945700 1 --2- 172.21.15.46:0/78943533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6d884a0 0x55a6f6f0b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.813+0000 7f85da947700 1 -- 172.21.15.46:0/78943533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6d884a0 msgr2=0x55a6f6f0b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.813+0000 7f85da947700 1 --2- 172.21.15.46:0/78943533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6d884a0 0x55a6f6f0b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.813+0000 7f85da947700 1 -- 172.21.15.46:0/78943533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6e02d00 msgr2=0x55a6f6f0b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.813+0000 7f85da947700 1 --2- 172.21.15.46:0/78943533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6e02d00 0x55a6f6f0b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.813+0000 7f85da947700 1 -- 172.21.15.46:0/78943533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f6dbe500 con 0x55a6f6f05400 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.813+0000 7f85da947700 1 --2- 172.21.15.46:0/78943533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6f05400 0x55a6f6f05810 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0a6980 tx=0x7f85cc01da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c5a3517bb750b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.813+0000 7f85c8d47700 1 -- 172.21.15.46:0/78943533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc0ab000 con 0x55a6f6f05400 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.813+0000 7f85c8d47700 1 -- 172.21.15.46:0/78943533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85cc0aa900 con 0x55a6f6f05400 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.813+0000 7f85c8d47700 1 -- 172.21.15.46:0/78943533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc0aab60 con 0x55a6f6f05400 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.814+0000 7f85eadc0b80 1 -- 172.21.15.46:0/78943533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6f05400 msgr2=0x55a6f6f05810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.814+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/78943533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6f05400 0x55a6f6f05810 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0a6980 tx=0x7f85cc01da20 comp rx=0 tx=0).stop 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.814+0000 7f85eadc0b80 1 -- 172.21.15.46:0/78943533 shutdown_connections 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.814+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/78943533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6f05400 0x55a6f6f05810 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.814+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/78943533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6e02d00 0x55a6f6f0b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.814+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/78943533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6d884a0 0x55a6f6f0b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.814+0000 7f85eadc0b80 1 -- 172.21.15.46:0/78943533 >> 172.21.15.46:0/78943533 conn(0x55a6f6d8bf70 msgr2=0x55a6f6f00220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.814+0000 7f85eadc0b80 1 -- 172.21.15.46:0/78943533 shutdown_connections 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.814+0000 7f85eadc0b80 1 -- 172.21.15.46:0/78943533 wait complete. 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85eadc0b80 1 -- start start 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85eadc0b80 1 Processor -- start 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85eadc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6d884a0 0x55a6f6f149e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85eadc0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f14f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85eadc0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6f05400 0x55a6f6f0ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85eadc0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a6f6e280a0 con 0x55a6f6e02d00 2024-05-09T00:22:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85eadc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a6f6dc2b20 con 0x55a6f6d884a0 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85eadc0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a6f6f05fb0 con 0x55a6f6f05400 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85eadc0b80 10 client.0 _ll_register_callbacks cb 0x55a6f6c996c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85d9945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f14f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85d9945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f14f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46218/0 (socket says 172.21.15.46:46218) 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85d9945700 1 -- 172.21.15.46:0/330814074 learned_addr learned my addr 172.21.15.46:0/330814074 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85da146700 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6d884a0 0x55a6f6f149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.815+0000 7f85da947700 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6f05400 0x55a6f6f0ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85d9945700 1 -- 172.21.15.46:0/330814074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6f05400 msgr2=0x55a6f6f0ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85d9945700 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6f05400 0x55a6f6f0ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85d9945700 1 -- 172.21.15.46:0/330814074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6d884a0 msgr2=0x55a6f6f149e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85d9945700 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6d884a0 0x55a6f6f149e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85d9945700 1 -- 172.21.15.46:0/330814074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f6ca3490 con 0x55a6f6e02d00 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85da146700 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6d884a0 0x55a6f6f149e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:22:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85d9945700 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f14f20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f85c400b9b0 tx=0x7f85c400bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4094be4cc98cb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85bdffb700 1 -- 172.21.15.46:0/330814074 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4003d00 con 0x55a6f6e02d00 2024-05-09T00:22:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85bdffb700 1 -- 172.21.15.46:0/330814074 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f85c4004830 con 0x55a6f6e02d00 2024-05-09T00:22:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85eadc0b80 1 -- 172.21.15.46:0/330814074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85cc0aa900 con 0x55a6f6e02d00 2024-05-09T00:22:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85bdffb700 1 -- 172.21.15.46:0/330814074 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f85c4003ea0 con 0x55a6f6e02d00 2024-05-09T00:22:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85eadc0b80 10 client.17154 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.816+0000 7f85eadc0b80 1 -- 172.21.15.46:0/330814074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a6f6f1c660 con 0x55a6f6e02d00 2024-05-09T00:22:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.817+0000 7f85bdffb700 1 -- 172.21.15.46:0/330814074 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85cc0aa900 con 0x55a6f6e02d00 2024-05-09T00:22:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.817+0000 7f85bdffb700 1 -- 172.21.15.46:0/330814074 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f85c40054a0 con 0x55a6f6e02d00 2024-05-09T00:22:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.817+0000 7f85eadc0b80 10 client.17154 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.817+0000 7f85eadc0b80 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:22:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.817+0000 7f85eadc0b80 20 client.17154 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.817+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a6f6f1eca0 0x55a6f6f210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.817+0000 7f85eadc0b80 4 client.17154 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:22:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.817+0000 7f85eadc0b80 1 -- 172.21.15.46:0/330814074 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6f6e35a80 con 0x55a6f6f1eca0 2024-05-09T00:22:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.817+0000 7f85da146700 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a6f6f1eca0 0x55a6f6f210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.818+0000 7f85bdffb700 10 client.17154 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.818+0000 7f85da146700 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a6f6f1eca0 0x55a6f6f210c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e621f835e9bf97 server_cookie=5ab568bb0d095ab2 in_seq=0 out_seq=0 2024-05-09T00:22:02.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85bdffb700 1 -- 172.21.15.46:0/330814074 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a6f6e35a80 con 0x55a6f6f1eca0 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85bdffb700 10 client.17154 handle_command_reply: tid=0 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85eadc0b80 1 client.17154 shutdown 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85eadc0b80 1 -- 172.21.15.46:0/330814074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 msgr2=0x55a6f6f14f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f14f20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f85c400b9b0 tx=0x7f85c400bcc0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85eadc0b80 1 -- 172.21.15.46:0/330814074 shutdown_connections 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a6f6f1eca0 0x55a6f6f210c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6f05400 0x55a6f6f0ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6d884a0 0x55a6f6f149e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/330814074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f14f20 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85eadc0b80 1 -- 172.21.15.46:0/330814074 >> 172.21.15.46:0/330814074 conn(0x55a6f6d8bf70 msgr2=0x55a6f6ef9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85bdffb700 0 client.17154 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85eadc0b80 1 -- 172.21.15.46:0/330814074 shutdown_connections 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85eadc0b80 1 -- 172.21.15.46:0/330814074 wait complete. 2024-05-09T00:22:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.819+0000 7f85eadc0b80 20 client.17154 trim_cache size 0 max 16384 2024-05-09T00:22:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.839+0000 7f85eadc0b80 1 Processor -- start 2024-05-09T00:22:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.839+0000 7f85eadc0b80 1 -- start start 2024-05-09T00:22:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85eadc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6d884a0 0x55a6f6f0da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85eadc0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f26860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85eadc0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6f05400 0x55a6f6f26da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85eadc0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85d41b1470 con 0x55a6f6e02d00 2024-05-09T00:22:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85eadc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85d41aed00 con 0x55a6f6d884a0 2024-05-09T00:22:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85eadc0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f85d41acb50 con 0x55a6f6f05400 2024-05-09T00:22:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85d9945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f26860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85d9945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f26860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46230/0 (socket says 172.21.15.46:46230) 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85d9945700 1 -- 172.21.15.46:0/940146075 learned_addr learned my addr 172.21.15.46:0/940146075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85da146700 1 --2- 172.21.15.46:0/940146075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6d884a0 0x55a6f6f0da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85da947700 1 --2- 172.21.15.46:0/940146075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6f05400 0x55a6f6f26da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85d9945700 1 -- 172.21.15.46:0/940146075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6f05400 msgr2=0x55a6f6f26da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85d9945700 1 --2- 172.21.15.46:0/940146075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6f05400 0x55a6f6f26da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85d9945700 1 -- 172.21.15.46:0/940146075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6d884a0 msgr2=0x55a6f6f0da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85d9945700 1 --2- 172.21.15.46:0/940146075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6d884a0 0x55a6f6f0da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85d9945700 1 -- 172.21.15.46:0/940146075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f6efc9d0 con 0x55a6f6e02d00 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85da947700 1 --2- 172.21.15.46:0/940146075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6f05400 0x55a6f6f26da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85d9945700 1 --2- 172.21.15.46:0/940146075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f26860 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f85c401b540 tx=0x7f85c400b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1619de30202d2232 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85bffff700 1 -- 172.21.15.46:0/940146075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4005040 con 0x55a6f6e02d00 2024-05-09T00:22:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85bffff700 1 -- 172.21.15.46:0/940146075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85c4005690 con 0x55a6f6e02d00 2024-05-09T00:22:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.840+0000 7f85bffff700 1 -- 172.21.15.46:0/940146075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4004800 con 0x55a6f6e02d00 2024-05-09T00:22:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.841+0000 7f85eadc0b80 1 -- 172.21.15.46:0/940146075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 msgr2=0x55a6f6f26860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.841+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/940146075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f26860 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f85c401b540 tx=0x7f85c400b4d0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.841+0000 7f85eadc0b80 1 -- 172.21.15.46:0/940146075 shutdown_connections 2024-05-09T00:22:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.841+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/940146075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6f05400 0x55a6f6f26da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.841+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/940146075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6d884a0 0x55a6f6f0da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.841+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/940146075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f26860 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.841+0000 7f85eadc0b80 1 -- 172.21.15.46:0/940146075 >> 172.21.15.46:0/940146075 conn(0x55a6f6d8bf70 msgr2=0x55a6f6f001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.841+0000 7f85eadc0b80 1 -- 172.21.15.46:0/940146075 shutdown_connections 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.841+0000 7f85eadc0b80 1 -- 172.21.15.46:0/940146075 wait complete. 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85eadc0b80 1 -- start start 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85eadc0b80 1 Processor -- start 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85eadc0b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6d884a0 0x55a6f6f1ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85eadc0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85eadc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6f05400 0x55a6f6f18620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85eadc0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85d41ad1c0 con 0x55a6f6e02d00 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85eadc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85d41ae630 con 0x55a6f6f05400 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85eadc0b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f85d41ad890 con 0x55a6f6d884a0 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85eadc0b80 10 client.0 _ll_register_callbacks cb 0x55a6f6c996c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85d9945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85d9945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f180e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46236/0 (socket says 172.21.15.46:46236) 2024-05-09T00:22:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85d9945700 1 -- 172.21.15.46:0/2480699137 learned_addr learned my addr 172.21.15.46:0/2480699137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85da947700 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6f05400 0x55a6f6f18620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.842+0000 7f85da146700 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6d884a0 0x55a6f6f1ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85d9945700 1 -- 172.21.15.46:0/2480699137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6d884a0 msgr2=0x55a6f6f1ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85d9945700 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6d884a0 0x55a6f6f1ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85d9945700 1 -- 172.21.15.46:0/2480699137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6f05400 msgr2=0x55a6f6f18620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85d9945700 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6f05400 0x55a6f6f18620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85d9945700 1 -- 172.21.15.46:0/2480699137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f6f18b60 con 0x55a6f6e02d00 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85d9945700 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f180e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f85c400b9b0 tx=0x7f85c4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d364ba75268b2e25 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85bdffb700 1 -- 172.21.15.46:0/2480699137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4005410 con 0x55a6f6e02d00 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85bdffb700 1 -- 172.21.15.46:0/2480699137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f85c40045a0 con 0x55a6f6e02d00 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85eadc0b80 1 -- 172.21.15.46:0/2480699137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c4005690 con 0x55a6f6e02d00 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85bdffb700 1 -- 172.21.15.46:0/2480699137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f85c40047a0 con 0x55a6f6e02d00 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85eadc0b80 10 client.17160 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85eadc0b80 1 -- 172.21.15.46:0/2480699137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a6f6f88f60 con 0x55a6f6e02d00 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.843+0000 7f85bdffb700 1 -- 172.21.15.46:0/2480699137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85c4005690 con 0x55a6f6e02d00 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.844+0000 7f85bdffb700 1 -- 172.21.15.46:0/2480699137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f85c4026a30 con 0x55a6f6e02d00 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.844+0000 7f85eadc0b80 10 client.17160 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.844+0000 7f85eadc0b80 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.844+0000 7f85eadc0b80 20 client.17160 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.844+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a6f6f89600 0x55a6f6f8ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.844+0000 7f85eadc0b80 4 client.17160 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.844+0000 7f85eadc0b80 1 -- 172.21.15.46:0/2480699137 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a6f6e2fb20 con 0x55a6f6f89600 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.844+0000 7f85da146700 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a6f6f89600 0x55a6f6f8ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.844+0000 7f85bdffb700 10 client.17160 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.844+0000 7f85da146700 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a6f6f89600 0x55a6f6f8ba20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47151af358e572c0 server_cookie=c9c9fc9630a724d5 in_seq=0 out_seq=0 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85bdffb700 1 -- 172.21.15.46:0/2480699137 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a6f6e2fb20 con 0x55a6f6f89600 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85bdffb700 10 client.17160 handle_command_reply: tid=0 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85eadc0b80 1 client.17160 shutdown 2024-05-09T00:22:02.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85eadc0b80 1 -- 172.21.15.46:0/2480699137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 msgr2=0x55a6f6f180e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:02.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f180e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f85c400b9b0 tx=0x7f85c4005840 comp rx=0 tx=0).stop 2024-05-09T00:22:02.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85eadc0b80 1 -- 172.21.15.46:0/2480699137 shutdown_connections 2024-05-09T00:22:02.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a6f6f89600 0x55a6f6f8ba20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a6f6f05400 0x55a6f6f18620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6f6d884a0 0x55a6f6f1ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85eadc0b80 1 --2- 172.21.15.46:0/2480699137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f6e02d00 0x55a6f6f180e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85bdffb700 0 client.17160 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:02.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85eadc0b80 1 -- 172.21.15.46:0/2480699137 >> 172.21.15.46:0/2480699137 conn(0x55a6f6d8bf70 msgr2=0x55a6f6f09390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:02.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85eadc0b80 1 -- 172.21.15.46:0/2480699137 shutdown_connections 2024-05-09T00:22:02.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85eadc0b80 1 -- 172.21.15.46:0/2480699137 wait complete. 2024-05-09T00:22:02.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.845+0000 7f85eadc0b80 20 client.17160 trim_cache size 0 max 16384 2024-05-09T00:22:02.851 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:22:02.851 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:22:02.851 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:22:02.851 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:22:02.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.846+0000 7f85dc3aa700 1 -- 172.21.15.46:0/1261207559 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f85c00b2900 msgr2=0x7f85c00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:02.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.846+0000 7f85dc3aa700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f85c00b2900 0x7f85c00b4e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f85d000f4d0 tx=0x7f85d0005f90 comp rx=0 tx=0).stop 2024-05-09T00:22:02.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.846+0000 7f85dc3aa700 1 -- 172.21.15.46:0/1261207559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4109870 msgr2=0x7f85d419f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:02.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.846+0000 7f85dc3aa700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4109870 0x7f85d419f640 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009fd0 tx=0x7f85cc007a00 comp rx=0 tx=0).stop 2024-05-09T00:22:02.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.846+0000 7f85dc3aa700 1 -- 172.21.15.46:0/1261207559 shutdown_connections 2024-05-09T00:22:02.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.846+0000 7f85dc3aa700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f85c00b2900 0x7f85c00b4e10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.846+0000 7f85dc3aa700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4109870 0x7f85d419f640 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.846+0000 7f85dc3aa700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85d41045f0 0x7f85d419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.846+0000 7f85dc3aa700 1 --2- 172.21.15.46:0/1261207559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85d4108f20 0x7f85d419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:02.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.846+0000 7f85dc3aa700 1 -- 172.21.15.46:0/1261207559 >> 172.21.15.46:0/1261207559 conn(0x7f85d4100040 msgr2=0x7f85d4108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:02.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.846+0000 7f85dc3aa700 1 -- 172.21.15.46:0/1261207559 shutdown_connections 2024-05-09T00:22:02.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:02.846+0000 7f85dc3aa700 1 -- 172.21.15.46:0/1261207559 wait complete. 2024-05-09T00:22:02.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:02 smithi169 ceph-mon[34777]: pgmap v696: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:22:02.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:02 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:22:02.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:02 smithi142 ceph-mon[34533]: pgmap v696: 97 pgs: 97 active+clean; 6.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:22:02.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:02 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:22:03.659 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:22:03.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:22:03.916 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:22:04.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:04 smithi169 ceph-mon[34777]: pgmap v697: 97 pgs: 97 active+clean; 6.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:22:04.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:04 smithi142 ceph-mon[34533]: pgmap v697: 97 pgs: 97 active+clean; 6.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:22:05.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:04 smithi046 ceph-mon[30151]: pgmap v697: 97 pgs: 97 active+clean; 6.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:22:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.762+0000 7fc0affa3700 1 -- 172.21.15.46:0/3424648333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0a8110c30 msgr2=0x7fc0a8113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.762+0000 7fc0affa3700 1 --2- 172.21.15.46:0/3424648333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0a8110c30 0x7fc0a8113050 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000b0a0 tx=0x7fc0a000b3b0 comp rx=0 tx=0).stop 2024-05-09T00:22:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.762+0000 7fc0affa3700 1 -- 172.21.15.46:0/3424648333 shutdown_connections 2024-05-09T00:22:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.762+0000 7fc0affa3700 1 --2- 172.21.15.46:0/3424648333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0a8110c30 0x7fc0a8113050 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.762+0000 7fc0affa3700 1 --2- 172.21.15.46:0/3424648333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a8103080 0x7fc0a8103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.762+0000 7fc0affa3700 1 --2- 172.21.15.46:0/3424648333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a81039d0 0x7fc0a81106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.762+0000 7fc0affa3700 1 -- 172.21.15.46:0/3424648333 >> 172.21.15.46:0/3424648333 conn(0x7fc0a80fedf0 msgr2=0x7fc0a8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.762+0000 7fc0affa3700 1 -- 172.21.15.46:0/3424648333 shutdown_connections 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.762+0000 7fc0affa3700 1 -- 172.21.15.46:0/3424648333 wait complete. 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.763+0000 7fc0affa3700 1 Processor -- start 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.763+0000 7fc0affa3700 1 -- start start 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.763+0000 7fc0affa3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a8103080 0x7fc0a819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.763+0000 7fc0affa3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0a81039d0 0x7fc0a819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.763+0000 7fc0affa3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a8110c30 0x7fc0a81a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.763+0000 7fc0affa3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a81166b0 con 0x7fc0a81039d0 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.763+0000 7fc0affa3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a8116530 con 0x7fc0a8103080 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.763+0000 7fc0affa3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a8116830 con 0x7fc0a8110c30 2024-05-09T00:22:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.763+0000 7fc0ad53e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0a81039d0 0x7fc0a819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.763+0000 7fc0ad53e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0a81039d0 0x7fc0a819d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46256/0 (socket says 172.21.15.46:46256) 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.763+0000 7fc0ad53e700 1 -- 172.21.15.46:0/2757362848 learned_addr learned my addr 172.21.15.46:0/2757362848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.763+0000 7fc0add3f700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a8103080 0x7fc0a819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.764+0000 7fc0ae540700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a8110c30 0x7fc0a81a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.764+0000 7fc0ad53e700 1 -- 172.21.15.46:0/2757362848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a8110c30 msgr2=0x7fc0a81a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.764+0000 7fc0ad53e700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a8110c30 0x7fc0a81a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.764+0000 7fc0ad53e700 1 -- 172.21.15.46:0/2757362848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a8103080 msgr2=0x7fc0a819cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.764+0000 7fc0ad53e700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a8103080 0x7fc0a819cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.764+0000 7fc0ad53e700 1 -- 172.21.15.46:0/2757362848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a81a1e00 con 0x7fc0a81039d0 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.764+0000 7fc0ad53e700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0a81039d0 0x7fc0a819d580 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc09800c9b0 tx=0x7fc09800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.764+0000 7fc09effd700 1 -- 172.21.15.46:0/2757362848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098007aa0 con 0x7fc0a81039d0 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.764+0000 7fc09effd700 1 -- 172.21.15.46:0/2757362848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc098007c40 con 0x7fc0a81039d0 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.764+0000 7fc0affa3700 1 -- 172.21.15.46:0/2757362848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a81a20f0 con 0x7fc0a81039d0 2024-05-09T00:22:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.764+0000 7fc09effd700 1 -- 172.21.15.46:0/2757362848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098018660 con 0x7fc0a81039d0 2024-05-09T00:22:05.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.764+0000 7fc0affa3700 1 -- 172.21.15.46:0/2757362848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a81a9ac0 con 0x7fc0a81039d0 2024-05-09T00:22:05.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.765+0000 7fc0affa3700 1 -- 172.21.15.46:0/2757362848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a80512c0 con 0x7fc0a81039d0 2024-05-09T00:22:05.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.767+0000 7fc09effd700 1 -- 172.21.15.46:0/2757362848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc09800f450 con 0x7fc0a81039d0 2024-05-09T00:22:05.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.768+0000 7fc09effd700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0940b2900 0x7fc0940b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:05.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.768+0000 7fc0add3f700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0940b2900 0x7fc0940b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:05.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.768+0000 7fc09effd700 1 -- 172.21.15.46:0/2757362848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc0980e23f0 con 0x7fc0a81039d0 2024-05-09T00:22:05.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.769+0000 7fc0add3f700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0940b2900 0x7fc0940b4e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4005950 tx=0x7fc0a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:05.772+0000 7fc09effd700 1 -- 172.21.15.46:0/2757362848 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc098014030 con 0x7fc0a81039d0 2024-05-09T00:22:06.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.034+0000 7fc0affa3700 1 -- 172.21.15.46:0/2757362848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0a802e5f0 con 0x7fc0a81039d0 2024-05-09T00:22:06.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.035+0000 7fc09effd700 1 -- 172.21.15.46:0/2757362848 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7fc0980aa150 con 0x7fc0a81039d0 2024-05-09T00:22:06.036 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:22:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:22:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.037+0000 7fc0affa3700 1 -- 172.21.15.46:0/2757362848 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0940b2900 msgr2=0x7fc0940b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.037+0000 7fc0affa3700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0940b2900 0x7fc0940b4e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4005950 tx=0x7fc0a400b410 comp rx=0 tx=0).stop 2024-05-09T00:22:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.038+0000 7fc0affa3700 1 -- 172.21.15.46:0/2757362848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0a81039d0 msgr2=0x7fc0a819d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.038+0000 7fc0affa3700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0a81039d0 0x7fc0a819d580 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc09800c9b0 tx=0x7fc09800ccc0 comp rx=0 tx=0).stop 2024-05-09T00:22:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.038+0000 7fc0affa3700 1 -- 172.21.15.46:0/2757362848 shutdown_connections 2024-05-09T00:22:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.038+0000 7fc0affa3700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0940b2900 0x7fc0940b4e10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.038+0000 7fc0affa3700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a8110c30 0x7fc0a81a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.038+0000 7fc0affa3700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a8103080 0x7fc0a819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.038+0000 7fc0affa3700 1 --2- 172.21.15.46:0/2757362848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0a81039d0 0x7fc0a819d580 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.038+0000 7fc0affa3700 1 -- 172.21.15.46:0/2757362848 >> 172.21.15.46:0/2757362848 conn(0x7fc0a80fedf0 msgr2=0x7fc0a81126b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.038+0000 7fc0affa3700 1 -- 172.21.15.46:0/2757362848 shutdown_connections 2024-05-09T00:22:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:06.038+0000 7fc0affa3700 1 -- 172.21.15.46:0/2757362848 wait complete. 2024-05-09T00:22:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:22:06.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:06 smithi046 ceph-mon[30151]: pgmap v698: 97 pgs: 97 active+clean; 6.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:06.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:06 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2757362848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:22:06.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:06 smithi169 ceph-mon[34777]: pgmap v698: 97 pgs: 97 active+clean; 6.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:06.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:06 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2757362848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:22:06.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:22:06.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:06 smithi142 ceph-mon[34533]: pgmap v698: 97 pgs: 97 active+clean; 6.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:06.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:06 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2757362848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:22:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:22:08.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:08 smithi046 ceph-mon[30151]: pgmap v699: 97 pgs: 97 active+clean; 6.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:22:08.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:08 smithi169 ceph-mon[34777]: pgmap v699: 97 pgs: 97 active+clean; 6.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:22:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.898+0000 7fc022220700 1 -- 172.21.15.46:0/975855643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc01c10e920 msgr2=0x7fc01c110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.898+0000 7fc022220700 1 --2- 172.21.15.46:0/975855643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc01c10e920 0x7fc01c110e00 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc01400b0a0 tx=0x7fc01400b3b0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.898+0000 7fc022220700 1 -- 172.21.15.46:0/975855643 shutdown_connections 2024-05-09T00:22:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.898+0000 7fc022220700 1 --2- 172.21.15.46:0/975855643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc01c10e920 0x7fc01c110e00 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.898+0000 7fc022220700 1 --2- 172.21.15.46:0/975855643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c1054f0 0x7fc01c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.898+0000 7fc022220700 1 --2- 172.21.15.46:0/975855643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01c105e40 0x7fc01c10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.898+0000 7fc022220700 1 -- 172.21.15.46:0/975855643 >> 172.21.15.46:0/975855643 conn(0x7fc01c078e00 msgr2=0x7fc01c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.899+0000 7fc022220700 1 -- 172.21.15.46:0/975855643 shutdown_connections 2024-05-09T00:22:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.899+0000 7fc022220700 1 -- 172.21.15.46:0/975855643 wait complete. 2024-05-09T00:22:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.899+0000 7fc022220700 1 Processor -- start 2024-05-09T00:22:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.899+0000 7fc022220700 1 -- start start 2024-05-09T00:22:08.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc022220700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc01c1054f0 0x7fc01c19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc022220700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c105e40 0x7fc01c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc022220700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01c10e920 0x7fc01c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc022220700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1144f0 con 0x7fc01c1054f0 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc022220700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c114370 con 0x7fc01c105e40 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc022220700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c114670 con 0x7fc01c10e920 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc01b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc01c1054f0 0x7fc01c19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc01b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc01c1054f0 0x7fc01c19aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36150/0 (socket says 172.21.15.46:36150) 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc01b7fe700 1 -- 172.21.15.46:0/548937777 learned_addr learned my addr 172.21.15.46:0/548937777 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc01bfff700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01c10e920 0x7fc01c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc01affd700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c105e40 0x7fc01c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc01b7fe700 1 -- 172.21.15.46:0/548937777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01c10e920 msgr2=0x7fc01c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc01b7fe700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01c10e920 0x7fc01c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc01b7fe700 1 -- 172.21.15.46:0/548937777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c105e40 msgr2=0x7fc01c19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc01b7fe700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c105e40 0x7fc01c19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc01b7fe700 1 -- 172.21.15.46:0/548937777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c19fd70 con 0x7fc01c1054f0 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc01bfff700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01c10e920 0x7fc01c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.900+0000 7fc01b7fe700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc01c1054f0 0x7fc01c19aea0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00b530 tx=0x7fc00c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:08.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.901+0000 7fc018ff9700 1 -- 172.21.15.46:0/548937777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c015020 con 0x7fc01c1054f0 2024-05-09T00:22:08.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.901+0000 7fc018ff9700 1 -- 172.21.15.46:0/548937777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc00c011ce0 con 0x7fc01c1054f0 2024-05-09T00:22:08.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.901+0000 7fc018ff9700 1 -- 172.21.15.46:0/548937777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c010d20 con 0x7fc01c1054f0 2024-05-09T00:22:08.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.901+0000 7fc022220700 1 -- 172.21.15.46:0/548937777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c1a0060 con 0x7fc01c1054f0 2024-05-09T00:22:08.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.901+0000 7fc022220700 1 -- 172.21.15.46:0/548937777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c1a7980 con 0x7fc01c1054f0 2024-05-09T00:22:08.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.904+0000 7fc018ff9700 1 -- 172.21.15.46:0/548937777 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc00c018020 con 0x7fc01c1054f0 2024-05-09T00:22:08.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.904+0000 7fc018ff9700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0040b2900 0x7fc0040b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.904+0000 7fc030c36b80 1 Processor -- start 2024-05-09T00:22:08.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc01affd700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0040b2900 0x7fc0040b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc030c36b80 1 -- start start 2024-05-09T00:22:08.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc018ff9700 1 -- 172.21.15.46:0/548937777 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc00c0e19d0 con 0x7fc01c1054f0 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc030c36b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d48e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc030c36b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d30b4a0 0x564b9d48ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc030c36b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d4886a0 0x564b9d488ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc030c36b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564b9d3aaff0 con 0x564b9d30b4a0 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc030c36b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564b9d3497f0 con 0x564b9d4886a0 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc030c36b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564b9d3a6fb0 con 0x564b9d3f5160 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc01b7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d48e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc01affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d30b4a0 0x564b9d48ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc01b7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d48e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.46:50176/0 (socket says 172.21.15.46:50176) 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc01b7fe700 1 -- 172.21.15.46:0/207543027 learned_addr learned my addr 172.21.15.46:0/207543027 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc01affd700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0040b2900 0x7fc0040b4e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc010005fd0 tx=0x7fc01000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc01bfff700 1 --2- 172.21.15.46:0/207543027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d4886a0 0x564b9d488ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc01affd700 1 -- 172.21.15.46:0/207543027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 msgr2=0x564b9d48e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc01affd700 1 --2- 172.21.15.46:0/207543027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d48e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc01affd700 1 -- 172.21.15.46:0/207543027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d4886a0 msgr2=0x564b9d488ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.905+0000 7fc01affd700 1 --2- 172.21.15.46:0/207543027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d4886a0 0x564b9d488ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.906+0000 7fc01affd700 1 -- 172.21.15.46:0/207543027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b9d341370 con 0x564b9d30b4a0 2024-05-09T00:22:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.906+0000 7fc01b7fe700 1 --2- 172.21.15.46:0/207543027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d48e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:22:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.906+0000 7fc01affd700 1 --2- 172.21.15.46:0/207543027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d30b4a0 0x564b9d48ea30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc01c110700 tx=0x7fc010011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8523234979d04b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.906+0000 7fc001ffb700 1 -- 172.21.15.46:0/207543027 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010021070 con 0x564b9d30b4a0 2024-05-09T00:22:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.906+0000 7fc001ffb700 1 -- 172.21.15.46:0/207543027 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc01000ee90 con 0x564b9d30b4a0 2024-05-09T00:22:08.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.906+0000 7fc001ffb700 1 -- 172.21.15.46:0/207543027 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01000f620 con 0x564b9d30b4a0 2024-05-09T00:22:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.907+0000 7fc030c36b80 1 -- 172.21.15.46:0/207543027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d30b4a0 msgr2=0x564b9d48ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.907+0000 7fc030c36b80 1 --2- 172.21.15.46:0/207543027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d30b4a0 0x564b9d48ea30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc01c110700 tx=0x7fc010011000 comp rx=0 tx=0).stop 2024-05-09T00:22:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.907+0000 7fc030c36b80 1 -- 172.21.15.46:0/207543027 shutdown_connections 2024-05-09T00:22:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.907+0000 7fc030c36b80 1 --2- 172.21.15.46:0/207543027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d4886a0 0x564b9d488ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.907+0000 7fc030c36b80 1 --2- 172.21.15.46:0/207543027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d48e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.907+0000 7fc030c36b80 1 --2- 172.21.15.46:0/207543027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d30b4a0 0x564b9d48ea30 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.907+0000 7fc030c36b80 1 -- 172.21.15.46:0/207543027 >> 172.21.15.46:0/207543027 conn(0x564b9d36b6a0 msgr2=0x564b9d483400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.907+0000 7fc030c36b80 1 -- 172.21.15.46:0/207543027 shutdown_connections 2024-05-09T00:22:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.907+0000 7fc030c36b80 1 -- 172.21.15.46:0/207543027 wait complete. 2024-05-09T00:22:08.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.907+0000 7fc030c36b80 1 -- start start 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc030c36b80 1 Processor -- start 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc030c36b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 0x564b9d492120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc030c36b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d492660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc030c36b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d492ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc030c36b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564b9d226ba0 con 0x564b9d4886a0 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc030c36b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564b9d365150 con 0x564b9d30b4a0 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc030c36b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564b9d4951d0 con 0x564b9d3f5160 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc030c36b80 10 client.0 _ll_register_callbacks cb 0x564b9d21c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc01bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d492ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc01bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d492ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36158/0 (socket says 172.21.15.46:36158) 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc01bfff700 1 -- 172.21.15.46:0/4263226601 learned_addr learned my addr 172.21.15.46:0/4263226601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc01b7fe700 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 0x564b9d492120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc01affd700 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d492660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc01bfff700 1 -- 172.21.15.46:0/4263226601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 msgr2=0x564b9d492660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.908+0000 7fc01bfff700 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d492660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.909+0000 7fc01bfff700 1 -- 172.21.15.46:0/4263226601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 msgr2=0x564b9d492120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.909+0000 7fc01bfff700 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 0x564b9d492120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.909+0000 7fc01bfff700 1 -- 172.21.15.46:0/4263226601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b9d226490 con 0x564b9d4886a0 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.909+0000 7fc01bfff700 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d492ba0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fc01400b0a0 tx=0x7fc014004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbeea55611cdfb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.909+0000 7fbffaffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.909+0000 7fbffaffd700 1 -- 172.21.15.46:0/4263226601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0140079d0 con 0x564b9d4886a0 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.909+0000 7fbffaffd700 1 -- 172.21.15.46:0/4263226601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc014007b70 con 0x564b9d4886a0 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.909+0000 7fc030c36b80 1 -- 172.21.15.46:0/4263226601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01000ee90 con 0x564b9d4886a0 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.909+0000 7fbffaffd700 1 -- 172.21.15.46:0/4263226601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc014007d70 con 0x564b9d4886a0 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.909+0000 7fc030c36b80 10 client.17178 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.909+0000 7fc030c36b80 1 -- 172.21.15.46:0/4263226601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b9d49fbc0 con 0x564b9d4886a0 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.909+0000 7fbffaffd700 1 -- 172.21.15.46:0/4263226601 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc01000ee90 con 0x564b9d4886a0 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.910+0000 7fbffaffd700 1 -- 172.21.15.46:0/4263226601 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc014011070 con 0x564b9d4886a0 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.910+0000 7fc030c36b80 10 client.17178 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.910+0000 7fc030c36b80 10 client.17178 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:22:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.910+0000 7fc030c36b80 20 client.17178 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.910+0000 7fc030c36b80 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x564b9d4a21a0 0x564b9d4a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.910+0000 7fc030c36b80 4 client.17178 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:22:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.910+0000 7fc030c36b80 1 -- 172.21.15.46:0/4263226601 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b9d3b89d0 con 0x564b9d4a21a0 2024-05-09T00:22:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.910+0000 7fc01b7fe700 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x564b9d4a21a0 0x564b9d4a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.911+0000 7fc01b7fe700 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x564b9d4a21a0 0x564b9d4a4600 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f38d3bcaac6fb3fe server_cookie=9f70fc09e7a467fb in_seq=0 out_seq=0 2024-05-09T00:22:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.911+0000 7fbffaffd700 10 client.17178 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:22:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.911+0000 7fbffaffd700 1 -- 172.21.15.46:0/4263226601 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b9d3b89d0 con 0x564b9d4a21a0 2024-05-09T00:22:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.911+0000 7fbffaffd700 10 client.17178 handle_command_reply: tid=0 2024-05-09T00:22:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.911+0000 7fc030c36b80 1 client.17178 shutdown 2024-05-09T00:22:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.912+0000 7fc030c36b80 1 -- 172.21.15.46:0/4263226601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 msgr2=0x564b9d492ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.912+0000 7fc030c36b80 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d492ba0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fc01400b0a0 tx=0x7fc014004d30 comp rx=0 tx=0).stop 2024-05-09T00:22:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.912+0000 7fc030c36b80 1 -- 172.21.15.46:0/4263226601 shutdown_connections 2024-05-09T00:22:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.912+0000 7fc030c36b80 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x564b9d4a21a0 0x564b9d4a4600 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.912+0000 7fc030c36b80 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d492ba0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.912+0000 7fc030c36b80 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 0x564b9d492120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.912+0000 7fc030c36b80 1 --2- 172.21.15.46:0/4263226601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d492660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.912+0000 7fc030c36b80 1 -- 172.21.15.46:0/4263226601 >> 172.21.15.46:0/4263226601 conn(0x564b9d36b6a0 msgr2=0x564b9d4831c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.912+0000 7fbffaffd700 0 client.17178 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:22:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.912+0000 7fc030c36b80 1 -- 172.21.15.46:0/4263226601 shutdown_connections 2024-05-09T00:22:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.912+0000 7fc030c36b80 1 -- 172.21.15.46:0/4263226601 wait complete. 2024-05-09T00:22:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.912+0000 7fc030c36b80 20 client.17178 trim_cache size 0 max 16384 2024-05-09T00:22:08.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.927+0000 7fc030c36b80 1 Processor -- start 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.927+0000 7fc030c36b80 1 -- start start 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.927+0000 7fc030c36b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 0x564b9d48c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.927+0000 7fc030c36b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d49e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.927+0000 7fc030c36b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d4a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.927+0000 7fc030c36b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1b1580 con 0x564b9d4886a0 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.927+0000 7fc030c36b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1aee10 con 0x564b9d30b4a0 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.927+0000 7fc030c36b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1acc40 con 0x564b9d3f5160 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.927+0000 7fc01bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d4a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.927+0000 7fc01bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d4a9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36174/0 (socket says 172.21.15.46:36174) 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.927+0000 7fc01bfff700 1 -- 172.21.15.46:0/750236751 learned_addr learned my addr 172.21.15.46:0/750236751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.928+0000 7fc01b7fe700 1 --2- 172.21.15.46:0/750236751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 0x564b9d48c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.928+0000 7fc01affd700 1 --2- 172.21.15.46:0/750236751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d49e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.928+0000 7fc01bfff700 1 -- 172.21.15.46:0/750236751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 msgr2=0x564b9d49e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.928+0000 7fc01bfff700 1 --2- 172.21.15.46:0/750236751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d49e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.928+0000 7fc01bfff700 1 -- 172.21.15.46:0/750236751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 msgr2=0x564b9d48c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.928+0000 7fc01bfff700 1 --2- 172.21.15.46:0/750236751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 0x564b9d48c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.928+0000 7fc01bfff700 1 -- 172.21.15.46:0/750236751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b9d4906b0 con 0x564b9d4886a0 2024-05-09T00:22:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.928+0000 7fc01bfff700 1 --2- 172.21.15.46:0/750236751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d4a9dc0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc014009900 tx=0x7fc014007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19abe1df60450517 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.928+0000 7fc0017fa700 1 -- 172.21.15.46:0/750236751 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014036070 con 0x564b9d4886a0 2024-05-09T00:22:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.928+0000 7fc0017fa700 1 -- 172.21.15.46:0/750236751 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc014005c60 con 0x564b9d4886a0 2024-05-09T00:22:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.928+0000 7fc0017fa700 1 -- 172.21.15.46:0/750236751 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014032460 con 0x564b9d4886a0 2024-05-09T00:22:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.929+0000 7fc030c36b80 1 -- 172.21.15.46:0/750236751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 msgr2=0x564b9d4a9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.929+0000 7fc030c36b80 1 --2- 172.21.15.46:0/750236751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d4a9dc0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc014009900 tx=0x7fc014007f90 comp rx=0 tx=0).stop 2024-05-09T00:22:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.929+0000 7fc030c36b80 1 -- 172.21.15.46:0/750236751 shutdown_connections 2024-05-09T00:22:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.929+0000 7fc030c36b80 1 --2- 172.21.15.46:0/750236751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d4a9dc0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.929+0000 7fc030c36b80 1 --2- 172.21.15.46:0/750236751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 0x564b9d48c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.929+0000 7fc030c36b80 1 --2- 172.21.15.46:0/750236751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d49e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.929+0000 7fc030c36b80 1 -- 172.21.15.46:0/750236751 >> 172.21.15.46:0/750236751 conn(0x564b9d36b6a0 msgr2=0x564b9d48be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.929+0000 7fc030c36b80 1 -- 172.21.15.46:0/750236751 shutdown_connections 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.929+0000 7fc030c36b80 1 -- 172.21.15.46:0/750236751 wait complete. 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.929+0000 7fc030c36b80 1 -- start start 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc030c36b80 1 Processor -- start 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc030c36b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 0x564b9d49a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc030c36b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d49a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc030c36b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d48e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc030c36b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ad2d0 con 0x564b9d4886a0 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc030c36b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ae740 con 0x564b9d30b4a0 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc030c36b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ad9a0 con 0x564b9d3f5160 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc030c36b80 10 client.0 _ll_register_callbacks cb 0x564b9d21c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc01bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d48e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc01bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d48e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36186/0 (socket says 172.21.15.46:36186) 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc01bfff700 1 -- 172.21.15.46:0/2542370054 learned_addr learned my addr 172.21.15.46:0/2542370054 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc01affd700 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d49a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc01b7fe700 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 0x564b9d49a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc01bfff700 1 -- 172.21.15.46:0/2542370054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 msgr2=0x564b9d49a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc01bfff700 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d49a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc01bfff700 1 -- 172.21.15.46:0/2542370054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 msgr2=0x564b9d49a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc01bfff700 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 0x564b9d49a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc01bfff700 1 -- 172.21.15.46:0/2542370054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b9d4900e0 con 0x564b9d4886a0 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.930+0000 7fc01affd700 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d49a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.931+0000 7fc01bfff700 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d48e4f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc014009560 tx=0x7fc01400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd803050dfcb4094 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.931+0000 7fbffaffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.931+0000 7fbffaffd700 1 -- 172.21.15.46:0/2542370054 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014036040 con 0x564b9d4886a0 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.931+0000 7fbffaffd700 1 -- 172.21.15.46:0/2542370054 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fc0140077d0 con 0x564b9d4886a0 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.931+0000 7fc030c36b80 1 -- 172.21.15.46:0/2542370054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc014005c60 con 0x564b9d4886a0 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.931+0000 7fbffaffd700 1 -- 172.21.15.46:0/2542370054 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc0140079d0 con 0x564b9d4886a0 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.931+0000 7fc030c36b80 10 client.17184 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.931+0000 7fc030c36b80 1 -- 172.21.15.46:0/2542370054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b9d50be10 con 0x564b9d4886a0 2024-05-09T00:22:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.931+0000 7fbffaffd700 1 -- 172.21.15.46:0/2542370054 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc014005c60 con 0x564b9d4886a0 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.931+0000 7fbffaffd700 1 -- 172.21.15.46:0/2542370054 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc014011050 con 0x564b9d4886a0 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.932+0000 7fc030c36b80 10 client.17184 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.932+0000 7fc030c36b80 10 client.17184 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.932+0000 7fc030c36b80 20 client.17184 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.932+0000 7fc030c36b80 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x564b9d50c270 0x564b9d50e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.932+0000 7fc030c36b80 4 client.17184 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.932+0000 7fc030c36b80 1 -- 172.21.15.46:0/2542370054 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b9d3b2a70 con 0x564b9d50c270 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.932+0000 7fc01b7fe700 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x564b9d50c270 0x564b9d50e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.932+0000 7fc01b7fe700 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x564b9d50c270 0x564b9d50e690 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7d373a8575171f5 server_cookie=463ec1506b11616c in_seq=0 out_seq=0 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.932+0000 7fbffaffd700 10 client.17184 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fbffaffd700 1 -- 172.21.15.46:0/2542370054 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b9d3b2a70 con 0x564b9d50c270 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fbffaffd700 10 client.17184 handle_command_reply: tid=0 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fc030c36b80 1 client.17184 shutdown 2024-05-09T00:22:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fc030c36b80 1 -- 172.21.15.46:0/2542370054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 msgr2=0x564b9d48e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fc030c36b80 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d48e4f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc014009560 tx=0x7fc01400b860 comp rx=0 tx=0).stop 2024-05-09T00:22:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fc030c36b80 1 -- 172.21.15.46:0/2542370054 shutdown_connections 2024-05-09T00:22:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fc030c36b80 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x564b9d50c270 0x564b9d50e690 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fc030c36b80 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b9d4886a0 0x564b9d48e4f0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fc030c36b80 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b9d30b4a0 0x564b9d49a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fc030c36b80 1 --2- 172.21.15.46:0/2542370054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9d3f5160 0x564b9d49a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fc030c36b80 1 -- 172.21.15.46:0/2542370054 >> 172.21.15.46:0/2542370054 conn(0x564b9d36b6a0 msgr2=0x564b9d487690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fbffaffd700 0 client.17184 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:22:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fc030c36b80 1 -- 172.21.15.46:0/2542370054 shutdown_connections 2024-05-09T00:22:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fc030c36b80 1 -- 172.21.15.46:0/2542370054 wait complete. 2024-05-09T00:22:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.933+0000 7fc030c36b80 20 client.17184 trim_cache size 0 max 16384 2024-05-09T00:22:08.939 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:22:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.934+0000 7fc022220700 1 -- 172.21.15.46:0/548937777 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0040b2900 msgr2=0x7fc0040b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.934+0000 7fc022220700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0040b2900 0x7fc0040b4e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc010005fd0 tx=0x7fc01000a560 comp rx=0 tx=0).stop 2024-05-09T00:22:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.934+0000 7fc022220700 1 -- 172.21.15.46:0/548937777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc01c1054f0 msgr2=0x7fc01c19aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.934+0000 7fc022220700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc01c1054f0 0x7fc01c19aea0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00b530 tx=0x7fc00c00b8f0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.934+0000 7fc022220700 1 -- 172.21.15.46:0/548937777 shutdown_connections 2024-05-09T00:22:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.934+0000 7fc022220700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fc0040b2900 0x7fc0040b4e10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.934+0000 7fc022220700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01c10e920 0x7fc01c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.934+0000 7fc022220700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc01c1054f0 0x7fc01c19aea0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.934+0000 7fc022220700 1 --2- 172.21.15.46:0/548937777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc01c105e40 0x7fc01c19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.934+0000 7fc022220700 1 -- 172.21.15.46:0/548937777 >> 172.21.15.46:0/548937777 conn(0x7fc01c078e00 msgr2=0x7fc01c1000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.934+0000 7fc022220700 1 -- 172.21.15.46:0/548937777 shutdown_connections 2024-05-09T00:22:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:08.934+0000 7fc022220700 1 -- 172.21.15.46:0/548937777 wait complete. 2024-05-09T00:22:08.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:08 smithi142 ceph-mon[34533]: pgmap v699: 97 pgs: 97 active+clean; 6.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:22:09.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:22:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:22:10.813 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:10 smithi046 ceph-mon[30151]: pgmap v700: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:10.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:10 smithi169 ceph-mon[34777]: pgmap v700: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:10.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:10 smithi142 ceph-mon[34533]: pgmap v700: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.939+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/4072355831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a4108d40 msgr2=0x7ff9a4109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.939+0000 7ff9a8b9e700 1 --2- 172.21.15.46:0/4072355831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a4108d40 0x7ff9a4109150 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0099c0 tx=0x7ff98c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.939+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/4072355831 shutdown_connections 2024-05-09T00:22:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.939+0000 7ff9a8b9e700 1 --2- 172.21.15.46:0/4072355831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9a4109690 0x7ff9a4110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.939+0000 7ff9a8b9e700 1 --2- 172.21.15.46:0/4072355831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9a4104410 0x7ff9a4104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.939+0000 7ff9a8b9e700 1 --2- 172.21.15.46:0/4072355831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a4108d40 0x7ff9a4109150 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.939+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/4072355831 >> 172.21.15.46:0/4072355831 conn(0x7ff9a4079d50 msgr2=0x7ff9a407a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.939+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/4072355831 shutdown_connections 2024-05-09T00:22:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.939+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/4072355831 wait complete. 2024-05-09T00:22:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff9a8b9e700 1 Processor -- start 2024-05-09T00:22:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff9a8b9e700 1 -- start start 2024-05-09T00:22:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff9a8b9e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a4104410 0x7ff9a4075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff9a8b9e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9a4108d40 0x7ff9a4076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff9a8b9e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9a4109690 0x7ff9a4076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff9a8b9e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41142f0 con 0x7ff9a4104410 2024-05-09T00:22:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff9a8b9e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4114170 con 0x7ff9a4109690 2024-05-09T00:22:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff9a8b9e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4114470 con 0x7ff9a4108d40 2024-05-09T00:22:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff9a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a4104410 0x7ff9a4075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff9a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a4104410 0x7ff9a4075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36194/0 (socket says 172.21.15.46:36194) 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff9a37fe700 1 -- 172.21.15.46:0/948108554 learned_addr learned my addr 172.21.15.46:0/948108554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff9a3fff700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9a4109690 0x7ff9a4076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.940+0000 7ff99affd700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9a4108d40 0x7ff9a4076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.941+0000 7ff9a37fe700 1 -- 172.21.15.46:0/948108554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9a4108d40 msgr2=0x7ff9a4076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.941+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9a4108d40 0x7ff9a4076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.941+0000 7ff9a37fe700 1 -- 172.21.15.46:0/948108554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9a4109690 msgr2=0x7ff9a4076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.941+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9a4109690 0x7ff9a4076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.941+0000 7ff9a37fe700 1 -- 172.21.15.46:0/948108554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a41a9060 con 0x7ff9a4104410 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.941+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a4104410 0x7ff9a4075ac0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c920 tx=0x7ff99400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.941+0000 7ff9a17fa700 1 -- 172.21.15.46:0/948108554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994004a30 con 0x7ff9a4104410 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.941+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/948108554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a41a9350 con 0x7ff9a4104410 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.941+0000 7ff9a17fa700 1 -- 172.21.15.46:0/948108554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9940054d0 con 0x7ff9a4104410 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.941+0000 7ff9a17fa700 1 -- 172.21.15.46:0/948108554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994007bd0 con 0x7ff9a4104410 2024-05-09T00:22:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.941+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/948108554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a41a9860 con 0x7ff9a4104410 2024-05-09T00:22:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.945+0000 7ff9a17fa700 1 -- 172.21.15.46:0/948108554 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff99400f450 con 0x7ff9a4104410 2024-05-09T00:22:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9a17fa700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9900b2880 0x7ff9900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff99affd700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9900b2880 0x7ff9900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9a17fa700 1 -- 172.21.15.46:0/948108554 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff9940e1e00 con 0x7ff9a4104410 2024-05-09T00:22:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9b8816b80 1 Processor -- start 2024-05-09T00:22:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9b8816b80 1 -- start start 2024-05-09T00:22:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9b8816b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 0x55f969fd0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9b8816b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969e4d4a0 0x55f969fd0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9b8816b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969fca800 0x55f969fcac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff99affd700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9900b2880 0x7ff9900b4d90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff98c006010 tx=0x7ff98c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9b8816b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f969eecff0 con 0x55f969f37160 2024-05-09T00:22:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9b8816b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f969e8b7f0 con 0x55f969fca800 2024-05-09T00:22:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9b8816b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f969ee8fb0 con 0x55f969e4d4a0 2024-05-09T00:22:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 0x55f969fd0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 0x55f969fd0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36200/0 (socket says 172.21.15.46:36200) 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.946+0000 7ff9a37fe700 1 -- 172.21.15.46:0/2295015370 learned_addr learned my addr 172.21.15.46:0/2295015370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff9a3fff700 1 --2- 172.21.15.46:0/2295015370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969fca800 0x55f969fcac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff99affd700 1 --2- 172.21.15.46:0/2295015370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969e4d4a0 0x55f969fd0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff9a37fe700 1 -- 172.21.15.46:0/2295015370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969e4d4a0 msgr2=0x55f969fd0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/2295015370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969e4d4a0 0x55f969fd0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff9a37fe700 1 -- 172.21.15.46:0/2295015370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969fca800 msgr2=0x55f969fcac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/2295015370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969fca800 0x55f969fcac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff9a37fe700 1 -- 172.21.15.46:0/2295015370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f969e83370 con 0x55f969f37160 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff99affd700 1 --2- 172.21.15.46:0/2295015370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969e4d4a0 0x55f969fd0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff9a3fff700 1 --2- 172.21.15.46:0/2295015370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969fca800 0x55f969fcac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/2295015370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 0x55f969fd0640 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff9940a9370 tx=0x7ff994012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1186d9d692740f22 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff999ffb700 1 -- 172.21.15.46:0/2295015370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99401d040 con 0x55f969f37160 2024-05-09T00:22:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff999ffb700 1 -- 172.21.15.46:0/2295015370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9940ad790 con 0x55f969f37160 2024-05-09T00:22:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.947+0000 7ff999ffb700 1 -- 172.21.15.46:0/2295015370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940a9640 con 0x55f969f37160 2024-05-09T00:22:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.948+0000 7ff9b8816b80 1 -- 172.21.15.46:0/2295015370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 msgr2=0x55f969fd0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.948+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/2295015370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 0x55f969fd0640 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff9940a9370 tx=0x7ff994012000 comp rx=0 tx=0).stop 2024-05-09T00:22:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.948+0000 7ff9b8816b80 1 -- 172.21.15.46:0/2295015370 shutdown_connections 2024-05-09T00:22:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.948+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/2295015370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969fca800 0x55f969fcac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.948+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/2295015370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 0x55f969fd0640 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.948+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/2295015370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969e4d4a0 0x55f969fd0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.948+0000 7ff9b8816b80 1 -- 172.21.15.46:0/2295015370 >> 172.21.15.46:0/2295015370 conn(0x55f969ead6a0 msgr2=0x55f969fc5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.948+0000 7ff9b8816b80 1 -- 172.21.15.46:0/2295015370 shutdown_connections 2024-05-09T00:22:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.948+0000 7ff9b8816b80 1 -- 172.21.15.46:0/2295015370 wait complete. 2024-05-09T00:22:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.948+0000 7ff9b8816b80 1 -- start start 2024-05-09T00:22:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff9b8816b80 1 Processor -- start 2024-05-09T00:22:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff9b8816b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fd4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff9b8816b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969f37160 0x55f969fd4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff9b8816b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 0x55f969fd4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff9b8816b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f969d68ba0 con 0x55f969fca800 2024-05-09T00:22:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff9b8816b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f969ea7150 con 0x55f969e4d4a0 2024-05-09T00:22:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff9b8816b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f969fcb5f0 con 0x55f969f37160 2024-05-09T00:22:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff9b8816b80 10 client.0 _ll_register_callbacks cb 0x55f969d5e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff9a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 0x55f969fd4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff9a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 0x55f969fd4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36204/0 (socket says 172.21.15.46:36204) 2024-05-09T00:22:11.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff9a3fff700 1 -- 172.21.15.46:0/3393438099 learned_addr learned my addr 172.21.15.46:0/3393438099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff99affd700 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969f37160 0x55f969fd4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.949+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fd4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff9a3fff700 1 -- 172.21.15.46:0/3393438099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969f37160 msgr2=0x55f969fd4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff9a3fff700 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969f37160 0x55f969fd4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff9a3fff700 1 -- 172.21.15.46:0/3393438099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 msgr2=0x55f969fd4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff9a3fff700 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fd4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff9a3fff700 1 -- 172.21.15.46:0/3393438099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f969d68490 con 0x55f969fca800 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fd4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff9a3fff700 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 0x55f969fd4e80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff99c008e90 tx=0x7ff99c0114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a683cd70943cac03 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff982ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff982ffd700 1 -- 172.21.15.46:0/3393438099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c011cb0 con 0x55f969fca800 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff982ffd700 1 -- 172.21.15.46:0/3393438099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff99c016480 con 0x55f969fca800 2024-05-09T00:22:11.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3393438099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9940ad790 con 0x55f969fca800 2024-05-09T00:22:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff982ffd700 1 -- 172.21.15.46:0/3393438099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff99c011e50 con 0x55f969fca800 2024-05-09T00:22:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff9b8816b80 10 client.17196 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3393438099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f969fe1ca0 con 0x55f969fca800 2024-05-09T00:22:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.950+0000 7ff982ffd700 1 -- 172.21.15.46:0/3393438099 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9940ad790 con 0x55f969fca800 2024-05-09T00:22:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.951+0000 7ff982ffd700 1 -- 172.21.15.46:0/3393438099 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff99c01f960 con 0x55f969fca800 2024-05-09T00:22:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.951+0000 7ff9b8816b80 10 client.17196 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.951+0000 7ff9b8816b80 10 client.17196 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:22:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.951+0000 7ff9b8816b80 20 client.17196 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.951+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f969fe4330 0x55f969fe6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.951+0000 7ff9b8816b80 4 client.17196 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:22:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.951+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3393438099 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f969efa9d0 con 0x55f969fe4330 2024-05-09T00:22:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.951+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f969fe4330 0x55f969fe6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.951+0000 7ff982ffd700 10 client.17196 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:22:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.951+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f969fe4330 0x55f969fe6750 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b9a6513fc5bd857 server_cookie=e0fbd044e460fa36 in_seq=0 out_seq=0 2024-05-09T00:22:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.952+0000 7ff982ffd700 1 -- 172.21.15.46:0/3393438099 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f969efa9d0 con 0x55f969fe4330 2024-05-09T00:22:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.952+0000 7ff982ffd700 10 client.17196 handle_command_reply: tid=0 2024-05-09T00:22:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.952+0000 7ff9b8816b80 1 client.17196 shutdown 2024-05-09T00:22:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.953+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3393438099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 msgr2=0x55f969fd4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.953+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 0x55f969fd4e80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff99c008e90 tx=0x7ff99c0114a0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.953+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3393438099 shutdown_connections 2024-05-09T00:22:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.953+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f969fe4330 0x55f969fe6750 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.953+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 0x55f969fd4e80 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.953+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fd4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.953+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/3393438099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969f37160 0x55f969fd4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.953+0000 7ff982ffd700 0 client.17196 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:22:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.953+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3393438099 >> 172.21.15.46:0/3393438099 conn(0x55f969ead6a0 msgr2=0x55f969fc5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.953+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3393438099 shutdown_connections 2024-05-09T00:22:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.953+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3393438099 wait complete. 2024-05-09T00:22:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.953+0000 7ff9b8816b80 20 client.17196 trim_cache size 0 max 16384 2024-05-09T00:22:11.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.968+0000 7ff9b8816b80 1 Processor -- start 2024-05-09T00:22:11.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.968+0000 7ff9b8816b80 1 -- start start 2024-05-09T00:22:11.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.968+0000 7ff9b8816b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fcfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.968+0000 7ff9b8816b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 0x55f969fd1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.968+0000 7ff9b8816b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969fca800 0x55f969fe0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.968+0000 7ff9b8816b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b3630 con 0x55f969f37160 2024-05-09T00:22:11.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.968+0000 7ff9b8816b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b0ec0 con 0x55f969e4d4a0 2024-05-09T00:22:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.968+0000 7ff9b8816b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41aecf0 con 0x55f969fca800 2024-05-09T00:22:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.968+0000 7ff99affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 0x55f969fd1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.968+0000 7ff99affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 0x55f969fd1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36216/0 (socket says 172.21.15.46:36216) 2024-05-09T00:22:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.968+0000 7ff99affd700 1 -- 172.21.15.46:0/2159854928 learned_addr learned my addr 172.21.15.46:0/2159854928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.968+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/2159854928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fcfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.969+0000 7ff9a3fff700 1 --2- 172.21.15.46:0/2159854928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969fca800 0x55f969fe0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.969+0000 7ff99affd700 1 -- 172.21.15.46:0/2159854928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969fca800 msgr2=0x55f969fe0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.969+0000 7ff99affd700 1 --2- 172.21.15.46:0/2159854928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969fca800 0x55f969fe0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.969+0000 7ff99affd700 1 -- 172.21.15.46:0/2159854928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 msgr2=0x55f969fcfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:11.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.969+0000 7ff99affd700 1 --2- 172.21.15.46:0/2159854928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fcfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.969+0000 7ff99affd700 1 -- 172.21.15.46:0/2159854928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f969fc1b60 con 0x55f969f37160 2024-05-09T00:22:11.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.969+0000 7ff99affd700 1 --2- 172.21.15.46:0/2159854928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 0x55f969fd1e20 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff9a406b2c0 tx=0x7ff98c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904c2acd2feda5d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:11.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.969+0000 7ff9997fa700 1 -- 172.21.15.46:0/2159854928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c034070 con 0x55f969f37160 2024-05-09T00:22:11.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.969+0000 7ff9997fa700 1 -- 172.21.15.46:0/2159854928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff98c00bc40 con 0x55f969f37160 2024-05-09T00:22:11.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.969+0000 7ff9997fa700 1 -- 172.21.15.46:0/2159854928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00fc10 con 0x55f969f37160 2024-05-09T00:22:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.970+0000 7ff9b8816b80 1 -- 172.21.15.46:0/2159854928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 msgr2=0x55f969fd1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.970+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/2159854928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 0x55f969fd1e20 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff9a406b2c0 tx=0x7ff98c01e000 comp rx=0 tx=0).stop 2024-05-09T00:22:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.970+0000 7ff9b8816b80 1 -- 172.21.15.46:0/2159854928 shutdown_connections 2024-05-09T00:22:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.970+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/2159854928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969fca800 0x55f969fe0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.970+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/2159854928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fcfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.970+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/2159854928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969f37160 0x55f969fd1e20 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.970+0000 7ff9b8816b80 1 -- 172.21.15.46:0/2159854928 >> 172.21.15.46:0/2159854928 conn(0x55f969ead6a0 msgr2=0x55f969fcf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.970+0000 7ff9b8816b80 1 -- 172.21.15.46:0/2159854928 shutdown_connections 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.970+0000 7ff9b8816b80 1 -- 172.21.15.46:0/2159854928 wait complete. 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.970+0000 7ff9b8816b80 1 -- start start 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff9b8816b80 1 Processor -- start 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff9b8816b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fdcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff9b8816b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969f37160 0x55f969feb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff9b8816b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 0x55f969febbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff9b8816b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41af380 con 0x55f969fca800 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff9b8816b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41b07f0 con 0x55f969e4d4a0 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff9b8816b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a41afa50 con 0x55f969f37160 2024-05-09T00:22:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff9b8816b80 10 client.0 _ll_register_callbacks cb 0x55f969d5e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff9a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 0x55f969febbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff99affd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969f37160 0x55f969feb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff9a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 0x55f969febbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36224/0 (socket says 172.21.15.46:36224) 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff9a3fff700 1 -- 172.21.15.46:0/3219409404 learned_addr learned my addr 172.21.15.46:0/3219409404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/3219409404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fdcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff99affd700 1 -- 172.21.15.46:0/3219409404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 msgr2=0x55f969fdcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff99affd700 1 --2- 172.21.15.46:0/3219409404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fdcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff99affd700 1 -- 172.21.15.46:0/3219409404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 msgr2=0x55f969febbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff99affd700 1 --2- 172.21.15.46:0/3219409404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 0x55f969febbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.971+0000 7ff99affd700 1 -- 172.21.15.46:0/3219409404 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f969fec370 con 0x55f969f37160 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.972+0000 7ff99affd700 1 --2- 172.21.15.46:0/3219409404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969f37160 0x55f969feb580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00b580 tx=0x7ff98c005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7785d8f20ab64ecc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.972+0000 7ff982ffd700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-05-09T00:22:11.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.972+0000 7ff982ffd700 1 -- 172.21.15.46:0/3219409404 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c03d060 con 0x55f969f37160 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.972+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3219409404 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff98c00bc40 con 0x55f969f37160 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.972+0000 7ff982ffd700 1 -- 172.21.15.46:0/3219409404 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff98c00fa10 con 0x55f969f37160 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.972+0000 7ff982ffd700 1 -- 172.21.15.46:0/3219409404 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff98c034070 con 0x55f969f37160 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.972+0000 7ff9b8816b80 10 client.26677 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.972+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3219409404 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f96a04eb20 con 0x55f969f37160 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.972+0000 7ff982ffd700 1 -- 172.21.15.46:0/3219409404 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff98c00bc40 con 0x55f969f37160 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.973+0000 7ff982ffd700 1 -- 172.21.15.46:0/3219409404 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff98c022070 con 0x55f969f37160 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.973+0000 7ff9b8816b80 10 client.26677 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.973+0000 7ff9b8816b80 10 client.26677 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.973+0000 7ff9b8816b80 20 client.26677 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.973+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/3219409404 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f96a04f080 0x55f96a0514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.973+0000 7ff9b8816b80 4 client.26677 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:22:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.973+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3219409404 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f969ef4a70 con 0x55f96a04f080 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.973+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/3219409404 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f96a04f080 0x55f96a0514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.973+0000 7ff982ffd700 10 client.26677 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.973+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/3219409404 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f96a04f080 0x55f96a0514a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8134396712a4699 server_cookie=ffcbd526bf026acb in_seq=0 out_seq=0 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff982ffd700 1 -- 172.21.15.46:0/3219409404 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f969ef4a70 con 0x55f96a04f080 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff982ffd700 10 client.26677 handle_command_reply: tid=0 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff9b8816b80 1 client.26677 shutdown 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3219409404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969f37160 msgr2=0x55f969feb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/3219409404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969f37160 0x55f969feb580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00b580 tx=0x7ff98c005a90 comp rx=0 tx=0).stop 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3219409404 shutdown_connections 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/3219409404 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f96a04f080 0x55f96a0514a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/3219409404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f969fca800 0x55f969febbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/3219409404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f969e4d4a0 0x55f969fdcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff982ffd700 0 client.26677 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:22:11.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff9b8816b80 1 --2- 172.21.15.46:0/3219409404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f969f37160 0x55f969feb580 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3219409404 >> 172.21.15.46:0/3219409404 conn(0x55f969ead6a0 msgr2=0x55f969fce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:11.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3219409404 shutdown_connections 2024-05-09T00:22:11.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff9b8816b80 1 -- 172.21.15.46:0/3219409404 wait complete. 2024-05-09T00:22:11.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.974+0000 7ff9b8816b80 20 client.26677 trim_cache size 0 max 16384 2024-05-09T00:22:11.981 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:22:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.975+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/948108554 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9900b2880 msgr2=0x7ff9900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.975+0000 7ff9a8b9e700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9900b2880 0x7ff9900b4d90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff98c006010 tx=0x7ff98c000bc0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.975+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/948108554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a4104410 msgr2=0x7ff9a4075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.975+0000 7ff9a8b9e700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a4104410 0x7ff9a4075ac0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c920 tx=0x7ff99400cc30 comp rx=0 tx=0).stop 2024-05-09T00:22:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.975+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/948108554 shutdown_connections 2024-05-09T00:22:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.975+0000 7ff9a8b9e700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff9900b2880 0x7ff9900b4d90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.975+0000 7ff9a8b9e700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9a4109690 0x7ff9a4076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.975+0000 7ff9a8b9e700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9a4104410 0x7ff9a4075ac0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.975+0000 7ff9a8b9e700 1 --2- 172.21.15.46:0/948108554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9a4108d40 0x7ff9a4076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.975+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/948108554 >> 172.21.15.46:0/948108554 conn(0x7ff9a4079d50 msgr2=0x7ff9a4101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.975+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/948108554 shutdown_connections 2024-05-09T00:22:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:11.975+0000 7ff9a8b9e700 1 -- 172.21.15.46:0/948108554 wait complete. 2024-05-09T00:22:12.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:22:12.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:12 smithi169 ceph-mon[34777]: pgmap v701: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:22:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:22:12.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:12 smithi142 ceph-mon[34533]: pgmap v701: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:22:13.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:12 smithi046 ceph-mon[30151]: pgmap v701: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:22:14.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.831+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2224257715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dcc108760 msgr2=0x7f9dcc1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:14.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.831+0000 7f9dd47cc700 1 --2- 172.21.15.46:0/2224257715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dcc108760 0x7f9dcc1100c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400b0d0 tx=0x7f9dc400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.833 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:14 smithi046 ceph-mon[30151]: pgmap v702: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:22:14.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.832+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2224257715 shutdown_connections 2024-05-09T00:22:14.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.832+0000 7f9dd47cc700 1 --2- 172.21.15.46:0/2224257715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dcc108760 0x7f9dcc1100c0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.832+0000 7f9dd47cc700 1 --2- 172.21.15.46:0/2224257715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9dcc1034e0 0x7f9dcc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.832+0000 7f9dd47cc700 1 --2- 172.21.15.46:0/2224257715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dcc107e10 0x7f9dcc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.832+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2224257715 >> 172.21.15.46:0/2224257715 conn(0x7f9dcc0fef30 msgr2=0x7f9dcc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.832+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2224257715 shutdown_connections 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.832+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2224257715 wait complete. 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd47cc700 1 Processor -- start 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd47cc700 1 -- start start 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd47cc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9dcc1034e0 0x7f9dcc19d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd47cc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dcc107e10 0x7f9dcc19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd47cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dcc108760 0x7f9dcc1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd47cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc113390 con 0x7f9dcc108760 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd47cc700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc113210 con 0x7f9dcc107e10 2024-05-09T00:22:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd47cc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc113510 con 0x7f9dcc1034e0 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd2d69700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dcc108760 0x7f9dcc1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd2d69700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dcc108760 0x7f9dcc1a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36244/0 (socket says 172.21.15.46:36244) 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd2d69700 1 -- 172.21.15.46:0/2261909217 learned_addr learned my addr 172.21.15.46:0/2261909217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd2568700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9dcc1034e0 0x7f9dcc19d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.833+0000 7f9dd1d67700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dcc107e10 0x7f9dcc19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.834+0000 7f9dd2d69700 1 -- 172.21.15.46:0/2261909217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9dcc1034e0 msgr2=0x7f9dcc19d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.834+0000 7f9dd2d69700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9dcc1034e0 0x7f9dcc19d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.834+0000 7f9dd2d69700 1 -- 172.21.15.46:0/2261909217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dcc107e10 msgr2=0x7f9dcc19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.834+0000 7f9dd2d69700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dcc107e10 0x7f9dcc19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.834+0000 7f9dd2d69700 1 -- 172.21.15.46:0/2261909217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc1a1f60 con 0x7f9dcc108760 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.834+0000 7f9dd2d69700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dcc108760 0x7f9dcc1a18c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40095c0 tx=0x7f9dc4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.834+0000 7f9dc37fe700 1 -- 172.21.15.46:0/2261909217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc401f070 con 0x7f9dcc108760 2024-05-09T00:22:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.834+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2261909217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dcc1a21f0 con 0x7f9dcc108760 2024-05-09T00:22:14.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.834+0000 7f9dc37fe700 1 -- 172.21.15.46:0/2261909217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc4007e20 con 0x7f9dcc108760 2024-05-09T00:22:14.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.834+0000 7f9dc37fe700 1 -- 172.21.15.46:0/2261909217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc401b4a0 con 0x7f9dcc108760 2024-05-09T00:22:14.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.834+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2261909217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dcc1a9b70 con 0x7f9dcc108760 2024-05-09T00:22:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.838+0000 7f9dc37fe700 1 -- 172.21.15.46:0/2261909217 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9dc4022070 con 0x7f9dcc108760 2024-05-09T00:22:14.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9dc37fe700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9db80b2900 0x7f9db80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9de31e2b80 1 Processor -- start 2024-05-09T00:22:14.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9dd2568700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9db80b2900 0x7f9db80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9dc37fe700 1 -- 172.21.15.46:0/2261909217 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9dc400f020 con 0x7f9dcc108760 2024-05-09T00:22:14.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9de31e2b80 1 -- start start 2024-05-09T00:22:14.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9de31e2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4751aed00 0x55e4752b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9de31e2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751344a0 0x55e4752b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9de31e2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4752b1400 0x55e4752b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9de31e2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e4751d40a0 con 0x55e4752b1400 2024-05-09T00:22:14.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9de31e2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e4751799a0 con 0x55e4751aed00 2024-05-09T00:22:14.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9de31e2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e4751d0060 con 0x55e4751344a0 2024-05-09T00:22:14.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9dd2568700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9db80b2900 0x7f9db80b4e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8007be0 tx=0x7f9dc80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:14.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9dd2d69700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4752b1400 0x55e4752b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9dd2d69700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4752b1400 0x55e4752b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36260/0 (socket says 172.21.15.46:36260) 2024-05-09T00:22:14.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9dd2d69700 1 -- 172.21.15.46:0/4166534487 learned_addr learned my addr 172.21.15.46:0/4166534487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:14.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.839+0000 7f9dd2568700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4751aed00 0x55e4752b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.840+0000 7f9dd1d67700 1 --2- 172.21.15.46:0/4166534487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751344a0 0x55e4752b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.840+0000 7f9dd2d69700 1 -- 172.21.15.46:0/4166534487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751344a0 msgr2=0x55e4752b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:14.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.840+0000 7f9dd2d69700 1 --2- 172.21.15.46:0/4166534487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751344a0 0x55e4752b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.840+0000 7f9dd2d69700 1 -- 172.21.15.46:0/4166534487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4751aed00 msgr2=0x55e4752b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:14.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.840+0000 7f9dd2d69700 1 --2- 172.21.15.46:0/4166534487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4751aed00 0x55e4752b7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.840+0000 7f9dd2d69700 1 -- 172.21.15.46:0/4166534487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47516a500 con 0x55e4752b1400 2024-05-09T00:22:14.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.840+0000 7f9dd2d69700 1 --2- 172.21.15.46:0/4166534487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4752b1400 0x55e4752b1810 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40aa9d0 tx=0x7f9dc40aa410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6df02f11afb8925 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:14.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.840+0000 7f9dc0ff9700 1 -- 172.21.15.46:0/4166534487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40aa550 con 0x55e4752b1400 2024-05-09T00:22:14.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.840+0000 7f9dc0ff9700 1 -- 172.21.15.46:0/4166534487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc40ab140 con 0x55e4752b1400 2024-05-09T00:22:14.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.840+0000 7f9dc0ff9700 1 -- 172.21.15.46:0/4166534487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4003db0 con 0x55e4752b1400 2024-05-09T00:22:14.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.841+0000 7f9de31e2b80 1 -- 172.21.15.46:0/4166534487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4752b1400 msgr2=0x55e4752b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:14.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.841+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/4166534487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4752b1400 0x55e4752b1810 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40aa9d0 tx=0x7f9dc40aa410 comp rx=0 tx=0).stop 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.841+0000 7f9de31e2b80 1 -- 172.21.15.46:0/4166534487 shutdown_connections 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.841+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/4166534487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4752b1400 0x55e4752b1810 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.841+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/4166534487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4751aed00 0x55e4752b7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.841+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/4166534487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751344a0 0x55e4752b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.841+0000 7f9de31e2b80 1 -- 172.21.15.46:0/4166534487 >> 172.21.15.46:0/4166534487 conn(0x55e475137f70 msgr2=0x55e4752ac220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.841+0000 7f9de31e2b80 1 -- 172.21.15.46:0/4166534487 shutdown_connections 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.841+0000 7f9de31e2b80 1 -- 172.21.15.46:0/4166534487 wait complete. 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9de31e2b80 1 -- start start 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9de31e2b80 1 Processor -- start 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9de31e2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 0x55e4752c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9de31e2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4751aed00 0x55e4752c1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9de31e2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4752b1400 0x55e4752bafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9de31e2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e4752b1fb0 con 0x55e4751344a0 2024-05-09T00:22:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9de31e2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e47516eb20 con 0x55e4751aed00 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9de31e2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e4752b2130 con 0x55e4752b1400 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9de31e2b80 10 client.0 _ll_register_callbacks cb 0x55e4750456c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9dd2568700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 0x55e4752c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9dd2568700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 0x55e4752c0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36272/0 (socket says 172.21.15.46:36272) 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9dd2568700 1 -- 172.21.15.46:0/2618934724 learned_addr learned my addr 172.21.15.46:0/2618934724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9dd2d69700 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4752b1400 0x55e4752bafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.842+0000 7f9dd1d67700 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4751aed00 0x55e4752c1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9dd2568700 1 -- 172.21.15.46:0/2618934724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4752b1400 msgr2=0x55e4752bafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9dd2568700 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4752b1400 0x55e4752bafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9dd2568700 1 -- 172.21.15.46:0/2618934724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4751aed00 msgr2=0x55e4752c1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9dd2568700 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4751aed00 0x55e4752c1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9dd2568700 1 -- 172.21.15.46:0/2618934724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47504f490 con 0x55e4751344a0 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9dd2568700 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 0x55e4752c0bd0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80042e0 tx=0x7f9dc80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90f79e309dbc358 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:14.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9db5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9db5ffb700 1 -- 172.21.15.46:0/2618934724 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8003b10 con 0x55e4751344a0 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9db5ffb700 1 -- 172.21.15.46:0/2618934724 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9dc8003cb0 con 0x55e4751344a0 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9de31e2b80 1 -- 172.21.15.46:0/2618934724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dc40ab140 con 0x55e4751344a0 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9db5ffb700 1 -- 172.21.15.46:0/2618934724 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9dc8011440 con 0x55e4751344a0 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9de31e2b80 10 client.17214 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9de31e2b80 1 -- 172.21.15.46:0/2618934724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e4752c8850 con 0x55e4751344a0 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.843+0000 7f9db5ffb700 1 -- 172.21.15.46:0/2618934724 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc40ab140 con 0x55e4751344a0 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.844+0000 7f9db5ffb700 1 -- 172.21.15.46:0/2618934724 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9dc80295b0 con 0x55e4751344a0 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.844+0000 7f9de31e2b80 10 client.17214 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.844+0000 7f9de31e2b80 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.844+0000 7f9de31e2b80 20 client.17214 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.844+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e4752caee0 0x55e4752cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.844+0000 7f9de31e2b80 4 client.17214 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:22:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.844+0000 7f9de31e2b80 1 -- 172.21.15.46:0/2618934724 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4751e1a80 con 0x55e4752caee0 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.844+0000 7f9dd1d67700 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e4752caee0 0x55e4752cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.845+0000 7f9dd1d67700 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e4752caee0 0x55e4752cd300 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea9943b03e26e87 server_cookie=cd35501699680cfc in_seq=0 out_seq=0 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.845+0000 7f9db5ffb700 10 client.17214 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.845+0000 7f9db5ffb700 1 -- 172.21.15.46:0/2618934724 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4751e1a80 con 0x55e4752caee0 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.845+0000 7f9db5ffb700 10 client.17214 handle_command_reply: tid=0 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9de31e2b80 1 client.17214 shutdown 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9de31e2b80 1 -- 172.21.15.46:0/2618934724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 msgr2=0x55e4752c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 0x55e4752c0bd0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80042e0 tx=0x7f9dc80047c0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9de31e2b80 1 -- 172.21.15.46:0/2618934724 shutdown_connections 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e4752caee0 0x55e4752cd300 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4752b1400 0x55e4752bafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 0x55e4752c0bd0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/2618934724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4751aed00 0x55e4752c1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9de31e2b80 1 -- 172.21.15.46:0/2618934724 >> 172.21.15.46:0/2618934724 conn(0x55e475137f70 msgr2=0x55e4752a5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:14.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9db5ffb700 0 client.17214 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:14.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9de31e2b80 1 -- 172.21.15.46:0/2618934724 shutdown_connections 2024-05-09T00:22:14.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9de31e2b80 1 -- 172.21.15.46:0/2618934724 wait complete. 2024-05-09T00:22:14.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.846+0000 7f9de31e2b80 20 client.17214 trim_cache size 0 max 16384 2024-05-09T00:22:14.855 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:14 smithi169 ceph-mon[34777]: pgmap v702: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:22:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.861+0000 7f9de31e2b80 1 Processor -- start 2024-05-09T00:22:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.861+0000 7f9de31e2b80 1 -- start start 2024-05-09T00:22:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9de31e2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 0x55e4752b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9de31e2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751aed00 0x55e4752d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9de31e2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4752b1400 0x55e4752d3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9de31e2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1b3730 con 0x55e4751344a0 2024-05-09T00:22:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9de31e2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1b0fc0 con 0x55e4752b1400 2024-05-09T00:22:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9de31e2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1aee10 con 0x55e4751aed00 2024-05-09T00:22:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd2568700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 0x55e4752b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd2568700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 0x55e4752b67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36278/0 (socket says 172.21.15.46:36278) 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd2568700 1 -- 172.21.15.46:0/911756653 learned_addr learned my addr 172.21.15.46:0/911756653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd2d69700 1 --2- 172.21.15.46:0/911756653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4752b1400 0x55e4752d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd1d67700 1 --2- 172.21.15.46:0/911756653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751aed00 0x55e4752d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd2568700 1 -- 172.21.15.46:0/911756653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751aed00 msgr2=0x55e4752d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd2568700 1 --2- 172.21.15.46:0/911756653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751aed00 0x55e4752d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd2568700 1 -- 172.21.15.46:0/911756653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4752b1400 msgr2=0x55e4752d3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd2568700 1 --2- 172.21.15.46:0/911756653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4752b1400 0x55e4752d3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd2568700 1 -- 172.21.15.46:0/911756653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4752a89d0 con 0x55e4751344a0 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd2d69700 1 --2- 172.21.15.46:0/911756653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4752b1400 0x55e4752d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd1d67700 1 --2- 172.21.15.46:0/911756653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751aed00 0x55e4752d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9dd2568700 1 --2- 172.21.15.46:0/911756653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 0x55e4752b67d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8020430 tx=0x7f9dc8010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295cf44c05e22fb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9db7fff700 1 -- 172.21.15.46:0/911756653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8029e10 con 0x55e4751344a0 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9db7fff700 1 -- 172.21.15.46:0/911756653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc8024030 con 0x55e4751344a0 2024-05-09T00:22:14.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.862+0000 7f9db7fff700 1 -- 172.21.15.46:0/911756653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80118d0 con 0x55e4751344a0 2024-05-09T00:22:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.863+0000 7f9de31e2b80 1 -- 172.21.15.46:0/911756653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 msgr2=0x55e4752b67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.863+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/911756653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 0x55e4752b67d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8020430 tx=0x7f9dc8010c70 comp rx=0 tx=0).stop 2024-05-09T00:22:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.863+0000 7f9de31e2b80 1 -- 172.21.15.46:0/911756653 shutdown_connections 2024-05-09T00:22:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.863+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/911756653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4752b1400 0x55e4752d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.863+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/911756653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751344a0 0x55e4752b67d0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.863+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/911756653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751aed00 0x55e4752d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.863+0000 7f9de31e2b80 1 -- 172.21.15.46:0/911756653 >> 172.21.15.46:0/911756653 conn(0x55e475137f70 msgr2=0x55e4752b60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.863+0000 7f9de31e2b80 1 -- 172.21.15.46:0/911756653 shutdown_connections 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.863+0000 7f9de31e2b80 1 -- 172.21.15.46:0/911756653 wait complete. 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9de31e2b80 1 -- start start 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9de31e2b80 1 Processor -- start 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9de31e2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751344a0 0x55e47532da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9de31e2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751aed00 0x55e47532df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9de31e2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4752b1400 0x55e4752c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9de31e2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1af480 con 0x55e4751aed00 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9de31e2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1b08f0 con 0x55e4752b1400 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9de31e2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc1afb50 con 0x55e4751344a0 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9de31e2b80 10 client.0 _ll_register_callbacks cb 0x55e4750456c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9dd1d67700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751aed00 0x55e47532df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9dd1d67700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751aed00 0x55e47532df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36280/0 (socket says 172.21.15.46:36280) 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9dd1d67700 1 -- 172.21.15.46:0/1665381090 learned_addr learned my addr 172.21.15.46:0/1665381090 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9dd2568700 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751344a0 0x55e47532da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.864+0000 7f9dd2d69700 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4752b1400 0x55e4752c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9dd1d67700 1 -- 172.21.15.46:0/1665381090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751344a0 msgr2=0x55e47532da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9dd1d67700 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751344a0 0x55e47532da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9dd1d67700 1 -- 172.21.15.46:0/1665381090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4752b1400 msgr2=0x55e4752c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9dd1d67700 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4752b1400 0x55e4752c43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9dd1d67700 1 -- 172.21.15.46:0/1665381090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4752c4a50 con 0x55e4751aed00 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9dd1d67700 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751aed00 0x55e47532df80 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc000c00 tx=0x7f9dbc004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac03129c4b6bf2b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9db5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9db5ffb700 1 -- 172.21.15.46:0/1665381090 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc024060 con 0x55e4751aed00 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9db5ffb700 1 -- 172.21.15.46:0/1665381090 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9dbc003df0 con 0x55e4751aed00 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9de31e2b80 1 -- 172.21.15.46:0/1665381090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dc8024030 con 0x55e4751aed00 2024-05-09T00:22:14.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9db5ffb700 1 -- 172.21.15.46:0/1665381090 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9dbc01d450 con 0x55e4751aed00 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9de31e2b80 10 client.17220 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9de31e2b80 1 -- 172.21.15.46:0/1665381090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e475334ff0 con 0x55e4751aed00 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9db5ffb700 1 -- 172.21.15.46:0/1665381090 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc8024030 con 0x55e4751aed00 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.865+0000 7f9db5ffb700 1 -- 172.21.15.46:0/1665381090 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9dbc029070 con 0x55e4751aed00 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.866+0000 7f9de31e2b80 10 client.17220 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.866+0000 7f9de31e2b80 10 client.17220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.866+0000 7f9de31e2b80 20 client.17220 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.866+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e475335650 0x55e475337a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.866+0000 7f9de31e2b80 4 client.17220 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.866+0000 7f9de31e2b80 1 -- 172.21.15.46:0/1665381090 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4751dbb20 con 0x55e475335650 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.866+0000 7f9dd2568700 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e475335650 0x55e475337a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.866+0000 7f9dd2568700 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e475335650 0x55e475337a70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2784faa5f8b411d2 server_cookie=72ba54b603375c89 in_seq=0 out_seq=0 2024-05-09T00:22:14.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.866+0000 7f9db5ffb700 10 client.17220 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9db5ffb700 1 -- 172.21.15.46:0/1665381090 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4751dbb20 con 0x55e475335650 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9db5ffb700 10 client.17220 handle_command_reply: tid=0 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9de31e2b80 1 client.17220 shutdown 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9de31e2b80 1 -- 172.21.15.46:0/1665381090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751aed00 msgr2=0x55e47532df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751aed00 0x55e47532df80 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc000c00 tx=0x7f9dbc004480 comp rx=0 tx=0).stop 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9de31e2b80 1 -- 172.21.15.46:0/1665381090 shutdown_connections 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55e475335650 0x55e475337a70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4752b1400 0x55e4752c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e4751344a0 0x55e47532da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9db5ffb700 0 client.17220 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9de31e2b80 1 --2- 172.21.15.46:0/1665381090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4751aed00 0x55e47532df80 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9de31e2b80 1 -- 172.21.15.46:0/1665381090 >> 172.21.15.46:0/1665381090 conn(0x55e475137f70 msgr2=0x55e4752a5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:14.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9de31e2b80 1 -- 172.21.15.46:0/1665381090 shutdown_connections 2024-05-09T00:22:14.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9de31e2b80 1 -- 172.21.15.46:0/1665381090 wait complete. 2024-05-09T00:22:14.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.867+0000 7f9de31e2b80 20 client.17220 trim_cache size 0 max 16384 2024-05-09T00:22:14.873 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:22:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.868+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2261909217 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9db80b2900 msgr2=0x7f9db80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.868+0000 7f9dd47cc700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9db80b2900 0x7f9db80b4e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8007be0 tx=0x7f9dc80073d0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.868+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2261909217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dcc108760 msgr2=0x7f9dcc1a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.868+0000 7f9dd47cc700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dcc108760 0x7f9dcc1a18c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40095c0 tx=0x7f9dc4007a20 comp rx=0 tx=0).stop 2024-05-09T00:22:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.868+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2261909217 shutdown_connections 2024-05-09T00:22:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.868+0000 7f9dd47cc700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f9db80b2900 0x7f9db80b4e10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.868+0000 7f9dd47cc700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dcc108760 0x7f9dcc1a18c0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.868+0000 7f9dd47cc700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9dcc1034e0 0x7f9dcc19d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.868+0000 7f9dd47cc700 1 --2- 172.21.15.46:0/2261909217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dcc107e10 0x7f9dcc19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.868+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2261909217 >> 172.21.15.46:0/2261909217 conn(0x7f9dcc0fef30 msgr2=0x7f9dcc10ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.868+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2261909217 shutdown_connections 2024-05-09T00:22:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:14.868+0000 7f9dd47cc700 1 -- 172.21.15.46:0/2261909217 wait complete. 2024-05-09T00:22:14.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:14 smithi142 ceph-mon[34533]: pgmap v702: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:22:16.719 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:22:16.719 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:22:16.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:16 smithi142 ceph-mon[34533]: pgmap v703: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:22:16.986 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:22:17.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:16 smithi046 ceph-mon[30151]: pgmap v703: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:22:17.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:16 smithi169 ceph-mon[34777]: pgmap v703: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:22:18.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.821+0000 7f338410a700 1 -- 172.21.15.46:0/1073365936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c1034e0 msgr2=0x7f337c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:18.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.821+0000 7f338410a700 1 --2- 172.21.15.46:0/1073365936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c1034e0 0x7f337c1038f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f33780099c0 tx=0x7f3378009cd0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.821+0000 7f338410a700 1 -- 172.21.15.46:0/1073365936 shutdown_connections 2024-05-09T00:22:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.821+0000 7f338410a700 1 --2- 172.21.15.46:0/1073365936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f337c108760 0x7f337c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.821+0000 7f338410a700 1 --2- 172.21.15.46:0/1073365936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c1034e0 0x7f337c1038f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.821+0000 7f338410a700 1 --2- 172.21.15.46:0/1073365936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f337c107e10 0x7f337c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.821+0000 7f338410a700 1 -- 172.21.15.46:0/1073365936 >> 172.21.15.46:0/1073365936 conn(0x7f337c0fef30 msgr2=0x7f337c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:18.823 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:18 smithi046 ceph-mon[30151]: pgmap v704: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:22:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.822+0000 7f338410a700 1 -- 172.21.15.46:0/1073365936 shutdown_connections 2024-05-09T00:22:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.822+0000 7f338410a700 1 -- 172.21.15.46:0/1073365936 wait complete. 2024-05-09T00:22:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.823+0000 7f338410a700 1 Processor -- start 2024-05-09T00:22:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.823+0000 7f338410a700 1 -- start start 2024-05-09T00:22:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.823+0000 7f338410a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f337c1034e0 0x7f337c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.823+0000 7f338410a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f337c107e10 0x7f337c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.823+0000 7f338410a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c108760 0x7f337c1a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.823+0000 7f338410a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1133b0 con 0x7f337c108760 2024-05-09T00:22:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.823+0000 7f338410a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f337c113230 con 0x7f337c1034e0 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.823+0000 7f338410a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f337c113530 con 0x7f337c107e10 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f3381ea6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f337c1034e0 0x7f337c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.823+0000 7f33826a7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c108760 0x7f337c1a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f3381ea6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f337c1034e0 0x7f337c19bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:59844/0 (socket says 172.21.15.46:59844) 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f3381ea6700 1 -- 172.21.15.46:0/2991966079 learned_addr learned my addr 172.21.15.46:0/2991966079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f33816a5700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f337c107e10 0x7f337c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f33826a7700 1 -- 172.21.15.46:0/2991966079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f337c107e10 msgr2=0x7f337c19c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f33826a7700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f337c107e10 0x7f337c19c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f33826a7700 1 -- 172.21.15.46:0/2991966079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f337c1034e0 msgr2=0x7f337c19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f33826a7700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f337c1034e0 0x7f337c19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f33826a7700 1 -- 172.21.15.46:0/2991966079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c19ca60 con 0x7f337c108760 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f33826a7700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c108760 0x7f337c1a6cf0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f337400ab40 tx=0x7f337400ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:18.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f3372ffd700 1 -- 172.21.15.46:0/2991966079 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374017020 con 0x7f337c108760 2024-05-09T00:22:18.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f338410a700 1 -- 172.21.15.46:0/2991966079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c1a7350 con 0x7f337c108760 2024-05-09T00:22:18.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f3372ffd700 1 -- 172.21.15.46:0/2991966079 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3374004bb0 con 0x7f337c108760 2024-05-09T00:22:18.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f3372ffd700 1 -- 172.21.15.46:0/2991966079 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374005280 con 0x7f337c108760 2024-05-09T00:22:18.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.824+0000 7f338410a700 1 -- 172.21.15.46:0/2991966079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c1a7860 con 0x7f337c108760 2024-05-09T00:22:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.828+0000 7f3372ffd700 1 -- 172.21.15.46:0/2991966079 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3374005420 con 0x7f337c108760 2024-05-09T00:22:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.829+0000 7f3372ffd700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33680b2880 0x7f33680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.829+0000 7f3392b20b80 1 Processor -- start 2024-05-09T00:22:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.829+0000 7f3381ea6700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33680b2880 0x7f33680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.829+0000 7f3372ffd700 1 -- 172.21.15.46:0/2991966079 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f33740e1790 con 0x7f337c108760 2024-05-09T00:22:18.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.829+0000 7f3392b20b80 1 -- start start 2024-05-09T00:22:18.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.829+0000 7f3392b20b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.829+0000 7f3392b20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 0x55d3cbc0aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.829+0000 7f3381ea6700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33680b2880 0x7f33680b4d90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3378009750 tx=0x7f3378005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:18.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.829+0000 7f3392b20b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc04f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.829+0000 7f3392b20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d3cbb23070 con 0x55d3cba83520 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f3392b20b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d3cbac1870 con 0x55d3cbc04b30 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f3392b20b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d3cbb1f030 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f33816a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 0x55d3cbc0aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f33816a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 0x55d3cbc0aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56806/0 (socket says 172.21.15.46:56806) 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f33816a5700 1 -- 172.21.15.46:0/4066374217 learned_addr learned my addr 172.21.15.46:0/4066374217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f3381ea6700 1 --2- 172.21.15.46:0/4066374217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f33826a7700 1 --2- 172.21.15.46:0/4066374217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc04f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f33816a5700 1 -- 172.21.15.46:0/4066374217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cbb6d1e0 msgr2=0x55d3cbc0a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f33816a5700 1 --2- 172.21.15.46:0/4066374217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f33816a5700 1 -- 172.21.15.46:0/4066374217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 msgr2=0x55d3cbc04f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f33816a5700 1 --2- 172.21.15.46:0/4066374217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc04f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f33816a5700 1 -- 172.21.15.46:0/4066374217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3cbab93f0 con 0x55d3cba83520 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f33826a7700 1 --2- 172.21.15.46:0/4066374217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc04f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:22:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f33816a5700 1 --2- 172.21.15.46:0/4066374217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 0x55d3cbc0aec0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f336c00ac40 tx=0x7f336c00af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73349a3d3823ccc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:18.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.830+0000 7f3367fff700 1 -- 172.21.15.46:0/4066374217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c01d070 con 0x55d3cba83520 2024-05-09T00:22:18.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.831+0000 7f3367fff700 1 -- 172.21.15.46:0/4066374217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336c00fde0 con 0x55d3cba83520 2024-05-09T00:22:18.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.831+0000 7f3367fff700 1 -- 172.21.15.46:0/4066374217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c010820 con 0x55d3cba83520 2024-05-09T00:22:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.831+0000 7f3392b20b80 1 -- 172.21.15.46:0/4066374217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 msgr2=0x55d3cbc0aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.831+0000 7f3392b20b80 1 --2- 172.21.15.46:0/4066374217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 0x55d3cbc0aec0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f336c00ac40 tx=0x7f336c00af50 comp rx=0 tx=0).stop 2024-05-09T00:22:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.831+0000 7f3392b20b80 1 -- 172.21.15.46:0/4066374217 shutdown_connections 2024-05-09T00:22:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.831+0000 7f3392b20b80 1 --2- 172.21.15.46:0/4066374217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc04f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.831+0000 7f3392b20b80 1 --2- 172.21.15.46:0/4066374217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.831+0000 7f3392b20b80 1 --2- 172.21.15.46:0/4066374217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 0x55d3cbc0aec0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.831+0000 7f3392b20b80 1 -- 172.21.15.46:0/4066374217 >> 172.21.15.46:0/4066374217 conn(0x55d3cbae3720 msgr2=0x55d3cbbfb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.832+0000 7f3392b20b80 1 -- 172.21.15.46:0/4066374217 shutdown_connections 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.832+0000 7f3392b20b80 1 -- 172.21.15.46:0/4066374217 wait complete. 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.832+0000 7f3392b20b80 1 -- start start 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.832+0000 7f3392b20b80 1 Processor -- start 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3392b20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 0x55d3cbc0e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3392b20b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3392b20b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cbc04b30 0x55d3cbc0f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3392b20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d3cb99ec20 con 0x55d3cba83520 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3392b20b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d3cbadd1d0 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3392b20b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d3cbc11660 con 0x55d3cbc04b30 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3392b20b80 10 client.0 _ll_register_callbacks cb 0x55d3cb994740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3381ea6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 0x55d3cbc0e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3381ea6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 0x55d3cbc0e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56818/0 (socket says 172.21.15.46:56818) 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3381ea6700 1 -- 172.21.15.46:0/718541798 learned_addr learned my addr 172.21.15.46:0/718541798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f33826a7700 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cbc04b30 0x55d3cbc0f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f33816a5700 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3381ea6700 1 -- 172.21.15.46:0/718541798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cbc04b30 msgr2=0x55d3cbc0f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3381ea6700 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cbc04b30 0x55d3cbc0f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3381ea6700 1 -- 172.21.15.46:0/718541798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbb6d1e0 msgr2=0x55d3cbc0ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3381ea6700 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3381ea6700 1 -- 172.21.15.46:0/718541798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3cb99e510 con 0x55d3cba83520 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f33816a5700 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f3381ea6700 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 0x55d3cbc0e690 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f337800b580 tx=0x7f337801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad36abe10bc490e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.833+0000 7f33657fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f33657fa700 1 -- 172.21.15.46:0/718541798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337800f660 con 0x55d3cba83520 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f33657fa700 1 -- 172.21.15.46:0/718541798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f337800f800 con 0x55d3cba83520 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f3392b20b80 1 -- 172.21.15.46:0/718541798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f336c00fde0 con 0x55d3cba83520 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f33657fa700 1 -- 172.21.15.46:0/718541798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f337800fa40 con 0x55d3cba83520 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f3392b20b80 10 client.17232 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:18.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f3392b20b80 1 -- 172.21.15.46:0/718541798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d3cbc1c070 con 0x55d3cba83520 2024-05-09T00:22:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f33657fa700 1 -- 172.21.15.46:0/718541798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336c00fde0 con 0x55d3cba83520 2024-05-09T00:22:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f33657fa700 1 -- 172.21.15.46:0/718541798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3378022070 con 0x55d3cba83520 2024-05-09T00:22:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f3392b20b80 10 client.17232 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f3392b20b80 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:22:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f3392b20b80 20 client.17232 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f3392b20b80 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d3cbc1e650 0x55d3cbc20a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f3392b20b80 4 client.17232 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:22:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.834+0000 7f3392b20b80 1 -- 172.21.15.46:0/718541798 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3cbb30a50 con 0x55d3cbc1e650 2024-05-09T00:22:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.835+0000 7f33816a5700 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d3cbc1e650 0x55d3cbc20a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.835+0000 7f33657fa700 10 client.17232 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.835+0000 7f33816a5700 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d3cbc1e650 0x55d3cbc20a90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7de9ac5878bee47b server_cookie=8771a73d4f70e51b in_seq=0 out_seq=0 2024-05-09T00:22:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f33657fa700 1 -- 172.21.15.46:0/718541798 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d3cbb30a50 con 0x55d3cbc1e650 2024-05-09T00:22:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f33657fa700 10 client.17232 handle_command_reply: tid=0 2024-05-09T00:22:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f3392b20b80 1 client.17232 shutdown 2024-05-09T00:22:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f3392b20b80 1 -- 172.21.15.46:0/718541798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 msgr2=0x55d3cbc0e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f3392b20b80 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 0x55d3cbc0e690 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f337800b580 tx=0x7f337801e000 comp rx=0 tx=0).stop 2024-05-09T00:22:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f3392b20b80 1 -- 172.21.15.46:0/718541798 shutdown_connections 2024-05-09T00:22:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f3392b20b80 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d3cbc1e650 0x55d3cbc20a90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f3392b20b80 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cbc04b30 0x55d3cbc0f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f3392b20b80 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cba83520 0x55d3cbc0e690 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f3392b20b80 1 --2- 172.21.15.46:0/718541798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f33657fa700 0 client.17232 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:18.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f3392b20b80 1 -- 172.21.15.46:0/718541798 >> 172.21.15.46:0/718541798 conn(0x55d3cbae3720 msgr2=0x55d3cbbfb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:18.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f3392b20b80 1 -- 172.21.15.46:0/718541798 shutdown_connections 2024-05-09T00:22:18.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.836+0000 7f3392b20b80 1 -- 172.21.15.46:0/718541798 wait complete. 2024-05-09T00:22:18.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.837+0000 7f3392b20b80 20 client.17232 trim_cache size 0 max 16384 2024-05-09T00:22:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.857+0000 7f3392b20b80 1 Processor -- start 2024-05-09T00:22:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.857+0000 7f3392b20b80 1 -- start start 2024-05-09T00:22:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f3392b20b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cba83520 0x55d3cbc08cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f3392b20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f3392b20b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc1a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f3392b20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1af260 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f3392b20b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1acaf0 con 0x55d3cbc04b30 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f3392b20b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1aa920 con 0x55d3cba83520 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f33816a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f33816a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56822/0 (socket says 172.21.15.46:56822) 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f33816a5700 1 -- 172.21.15.46:0/2646419821 learned_addr learned my addr 172.21.15.46:0/2646419821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f3381ea6700 1 --2- 172.21.15.46:0/2646419821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cba83520 0x55d3cbc08cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f33826a7700 1 --2- 172.21.15.46:0/2646419821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc1a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f33816a5700 1 -- 172.21.15.46:0/2646419821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cba83520 msgr2=0x55d3cbc08cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f33816a5700 1 --2- 172.21.15.46:0/2646419821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cba83520 0x55d3cbc08cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f33816a5700 1 -- 172.21.15.46:0/2646419821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 msgr2=0x55d3cbc1a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f33816a5700 1 --2- 172.21.15.46:0/2646419821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc1a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f33816a5700 1 -- 172.21.15.46:0/2646419821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3cbbfcd80 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f3381ea6700 1 --2- 172.21.15.46:0/2646419821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cba83520 0x55d3cbc08cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:22:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f33826a7700 1 --2- 172.21.15.46:0/2646419821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc1a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:22:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.858+0000 7f33816a5700 1 --2- 172.21.15.46:0/2646419821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0ef50 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f336c00a840 tx=0x7f336c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50800ef6ccea135 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.859+0000 7f33677fe700 1 -- 172.21.15.46:0/2646419821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c01d040 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.859+0000 7f33677fe700 1 -- 172.21.15.46:0/2646419821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336c004020 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.859+0000 7f33677fe700 1 -- 172.21.15.46:0/2646419821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c0049e0 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.859+0000 7f3392b20b80 1 -- 172.21.15.46:0/2646419821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 msgr2=0x55d3cbc0ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.859+0000 7f3392b20b80 1 --2- 172.21.15.46:0/2646419821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0ef50 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f336c00a840 tx=0x7f336c00a7d0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.859+0000 7f3392b20b80 1 -- 172.21.15.46:0/2646419821 shutdown_connections 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.859+0000 7f3392b20b80 1 --2- 172.21.15.46:0/2646419821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc1a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.859+0000 7f3392b20b80 1 --2- 172.21.15.46:0/2646419821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cba83520 0x55d3cbc08cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.859+0000 7f3392b20b80 1 --2- 172.21.15.46:0/2646419821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc0ef50 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.859+0000 7f3392b20b80 1 -- 172.21.15.46:0/2646419821 >> 172.21.15.46:0/2646419821 conn(0x55d3cbae3720 msgr2=0x55d3cbc04190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.860+0000 7f3392b20b80 1 -- 172.21.15.46:0/2646419821 shutdown_connections 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.860+0000 7f3392b20b80 1 -- 172.21.15.46:0/2646419821 wait complete. 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.860+0000 7f3392b20b80 1 -- start start 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.860+0000 7f3392b20b80 1 Processor -- start 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f3392b20b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cba83520 0x55d3cbc16820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f3392b20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc16d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f3392b20b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc137f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f3392b20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1aafb0 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f3392b20b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1ac420 con 0x55d3cbc04b30 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f3392b20b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1ab680 con 0x55d3cba83520 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f3392b20b80 10 client.0 _ll_register_callbacks cb 0x55d3cb994740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33816a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc16d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33816a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc16d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56830/0 (socket says 172.21.15.46:56830) 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33816a5700 1 -- 172.21.15.46:0/3153956973 learned_addr learned my addr 172.21.15.46:0/3153956973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f3381ea6700 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cba83520 0x55d3cbc16820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33826a7700 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33816a5700 1 -- 172.21.15.46:0/3153956973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cba83520 msgr2=0x55d3cbc16820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33816a5700 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cba83520 0x55d3cbc16820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33816a5700 1 -- 172.21.15.46:0/3153956973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 msgr2=0x55d3cbc137f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33816a5700 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc137f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33816a5700 1 -- 172.21.15.46:0/3153956973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3cbc13d30 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33826a7700 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc137f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33816a5700 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc16d60 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f336c005950 tx=0x7f336c00ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd3b114ded053dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33657fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.861+0000 7f33657fa700 1 -- 172.21.15.46:0/3153956973 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c01d070 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f3392b20b80 1 -- 172.21.15.46:0/3153956973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f336c004020 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f33657fa700 1 -- 172.21.15.46:0/3153956973 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f336c0041d0 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f33657fa700 1 -- 172.21.15.46:0/3153956973 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f336c0043d0 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f3392b20b80 10 client.17238 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f3392b20b80 1 -- 172.21.15.46:0/3153956973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d3cbc89520 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f33657fa700 1 -- 172.21.15.46:0/3153956973 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336c004020 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f33657fa700 1 -- 172.21.15.46:0/3153956973 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f336c0039d0 con 0x55d3cbb6d1e0 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f3392b20b80 10 client.17238 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f3392b20b80 10 client.17238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:22:18.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f3392b20b80 20 client.17238 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:18.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f3392b20b80 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d3cbc89a40 0x55d3cbc8be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:18.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f3392b20b80 4 client.17238 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:22:18.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f3392b20b80 1 -- 172.21.15.46:0/3153956973 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d3cbb2aaf0 con 0x55d3cbc89a40 2024-05-09T00:22:18.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.862+0000 7f3381ea6700 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d3cbc89a40 0x55d3cbc8be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:18.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.863+0000 7f3381ea6700 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d3cbc89a40 0x55d3cbc8be60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126a201a211aea0e server_cookie=d129347a8bc741f2 in_seq=0 out_seq=0 2024-05-09T00:22:18.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.863+0000 7f33657fa700 10 client.17238 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.864+0000 7f33657fa700 1 -- 172.21.15.46:0/3153956973 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d3cbb2aaf0 con 0x55d3cbc89a40 2024-05-09T00:22:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.864+0000 7f33657fa700 10 client.17238 handle_command_reply: tid=0 2024-05-09T00:22:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.864+0000 7f3392b20b80 1 client.17238 shutdown 2024-05-09T00:22:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.864+0000 7f3392b20b80 1 -- 172.21.15.46:0/3153956973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 msgr2=0x55d3cbc16d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.864+0000 7f3392b20b80 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc16d60 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f336c005950 tx=0x7f336c00ff50 comp rx=0 tx=0).stop 2024-05-09T00:22:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.864+0000 7f3392b20b80 1 -- 172.21.15.46:0/3153956973 shutdown_connections 2024-05-09T00:22:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.864+0000 7f3392b20b80 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d3cbc89a40 0x55d3cbc8be60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.864+0000 7f3392b20b80 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cbc04b30 0x55d3cbc137f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.864+0000 7f3392b20b80 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d3cba83520 0x55d3cbc16820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.864+0000 7f3392b20b80 1 --2- 172.21.15.46:0/3153956973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d3cbb6d1e0 0x55d3cbc16d60 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.864+0000 7f33657fa700 0 client.17238 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:18.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.864+0000 7f3392b20b80 1 -- 172.21.15.46:0/3153956973 >> 172.21.15.46:0/3153956973 conn(0x55d3cbae3720 msgr2=0x55d3cbbfa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:18.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f3392b20b80 1 -- 172.21.15.46:0/3153956973 shutdown_connections 2024-05-09T00:22:18.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f3392b20b80 1 -- 172.21.15.46:0/3153956973 wait complete. 2024-05-09T00:22:18.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f3392b20b80 20 client.17238 trim_cache size 0 max 16384 2024-05-09T00:22:18.869 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:22:18.869 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:22:18.869 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "d73b3d0d-e151-443f-8bdb-614bd0edc8cc", 2024-05-09T00:22:18.869 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:22:18.869 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:22:18.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f338410a700 1 -- 172.21.15.46:0/2991966079 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33680b2880 msgr2=0x7f33680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:18.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f338410a700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33680b2880 0x7f33680b4d90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3378009750 tx=0x7f3378005fb0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f338410a700 1 -- 172.21.15.46:0/2991966079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c108760 msgr2=0x7f337c1a6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:18.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f338410a700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c108760 0x7f337c1a6cf0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f337400ab40 tx=0x7f337400ae50 comp rx=0 tx=0).stop 2024-05-09T00:22:18.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f338410a700 1 -- 172.21.15.46:0/2991966079 shutdown_connections 2024-05-09T00:22:18.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f338410a700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f33680b2880 0x7f33680b4d90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f338410a700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c108760 0x7f337c1a6cf0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f338410a700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f337c1034e0 0x7f337c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f338410a700 1 --2- 172.21.15.46:0/2991966079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f337c107e10 0x7f337c19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f338410a700 1 -- 172.21.15.46:0/2991966079 >> 172.21.15.46:0/2991966079 conn(0x7f337c0fef30 msgr2=0x7f337c1071d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.865+0000 7f338410a700 1 -- 172.21.15.46:0/2991966079 shutdown_connections 2024-05-09T00:22:18.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:18.866+0000 7f338410a700 1 -- 172.21.15.46:0/2991966079 wait complete. 2024-05-09T00:22:18.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:18 smithi142 ceph-mon[34533]: pgmap v704: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:22:19.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:18 smithi169 ceph-mon[34777]: pgmap v704: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:22:19.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:22:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:22:19.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:19 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:22:19.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:19 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:22:19.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:19 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:22:19.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:19 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:22:20.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:19 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:22:20.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:19 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:22:20.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:19 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:22:20.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:19 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:22:20.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:19 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:22:20.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:19 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:22:20.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:19 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:22:20.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:19 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:22:20.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:20 smithi142 ceph-mon[34533]: pgmap v705: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:22:21.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:20 smithi046 ceph-mon[30151]: pgmap v705: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:22:21.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:20 smithi169 ceph-mon[34777]: pgmap v705: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:22:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.752+0000 7fb59fcf6700 1 -- 172.21.15.46:0/1579714513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb598109890 msgr2=0x7fb5981111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.752+0000 7fb59fcf6700 1 --2- 172.21.15.46:0/1579714513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb598109890 0x7fb5981111f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb5900099c0 tx=0x7fb590009cd0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.752+0000 7fb59fcf6700 1 -- 172.21.15.46:0/1579714513 shutdown_connections 2024-05-09T00:22:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.752+0000 7fb59fcf6700 1 --2- 172.21.15.46:0/1579714513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb598109890 0x7fb5981111f0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.752+0000 7fb59fcf6700 1 --2- 172.21.15.46:0/1579714513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb598102b20 0x7fb598102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.752+0000 7fb59fcf6700 1 --2- 172.21.15.46:0/1579714513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb598108f40 0x7fb598109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.752+0000 7fb59fcf6700 1 -- 172.21.15.46:0/1579714513 >> 172.21.15.46:0/1579714513 conn(0x7fb5980fe4e0 msgr2=0x7fb598100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.752+0000 7fb59fcf6700 1 -- 172.21.15.46:0/1579714513 shutdown_connections 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.752+0000 7fb59fcf6700 1 -- 172.21.15.46:0/1579714513 wait complete. 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.752+0000 7fb59fcf6700 1 Processor -- start 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59fcf6700 1 -- start start 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59fcf6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb598102b20 0x7fb598103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59fcf6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb598108f40 0x7fb598103fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59fcf6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb598109890 0x7fb5981044f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59fcf6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb598114520 con 0x7fb598109890 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59fcf6700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981143a0 con 0x7fb598102b20 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59fcf6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981146a0 con 0x7fb598108f40 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59e293700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb598109890 0x7fb5981044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59e293700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb598109890 0x7fb5981044f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56846/0 (socket says 172.21.15.46:56846) 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59e293700 1 -- 172.21.15.46:0/2657414751 learned_addr learned my addr 172.21.15.46:0/2657414751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59d291700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb598108f40 0x7fb598103fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59da92700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb598102b20 0x7fb598103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59e293700 1 -- 172.21.15.46:0/2657414751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb598108f40 msgr2=0x7fb598103fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.753+0000 7fb59e293700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb598108f40 0x7fb598103fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.754+0000 7fb59e293700 1 -- 172.21.15.46:0/2657414751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb598102b20 msgr2=0x7fb598103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.754+0000 7fb59e293700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb598102b20 0x7fb598103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.754+0000 7fb59e293700 1 -- 172.21.15.46:0/2657414751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5981a90e0 con 0x7fb598109890 2024-05-09T00:22:21.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.754+0000 7fb59e293700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb598109890 0x7fb5981044f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb59000bff0 tx=0x7fb5900036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:21.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.754+0000 7fb58effd700 1 -- 172.21.15.46:0/2657414751 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590026070 con 0x7fb598109890 2024-05-09T00:22:21.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.754+0000 7fb59fcf6700 1 -- 172.21.15.46:0/2657414751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5981a9370 con 0x7fb598109890 2024-05-09T00:22:21.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.754+0000 7fb58effd700 1 -- 172.21.15.46:0/2657414751 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb590004c90 con 0x7fb598109890 2024-05-09T00:22:21.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.754+0000 7fb58effd700 1 -- 172.21.15.46:0/2657414751 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590003dd0 con 0x7fb598109890 2024-05-09T00:22:21.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.754+0000 7fb59fcf6700 1 -- 172.21.15.46:0/2657414751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5981a97c0 con 0x7fb598109890 2024-05-09T00:22:21.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.757+0000 7fb58effd700 1 -- 172.21.15.46:0/2657414751 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb590029070 con 0x7fb598109890 2024-05-09T00:22:21.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.757+0000 7fb58effd700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb5840b2880 0x7fb5840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.758+0000 7fb58effd700 1 -- 172.21.15.46:0/2657414751 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb5900e2e30 con 0x7fb598109890 2024-05-09T00:22:21.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.758+0000 7fb59da92700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb5840b2880 0x7fb5840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.758+0000 7fb59da92700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb5840b2880 0x7fb5840b4d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb59806bc50 tx=0x7fb59400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:21.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.758+0000 7fb5ae70cb80 1 Processor -- start 2024-05-09T00:22:21.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.758+0000 7fb5ae70cb80 1 -- start start 2024-05-09T00:22:21.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb5ae70cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb5ae70cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dced714a0 0x563dceef4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb5ae70cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dceeee6a0 0x563dceeeead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb5ae70cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563dcee10ff0 con 0x563dcee5b160 2024-05-09T00:22:21.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb5ae70cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563dcedaf7f0 con 0x563dced714a0 2024-05-09T00:22:21.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb5ae70cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563dcee0cfb0 con 0x563dceeee6a0 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb59da92700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb59da92700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56852/0 (socket says 172.21.15.46:56852) 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb59da92700 1 -- 172.21.15.46:0/11878138 learned_addr learned my addr 172.21.15.46:0/11878138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb59e293700 1 --2- 172.21.15.46:0/11878138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dceeee6a0 0x563dceeeead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb59d291700 1 --2- 172.21.15.46:0/11878138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dced714a0 0x563dceef4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb59da92700 1 -- 172.21.15.46:0/11878138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dceeee6a0 msgr2=0x563dceeeead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb59da92700 1 --2- 172.21.15.46:0/11878138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dceeee6a0 0x563dceeeead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb59da92700 1 -- 172.21.15.46:0/11878138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dced714a0 msgr2=0x563dceef4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb59da92700 1 --2- 172.21.15.46:0/11878138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dced714a0 0x563dceef4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb59da92700 1 -- 172.21.15.46:0/11878138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dceda7370 con 0x563dcee5b160 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.759+0000 7fb59da92700 1 --2- 172.21.15.46:0/11878138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef44f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb59400a420 tx=0x7fb594012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42dbf591e23fbbbc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.760+0000 7fb583fff700 1 -- 172.21.15.46:0/11878138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594026070 con 0x563dcee5b160 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.760+0000 7fb583fff700 1 -- 172.21.15.46:0/11878138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb594016e40 con 0x563dcee5b160 2024-05-09T00:22:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.760+0000 7fb583fff700 1 -- 172.21.15.46:0/11878138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594014970 con 0x563dcee5b160 2024-05-09T00:22:21.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.761+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/11878138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 msgr2=0x563dceef44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:21.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.761+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/11878138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef44f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb59400a420 tx=0x7fb594012000 comp rx=0 tx=0).stop 2024-05-09T00:22:21.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.761+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/11878138 shutdown_connections 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.761+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/11878138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dceeee6a0 0x563dceeeead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.761+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/11878138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef44f0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.761+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/11878138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dced714a0 0x563dceef4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.761+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/11878138 >> 172.21.15.46:0/11878138 conn(0x563dcedd16a0 msgr2=0x563dceee9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.761+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/11878138 shutdown_connections 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.761+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/11878138 wait complete. 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.761+0000 7fb5ae70cb80 1 -- start start 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.761+0000 7fb5ae70cb80 1 Processor -- start 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb5ae70cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dced714a0 0x563dceef8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb5ae70cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb5ae70cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dceeee6a0 0x563dceef8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb5ae70cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563dcec8cba0 con 0x563dcee5b160 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb5ae70cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563dcedcb150 con 0x563dceeee6a0 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb5ae70cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563dceefb1d0 con 0x563dced714a0 2024-05-09T00:22:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb5ae70cb80 10 client.0 _ll_register_callbacks cb 0x563dcec826c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb59d291700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb59d291700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56858/0 (socket says 172.21.15.46:56858) 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb59d291700 1 -- 172.21.15.46:0/1673655187 learned_addr learned my addr 172.21.15.46:0/1673655187 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb59da92700 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dced714a0 0x563dceef8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb59e293700 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dceeee6a0 0x563dceef8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb59d291700 1 -- 172.21.15.46:0/1673655187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dced714a0 msgr2=0x563dceef8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb59d291700 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dced714a0 0x563dceef8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb59d291700 1 -- 172.21.15.46:0/1673655187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dceeee6a0 msgr2=0x563dceef8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb59d291700 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dceeee6a0 0x563dceef8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb59d291700 1 -- 172.21.15.46:0/1673655187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dcec8c490 con 0x563dcee5b160 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb59d291700 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef8780 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fb588011660 tx=0x7fb588011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf37b0bb65c8b12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.762+0000 7fb5817fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5817fa700 1 -- 172.21.15.46:0/1673655187 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588004230 con 0x563dcee5b160 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5817fa700 1 -- 172.21.15.46:0/1673655187 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb5880043d0 con 0x563dcee5b160 2024-05-09T00:22:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1673655187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563dcee1e9d0 con 0x563dcee5b160 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5817fa700 1 -- 172.21.15.46:0/1673655187 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb588012720 con 0x563dcee5b160 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5ae70cb80 10 client.17250 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1673655187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563dcef05c70 con 0x563dcee5b160 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5817fa700 1 -- 172.21.15.46:0/1673655187 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563dcee1e9d0 con 0x563dcee5b160 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5817fa700 1 -- 172.21.15.46:0/1673655187 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb588003bb0 con 0x563dcee5b160 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5ae70cb80 10 client.17250 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5ae70cb80 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5ae70cb80 20 client.17250 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dcef082a0 0x563dcef0a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5ae70cb80 4 client.17250 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.763+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1673655187 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dcee18a70 con 0x563dcef082a0 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.764+0000 7fb59da92700 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dcef082a0 0x563dcef0a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.764+0000 7fb59da92700 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dcef082a0 0x563dcef0a6f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d1028896b6ad62e server_cookie=854a6ec29e33dffd in_seq=0 out_seq=0 2024-05-09T00:22:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.764+0000 7fb5817fa700 10 client.17250 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5817fa700 1 -- 172.21.15.46:0/1673655187 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563dcee18a70 con 0x563dcef082a0 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5817fa700 10 client.17250 handle_command_reply: tid=0 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5ae70cb80 1 client.17250 shutdown 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1673655187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 msgr2=0x563dceef8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef8780 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fb588011660 tx=0x7fb588011970 comp rx=0 tx=0).stop 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1673655187 shutdown_connections 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dcef082a0 0x563dcef0a6f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dceeee6a0 0x563dceef8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dced714a0 0x563dceef8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/1673655187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef8780 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5817fa700 0 client.17250 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1673655187 >> 172.21.15.46:0/1673655187 conn(0x563dcedd16a0 msgr2=0x563dceee91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1673655187 shutdown_connections 2024-05-09T00:22:21.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1673655187 wait complete. 2024-05-09T00:22:21.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.765+0000 7fb5ae70cb80 20 client.17250 trim_cache size 0 max 16384 2024-05-09T00:22:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.786+0000 7fb5ae70cb80 1 Processor -- start 2024-05-09T00:22:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.786+0000 7fb5ae70cb80 1 -- start start 2024-05-09T00:22:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb5ae70cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dced714a0 0x563dceef3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb5ae70cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb5ae70cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dceeee6a0 0x563dceef98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb5ae70cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981b3620 con 0x563dcee5b160 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb5ae70cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981b0eb0 con 0x563dced714a0 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb5ae70cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981aece0 con 0x563dceeee6a0 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb59d291700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb59d291700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56870/0 (socket says 172.21.15.46:56870) 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb59d291700 1 -- 172.21.15.46:0/2133856576 learned_addr learned my addr 172.21.15.46:0/2133856576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb59da92700 1 --2- 172.21.15.46:0/2133856576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dced714a0 0x563dceef3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb59e293700 1 --2- 172.21.15.46:0/2133856576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dceeee6a0 0x563dceef98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb59d291700 1 -- 172.21.15.46:0/2133856576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dceeee6a0 msgr2=0x563dceef98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb59d291700 1 --2- 172.21.15.46:0/2133856576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dceeee6a0 0x563dceef98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb59d291700 1 -- 172.21.15.46:0/2133856576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dced714a0 msgr2=0x563dceef3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb59d291700 1 --2- 172.21.15.46:0/2133856576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dced714a0 0x563dceef3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb59d291700 1 -- 172.21.15.46:0/2133856576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dceee8fb0 con 0x563dcee5b160 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb59da92700 1 --2- 172.21.15.46:0/2133856576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dced714a0 0x563dceef3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.787+0000 7fb59e293700 1 --2- 172.21.15.46:0/2133856576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dceeee6a0 0x563dceef98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:22:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.788+0000 7fb59d291700 1 --2- 172.21.15.46:0/2133856576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef8b00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb5880245c0 tx=0x7fb588024640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c476ee3da667138 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.788+0000 7fb5837fe700 1 -- 172.21.15.46:0/2133856576 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588012ac0 con 0x563dcee5b160 2024-05-09T00:22:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.788+0000 7fb5837fe700 1 -- 172.21.15.46:0/2133856576 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb588024ce0 con 0x563dcee5b160 2024-05-09T00:22:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.788+0000 7fb5837fe700 1 -- 172.21.15.46:0/2133856576 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588003680 con 0x563dcee5b160 2024-05-09T00:22:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.789+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/2133856576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 msgr2=0x563dceef8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.789+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/2133856576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef8b00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb5880245c0 tx=0x7fb588024640 comp rx=0 tx=0).stop 2024-05-09T00:22:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.789+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/2133856576 shutdown_connections 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.789+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/2133856576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dceeee6a0 0x563dceef98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.789+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/2133856576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dced714a0 0x563dceef3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.789+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/2133856576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dceef8b00 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.789+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/2133856576 >> 172.21.15.46:0/2133856576 conn(0x563dcedd16a0 msgr2=0x563dceef3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.789+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/2133856576 shutdown_connections 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.789+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/2133856576 wait complete. 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.789+0000 7fb5ae70cb80 1 -- start start 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.789+0000 7fb5ae70cb80 1 Processor -- start 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb5ae70cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dced714a0 0x563dcef002b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb5ae70cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dcef007f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb5ae70cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dceeee6a0 0x563dceefd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb5ae70cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981af370 con 0x563dcee5b160 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb5ae70cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981b07e0 con 0x563dceeee6a0 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb5ae70cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981afa40 con 0x563dced714a0 2024-05-09T00:22:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb5ae70cb80 10 client.0 _ll_register_callbacks cb 0x563dcec826c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb59d291700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dcef007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb59d291700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dcef007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56872/0 (socket says 172.21.15.46:56872) 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb59d291700 1 -- 172.21.15.46:0/1788224871 learned_addr learned my addr 172.21.15.46:0/1788224871 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb59da92700 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dced714a0 0x563dcef002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb59e293700 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dceeee6a0 0x563dceefd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb59d291700 1 -- 172.21.15.46:0/1788224871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dced714a0 msgr2=0x563dcef002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb59d291700 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dced714a0 0x563dcef002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb59d291700 1 -- 172.21.15.46:0/1788224871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dceeee6a0 msgr2=0x563dceefd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb59d291700 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dceeee6a0 0x563dceefd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb59d291700 1 -- 172.21.15.46:0/1788224871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dceefd7c0 con 0x563dcee5b160 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.790+0000 7fb59e293700 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dceeee6a0 0x563dceefd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb59d291700 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dcef007f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb588011e40 tx=0x7fb588005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7861f3e8ed9408 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5817fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5817fa700 1 -- 172.21.15.46:0/1788224871 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5880128c0 con 0x563dcee5b160 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5817fa700 1 -- 172.21.15.46:0/1788224871 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fb588012a60 con 0x563dcee5b160 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1788224871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb588024ce0 con 0x563dcee5b160 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5817fa700 1 -- 172.21.15.46:0/1788224871 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb588012c60 con 0x563dcee5b160 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5ae70cb80 10 client.17256 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1788224871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563dcef73050 con 0x563dcee5b160 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5817fa700 1 -- 172.21.15.46:0/1788224871 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb588024ce0 con 0x563dcee5b160 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5817fa700 1 -- 172.21.15.46:0/1788224871 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb588034070 con 0x563dcee5b160 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5ae70cb80 10 client.17256 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5ae70cb80 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5ae70cb80 20 client.17256 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dcef73670 0x563dcef75a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5ae70cb80 4 client.17256 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.791+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1788224871 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563dcec8c970 con 0x563dcef73670 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.792+0000 7fb59da92700 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dcef73670 0x563dcef75a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.792+0000 7fb59da92700 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dcef73670 0x563dcef75a90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bcdb6d44e05f519 server_cookie=92c6054ec1f757ef in_seq=0 out_seq=0 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.792+0000 7fb5817fa700 10 client.17256 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.792+0000 7fb5817fa700 1 -- 172.21.15.46:0/1788224871 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563dcec8c970 con 0x563dcef73670 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.792+0000 7fb5817fa700 10 client.17256 handle_command_reply: tid=0 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.792+0000 7fb5ae70cb80 1 client.17256 shutdown 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1788224871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 msgr2=0x563dcef007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dcef007f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb588011e40 tx=0x7fb588005990 comp rx=0 tx=0).stop 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1788224871 shutdown_connections 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dcef73670 0x563dcef75a90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dceeee6a0 0x563dceefd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dced714a0 0x563dcef002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb5817fa700 0 client.17256 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb5ae70cb80 1 --2- 172.21.15.46:0/1788224871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dcee5b160 0x563dcef007f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1788224871 >> 172.21.15.46:0/1788224871 conn(0x563dcedd16a0 msgr2=0x563dceeed690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1788224871 shutdown_connections 2024-05-09T00:22:21.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb5ae70cb80 1 -- 172.21.15.46:0/1788224871 wait complete. 2024-05-09T00:22:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb5ae70cb80 20 client.17256 trim_cache size 0 max 16384 2024-05-09T00:22:21.797 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:22:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T00:22:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T00:22:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "d73b3d0d-e151-443f-8bdb-614bd0edc8cc": { 2024-05-09T00:22:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T00:22:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "d73b3d0d-e151-443f-8bdb-614bd0edc8cc", 2024-05-09T00:22:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T00:22:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:22:21.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:22:21.797 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:22:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb59fcf6700 1 -- 172.21.15.46:0/2657414751 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb5840b2880 msgr2=0x7fb5840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb59fcf6700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb5840b2880 0x7fb5840b4d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb59806bc50 tx=0x7fb59400c040 comp rx=0 tx=0).stop 2024-05-09T00:22:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb59fcf6700 1 -- 172.21.15.46:0/2657414751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb598109890 msgr2=0x7fb5981044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.793+0000 7fb59fcf6700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb598109890 0x7fb5981044f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb59000bff0 tx=0x7fb5900036f0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.794+0000 7fb59fcf6700 1 -- 172.21.15.46:0/2657414751 shutdown_connections 2024-05-09T00:22:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.794+0000 7fb59fcf6700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fb5840b2880 0x7fb5840b4d90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.794+0000 7fb59fcf6700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb598109890 0x7fb5981044f0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.794+0000 7fb59fcf6700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb598102b20 0x7fb598103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.794+0000 7fb59fcf6700 1 --2- 172.21.15.46:0/2657414751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb598108f40 0x7fb598103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:21.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.794+0000 7fb59fcf6700 1 -- 172.21.15.46:0/2657414751 >> 172.21.15.46:0/2657414751 conn(0x7fb5980fe4e0 msgr2=0x7fb598108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:21.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.794+0000 7fb59fcf6700 1 -- 172.21.15.46:0/2657414751 shutdown_connections 2024-05-09T00:22:21.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:21.794+0000 7fb59fcf6700 1 -- 172.21.15.46:0/2657414751 wait complete. 2024-05-09T00:22:22.638 INFO:tasks.cephfs.filesystem:scrub status for tag:d73b3d0d-e151-443f-8bdb-614bd0edc8cc - {'path': '/', 'tag': 'd73b3d0d-e151-443f-8bdb-614bd0edc8cc', 'options': 'recursive,force'} 2024-05-09T00:22:22.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:22 smithi142 ceph-mon[34533]: pgmap v706: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:22:22.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:22 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:22:23.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:22 smithi046 ceph-mon[30151]: pgmap v706: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:22:23.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:22 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:22:23.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:22 smithi169 ceph-mon[34777]: pgmap v706: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:22:23.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:22 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:22:24.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:24 smithi142 ceph-mon[34533]: pgmap v707: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:22:25.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:24 smithi046 ceph-mon[30151]: pgmap v707: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:22:25.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:24 smithi169 ceph-mon[34777]: pgmap v707: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:22:26.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:26 smithi142 ceph-mon[34533]: pgmap v708: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:27.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:26 smithi046 ceph-mon[30151]: pgmap v708: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:27.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:26 smithi169 ceph-mon[34777]: pgmap v708: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:28.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:28 smithi142 ceph-mon[34533]: pgmap v709: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:22:29.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:28 smithi046 ceph-mon[30151]: pgmap v709: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:22:29.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:28 smithi169 ceph-mon[34777]: pgmap v709: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:22:30.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:30 smithi142 ceph-mon[34533]: pgmap v710: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:22:31.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:30 smithi046 ceph-mon[30151]: pgmap v710: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:22:31.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:30 smithi169 ceph-mon[34777]: pgmap v710: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:22:32.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:32 smithi142 ceph-mon[34533]: pgmap v711: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:22:33.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:32 smithi046 ceph-mon[30151]: pgmap v711: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:22:33.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:32 smithi169 ceph-mon[34777]: pgmap v711: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:22:34.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:34 smithi142 ceph-mon[34533]: pgmap v712: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:22:35.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:34 smithi046 ceph-mon[30151]: pgmap v712: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:22:35.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:34 smithi169 ceph-mon[34777]: pgmap v712: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:22:36.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:36 smithi142 ceph-mon[34533]: pgmap v713: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:22:37.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:36 smithi046 ceph-mon[30151]: pgmap v713: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:22:37.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:36 smithi169 ceph-mon[34777]: pgmap v713: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:22:38.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:38 smithi142 ceph-mon[34533]: pgmap v714: 97 pgs: 97 active+clean; 6.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:22:39.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:38 smithi046 ceph-mon[30151]: pgmap v714: 97 pgs: 97 active+clean; 6.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:22:39.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:38 smithi169 ceph-mon[34777]: pgmap v714: 97 pgs: 97 active+clean; 6.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:22:40.944 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:40 smithi142 ceph-mon[34533]: pgmap v715: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:41.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:40 smithi046 ceph-mon[30151]: pgmap v715: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:41.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:40 smithi169 ceph-mon[34777]: pgmap v715: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:22:42.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:42 smithi142 ceph-mon[34533]: pgmap v716: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:22:43.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:42 smithi046 ceph-mon[30151]: pgmap v716: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:22:43.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:42 smithi169 ceph-mon[34777]: pgmap v716: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T00:22:44.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:44 smithi142 ceph-mon[34533]: pgmap v717: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:22:45.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:44 smithi046 ceph-mon[30151]: pgmap v717: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:22:45.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:44 smithi169 ceph-mon[34777]: pgmap v717: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:22:47.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:46 smithi046 ceph-mon[30151]: pgmap v718: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:22:47.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:46 smithi169 ceph-mon[34777]: pgmap v718: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:22:47.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:46 smithi142 ceph-mon[34533]: pgmap v718: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:22:49.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:48 smithi046 ceph-mon[30151]: pgmap v719: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:22:49.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:48 smithi169 ceph-mon[34777]: pgmap v719: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:22:49.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:48 smithi142 ceph-mon[34533]: pgmap v719: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:22:51.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:50 smithi046 ceph-mon[30151]: pgmap v720: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:22:51.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:50 smithi169 ceph-mon[34777]: pgmap v720: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:22:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:50 smithi142 ceph-mon[34533]: pgmap v720: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:22:52.640 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:22:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:22:53.036 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:52 smithi046 ceph-mon[30151]: pgmap v721: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:22:53.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:52 smithi169 ceph-mon[34777]: pgmap v721: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:22:53.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:52 smithi142 ceph-mon[34533]: pgmap v721: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:22:54.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:54 smithi046 ceph-mon[30151]: pgmap v722: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:22:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.930+0000 7f60b8155700 1 -- 172.21.15.46:0/2479558055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b0108f20 msgr2=0x7f60b0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.930+0000 7f60b8155700 1 --2- 172.21.15.46:0/2479558055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b0108f20 0x7f60b0109330 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f60a00099c0 tx=0x7f60a0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.930+0000 7f60b8155700 1 -- 172.21.15.46:0/2479558055 shutdown_connections 2024-05-09T00:22:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.930+0000 7f60b8155700 1 --2- 172.21.15.46:0/2479558055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b0109870 0x7f60b01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.930+0000 7f60b8155700 1 --2- 172.21.15.46:0/2479558055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b01045f0 0x7f60b0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.930+0000 7f60b8155700 1 --2- 172.21.15.46:0/2479558055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b0108f20 0x7f60b0109330 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.930+0000 7f60b8155700 1 -- 172.21.15.46:0/2479558055 >> 172.21.15.46:0/2479558055 conn(0x7f60b0100040 msgr2=0x7f60b0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.930+0000 7f60b8155700 1 -- 172.21.15.46:0/2479558055 shutdown_connections 2024-05-09T00:22:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.930+0000 7f60b8155700 1 -- 172.21.15.46:0/2479558055 wait complete. 2024-05-09T00:22:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.931+0000 7f60b8155700 1 Processor -- start 2024-05-09T00:22:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.931+0000 7f60b8155700 1 -- start start 2024-05-09T00:22:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.931+0000 7f60b8155700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b01045f0 0x7f60b019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.931+0000 7f60b8155700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b0108f20 0x7f60b019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.931+0000 7f60b8155700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b0109870 0x7f60b019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.931+0000 7f60b8155700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0114480 con 0x7f60b0108f20 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.931+0000 7f60b8155700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0114300 con 0x7f60b0109870 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.931+0000 7f60b56f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b0108f20 0x7f60b019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.931+0000 7f60b8155700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0114600 con 0x7f60b01045f0 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.931+0000 7f60b56f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b0108f20 0x7f60b019b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44532/0 (socket says 172.21.15.46:44532) 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.931+0000 7f60b56f0700 1 -- 172.21.15.46:0/1059808303 learned_addr learned my addr 172.21.15.46:0/1059808303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60b5ef1700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b01045f0 0x7f60b019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60b66f2700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b0109870 0x7f60b019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60b56f0700 1 -- 172.21.15.46:0/1059808303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b01045f0 msgr2=0x7f60b019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60b56f0700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b01045f0 0x7f60b019aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60b56f0700 1 -- 172.21.15.46:0/1059808303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b0109870 msgr2=0x7f60b019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60b56f0700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b0109870 0x7f60b019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60b56f0700 1 -- 172.21.15.46:0/1059808303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b019fd80 con 0x7f60b0108f20 2024-05-09T00:22:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60b56f0700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b0108f20 0x7f60b019b450 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f60a0006010 tx=0x7f60a000b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60a6ffd700 1 -- 172.21.15.46:0/1059808303 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a002c070 con 0x7f60b0108f20 2024-05-09T00:22:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60a6ffd700 1 -- 172.21.15.46:0/1059808303 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a0003680 con 0x7f60b0108f20 2024-05-09T00:22:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60b8155700 1 -- 172.21.15.46:0/1059808303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b01a0010 con 0x7f60b0108f20 2024-05-09T00:22:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60a6ffd700 1 -- 172.21.15.46:0/1059808303 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0004130 con 0x7f60b0108f20 2024-05-09T00:22:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.932+0000 7f60b8155700 1 -- 172.21.15.46:0/1059808303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b01a04f0 con 0x7f60b0108f20 2024-05-09T00:22:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.935+0000 7f60a6ffd700 1 -- 172.21.15.46:0/1059808303 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f60a002f070 con 0x7f60b0108f20 2024-05-09T00:22:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.936+0000 7f60a6ffd700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f609c0b2880 0x7f609c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.936+0000 7f60c6b6bb80 1 Processor -- start 2024-05-09T00:22:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.936+0000 7f60b5ef1700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f609c0b2880 0x7f609c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.936+0000 7f60a6ffd700 1 -- 172.21.15.46:0/1059808303 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f60a00e71d0 con 0x7f60b0108f20 2024-05-09T00:22:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.936+0000 7f60c6b6bb80 1 -- start start 2024-05-09T00:22:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60c6b6bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb8fd00 0x55d93cc98240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60b5ef1700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f609c0b2880 0x7f609c0b4d90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f60ac005950 tx=0x7f60ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60c6b6bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 0x55d93cc98780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60c6b6bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cc92400 0x55d93cc92810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60c6b6bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d93cbb50a0 con 0x55d93cb154a0 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60c6b6bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d93cb5a9a0 con 0x55d93cc92400 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60c6b6bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d93cbb1060 con 0x55d93cb8fd00 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60b56f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 0x55d93cc98780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60b56f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 0x55d93cc98780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44546/0 (socket says 172.21.15.46:44546) 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60b56f0700 1 -- 172.21.15.46:0/434477933 learned_addr learned my addr 172.21.15.46:0/434477933 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60b5ef1700 1 --2- 172.21.15.46:0/434477933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb8fd00 0x55d93cc98240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60b66f2700 1 --2- 172.21.15.46:0/434477933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cc92400 0x55d93cc92810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60b56f0700 1 -- 172.21.15.46:0/434477933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb8fd00 msgr2=0x55d93cc98240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60b56f0700 1 --2- 172.21.15.46:0/434477933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb8fd00 0x55d93cc98240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60b56f0700 1 -- 172.21.15.46:0/434477933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cc92400 msgr2=0x55d93cc92810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60b56f0700 1 --2- 172.21.15.46:0/434477933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cc92400 0x55d93cc92810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60b56f0700 1 -- 172.21.15.46:0/434477933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d93cb4b500 con 0x55d93cb154a0 2024-05-09T00:22:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.937+0000 7f60b56f0700 1 --2- 172.21.15.46:0/434477933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 0x55d93cc98780 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009990 tx=0x7f60a0009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad98c06758c2759 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.938+0000 7f609bfff700 1 -- 172.21.15.46:0/434477933 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00aef40 con 0x55d93cb154a0 2024-05-09T00:22:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.938+0000 7f609bfff700 1 -- 172.21.15.46:0/434477933 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a00b5110 con 0x55d93cb154a0 2024-05-09T00:22:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.938+0000 7f609bfff700 1 -- 172.21.15.46:0/434477933 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00aeaa0 con 0x55d93cb154a0 2024-05-09T00:22:54.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.939+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/434477933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 msgr2=0x55d93cc98780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.939+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/434477933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 0x55d93cc98780 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009990 tx=0x7f60a0009710 comp rx=0 tx=0).stop 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.939+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/434477933 shutdown_connections 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.939+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/434477933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cc92400 0x55d93cc92810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.939+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/434477933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb8fd00 0x55d93cc98240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.939+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/434477933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 0x55d93cc98780 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.939+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/434477933 >> 172.21.15.46:0/434477933 conn(0x55d93cb18f70 msgr2=0x55d93cc8d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.939+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/434477933 shutdown_connections 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.939+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/434477933 wait complete. 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.939+0000 7f60c6b6bb80 1 -- start start 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.939+0000 7f60c6b6bb80 1 Processor -- start 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60c6b6bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 0x55d93cca1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60c6b6bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cb8fd00 0x55d93cca1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60c6b6bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cc92400 0x55d93cc9beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60c6b6bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d93cbb50a0 con 0x55d93cb154a0 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60b5ef1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 0x55d93cca1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60b5ef1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 0x55d93cca1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44558/0 (socket says 172.21.15.46:44558) 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60b5ef1700 1 -- 172.21.15.46:0/3711341098 learned_addr learned my addr 172.21.15.46:0/3711341098 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3711341098 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d93cb4fb20 con 0x55d93cb8fd00 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3711341098 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d93cc92fb0 con 0x55d93cc92400 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60c6b6bb80 10 client.0 _ll_register_callbacks cb 0x55d93ca266c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60b66f2700 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cc92400 0x55d93cc9beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60b56f0700 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cb8fd00 0x55d93cca1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60b5ef1700 1 -- 172.21.15.46:0/3711341098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cc92400 msgr2=0x55d93cc9beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60b5ef1700 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cc92400 0x55d93cc9beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60b5ef1700 1 -- 172.21.15.46:0/3711341098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cb8fd00 msgr2=0x55d93cca1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60b5ef1700 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cb8fd00 0x55d93cca1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.940+0000 7f60b5ef1700 1 -- 172.21.15.46:0/3711341098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d93ca30490 con 0x55d93cb154a0 2024-05-09T00:22:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.941+0000 7f60b5ef1700 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 0x55d93cca1ab0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0116f0 tx=0x7f60ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3ddee390f23024 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.941+0000 7f60997fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.941+0000 7f60997fa700 1 -- 172.21.15.46:0/3711341098 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac028470 con 0x55d93cb154a0 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.941+0000 7f60997fa700 1 -- 172.21.15.46:0/3711341098 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f60ac028610 con 0x55d93cb154a0 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.941+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3711341098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a00b5110 con 0x55d93cb154a0 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.941+0000 7f60997fa700 1 -- 172.21.15.46:0/3711341098 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f60ac028810 con 0x55d93cb154a0 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.941+0000 7f60c6b6bb80 10 client.17268 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.941+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3711341098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d93cca9710 con 0x55d93cb154a0 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.941+0000 7f60997fa700 1 -- 172.21.15.46:0/3711341098 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a00b5110 con 0x55d93cb154a0 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.941+0000 7f60997fa700 1 -- 172.21.15.46:0/3711341098 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f60ac01e550 con 0x55d93cb154a0 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.942+0000 7f60c6b6bb80 10 client.17268 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.942+0000 7f60c6b6bb80 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.942+0000 7f60c6b6bb80 20 client.17268 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.942+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d93ccac130 0x55d93ccae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.942+0000 7f60c6b6bb80 4 client.17268 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:22:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.942+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3711341098 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d93cbc2a80 con 0x55d93ccac130 2024-05-09T00:22:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.942+0000 7f60b56f0700 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d93ccac130 0x55d93ccae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.942+0000 7f60997fa700 10 client.17268 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.942+0000 7f60b56f0700 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d93ccac130 0x55d93ccae600 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39d8f1f6bc881509 server_cookie=53596270d537a3c2 in_seq=0 out_seq=0 2024-05-09T00:22:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.943+0000 7f60997fa700 1 -- 172.21.15.46:0/3711341098 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d93cbc2a80 con 0x55d93ccac130 2024-05-09T00:22:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.943+0000 7f60997fa700 10 client.17268 handle_command_reply: tid=0 2024-05-09T00:22:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.943+0000 7f60c6b6bb80 1 client.17268 shutdown 2024-05-09T00:22:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.943+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3711341098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 msgr2=0x55d93cca1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.943+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 0x55d93cca1ab0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0116f0 tx=0x7f60ac0047c0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.944+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3711341098 shutdown_connections 2024-05-09T00:22:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.944+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d93ccac130 0x55d93ccae600 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.944+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cc92400 0x55d93cc9beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.944+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb154a0 0x55d93cca1ab0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.944+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/3711341098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cb8fd00 0x55d93cca1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.944+0000 7f60997fa700 0 client.17268 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.944+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3711341098 >> 172.21.15.46:0/3711341098 conn(0x55d93cb18f70 msgr2=0x55d93cc86ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.944+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3711341098 shutdown_connections 2024-05-09T00:22:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.944+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3711341098 wait complete. 2024-05-09T00:22:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.944+0000 7f60c6b6bb80 20 client.17268 trim_cache size 0 max 16384 2024-05-09T00:22:54.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.964+0000 7f60c6b6bb80 1 Processor -- start 2024-05-09T00:22:54.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.964+0000 7f60c6b6bb80 1 -- start start 2024-05-09T00:22:54.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.964+0000 7f60c6b6bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb154a0 0x55d93cc96310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.964+0000 7f60c6b6bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cb8fd00 0x55d93ccb3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.964+0000 7f60c6b6bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cc92400 0x55d93ccb4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.964+0000 7f60c6b6bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01af340 con 0x55d93cc92400 2024-05-09T00:22:54.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.964+0000 7f60c6b6bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01acbd0 con 0x55d93cb8fd00 2024-05-09T00:22:54.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.964+0000 7f60c6b6bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01aaa20 con 0x55d93cb154a0 2024-05-09T00:22:54.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.964+0000 7f60b66f2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cc92400 0x55d93ccb4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f60b66f2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cc92400 0x55d93ccb4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44560/0 (socket says 172.21.15.46:44560) 2024-05-09T00:22:54.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f60b66f2700 1 -- 172.21.15.46:0/3952918537 learned_addr learned my addr 172.21.15.46:0/3952918537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:54.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f60b5ef1700 1 --2- 172.21.15.46:0/3952918537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb154a0 0x55d93cc96310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f60b56f0700 1 --2- 172.21.15.46:0/3952918537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cb8fd00 0x55d93ccb3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f60b66f2700 1 -- 172.21.15.46:0/3952918537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb154a0 msgr2=0x55d93cc96310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:54.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f60b66f2700 1 --2- 172.21.15.46:0/3952918537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb154a0 0x55d93cc96310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f60b66f2700 1 -- 172.21.15.46:0/3952918537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cb8fd00 msgr2=0x55d93ccb3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:54.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f60b66f2700 1 --2- 172.21.15.46:0/3952918537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cb8fd00 0x55d93ccb3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f60b66f2700 1 -- 172.21.15.46:0/3952918537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d93cc899d0 con 0x55d93cc92400 2024-05-09T00:22:54.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f60b5ef1700 1 --2- 172.21.15.46:0/3952918537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb154a0 0x55d93cc96310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:22:54.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f60b56f0700 1 --2- 172.21.15.46:0/3952918537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cb8fd00 0x55d93ccb3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:22:54.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f60b66f2700 1 --2- 172.21.15.46:0/3952918537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cc92400 0x55d93ccb4300 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009fa0 tx=0x7f60a8009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00d5a92593839b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:54.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f609b7fe700 1 -- 172.21.15.46:0/3952918537 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a801b030 con 0x55d93cc92400 2024-05-09T00:22:54.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f609b7fe700 1 -- 172.21.15.46:0/3952918537 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a80178f0 con 0x55d93cc92400 2024-05-09T00:22:54.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.965+0000 7f609b7fe700 1 -- 172.21.15.46:0/3952918537 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a801fa00 con 0x55d93cc92400 2024-05-09T00:22:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.966+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3952918537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cc92400 msgr2=0x55d93ccb4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.966+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/3952918537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cc92400 0x55d93ccb4300 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009fa0 tx=0x7f60a8009100 comp rx=0 tx=0).stop 2024-05-09T00:22:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.966+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3952918537 shutdown_connections 2024-05-09T00:22:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.966+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/3952918537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cc92400 0x55d93ccb4300 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.966+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/3952918537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb154a0 0x55d93cc96310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.966+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/3952918537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cb8fd00 0x55d93ccb3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.966+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3952918537 >> 172.21.15.46:0/3952918537 conn(0x55d93cb18f70 msgr2=0x55d93cc95c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.966+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3952918537 shutdown_connections 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.966+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/3952918537 wait complete. 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.966+0000 7f60c6b6bb80 1 -- start start 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60c6b6bb80 1 Processor -- start 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60c6b6bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb154a0 0x55d93cca83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60c6b6bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb8fd00 0x55d93cca5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60c6b6bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cc92400 0x55d93cca5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60c6b6bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01ab090 con 0x55d93cb8fd00 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60c6b6bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01ac500 con 0x55d93cc92400 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60c6b6bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01ab760 con 0x55d93cb154a0 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60b56f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb8fd00 0x55d93cca5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60c6b6bb80 10 client.0 _ll_register_callbacks cb 0x55d93ca266c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60b56f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb8fd00 0x55d93cca5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44570/0 (socket says 172.21.15.46:44570) 2024-05-09T00:22:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60b56f0700 1 -- 172.21.15.46:0/2026092762 learned_addr learned my addr 172.21.15.46:0/2026092762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60b5ef1700 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb154a0 0x55d93cca83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.967+0000 7f60b66f2700 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cc92400 0x55d93cca5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60b56f0700 1 -- 172.21.15.46:0/2026092762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb154a0 msgr2=0x55d93cca83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60b56f0700 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb154a0 0x55d93cca83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60b56f0700 1 -- 172.21.15.46:0/2026092762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cc92400 msgr2=0x55d93cca5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:22:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60b56f0700 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cc92400 0x55d93cca5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60b56f0700 1 -- 172.21.15.46:0/2026092762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d93cca6390 con 0x55d93cb8fd00 2024-05-09T00:22:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60b56f0700 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb8fd00 0x55d93cca5770 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f60a00abab0 tx=0x7f60a0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804c7b94395b05a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60997fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:22:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60997fa700 1 -- 172.21.15.46:0/2026092762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00aef40 con 0x55d93cb8fd00 2024-05-09T00:22:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60997fa700 1 -- 172.21.15.46:0/2026092762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f60a00d10a0 con 0x55d93cb8fd00 2024-05-09T00:22:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/2026092762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a80178f0 con 0x55d93cb8fd00 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60997fa700 1 -- 172.21.15.46:0/2026092762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f60a00b5110 con 0x55d93cb8fd00 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60c6b6bb80 10 client.17274 fetch_fsmap learned FSMap version 26 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/2026092762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d93cd16750 con 0x55d93cb8fd00 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.968+0000 7f60997fa700 1 -- 172.21.15.46:0/2026092762 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a80178f0 con 0x55d93cb8fd00 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.969+0000 7f60997fa700 1 -- 172.21.15.46:0/2026092762 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f60a00aeaa0 con 0x55d93cb8fd00 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.969+0000 7f60c6b6bb80 10 client.17274 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.969+0000 7f60c6b6bb80 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.969+0000 7f60c6b6bb80 20 client.17274 populate_metadata read hostname 'smithi046' 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.969+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d93cd16cb0 0x55d93cd190d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.969+0000 7f60c6b6bb80 4 client.17274 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.969+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/2026092762 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d93cbbcb20 con 0x55d93cd16cb0 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.969+0000 7f60b5ef1700 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d93cd16cb0 0x55d93cd190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.969+0000 7f60b5ef1700 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d93cd16cb0 0x55d93cd190d0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce059646699064b server_cookie=c92f0f633c7d7a78 in_seq=0 out_seq=0 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.969+0000 7f60997fa700 10 client.17274 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60997fa700 1 -- 172.21.15.46:0/2026092762 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d93cbbcb20 con 0x55d93cd16cb0 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60997fa700 10 client.17274 handle_command_reply: tid=0 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60c6b6bb80 1 client.17274 shutdown 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/2026092762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb8fd00 msgr2=0x55d93cca5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb8fd00 0x55d93cca5770 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f60a00abab0 tx=0x7f60a0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/2026092762 shutdown_connections 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55d93cd16cb0 0x55d93cd190d0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d93cc92400 0x55d93cca5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d93cb154a0 0x55d93cca83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60997fa700 0 client.17274 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60c6b6bb80 1 --2- 172.21.15.46:0/2026092762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d93cb8fd00 0x55d93cca5770 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/2026092762 >> 172.21.15.46:0/2026092762 conn(0x55d93cb18f70 msgr2=0x55d93cc86ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/2026092762 shutdown_connections 2024-05-09T00:22:54.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60c6b6bb80 1 -- 172.21.15.46:0/2026092762 wait complete. 2024-05-09T00:22:54.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.970+0000 7f60c6b6bb80 20 client.17274 trim_cache size 0 max 16384 2024-05-09T00:22:54.975 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:22:54.975 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:22:54.975 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:22:54.975 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:22:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.971+0000 7f60b8155700 1 -- 172.21.15.46:0/1059808303 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f609c0b2880 msgr2=0x7f609c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.971+0000 7f60b8155700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f609c0b2880 0x7f609c0b4d90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f60ac005950 tx=0x7f60ac00b410 comp rx=0 tx=0).stop 2024-05-09T00:22:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.971+0000 7f60b8155700 1 -- 172.21.15.46:0/1059808303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b0108f20 msgr2=0x7f60b019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.971+0000 7f60b8155700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b0108f20 0x7f60b019b450 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f60a0006010 tx=0x7f60a000b9c0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.971+0000 7f60b8155700 1 -- 172.21.15.46:0/1059808303 shutdown_connections 2024-05-09T00:22:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.971+0000 7f60b8155700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f609c0b2880 0x7f609c0b4d90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.971+0000 7f60b8155700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60b0109870 0x7f60b019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.971+0000 7f60b8155700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60b01045f0 0x7f60b019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.971+0000 7f60b8155700 1 --2- 172.21.15.46:0/1059808303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b0108f20 0x7f60b019b450 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.971+0000 7f60b8155700 1 -- 172.21.15.46:0/1059808303 >> 172.21.15.46:0/1059808303 conn(0x7f60b0100040 msgr2=0x7f60b010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.971+0000 7f60b8155700 1 -- 172.21.15.46:0/1059808303 shutdown_connections 2024-05-09T00:22:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:54.971+0000 7f60b8155700 1 -- 172.21.15.46:0/1059808303 wait complete. 2024-05-09T00:22:55.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:54 smithi169 ceph-mon[34777]: pgmap v722: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:22:55.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:54 smithi142 ceph-mon[34533]: pgmap v722: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:22:55.833 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:22:55.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:22:56.099 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:22:57.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:56 smithi046 ceph-mon[30151]: pgmap v723: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:22:57.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:56 smithi169 ceph-mon[34777]: pgmap v723: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:22:57.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:56 smithi142 ceph-mon[34533]: pgmap v723: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:22:57.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.717+0000 7f44a06ae700 1 -- 172.21.15.46:0/1636576628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44981034c0 msgr2=0x7f44981038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:57.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.717+0000 7f44a06ae700 1 --2- 172.21.15.46:0/1636576628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44981034c0 0x7f44981038d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f44940099c0 tx=0x7f4494009cd0 comp rx=0 tx=0).stop 2024-05-09T00:22:57.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.717+0000 7f44a06ae700 1 -- 172.21.15.46:0/1636576628 shutdown_connections 2024-05-09T00:22:57.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.717+0000 7f44a06ae700 1 --2- 172.21.15.46:0/1636576628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498108740 0x7f44981100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:57.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.717+0000 7f44a06ae700 1 --2- 172.21.15.46:0/1636576628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44981034c0 0x7f44981038d0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:57.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.717+0000 7f44a06ae700 1 --2- 172.21.15.46:0/1636576628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4498107df0 0x7f4498108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:57.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.717+0000 7f44a06ae700 1 -- 172.21.15.46:0/1636576628 >> 172.21.15.46:0/1636576628 conn(0x7f44980fedf0 msgr2=0x7f4498101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:57.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.717+0000 7f44a06ae700 1 -- 172.21.15.46:0/1636576628 shutdown_connections 2024-05-09T00:22:57.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.717+0000 7f44a06ae700 1 -- 172.21.15.46:0/1636576628 wait complete. 2024-05-09T00:22:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.718+0000 7f44a06ae700 1 Processor -- start 2024-05-09T00:22:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.718+0000 7f44a06ae700 1 -- start start 2024-05-09T00:22:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.718+0000 7f44a06ae700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44981034c0 0x7f449819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.718+0000 7f44a06ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498107df0 0x7f449819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.718+0000 7f44a06ae700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498108740 0x7f449819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.718+0000 7f44a06ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4498113370 con 0x7f4498107df0 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.718+0000 7f44a06ae700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44981131f0 con 0x7f4498108740 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.718+0000 7f44a06ae700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44981134f0 con 0x7f44981034c0 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.718+0000 7f449dc49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498107df0 0x7f449819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.718+0000 7f449dc49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498107df0 0x7f449819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45940/0 (socket says 172.21.15.46:45940) 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.718+0000 7f449dc49700 1 -- 172.21.15.46:0/1819667857 learned_addr learned my addr 172.21.15.46:0/1819667857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.718+0000 7f449ec4b700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498108740 0x7f449819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.719+0000 7f449e44a700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44981034c0 0x7f449819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.719+0000 7f449dc49700 1 -- 172.21.15.46:0/1819667857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44981034c0 msgr2=0x7f449819ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.719+0000 7f449dc49700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44981034c0 0x7f449819ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.719+0000 7f449dc49700 1 -- 172.21.15.46:0/1819667857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498108740 msgr2=0x7f449819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.719+0000 7f449dc49700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498108740 0x7f449819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.719+0000 7f449dc49700 1 -- 172.21.15.46:0/1819667857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449819fd00 con 0x7f4498107df0 2024-05-09T00:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.719+0000 7f449dc49700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498107df0 0x7f449819b3d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f448c00d980 tx=0x7f448c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.719+0000 7f448b7fe700 1 -- 172.21.15.46:0/1819667857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c017070 con 0x7f4498107df0 2024-05-09T00:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.719+0000 7f44a06ae700 1 -- 172.21.15.46:0/1819667857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f449819fff0 con 0x7f4498107df0 2024-05-09T00:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.719+0000 7f448b7fe700 1 -- 172.21.15.46:0/1819667857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f448c00fd40 con 0x7f4498107df0 2024-05-09T00:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.719+0000 7f448b7fe700 1 -- 172.21.15.46:0/1819667857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c00e3b0 con 0x7f4498107df0 2024-05-09T00:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.719+0000 7f44a06ae700 1 -- 172.21.15.46:0/1819667857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44981a7910 con 0x7f4498107df0 2024-05-09T00:22:57.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.720+0000 7f44a06ae700 1 -- 172.21.15.46:0/1819667857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44980512c0 con 0x7f4498107df0 2024-05-09T00:22:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.723+0000 7f448b7fe700 1 -- 172.21.15.46:0/1819667857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f448c010610 con 0x7f4498107df0 2024-05-09T00:22:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.724+0000 7f448b7fe700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f44840b2880 0x7f44840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:22:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.724+0000 7f449e44a700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f44840b2880 0x7f44840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:22:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.724+0000 7f448b7fe700 1 -- 172.21.15.46:0/1819667857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f448c0e1ca0 con 0x7f4498107df0 2024-05-09T00:22:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.724+0000 7f449e44a700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f44840b2880 0x7f44840b4d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4494009990 tx=0x7f44940036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:22:57.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.728+0000 7f448b7fe700 1 -- 172.21.15.46:0/1819667857 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f448c0a9980 con 0x7f4498107df0 2024-05-09T00:22:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.990+0000 7f44a06ae700 1 -- 172.21.15.46:0/1819667857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f449819c190 con 0x7f4498107df0 2024-05-09T00:22:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.991+0000 7f448b7fe700 1 -- 172.21.15.46:0/1819667857 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f448c0a9280 con 0x7f4498107df0 2024-05-09T00:22:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:22:57.991 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:22:57.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.994+0000 7f44a06ae700 1 -- 172.21.15.46:0/1819667857 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f44840b2880 msgr2=0x7f44840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:57.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.994+0000 7f44a06ae700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f44840b2880 0x7f44840b4d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4494009990 tx=0x7f44940036e0 comp rx=0 tx=0).stop 2024-05-09T00:22:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.994+0000 7f44a06ae700 1 -- 172.21.15.46:0/1819667857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498107df0 msgr2=0x7f449819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:22:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.994+0000 7f44a06ae700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498107df0 0x7f449819b3d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f448c00d980 tx=0x7f448c00dc90 comp rx=0 tx=0).stop 2024-05-09T00:22:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.994+0000 7f44a06ae700 1 -- 172.21.15.46:0/1819667857 shutdown_connections 2024-05-09T00:22:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.994+0000 7f44a06ae700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f44840b2880 0x7f44840b4d90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.994+0000 7f44a06ae700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4498108740 0x7f449819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.994+0000 7f44a06ae700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44981034c0 0x7f449819ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.994+0000 7f44a06ae700 1 --2- 172.21.15.46:0/1819667857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498107df0 0x7f449819b3d0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:22:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.994+0000 7f44a06ae700 1 -- 172.21.15.46:0/1819667857 >> 172.21.15.46:0/1819667857 conn(0x7f44980fedf0 msgr2=0x7f44981063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:22:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.994+0000 7f44a06ae700 1 -- 172.21.15.46:0/1819667857 shutdown_connections 2024-05-09T00:22:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:22:57.994+0000 7f44a06ae700 1 -- 172.21.15.46:0/1819667857 wait complete. 2024-05-09T00:22:57.995 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:22:58.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:22:59.013 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:22:59.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:58 smithi046 ceph-mon[30151]: pgmap v724: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:22:59.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:22:58 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1819667857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:22:59.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:58 smithi169 ceph-mon[34777]: pgmap v724: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:22:59.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:22:58 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1819667857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:22:59.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:58 smithi142 ceph-mon[34533]: pgmap v724: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:22:59.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:22:58 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1819667857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:23:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.737+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/2996808474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f28108740 msgr2=0x7f2f281100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.737+0000 7f2f2f2b2700 1 --2- 172.21.15.46:0/2996808474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f28108740 0x7f2f281100a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000b0d0 tx=0x7f2f2000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.737+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/2996808474 shutdown_connections 2024-05-09T00:23:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.737+0000 7f2f2f2b2700 1 --2- 172.21.15.46:0/2996808474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f28108740 0x7f2f281100a0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.737+0000 7f2f2f2b2700 1 --2- 172.21.15.46:0/2996808474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f281034c0 0x7f2f281038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.737+0000 7f2f2f2b2700 1 --2- 172.21.15.46:0/2996808474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f28107df0 0x7f2f28108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.737+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/2996808474 >> 172.21.15.46:0/2996808474 conn(0x7f2f280fedf0 msgr2=0x7f2f28101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.737+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/2996808474 shutdown_connections 2024-05-09T00:23:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.737+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/2996808474 wait complete. 2024-05-09T00:23:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.738+0000 7f2f2f2b2700 1 Processor -- start 2024-05-09T00:23:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.738+0000 7f2f2f2b2700 1 -- start start 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.738+0000 7f2f2f2b2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f281034c0 0x7f2f2810c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.738+0000 7f2f2f2b2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f28107df0 0x7f2f2810c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.738+0000 7f2f2f2b2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f28108740 0x7f2f2810cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2f2b2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281133c0 con 0x7f2f28108740 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2f2b2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28113240 con 0x7f2f28107df0 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2f2b2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28113540 con 0x7f2f281034c0 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2d84f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f28108740 0x7f2f2810cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2d84f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f28108740 0x7f2f2810cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45954/0 (socket says 172.21.15.46:45954) 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2d84f700 1 -- 172.21.15.46:0/90978609 learned_addr learned my addr 172.21.15.46:0/90978609 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f281034c0 0x7f2f2810c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f28107df0 0x7f2f2810c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2d04e700 1 -- 172.21.15.46:0/90978609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f28107df0 msgr2=0x7f2f2810c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f28107df0 0x7f2f2810c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2d04e700 1 -- 172.21.15.46:0/90978609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f28108740 msgr2=0x7f2f2810cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f28108740 0x7f2f2810cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2d04e700 1 -- 172.21.15.46:0/90978609 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f281ad500 con 0x7f2f281034c0 2024-05-09T00:23:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f28107df0 0x7f2f2810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:23:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2d84f700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f28108740 0x7f2f2810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:23:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.739+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f281034c0 0x7f2f2810c320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400d950 tx=0x7f2f2400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.740+0000 7f2f1e7fc700 1 -- 172.21.15.46:0/90978609 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f24016070 con 0x7f2f281034c0 2024-05-09T00:23:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.740+0000 7f2f1e7fc700 1 -- 172.21.15.46:0/90978609 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f2400fd40 con 0x7f2f281034c0 2024-05-09T00:23:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.740+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/90978609 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f281ad7f0 con 0x7f2f281034c0 2024-05-09T00:23:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.740+0000 7f2f1e7fc700 1 -- 172.21.15.46:0/90978609 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2400e3b0 con 0x7f2f281034c0 2024-05-09T00:23:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.740+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/90978609 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f281add80 con 0x7f2f281034c0 2024-05-09T00:23:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.743+0000 7f2f1e7fc700 1 -- 172.21.15.46:0/90978609 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2f24019070 con 0x7f2f281034c0 2024-05-09T00:23:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.744+0000 7f2f1e7fc700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2f140b2900 0x7f2f140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.744+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2f140b2900 0x7f2f140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.744+0000 7f2f1e7fc700 1 -- 172.21.15.46:0/90978609 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f2f240e1bc0 con 0x7f2f281034c0 2024-05-09T00:23:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.745+0000 7f2f3dcc8b80 1 Processor -- start 2024-05-09T00:23:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.745+0000 7f2f3dcc8b80 1 -- start start 2024-05-09T00:23:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.745+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2f140b2900 0x7f2f140b4e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009680 tx=0x7f2f18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.745+0000 7f2f3dcc8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 0x55ca75e6e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.745+0000 7f2f3dcc8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 0x55ca75e6e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.745+0000 7f2f3dcc8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75e68400 0x55ca75e68810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.745+0000 7f2f3dcc8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ca75d8b0a0 con 0x55ca75d65d00 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.745+0000 7f2f3dcc8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ca75d309a0 con 0x55ca75ceb4a0 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.745+0000 7f2f3dcc8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ca75d87060 con 0x55ca75e68400 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.745+0000 7f2f2d04e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 0x55ca75e6e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.745+0000 7f2f2d04e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 0x55ca75e6e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45960/0 (socket says 172.21.15.46:45960) 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f2c84d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 0x55ca75e6e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f2d04e700 1 -- 172.21.15.46:0/2270568754 learned_addr learned my addr 172.21.15.46:0/2270568754 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f2d84f700 1 --2- 172.21.15.46:0/2270568754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75e68400 0x55ca75e68810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f2c84d700 1 -- 172.21.15.46:0/2270568754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75e68400 msgr2=0x55ca75e68810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/2270568754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75e68400 0x55ca75e68810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f2c84d700 1 -- 172.21.15.46:0/2270568754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 msgr2=0x55ca75e6e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/2270568754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 0x55ca75e6e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f2c84d700 1 -- 172.21.15.46:0/2270568754 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca75d21500 con 0x55ca75ceb4a0 2024-05-09T00:23:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/2270568754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 0x55ca75e6e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:23:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/2270568754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 0x55ca75e6e780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f28076b40 tx=0x7f2f18014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=486b1d0d3ed55da3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f137fe700 1 -- 172.21.15.46:0/2270568754 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1801a660 con 0x55ca75ceb4a0 2024-05-09T00:23:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f137fe700 1 -- 172.21.15.46:0/2270568754 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f1801acb0 con 0x55ca75ceb4a0 2024-05-09T00:23:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.746+0000 7f2f137fe700 1 -- 172.21.15.46:0/2270568754 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1801bdc0 con 0x55ca75ceb4a0 2024-05-09T00:23:00.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.748+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2270568754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 msgr2=0x55ca75e6e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:00.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.748+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2270568754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 0x55ca75e6e780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f28076b40 tx=0x7f2f18014000 comp rx=0 tx=0).stop 2024-05-09T00:23:00.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.748+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2270568754 shutdown_connections 2024-05-09T00:23:00.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.748+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2270568754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75e68400 0x55ca75e68810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.748+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2270568754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 0x55ca75e6e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.748+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2270568754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 0x55ca75e6e780 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.748+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2270568754 >> 172.21.15.46:0/2270568754 conn(0x55ca75ceef70 msgr2=0x55ca75e63220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.748+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2270568754 shutdown_connections 2024-05-09T00:23:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.748+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2270568754 wait complete. 2024-05-09T00:23:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.748+0000 7f2f3dcc8b80 1 -- start start 2024-05-09T00:23:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.748+0000 7f2f3dcc8b80 1 Processor -- start 2024-05-09T00:23:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f3dcc8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 0x55ca75e72050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f3dcc8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75d65d00 0x55ca75e72590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f3dcc8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75e68400 0x55ca75e72ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f3dcc8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ca75e69160 con 0x55ca75e68400 2024-05-09T00:23:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f3dcc8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ca75d25b20 con 0x55ca75ceb4a0 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f3dcc8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ca75e75020 con 0x55ca75d65d00 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f3dcc8b80 10 client.0 _ll_register_callbacks cb 0x55ca75bfc6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f2d04e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 0x55ca75e72050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f2d84f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75e68400 0x55ca75e72ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f2d04e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 0x55ca75e72050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:37008/0 (socket says 172.21.15.46:37008) 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f2d04e700 1 -- 172.21.15.46:0/2955134898 learned_addr learned my addr 172.21.15.46:0/2955134898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/2955134898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75d65d00 0x55ca75e72590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f2d04e700 1 -- 172.21.15.46:0/2955134898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75d65d00 msgr2=0x55ca75e72590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/2955134898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75d65d00 0x55ca75e72590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f2d04e700 1 -- 172.21.15.46:0/2955134898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75e68400 msgr2=0x55ca75e72ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/2955134898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75e68400 0x55ca75e72ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.749+0000 7f2f2d04e700 1 -- 172.21.15.46:0/2955134898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca75c06490 con 0x55ca75ceb4a0 2024-05-09T00:23:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.750+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/2955134898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 0x55ca75e72050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f240a6410 tx=0x7f2f24010840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9fa0ccb56d6f358 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.750+0000 7f2f10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.750+0000 7f2f10ff9700 1 -- 172.21.15.46:0/2955134898 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f24020040 con 0x55ca75ceb4a0 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.750+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2955134898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f1801acb0 con 0x55ca75ceb4a0 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.750+0000 7f2f10ff9700 1 -- 172.21.15.46:0/2955134898 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f2f240b1d80 con 0x55ca75ceb4a0 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.750+0000 7f2f10ff9700 1 -- 172.21.15.46:0/2955134898 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2f240a98a0 con 0x55ca75ceb4a0 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.750+0000 7f2f3dcc8b80 10 client.26636 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.750+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2955134898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca75e7fa30 con 0x55ca75ceb4a0 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.750+0000 7f2f10ff9700 1 -- 172.21.15.46:0/2955134898 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f240ca1f0 con 0x55ca75ceb4a0 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.751+0000 7f2f10ff9700 1 -- 172.21.15.46:0/2955134898 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2f240dfb40 con 0x55ca75ceb4a0 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.751+0000 7f2f3dcc8b80 10 client.26636 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.751+0000 7f2f3dcc8b80 10 client.26636 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.751+0000 7f2f3dcc8b80 20 client.26636 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.751+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2955134898 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca75e82160 0x55ca75e845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.751+0000 7f2f3dcc8b80 4 client.26636 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.751+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2955134898 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2f1801acb0 con 0x55ca75e82160 2024-05-09T00:23:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.751+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/2955134898 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca75e82160 0x55ca75e845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.752+0000 7f2f10ff9700 10 client.26636 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:23:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.752+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/2955134898 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca75e82160 0x55ca75e845b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cef85a2ce9f27ccc server_cookie=8c22aa7c3f451721 in_seq=0 out_seq=0 2024-05-09T00:23:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f10ff9700 1 -- 172.21.15.46:0/2955134898 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f2f1801acb0 con 0x55ca75e82160 2024-05-09T00:23:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f10ff9700 10 client.26636 handle_command_reply: tid=0 2024-05-09T00:23:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f3dcc8b80 1 client.26636 shutdown 2024-05-09T00:23:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2955134898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 msgr2=0x55ca75e72050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2955134898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 0x55ca75e72050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f240a6410 tx=0x7f2f24010840 comp rx=0 tx=0).stop 2024-05-09T00:23:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2955134898 shutdown_connections 2024-05-09T00:23:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2955134898 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca75e82160 0x55ca75e845b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2955134898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75e68400 0x55ca75e72ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2955134898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75ceb4a0 0x55ca75e72050 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2955134898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75d65d00 0x55ca75e72590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2955134898 >> 172.21.15.46:0/2955134898 conn(0x55ca75ceef70 msgr2=0x55ca75e5cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f10ff9700 0 client.26636 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:23:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2955134898 shutdown_connections 2024-05-09T00:23:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2955134898 wait complete. 2024-05-09T00:23:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.753+0000 7f2f3dcc8b80 20 client.26636 trim_cache size 0 max 16384 2024-05-09T00:23:00.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f3dcc8b80 1 Processor -- start 2024-05-09T00:23:00.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f3dcc8b80 1 -- start start 2024-05-09T00:23:00.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f3dcc8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75ceb4a0 0x55ca75e72910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f3dcc8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 0x55ca75e5d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f3dcc8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75e68400 0x55ca75e7de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f3dcc8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281b7b80 con 0x55ca75d65d00 2024-05-09T00:23:00.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f3dcc8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281b5410 con 0x55ca75e68400 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f3dcc8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281b3240 con 0x55ca75ceb4a0 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f2c84d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 0x55ca75e5d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f2c84d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 0x55ca75e5d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45978/0 (socket says 172.21.15.46:45978) 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f2c84d700 1 -- 172.21.15.46:0/1477971427 learned_addr learned my addr 172.21.15.46:0/1477971427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f2d84f700 1 --2- 172.21.15.46:0/1477971427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75e68400 0x55ca75e7de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.769+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/1477971427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75ceb4a0 0x55ca75e72910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.770+0000 7f2f2d84f700 1 -- 172.21.15.46:0/1477971427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75ceb4a0 msgr2=0x55ca75e72910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.770+0000 7f2f2d84f700 1 --2- 172.21.15.46:0/1477971427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75ceb4a0 0x55ca75e72910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.770+0000 7f2f2d84f700 1 -- 172.21.15.46:0/1477971427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 msgr2=0x55ca75e5d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.770+0000 7f2f2d84f700 1 --2- 172.21.15.46:0/1477971427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 0x55ca75e5d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.770+0000 7f2f2d84f700 1 -- 172.21.15.46:0/1477971427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca75e5f9d0 con 0x55ca75e68400 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.770+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/1477971427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 0x55ca75e5d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:23:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.770+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/1477971427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75ceb4a0 0x55ca75e72910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:00.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.770+0000 7f2f2d84f700 1 --2- 172.21.15.46:0/1477971427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75e68400 0x55ca75e7de20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009fd0 tx=0x7f2f20005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a0f2aac4b4025f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:00.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.770+0000 7f2f12ffd700 1 -- 172.21.15.46:0/1477971427 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20003c00 con 0x55ca75e68400 2024-05-09T00:23:00.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.770+0000 7f2f12ffd700 1 -- 172.21.15.46:0/1477971427 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f20003da0 con 0x55ca75e68400 2024-05-09T00:23:00.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.770+0000 7f2f12ffd700 1 -- 172.21.15.46:0/1477971427 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20005510 con 0x55ca75e68400 2024-05-09T00:23:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.771+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/1477971427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75e68400 msgr2=0x55ca75e7de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.771+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/1477971427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75e68400 0x55ca75e7de20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009fd0 tx=0x7f2f20005040 comp rx=0 tx=0).stop 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.771+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/1477971427 shutdown_connections 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.771+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/1477971427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75e68400 0x55ca75e7de20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.771+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/1477971427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75ceb4a0 0x55ca75e72910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.771+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/1477971427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75d65d00 0x55ca75e5d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.771+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/1477971427 >> 172.21.15.46:0/1477971427 conn(0x55ca75ceef70 msgr2=0x55ca75e5cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.772+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/1477971427 shutdown_connections 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.772+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/1477971427 wait complete. 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.772+0000 7f2f3dcc8b80 1 -- start start 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.772+0000 7f2f3dcc8b80 1 Processor -- start 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.772+0000 7f2f3dcc8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75ceb4a0 0x55ca75e7a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.772+0000 7f2f3dcc8b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75d65d00 0x55ca75e7a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f3dcc8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75e68400 0x55ca75e7ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f3dcc8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281b38d0 con 0x55ca75ceb4a0 2024-05-09T00:23:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f3dcc8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281b4d40 con 0x55ca75e68400 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f3dcc8b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281b3fa0 con 0x55ca75d65d00 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f3dcc8b80 10 client.0 _ll_register_callbacks cb 0x55ca75bfc6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2d04e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75ceb4a0 0x55ca75e7a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2d04e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75ceb4a0 0x55ca75e7a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45988/0 (socket says 172.21.15.46:45988) 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2d04e700 1 -- 172.21.15.46:0/2156397373 learned_addr learned my addr 172.21.15.46:0/2156397373 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2d84f700 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75e68400 0x55ca75e7ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75d65d00 0x55ca75e7a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2d04e700 1 -- 172.21.15.46:0/2156397373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75d65d00 msgr2=0x55ca75e7a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75d65d00 0x55ca75e7a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2d04e700 1 -- 172.21.15.46:0/2156397373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75e68400 msgr2=0x55ca75e7ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75e68400 0x55ca75e7ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2d04e700 1 -- 172.21.15.46:0/2156397373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca75e69600 con 0x55ca75ceb4a0 2024-05-09T00:23:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75d65d00 0x55ca75e7a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2d84f700 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75e68400 0x55ca75e7ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f2d04e700 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75ceb4a0 0x55ca75e7a150 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2f24000c00 tx=0x7f2f240a89c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35ca63841dac199 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.773+0000 7f2f10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.774+0000 7f2f10ff9700 1 -- 172.21.15.46:0/2156397373 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f24020020 con 0x55ca75ceb4a0 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.774+0000 7f2f10ff9700 1 -- 172.21.15.46:0/2156397373 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f2f2400f890 con 0x55ca75ceb4a0 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.774+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2156397373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f20003da0 con 0x55ca75ceb4a0 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.774+0000 7f2f10ff9700 1 -- 172.21.15.46:0/2156397373 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2f240ca1f0 con 0x55ca75ceb4a0 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.774+0000 7f2f3dcc8b80 10 client.17298 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.774+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2156397373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca75eec080 con 0x55ca75ceb4a0 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.774+0000 7f2f10ff9700 1 -- 172.21.15.46:0/2156397373 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f20003da0 con 0x55ca75ceb4a0 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.775+0000 7f2f10ff9700 1 -- 172.21.15.46:0/2156397373 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2f240a7e70 con 0x55ca75ceb4a0 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.775+0000 7f2f3dcc8b80 10 client.17298 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.775+0000 7f2f3dcc8b80 10 client.17298 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:23:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.775+0000 7f2f3dcc8b80 20 client.17298 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:00.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.775+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca75eec570 0x55ca75eee990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:00.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.775+0000 7f2f3dcc8b80 4 client.17298 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:23:00.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.775+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2156397373 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca75e63780 con 0x55ca75eec570 2024-05-09T00:23:00.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.775+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca75eec570 0x55ca75eee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:00.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f2c84d700 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca75eec570 0x55ca75eee990 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9cd074ab7ff7e878 server_cookie=6ef7c6bc5e53a20f in_seq=0 out_seq=0 2024-05-09T00:23:00.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f10ff9700 10 client.17298 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:23:00.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f10ff9700 1 -- 172.21.15.46:0/2156397373 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca75e63780 con 0x55ca75eec570 2024-05-09T00:23:00.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f10ff9700 10 client.17298 handle_command_reply: tid=0 2024-05-09T00:23:00.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f3dcc8b80 1 client.17298 shutdown 2024-05-09T00:23:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2156397373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75ceb4a0 msgr2=0x55ca75e7a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75ceb4a0 0x55ca75e7a150 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2f24000c00 tx=0x7f2f240a89c0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2156397373 shutdown_connections 2024-05-09T00:23:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca75eec570 0x55ca75eee990 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f10ff9700 0 client.17298 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:23:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca75e68400 0x55ca75e7ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca75ceb4a0 0x55ca75e7a150 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f3dcc8b80 1 --2- 172.21.15.46:0/2156397373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca75d65d00 0x55ca75e7a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2156397373 >> 172.21.15.46:0/2156397373 conn(0x55ca75ceef70 msgr2=0x55ca75e673f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2156397373 shutdown_connections 2024-05-09T00:23:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f3dcc8b80 1 -- 172.21.15.46:0/2156397373 wait complete. 2024-05-09T00:23:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.776+0000 7f2f3dcc8b80 20 client.17298 trim_cache size 0 max 16384 2024-05-09T00:23:00.781 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:23:00.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.777+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/90978609 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2f140b2900 msgr2=0x7f2f140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:00.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.777+0000 7f2f2f2b2700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2f140b2900 0x7f2f140b4e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009680 tx=0x7f2f18009260 comp rx=0 tx=0).stop 2024-05-09T00:23:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.777+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/90978609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f281034c0 msgr2=0x7f2f2810c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.777+0000 7f2f2f2b2700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f281034c0 0x7f2f2810c320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400d950 tx=0x7f2f2400dc60 comp rx=0 tx=0).stop 2024-05-09T00:23:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.777+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/90978609 shutdown_connections 2024-05-09T00:23:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.777+0000 7f2f2f2b2700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f2f140b2900 0x7f2f140b4e10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.777+0000 7f2f2f2b2700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f28108740 0x7f2f2810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.777+0000 7f2f2f2b2700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f281034c0 0x7f2f2810c320 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.777+0000 7f2f2f2b2700 1 --2- 172.21.15.46:0/90978609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f28107df0 0x7f2f2810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.777+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/90978609 >> 172.21.15.46:0/90978609 conn(0x7f2f280fedf0 msgr2=0x7f2f28101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.777+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/90978609 shutdown_connections 2024-05-09T00:23:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:00.777+0000 7f2f2f2b2700 1 -- 172.21.15.46:0/90978609 wait complete. 2024-05-09T00:23:01.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:00 smithi046 ceph-mon[30151]: pgmap v725: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:23:01.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:00 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:23:01.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:00 smithi169 ceph-mon[34777]: pgmap v725: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:23:01.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:00 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:23:01.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:00 smithi142 ceph-mon[34533]: pgmap v725: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:23:01.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:00 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:23:01.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:23:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:01.942 INFO:tasks.workunit.client.0.smithi046.stdout:499/600: open fd = 3 2024-05-09T00:23:01.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.943 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.943 INFO:tasks.workunit.client.0.smithi046.stdout:500/600: open fd = 3 2024-05-09T00:23:01.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.943 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.943 INFO:tasks.workunit.client.0.smithi046.stdout:501/600: open fd = 3 2024-05-09T00:23:01.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.943 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.943 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.943 INFO:tasks.workunit.client.0.smithi046.stdout:502/600: open fd = 3 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:503/600: open fd = 3 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:504/600: open fd = 3 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:505/600: open fd = 3 2024-05-09T00:23:01.944 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:506/600: open fd = 3 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:507/600: open fd = 3 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:508/600: open fd = 3 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.945 INFO:tasks.workunit.client.0.smithi046.stdout:509/600: open fd = 3 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:510/600: open fd = 3 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:511/600: open fd = 3 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:512/600: open fd = 3 2024-05-09T00:23:01.946 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:513/600: open fd = 3 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:514/600: open fd = 3 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:515/600: open fd = 3 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.947 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:516/600: open fd = 3 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:517/600: open fd = 3 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:518/600: open fd = 3 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:519/600: open fd = 3 2024-05-09T00:23:01.948 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:520/600: open fd = 3 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:521/600: open fd = 3 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:522/600: open fd = 3 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.949 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:523/600: open fd = 3 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:524/600: open fd = 3 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:525/600: open fd = 3 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.950 INFO:tasks.workunit.client.0.smithi046.stdout:526/600: open fd = 3 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:527/600: open fd = 3 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:528/600: open fd = 3 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:529/600: open fd = 3 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.951 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:530/600: open fd = 3 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:531/600: open fd = 3 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:532/600: open fd = 3 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.952 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.953 INFO:tasks.workunit.client.0.smithi046.stdout:533/600: open fd = 3 2024-05-09T00:23:01.953 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.953 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.953 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.953 INFO:tasks.workunit.client.0.smithi046.stdout:534/600: open fd = 3 2024-05-09T00:23:01.953 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.953 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.953 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.953 INFO:tasks.workunit.client.0.smithi046.stdout:535/600: open fd = 3 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:536/600: open fd = 3 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:537/600: open fd = 3 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:538/600: open fd = 3 2024-05-09T00:23:01.954 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:539/600: open fd = 3 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:540/600: open fd = 3 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:541/600: open fd = 3 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.955 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:542/600: open fd = 3 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:543/600: open fd = 3 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:544/600: open fd = 3 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:545/600: open fd = 3 2024-05-09T00:23:01.956 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:546/600: open fd = 3 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:547/600: open fd = 3 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:548/600: open fd = 3 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.957 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:549/600: open fd = 3 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:550/600: open fd = 3 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:551/600: open fd = 3 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.958 INFO:tasks.workunit.client.0.smithi046.stdout:552/600: open fd = 3 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:553/600: open fd = 3 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:554/600: open fd = 3 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:555/600: open fd = 3 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.959 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:556/600: open fd = 3 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:557/600: open fd = 3 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:558/600: open fd = 3 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.960 INFO:tasks.workunit.client.0.smithi046.stdout:559/600: open fd = 3 2024-05-09T00:23:01.961 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.961 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.961 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.961 INFO:tasks.workunit.client.0.smithi046.stdout:560/600: open fd = 3 2024-05-09T00:23:01.961 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.961 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:01.961 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:01.990 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:01 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:23:01.991 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:01 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:23:01.991 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:01 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:23:02.095 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:01 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:23:02.095 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:01 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:23:02.095 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:01 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:23:02.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:01 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:23:02.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:01 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:23:02.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:01 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:23:03.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:02 smithi046 ceph-mon[30151]: pgmap v726: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:03.107 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:02 smithi169 ceph-mon[34777]: pgmap v726: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:03.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:02 smithi142 ceph-mon[34533]: pgmap v726: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:03.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.667+0000 7f0b6cd30700 1 -- 172.21.15.46:0/3406525927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b681045e0 msgr2=0x7f0b681049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:03.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.667+0000 7f0b6cd30700 1 --2- 172.21.15.46:0/3406525927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b681045e0 0x7f0b681049f0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f0b580099c0 tx=0x7f0b58009cd0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.667+0000 7f0b6cd30700 1 -- 172.21.15.46:0/3406525927 shutdown_connections 2024-05-09T00:23:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.667+0000 7f0b6cd30700 1 --2- 172.21.15.46:0/3406525927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b68109860 0x7f0b681111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.667+0000 7f0b6cd30700 1 --2- 172.21.15.46:0/3406525927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b681045e0 0x7f0b681049f0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.667+0000 7f0b6cd30700 1 --2- 172.21.15.46:0/3406525927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b68108f10 0x7f0b68109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.667+0000 7f0b6cd30700 1 -- 172.21.15.46:0/3406525927 >> 172.21.15.46:0/3406525927 conn(0x7f0b680ffe70 msgr2=0x7f0b681022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.667+0000 7f0b6cd30700 1 -- 172.21.15.46:0/3406525927 shutdown_connections 2024-05-09T00:23:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.667+0000 7f0b6cd30700 1 -- 172.21.15.46:0/3406525927 wait complete. 2024-05-09T00:23:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.668+0000 7f0b6cd30700 1 Processor -- start 2024-05-09T00:23:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.668+0000 7f0b6cd30700 1 -- start start 2024-05-09T00:23:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b6cd30700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b681045e0 0x7f0b6819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b6cd30700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b68108f10 0x7f0b6819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b6cd30700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b68109860 0x7f0b6819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b6cd30700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68114500 con 0x7f0b68108f10 2024-05-09T00:23:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b6cd30700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68114380 con 0x7f0b68109860 2024-05-09T00:23:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b6cd30700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68114680 con 0x7f0b681045e0 2024-05-09T00:23:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b68108f10 0x7f0b6819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b68108f10 0x7f0b6819b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46006/0 (socket says 172.21.15.46:46006) 2024-05-09T00:23:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b65d9b700 1 -- 172.21.15.46:0/2272241140 learned_addr learned my addr 172.21.15.46:0/2272241140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b6659c700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b681045e0 0x7f0b6819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b66d9d700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b68109860 0x7f0b6819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b65d9b700 1 -- 172.21.15.46:0/2272241140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b681045e0 msgr2=0x7f0b6819ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b681045e0 0x7f0b6819ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b65d9b700 1 -- 172.21.15.46:0/2272241140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b68109860 msgr2=0x7f0b6819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b68109860 0x7f0b6819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b65d9b700 1 -- 172.21.15.46:0/2272241140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6819fc90 con 0x7f0b68108f10 2024-05-09T00:23:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b66d9d700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b68109860 0x7f0b6819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:23:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.669+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b68108f10 0x7f0b6819b410 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00b500 tx=0x7f0b5c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:03.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.670+0000 7f0b577fe700 1 -- 172.21.15.46:0/2272241140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c015020 con 0x7f0b68108f10 2024-05-09T00:23:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.670+0000 7f0b577fe700 1 -- 172.21.15.46:0/2272241140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c011ce0 con 0x7f0b68108f10 2024-05-09T00:23:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.670+0000 7f0b6cd30700 1 -- 172.21.15.46:0/2272241140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6819ff80 con 0x7f0b68108f10 2024-05-09T00:23:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.670+0000 7f0b577fe700 1 -- 172.21.15.46:0/2272241140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c010d40 con 0x7f0b68108f10 2024-05-09T00:23:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.670+0000 7f0b6cd30700 1 -- 172.21.15.46:0/2272241140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b681a04c0 con 0x7f0b68108f10 2024-05-09T00:23:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.673+0000 7f0b577fe700 1 -- 172.21.15.46:0/2272241140 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0b5c018020 con 0x7f0b68108f10 2024-05-09T00:23:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.673+0000 7f0b577fe700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0b500b2880 0x7f0b500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.673+0000 7f0b6659c700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0b500b2880 0x7f0b500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.673+0000 7f0b577fe700 1 -- 172.21.15.46:0/2272241140 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f0b5c0e1960 con 0x7f0b68108f10 2024-05-09T00:23:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b7b746b80 1 Processor -- start 2024-05-09T00:23:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b7b746b80 1 -- start start 2024-05-09T00:23:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b6659c700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0b500b2880 0x7f0b500b4d90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009990 tx=0x7f0b58000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b7b746b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920e62d00 0x55b920f6b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b7b746b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920de84a0 0x55b920f6b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b7b746b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920f65810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b7b746b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b920e880a0 con 0x55b920de84a0 2024-05-09T00:23:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b7b746b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b920e2d9a0 con 0x55b920f65400 2024-05-09T00:23:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b7b746b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b920e84060 con 0x55b920e62d00 2024-05-09T00:23:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920de84a0 0x55b920f6b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920de84a0 0x55b920f6b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46008/0 (socket says 172.21.15.46:46008) 2024-05-09T00:23:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b65d9b700 1 -- 172.21.15.46:0/3666399818 learned_addr learned my addr 172.21.15.46:0/3666399818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.674+0000 7f0b66d9d700 1 --2- 172.21.15.46:0/3666399818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920f65810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.675+0000 7f0b6659c700 1 --2- 172.21.15.46:0/3666399818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920e62d00 0x55b920f6b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.675+0000 7f0b65d9b700 1 -- 172.21.15.46:0/3666399818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920e62d00 msgr2=0x55b920f6b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.675+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/3666399818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920e62d00 0x55b920f6b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.675+0000 7f0b65d9b700 1 -- 172.21.15.46:0/3666399818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 msgr2=0x55b920f65810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.675+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/3666399818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920f65810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.675+0000 7f0b65d9b700 1 -- 172.21.15.46:0/3666399818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b920e1e500 con 0x55b920de84a0 2024-05-09T00:23:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.675+0000 7f0b6659c700 1 --2- 172.21.15.46:0/3666399818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920e62d00 0x55b920f6b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:03.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.675+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/3666399818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920de84a0 0x55b920f6b780 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0a5000 tx=0x7f0b5c004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137e496e42151b17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.675+0000 7f0b54ff9700 1 -- 172.21.15.46:0/3666399818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0a9680 con 0x55b920de84a0 2024-05-09T00:23:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.675+0000 7f0b54ff9700 1 -- 172.21.15.46:0/3666399818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0a8f80 con 0x55b920de84a0 2024-05-09T00:23:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.675+0000 7f0b54ff9700 1 -- 172.21.15.46:0/3666399818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0a81e0 con 0x55b920de84a0 2024-05-09T00:23:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.676+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3666399818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920de84a0 msgr2=0x55b920f6b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.676+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3666399818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920de84a0 0x55b920f6b780 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0a5000 tx=0x7f0b5c004860 comp rx=0 tx=0).stop 2024-05-09T00:23:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.676+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3666399818 shutdown_connections 2024-05-09T00:23:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.676+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3666399818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920f65810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.676+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3666399818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920e62d00 0x55b920f6b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.676+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3666399818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920de84a0 0x55b920f6b780 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.676+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3666399818 >> 172.21.15.46:0/3666399818 conn(0x55b920debf70 msgr2=0x55b920f60220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.676+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3666399818 shutdown_connections 2024-05-09T00:23:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.676+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3666399818 wait complete. 2024-05-09T00:23:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b7b746b80 1 -- start start 2024-05-09T00:23:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b7b746b80 1 Processor -- start 2024-05-09T00:23:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b7b746b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920de84a0 0x55b920f6ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b7b746b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f6f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b7b746b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920f65400 0x55b920f6f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b7b746b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b920f66160 con 0x55b920e62d00 2024-05-09T00:23:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b7b746b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b920e22b20 con 0x55b920de84a0 2024-05-09T00:23:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b7b746b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b920f72020 con 0x55b920f65400 2024-05-09T00:23:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b7b746b80 10 client.0 _ll_register_callbacks cb 0x55b920cf96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f6f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f6f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46018/0 (socket says 172.21.15.46:46018) 2024-05-09T00:23:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b65d9b700 1 -- 172.21.15.46:0/3765139920 learned_addr learned my addr 172.21.15.46:0/3765139920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b66d9d700 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920f65400 0x55b920f6f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.677+0000 7f0b6659c700 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920de84a0 0x55b920f6ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b65d9b700 1 -- 172.21.15.46:0/3765139920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920f65400 msgr2=0x55b920f6f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920f65400 0x55b920f6f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b65d9b700 1 -- 172.21.15.46:0/3765139920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920de84a0 msgr2=0x55b920f6ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920de84a0 0x55b920f6ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b65d9b700 1 -- 172.21.15.46:0/3765139920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b920d03490 con 0x55b920e62d00 2024-05-09T00:23:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f6f4a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0a8f10 tx=0x7f0b5c010750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26109003fa7c5780 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:03.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b4dffb700 1 -- 172.21.15.46:0/3765139920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c088060 con 0x55b920e62d00 2024-05-09T00:23:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b4dffb700 1 -- 172.21.15.46:0/3765139920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f0b5c0b1b60 con 0x55b920e62d00 2024-05-09T00:23:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3765139920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b5c0a8f80 con 0x55b920e62d00 2024-05-09T00:23:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b4dffb700 1 -- 172.21.15.46:0/3765139920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b5c0c9070 con 0x55b920e62d00 2024-05-09T00:23:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b7b746b80 10 client.17310 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3765139920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b920f80d80 con 0x55b920e62d00 2024-05-09T00:23:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b4dffb700 1 -- 172.21.15.46:0/3765139920 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0a8f80 con 0x55b920e62d00 2024-05-09T00:23:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.678+0000 7f0b4dffb700 1 -- 172.21.15.46:0/3765139920 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0b5c0a4980 con 0x55b920e62d00 2024-05-09T00:23:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.679+0000 7f0b7b746b80 10 client.17310 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.679+0000 7f0b7b746b80 10 client.17310 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:23:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.679+0000 7f0b7b746b80 20 client.17310 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.679+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b920f83250 0x55b920f85670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.679+0000 7f0b7b746b80 4 client.17310 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.679+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3765139920 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b920e95a80 con 0x55b920f83250 2024-05-09T00:23:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.679+0000 7f0b6659c700 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b920f83250 0x55b920f85670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.679+0000 7f0b4dffb700 10 client.17310 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:23:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.679+0000 7f0b6659c700 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b920f83250 0x55b920f85670 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1714a6ebbd635d8c server_cookie=9b19076ca40b5725 in_seq=0 out_seq=0 2024-05-09T00:23:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.680+0000 7f0b4dffb700 1 -- 172.21.15.46:0/3765139920 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b920e95a80 con 0x55b920f83250 2024-05-09T00:23:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.680+0000 7f0b4dffb700 10 client.17310 handle_command_reply: tid=0 2024-05-09T00:23:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.680+0000 7f0b7b746b80 1 client.17310 shutdown 2024-05-09T00:23:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.680+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3765139920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 msgr2=0x55b920f6f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.680+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f6f4a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0a8f10 tx=0x7f0b5c010750 comp rx=0 tx=0).stop 2024-05-09T00:23:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.680+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3765139920 shutdown_connections 2024-05-09T00:23:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.680+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b920f83250 0x55b920f85670 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.680+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920f65400 0x55b920f6f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.680+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920de84a0 0x55b920f6ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.680+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3765139920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f6f4a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.680+0000 7f0b4dffb700 0 client.17310 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:23:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.680+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3765139920 >> 172.21.15.46:0/3765139920 conn(0x55b920debf70 msgr2=0x55b920f59ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.681+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3765139920 shutdown_connections 2024-05-09T00:23:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.681+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3765139920 wait complete. 2024-05-09T00:23:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.681+0000 7f0b7b746b80 20 client.17310 trim_cache size 0 max 16384 2024-05-09T00:23:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b7b746b80 1 Processor -- start 2024-05-09T00:23:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b7b746b80 1 -- start start 2024-05-09T00:23:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b7b746b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920de84a0 0x55b920f5aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b7b746b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f7b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b7b746b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920f6e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b7b746b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681af390 con 0x55b920e62d00 2024-05-09T00:23:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b7b746b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681acc20 con 0x55b920f65400 2024-05-09T00:23:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b7b746b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681aaa50 con 0x55b920de84a0 2024-05-09T00:23:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f7b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f7b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46028/0 (socket says 172.21.15.46:46028) 2024-05-09T00:23:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b65d9b700 1 -- 172.21.15.46:0/57771181 learned_addr learned my addr 172.21.15.46:0/57771181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b66d9d700 1 --2- 172.21.15.46:0/57771181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920f6e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b6659c700 1 --2- 172.21.15.46:0/57771181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920de84a0 0x55b920f5aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b65d9b700 1 -- 172.21.15.46:0/57771181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920de84a0 msgr2=0x55b920f5aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/57771181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920de84a0 0x55b920f5aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b65d9b700 1 -- 172.21.15.46:0/57771181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 msgr2=0x55b920f6e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/57771181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920f6e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.696+0000 7f0b65d9b700 1 -- 172.21.15.46:0/57771181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b920f6baf0 con 0x55b920e62d00 2024-05-09T00:23:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.697+0000 7f0b6659c700 1 --2- 172.21.15.46:0/57771181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920de84a0 0x55b920f5aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.697+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/57771181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f7b1f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0a61f0 tx=0x7f0b5c004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c086f3b4b5db33c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.697+0000 7f0b4ffff700 1 -- 172.21.15.46:0/57771181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c088070 con 0x55b920e62d00 2024-05-09T00:23:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.697+0000 7f0b4ffff700 1 -- 172.21.15.46:0/57771181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0ad330 con 0x55b920e62d00 2024-05-09T00:23:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.697+0000 7f0b4ffff700 1 -- 172.21.15.46:0/57771181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0a4b50 con 0x55b920e62d00 2024-05-09T00:23:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.698+0000 7f0b7b746b80 1 -- 172.21.15.46:0/57771181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 msgr2=0x55b920f7b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.698+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/57771181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f7b1f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0a61f0 tx=0x7f0b5c004860 comp rx=0 tx=0).stop 2024-05-09T00:23:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.698+0000 7f0b7b746b80 1 -- 172.21.15.46:0/57771181 shutdown_connections 2024-05-09T00:23:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.698+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/57771181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920f6e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.698+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/57771181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920de84a0 0x55b920f5aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.698+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/57771181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f7b1f0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.698+0000 7f0b7b746b80 1 -- 172.21.15.46:0/57771181 >> 172.21.15.46:0/57771181 conn(0x55b920debf70 msgr2=0x55b920f68c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.698+0000 7f0b7b746b80 1 -- 172.21.15.46:0/57771181 shutdown_connections 2024-05-09T00:23:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.698+0000 7f0b7b746b80 1 -- 172.21.15.46:0/57771181 wait complete. 2024-05-09T00:23:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.698+0000 7f0b7b746b80 1 -- start start 2024-05-09T00:23:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.698+0000 7f0b7b746b80 1 Processor -- start 2024-05-09T00:23:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.698+0000 7f0b7b746b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920de84a0 0x55b920f85e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b7b746b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f863b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b7b746b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920fe4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b7b746b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681ab0e0 con 0x55b920e62d00 2024-05-09T00:23:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b7b746b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681ac550 con 0x55b920f65400 2024-05-09T00:23:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b7b746b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0b681ab7b0 con 0x55b920de84a0 2024-05-09T00:23:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b7b746b80 10 client.0 _ll_register_callbacks cb 0x55b920cf96c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f863b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46044/0 (socket says 172.21.15.46:46044) 2024-05-09T00:23:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b65d9b700 1 -- 172.21.15.46:0/3387514407 learned_addr learned my addr 172.21.15.46:0/3387514407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b6659c700 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920de84a0 0x55b920f85e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b66d9d700 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920fe4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b65d9b700 1 -- 172.21.15.46:0/3387514407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920de84a0 msgr2=0x55b920f85e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920de84a0 0x55b920f85e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b65d9b700 1 -- 172.21.15.46:0/3387514407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 msgr2=0x55b920fe4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920fe4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b65d9b700 1 -- 172.21.15.46:0/3387514407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b920fe4d10 con 0x55b920e62d00 2024-05-09T00:23:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b66d9d700 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920fe4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b65d9b700 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f863b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0a4d50 tx=0x7f0b5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b3b6b5f2755d47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b4dffb700 1 -- 172.21.15.46:0/3387514407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c088070 con 0x55b920e62d00 2024-05-09T00:23:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b4dffb700 1 -- 172.21.15.46:0/3387514407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f0b5c0a4b00 con 0x55b920e62d00 2024-05-09T00:23:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.699+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3387514407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b5c0ad330 con 0x55b920e62d00 2024-05-09T00:23:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.700+0000 7f0b4dffb700 1 -- 172.21.15.46:0/3387514407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b5c0b1de0 con 0x55b920e62d00 2024-05-09T00:23:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.700+0000 7f0b7b746b80 10 client.17316 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.700+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3387514407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b920fe4d10 con 0x55b920e62d00 2024-05-09T00:23:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.700+0000 7f0b4dffb700 1 -- 172.21.15.46:0/3387514407 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0ad330 con 0x55b920e62d00 2024-05-09T00:23:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.700+0000 7f0b4dffb700 1 -- 172.21.15.46:0/3387514407 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0b5c0ae4c0 con 0x55b920e62d00 2024-05-09T00:23:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.700+0000 7f0b7b746b80 10 client.17316 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.700+0000 7f0b7b746b80 10 client.17316 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:23:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.700+0000 7f0b7b746b80 20 client.17316 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.700+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b920fed4d0 0x55b920fef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.700+0000 7f0b7b746b80 4 client.17316 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:23:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.700+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3387514407 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b920e8fb20 con 0x55b920fed4d0 2024-05-09T00:23:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.700+0000 7f0b6659c700 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b920fed4d0 0x55b920fef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b4dffb700 10 client.17316 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:23:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b6659c700 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b920fed4d0 0x55b920fef8f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3866ad8d587e508e server_cookie=3fd3b9294af48744 in_seq=0 out_seq=0 2024-05-09T00:23:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b4dffb700 1 -- 172.21.15.46:0/3387514407 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b920e8fb20 con 0x55b920fed4d0 2024-05-09T00:23:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b4dffb700 10 client.17316 handle_command_reply: tid=0 2024-05-09T00:23:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b7b746b80 1 client.17316 shutdown 2024-05-09T00:23:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3387514407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 msgr2=0x55b920f863b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f863b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0a4d50 tx=0x7f0b5c004750 comp rx=0 tx=0).stop 2024-05-09T00:23:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3387514407 shutdown_connections 2024-05-09T00:23:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55b920fed4d0 0x55b920fef8f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b920f65400 0x55b920fe4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b920de84a0 0x55b920f85e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b7b746b80 1 --2- 172.21.15.46:0/3387514407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b920e62d00 0x55b920f863b0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3387514407 >> 172.21.15.46:0/3387514407 conn(0x55b920debf70 msgr2=0x55b920f68c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b4dffb700 0 client.17316 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:23:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3387514407 shutdown_connections 2024-05-09T00:23:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.701+0000 7f0b7b746b80 1 -- 172.21.15.46:0/3387514407 wait complete. 2024-05-09T00:23:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b7b746b80 20 client.17316 trim_cache size 0 max 16384 2024-05-09T00:23:03.712 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:23:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b6cd30700 1 -- 172.21.15.46:0/2272241140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0b500b2880 msgr2=0x7f0b500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b6cd30700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0b500b2880 0x7f0b500b4d90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009990 tx=0x7f0b58000bc0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b6cd30700 1 -- 172.21.15.46:0/2272241140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b68108f10 msgr2=0x7f0b6819b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b6cd30700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b68108f10 0x7f0b6819b410 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00b500 tx=0x7f0b5c00b810 comp rx=0 tx=0).stop 2024-05-09T00:23:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b6cd30700 1 -- 172.21.15.46:0/2272241140 shutdown_connections 2024-05-09T00:23:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b6cd30700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0b500b2880 0x7f0b500b4d90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b6cd30700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b68109860 0x7f0b6819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b6cd30700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b681045e0 0x7f0b6819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b6cd30700 1 --2- 172.21.15.46:0/2272241140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b68108f10 0x7f0b6819b410 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b6cd30700 1 -- 172.21.15.46:0/2272241140 >> 172.21.15.46:0/2272241140 conn(0x7f0b680ffe70 msgr2=0x7f0b6810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b6cd30700 1 -- 172.21.15.46:0/2272241140 shutdown_connections 2024-05-09T00:23:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:03.702+0000 7f0b6cd30700 1 -- 172.21.15.46:0/2272241140 wait complete. 2024-05-09T00:23:04.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:23:04.833 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:05.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:04 smithi046 ceph-mon[30151]: pgmap v727: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:23:05.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:04 smithi169 ceph-mon[34777]: pgmap v727: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:23:05.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:04 smithi142 ceph-mon[34533]: pgmap v727: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-05-09T00:23:06.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.532+0000 7ff5f59d3700 1 -- 172.21.15.46:0/102115362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5f01034c0 msgr2=0x7ff5f01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:06.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.532+0000 7ff5f59d3700 1 --2- 172.21.15.46:0/102115362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5f01034c0 0x7ff5f01038d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00099c0 tx=0x7ff5e0009cd0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.532+0000 7ff5f59d3700 1 -- 172.21.15.46:0/102115362 shutdown_connections 2024-05-09T00:23:06.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.532+0000 7ff5f59d3700 1 --2- 172.21.15.46:0/102115362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5f0108740 0x7ff5f01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.532+0000 7ff5f59d3700 1 --2- 172.21.15.46:0/102115362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5f01034c0 0x7ff5f01038d0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.532+0000 7ff5f59d3700 1 --2- 172.21.15.46:0/102115362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f0107df0 0x7ff5f0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.532+0000 7ff5f59d3700 1 -- 172.21.15.46:0/102115362 >> 172.21.15.46:0/102115362 conn(0x7ff5f00fedc0 msgr2=0x7ff5f01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.532+0000 7ff5f59d3700 1 -- 172.21.15.46:0/102115362 shutdown_connections 2024-05-09T00:23:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.532+0000 7ff5f59d3700 1 -- 172.21.15.46:0/102115362 wait complete. 2024-05-09T00:23:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.533+0000 7ff5f59d3700 1 Processor -- start 2024-05-09T00:23:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.533+0000 7ff5f59d3700 1 -- start start 2024-05-09T00:23:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.533+0000 7ff5f59d3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f01034c0 0x7ff5f019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.533+0000 7ff5f59d3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5f0107df0 0x7ff5f019c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.533+0000 7ff5f59d3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5f0108740 0x7ff5f019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.533+0000 7ff5f59d3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f01133c0 con 0x7ff5f0107df0 2024-05-09T00:23:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5f59d3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f0113240 con 0x7ff5f01034c0 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5f59d3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f0113540 con 0x7ff5f0108740 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5e67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5f0107df0 0x7ff5f019c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5e67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5f0107df0 0x7ff5f019c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48020/0 (socket says 172.21.15.46:48020) 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5e67fc700 1 -- 172.21.15.46:0/2635060886 learned_addr learned my addr 172.21.15.46:0/2635060886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5ef7fe700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5f0108740 0x7ff5f019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5eeffd700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f01034c0 0x7ff5f019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5e67fc700 1 -- 172.21.15.46:0/2635060886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5f0108740 msgr2=0x7ff5f019c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5e67fc700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5f0108740 0x7ff5f019c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5e67fc700 1 -- 172.21.15.46:0/2635060886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f01034c0 msgr2=0x7ff5f019bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5e67fc700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f01034c0 0x7ff5f019bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5e67fc700 1 -- 172.21.15.46:0/2635060886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f01a6f80 con 0x7ff5f0107df0 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5e67fc700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5f0107df0 0x7ff5f019c200 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c9e0 tx=0x7ff5d800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.534+0000 7ff5f49d1700 1 -- 172.21.15.46:0/2635060886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8007aa0 con 0x7ff5f0107df0 2024-05-09T00:23:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.535+0000 7ff5f49d1700 1 -- 172.21.15.46:0/2635060886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5d8007c40 con 0x7ff5f0107df0 2024-05-09T00:23:06.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.535+0000 7ff5f59d3700 1 -- 172.21.15.46:0/2635060886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f01a7270 con 0x7ff5f0107df0 2024-05-09T00:23:06.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.535+0000 7ff5f59d3700 1 -- 172.21.15.46:0/2635060886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f01a77b0 con 0x7ff5f0107df0 2024-05-09T00:23:06.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.535+0000 7ff5f49d1700 1 -- 172.21.15.46:0/2635060886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d800fe60 con 0x7ff5f0107df0 2024-05-09T00:23:06.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.537+0000 7ff5f49d1700 1 -- 172.21.15.46:0/2635060886 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff5d8004870 con 0x7ff5f0107df0 2024-05-09T00:23:06.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.538+0000 7ff5f49d1700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff5dc0b2880 0x7ff5dc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.538+0000 7ff5eeffd700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff5dc0b2880 0x7ff5dc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.538+0000 7ff5f49d1700 1 -- 172.21.15.46:0/2635060886 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff5d80e1fd0 con 0x7ff5f0107df0 2024-05-09T00:23:06.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.538+0000 7ff6043e9b80 1 Processor -- start 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff6043e9b80 1 -- start start 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff6043e9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdc4b160 0x555dfdce44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff6043e9b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdb614a0 0x555dfdce4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff6043e9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcdead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff6043e9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555dfdc00ff0 con 0x555dfdcde6a0 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff6043e9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555dfdb9f7f0 con 0x555dfdc4b160 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff6043e9b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555dfdbfcfb0 con 0x555dfdb614a0 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff5eeffd700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff5dc0b2880 0x7ff5dc0b4d90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000b580 tx=0x7ff5e00036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff5ef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcdead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff5ef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcdead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48028/0 (socket says 172.21.15.46:48028) 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff5ef7fe700 1 -- 172.21.15.46:0/3098326926 learned_addr learned my addr 172.21.15.46:0/3098326926 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff5e67fc700 1 --2- 172.21.15.46:0/3098326926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdb614a0 0x555dfdce4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff5eeffd700 1 --2- 172.21.15.46:0/3098326926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdc4b160 0x555dfdce44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff5ef7fe700 1 -- 172.21.15.46:0/3098326926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdb614a0 msgr2=0x555dfdce4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:06.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff5ef7fe700 1 --2- 172.21.15.46:0/3098326926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdb614a0 0x555dfdce4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff5ef7fe700 1 -- 172.21.15.46:0/3098326926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdc4b160 msgr2=0x555dfdce44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:06.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff5ef7fe700 1 --2- 172.21.15.46:0/3098326926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdc4b160 0x555dfdce44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.539+0000 7ff5ef7fe700 1 -- 172.21.15.46:0/3098326926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dfdb97370 con 0x555dfdcde6a0 2024-05-09T00:23:06.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.540+0000 7ff5ef7fe700 1 --2- 172.21.15.46:0/3098326926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcdead0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8003f10 tx=0x7ff5e8009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488915b09dbae785 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:06.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.540+0000 7ff5e5ffb700 1 -- 172.21.15.46:0/3098326926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8004bb0 con 0x555dfdcde6a0 2024-05-09T00:23:06.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.540+0000 7ff5e5ffb700 1 -- 172.21.15.46:0/3098326926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e8004d50 con 0x555dfdcde6a0 2024-05-09T00:23:06.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.540+0000 7ff5e5ffb700 1 -- 172.21.15.46:0/3098326926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8007dd0 con 0x555dfdcde6a0 2024-05-09T00:23:06.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.541+0000 7ff6043e9b80 1 -- 172.21.15.46:0/3098326926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 msgr2=0x555dfdcdead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:06.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.541+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/3098326926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcdead0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8003f10 tx=0x7ff5e8009ba0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.541+0000 7ff6043e9b80 1 -- 172.21.15.46:0/3098326926 shutdown_connections 2024-05-09T00:23:06.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.541+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/3098326926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcdead0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.541+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/3098326926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdc4b160 0x555dfdce44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.541+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/3098326926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdb614a0 0x555dfdce4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.541+0000 7ff6043e9b80 1 -- 172.21.15.46:0/3098326926 >> 172.21.15.46:0/3098326926 conn(0x555dfdbc16a0 msgr2=0x555dfdcd9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.541+0000 7ff6043e9b80 1 -- 172.21.15.46:0/3098326926 shutdown_connections 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.541+0000 7ff6043e9b80 1 -- 172.21.15.46:0/3098326926 wait complete. 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.541+0000 7ff6043e9b80 1 -- start start 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.541+0000 7ff6043e9b80 1 Processor -- start 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff6043e9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdb614a0 0x555dfdce80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff6043e9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdc4b160 0x555dfdce85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff6043e9b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdcde6a0 0x555dfdce8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff6043e9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555dfda7cba0 con 0x555dfdc4b160 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff6043e9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555dfdbbb150 con 0x555dfdb614a0 2024-05-09T00:23:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff6043e9b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555dfdceb1d0 con 0x555dfdcde6a0 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff6043e9b80 10 client.0 _ll_register_callbacks cb 0x555dfda726c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff5e67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdc4b160 0x555dfdce85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff5e67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdc4b160 0x555dfdce85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48036/0 (socket says 172.21.15.46:48036) 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff5e67fc700 1 -- 172.21.15.46:0/4237521892 learned_addr learned my addr 172.21.15.46:0/4237521892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff5eeffd700 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdb614a0 0x555dfdce80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff5ef7fe700 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdcde6a0 0x555dfdce8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff5e67fc700 1 -- 172.21.15.46:0/4237521892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdcde6a0 msgr2=0x555dfdce8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff5e67fc700 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdcde6a0 0x555dfdce8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff5e67fc700 1 -- 172.21.15.46:0/4237521892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdb614a0 msgr2=0x555dfdce80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff5e67fc700 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdb614a0 0x555dfdce80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.542+0000 7ff5e67fc700 1 -- 172.21.15.46:0/4237521892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dfda7c490 con 0x555dfdc4b160 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff5e67fc700 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdc4b160 0x555dfdce85f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80a92f0 tx=0x7ff5d80a8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e375198eac8f02 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff5ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff5ceffd700 1 -- 172.21.15.46:0/4237521892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80c3060 con 0x555dfdc4b160 2024-05-09T00:23:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff5ceffd700 1 -- 172.21.15.46:0/4237521892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff5d80a9850 con 0x555dfdc4b160 2024-05-09T00:23:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff6043e9b80 1 -- 172.21.15.46:0/4237521892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e8004d50 con 0x555dfdc4b160 2024-05-09T00:23:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff5ceffd700 1 -- 172.21.15.46:0/4237521892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5d80912d0 con 0x555dfdc4b160 2024-05-09T00:23:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff6043e9b80 10 client.17328 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff6043e9b80 1 -- 172.21.15.46:0/4237521892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555dfdcf5ba0 con 0x555dfdc4b160 2024-05-09T00:23:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff5ceffd700 1 -- 172.21.15.46:0/4237521892 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e8004d50 con 0x555dfdc4b160 2024-05-09T00:23:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff5ceffd700 1 -- 172.21.15.46:0/4237521892 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff5d80a4f60 con 0x555dfdc4b160 2024-05-09T00:23:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff6043e9b80 10 client.17328 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff6043e9b80 10 client.17328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff6043e9b80 20 client.17328 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555dfdcf81e0 0x555dfdcfa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff6043e9b80 4 client.17328 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:06.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.543+0000 7ff6043e9b80 1 -- 172.21.15.46:0/4237521892 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dfdc0e9d0 con 0x555dfdcf81e0 2024-05-09T00:23:06.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.544+0000 7ff5eeffd700 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555dfdcf81e0 0x555dfdcfa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.544+0000 7ff5ceffd700 10 client.17328 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:06.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.544+0000 7ff5eeffd700 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555dfdcf81e0 0x555dfdcfa690 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8fae359dadf90f3 server_cookie=299ebab0416e4c0 in_seq=0 out_seq=0 2024-05-09T00:23:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.545+0000 7ff5ceffd700 1 -- 172.21.15.46:0/4237521892 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555dfdc0e9d0 con 0x555dfdcf81e0 2024-05-09T00:23:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.545+0000 7ff5ceffd700 10 client.17328 handle_command_reply: tid=0 2024-05-09T00:23:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.545+0000 7ff6043e9b80 1 client.17328 shutdown 2024-05-09T00:23:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.545+0000 7ff6043e9b80 1 -- 172.21.15.46:0/4237521892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdc4b160 msgr2=0x555dfdce85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.545+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdc4b160 0x555dfdce85f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80a92f0 tx=0x7ff5d80a8de0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.545+0000 7ff6043e9b80 1 -- 172.21.15.46:0/4237521892 shutdown_connections 2024-05-09T00:23:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.545+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555dfdcf81e0 0x555dfdcfa690 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.545+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdcde6a0 0x555dfdce8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.545+0000 7ff5ceffd700 0 client.17328 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.545+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdb614a0 0x555dfdce80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.545+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/4237521892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdc4b160 0x555dfdce85f0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.545+0000 7ff6043e9b80 1 -- 172.21.15.46:0/4237521892 >> 172.21.15.46:0/4237521892 conn(0x555dfdbc16a0 msgr2=0x555dfdcd91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:06.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.546+0000 7ff6043e9b80 1 -- 172.21.15.46:0/4237521892 shutdown_connections 2024-05-09T00:23:06.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.546+0000 7ff6043e9b80 1 -- 172.21.15.46:0/4237521892 wait complete. 2024-05-09T00:23:06.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.546+0000 7ff6043e9b80 20 client.17328 trim_cache size 0 max 16384 2024-05-09T00:23:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff6043e9b80 1 Processor -- start 2024-05-09T00:23:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff6043e9b80 1 -- start start 2024-05-09T00:23:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff6043e9b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdb614a0 0x555dfdcfa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff6043e9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdc4b160 0x555dfdce2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff6043e9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcde0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff6043e9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d000fbe0 con 0x555dfdcde6a0 2024-05-09T00:23:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff6043e9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d000d470 con 0x555dfdc4b160 2024-05-09T00:23:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff6043e9b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d000b2a0 con 0x555dfdb614a0 2024-05-09T00:23:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff5ef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcde0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff5ef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcde0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48038/0 (socket says 172.21.15.46:48038) 2024-05-09T00:23:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff5ef7fe700 1 -- 172.21.15.46:0/2177524558 learned_addr learned my addr 172.21.15.46:0/2177524558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff5eeffd700 1 --2- 172.21.15.46:0/2177524558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdb614a0 0x555dfdcfa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.561+0000 7ff5e67fc700 1 --2- 172.21.15.46:0/2177524558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdc4b160 0x555dfdce2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.562+0000 7ff5ef7fe700 1 -- 172.21.15.46:0/2177524558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdb614a0 msgr2=0x555dfdcfa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:06.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.562+0000 7ff5ef7fe700 1 --2- 172.21.15.46:0/2177524558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdb614a0 0x555dfdcfa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.562+0000 7ff5ef7fe700 1 -- 172.21.15.46:0/2177524558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdc4b160 msgr2=0x555dfdce2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:06.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.562+0000 7ff5ef7fe700 1 --2- 172.21.15.46:0/2177524558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdc4b160 0x555dfdce2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.562+0000 7ff5ef7fe700 1 -- 172.21.15.46:0/2177524558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dfdcd2f60 con 0x555dfdcde6a0 2024-05-09T00:23:06.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.562+0000 7ff5e67fc700 1 --2- 172.21.15.46:0/2177524558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdc4b160 0x555dfdce2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:06.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.562+0000 7ff5ef7fe700 1 --2- 172.21.15.46:0/2177524558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcde0c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8003e60 tx=0x7ff5e8007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa3d3993aa37b73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:06.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.562+0000 7ff5e57fa700 1 -- 172.21.15.46:0/2177524558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8007940 con 0x555dfdcde6a0 2024-05-09T00:23:06.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.562+0000 7ff5e57fa700 1 -- 172.21.15.46:0/2177524558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e80044d0 con 0x555dfdcde6a0 2024-05-09T00:23:06.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.562+0000 7ff5e57fa700 1 -- 172.21.15.46:0/2177524558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80156c0 con 0x555dfdcde6a0 2024-05-09T00:23:06.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.563+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2177524558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 msgr2=0x555dfdcde0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:06.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.563+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/2177524558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcde0c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8003e60 tx=0x7ff5e8007690 comp rx=0 tx=0).stop 2024-05-09T00:23:06.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.563+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2177524558 shutdown_connections 2024-05-09T00:23:06.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.563+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/2177524558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcde0c0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.563+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/2177524558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdb614a0 0x555dfdcfa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.563+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/2177524558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdc4b160 0x555dfdce2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.563+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2177524558 >> 172.21.15.46:0/2177524558 conn(0x555dfdbc16a0 msgr2=0x555dfdce31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.563+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2177524558 shutdown_connections 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.563+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2177524558 wait complete. 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.563+0000 7ff6043e9b80 1 -- start start 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.563+0000 7ff6043e9b80 1 Processor -- start 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff6043e9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdb614a0 0x555dfdcfee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff6043e9b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdc4b160 0x555dfdcff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff6043e9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcff900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff6043e9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d000b930 con 0x555dfdcde6a0 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff6043e9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d000cda0 con 0x555dfdb614a0 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff6043e9b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d000c000 con 0x555dfdc4b160 2024-05-09T00:23:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff6043e9b80 10 client.0 _ll_register_callbacks cb 0x555dfda726c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff5ef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff5ef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcff900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48042/0 (socket says 172.21.15.46:48042) 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff5ef7fe700 1 -- 172.21.15.46:0/2079561656 learned_addr learned my addr 172.21.15.46:0/2079561656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff5e67fc700 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdc4b160 0x555dfdcff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff5eeffd700 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdb614a0 0x555dfdcfee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff5ef7fe700 1 -- 172.21.15.46:0/2079561656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdc4b160 msgr2=0x555dfdcff3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff5ef7fe700 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdc4b160 0x555dfdcff3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff5ef7fe700 1 -- 172.21.15.46:0/2079561656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdb614a0 msgr2=0x555dfdcfee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff5ef7fe700 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdb614a0 0x555dfdcfee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff5ef7fe700 1 -- 172.21.15.46:0/2079561656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dfdcef660 con 0x555dfdcde6a0 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff5ef7fe700 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcff900 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8004e20 tx=0x7ff5e8008ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bd206d1c45dba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.564+0000 7ff5ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.565+0000 7ff5ceffd700 1 -- 172.21.15.46:0/2079561656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8004bf0 con 0x555dfdcde6a0 2024-05-09T00:23:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.565+0000 7ff5ceffd700 1 -- 172.21.15.46:0/2079561656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff5e8015480 con 0x555dfdcde6a0 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.565+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2079561656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e80044d0 con 0x555dfdcde6a0 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.565+0000 7ff5ceffd700 1 -- 172.21.15.46:0/2079561656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5e8015680 con 0x555dfdcde6a0 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.565+0000 7ff6043e9b80 10 client.17334 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.565+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2079561656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555dfdcf08a0 con 0x555dfdcde6a0 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.565+0000 7ff5ceffd700 1 -- 172.21.15.46:0/2079561656 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e80044d0 con 0x555dfdcde6a0 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.565+0000 7ff5ceffd700 1 -- 172.21.15.46:0/2079561656 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff5e8030930 con 0x555dfdcde6a0 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.565+0000 7ff6043e9b80 10 client.17334 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.565+0000 7ff6043e9b80 10 client.17334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.565+0000 7ff6043e9b80 20 client.17334 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.565+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555dfdcf81e0 0x555dfdd64a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.566+0000 7ff6043e9b80 4 client.17334 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.566+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2079561656 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555dfdc08a70 con 0x555dfdcf81e0 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.566+0000 7ff5eeffd700 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555dfdcf81e0 0x555dfdd64a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.566+0000 7ff5ceffd700 10 client.17334 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.566+0000 7ff5eeffd700 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555dfdcf81e0 0x555dfdd64a50 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5477149d5db007 server_cookie=285faab1200096e0 in_seq=0 out_seq=0 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.566+0000 7ff5ceffd700 1 -- 172.21.15.46:0/2079561656 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555dfdc08a70 con 0x555dfdcf81e0 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.566+0000 7ff5ceffd700 10 client.17334 handle_command_reply: tid=0 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.566+0000 7ff6043e9b80 1 client.17334 shutdown 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.567+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2079561656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 msgr2=0x555dfdcff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.567+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcff900 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8004e20 tx=0x7ff5e8008ac0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.567+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2079561656 shutdown_connections 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.567+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x555dfdcf81e0 0x555dfdd64a50 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.567+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dfdcde6a0 0x555dfdcff900 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.567+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555dfdb614a0 0x555dfdcfee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.567+0000 7ff6043e9b80 1 --2- 172.21.15.46:0/2079561656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555dfdc4b160 0x555dfdcff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.567+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2079561656 >> 172.21.15.46:0/2079561656 conn(0x555dfdbc16a0 msgr2=0x555dfdcd9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.567+0000 7ff5ceffd700 0 client.17334 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.567+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2079561656 shutdown_connections 2024-05-09T00:23:06.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.567+0000 7ff6043e9b80 1 -- 172.21.15.46:0/2079561656 wait complete. 2024-05-09T00:23:06.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.567+0000 7ff6043e9b80 20 client.17334 trim_cache size 0 max 16384 2024-05-09T00:23:06.576 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:23:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.568+0000 7ff5f59d3700 1 -- 172.21.15.46:0/2635060886 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff5dc0b2880 msgr2=0x7ff5dc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.568+0000 7ff5f59d3700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff5dc0b2880 0x7ff5dc0b4d90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000b580 tx=0x7ff5e00036e0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.568+0000 7ff5f59d3700 1 -- 172.21.15.46:0/2635060886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5f0107df0 msgr2=0x7ff5f019c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.568+0000 7ff5f59d3700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5f0107df0 0x7ff5f019c200 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c9e0 tx=0x7ff5d800ccf0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.568+0000 7ff5f59d3700 1 -- 172.21.15.46:0/2635060886 shutdown_connections 2024-05-09T00:23:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.568+0000 7ff5f59d3700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff5dc0b2880 0x7ff5dc0b4d90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.568+0000 7ff5f59d3700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5f0108740 0x7ff5f019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.568+0000 7ff5f59d3700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f01034c0 0x7ff5f019bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.568+0000 7ff5f59d3700 1 --2- 172.21.15.46:0/2635060886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5f0107df0 0x7ff5f019c200 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.568+0000 7ff5f59d3700 1 -- 172.21.15.46:0/2635060886 >> 172.21.15.46:0/2635060886 conn(0x7ff5f00fedc0 msgr2=0x7ff5f01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.568+0000 7ff5f59d3700 1 -- 172.21.15.46:0/2635060886 shutdown_connections 2024-05-09T00:23:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:06.568+0000 7ff5f59d3700 1 -- 172.21.15.46:0/2635060886 wait complete. 2024-05-09T00:23:06.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:06 smithi046 ceph-mon[30151]: pgmap v728: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:23:07.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:06 smithi169 ceph-mon[34777]: pgmap v728: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:23:07.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:06 smithi142 ceph-mon[34533]: pgmap v728: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:23:08.356 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:23:08.357 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:23:08.623 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:09.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:08 smithi046 ceph-mon[30151]: pgmap v729: 97 pgs: 97 active+clean; 7.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:23:09.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:08 smithi169 ceph-mon[34777]: pgmap v729: 97 pgs: 97 active+clean; 7.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:23:09.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:08 smithi142 ceph-mon[34533]: pgmap v729: 97 pgs: 97 active+clean; 7.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:23:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.468+0000 7f47b2af9700 1 -- 172.21.15.46:0/1459484042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47ac109860 msgr2=0x7f47ac1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.468+0000 7f47b2af9700 1 --2- 172.21.15.46:0/1459484042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47ac109860 0x7f47ac1111c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f47a400b0d0 tx=0x7f47a400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.468+0000 7f47b2af9700 1 -- 172.21.15.46:0/1459484042 shutdown_connections 2024-05-09T00:23:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.468+0000 7f47b2af9700 1 --2- 172.21.15.46:0/1459484042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47ac109860 0x7f47ac1111c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.468+0000 7f47b2af9700 1 --2- 172.21.15.46:0/1459484042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ac1045e0 0x7f47ac1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.468+0000 7f47b2af9700 1 --2- 172.21.15.46:0/1459484042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac108f10 0x7f47ac109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.468+0000 7f47b2af9700 1 -- 172.21.15.46:0/1459484042 >> 172.21.15.46:0/1459484042 conn(0x7f47ac0ffe70 msgr2=0x7f47ac1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.468+0000 7f47b2af9700 1 -- 172.21.15.46:0/1459484042 shutdown_connections 2024-05-09T00:23:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.468+0000 7f47b2af9700 1 -- 172.21.15.46:0/1459484042 wait complete. 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47b2af9700 1 Processor -- start 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47b2af9700 1 -- start start 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47b2af9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ac1045e0 0x7f47ac19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47b2af9700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac108f10 0x7f47ac19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47b2af9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47ac109860 0x7f47ac19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47b2af9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac114490 con 0x7f47ac109860 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47b2af9700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac114310 con 0x7f47ac108f10 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47b2af9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac114610 con 0x7f47ac1045e0 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47b1096700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47ac109860 0x7f47ac19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47b0895700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ac1045e0 0x7f47ac19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47b1096700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47ac109860 0x7f47ac19f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48064/0 (socket says 172.21.15.46:48064) 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47b1096700 1 -- 172.21.15.46:0/2085314470 learned_addr learned my addr 172.21.15.46:0/2085314470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.469+0000 7f47abfff700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac108f10 0x7f47ac19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.470+0000 7f47b1096700 1 -- 172.21.15.46:0/2085314470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ac1045e0 msgr2=0x7f47ac19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.470+0000 7f47b1096700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ac1045e0 0x7f47ac19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.470+0000 7f47b1096700 1 -- 172.21.15.46:0/2085314470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac108f10 msgr2=0x7f47ac19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.470+0000 7f47b1096700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac108f10 0x7f47ac19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.470+0000 7f47b1096700 1 -- 172.21.15.46:0/2085314470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ac19fc40 con 0x7f47ac109860 2024-05-09T00:23:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.470+0000 7f47b1096700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47ac109860 0x7f47ac19f650 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009fd0 tx=0x7f47a4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.470+0000 7f47a9ffb700 1 -- 172.21.15.46:0/2085314470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a401f070 con 0x7f47ac109860 2024-05-09T00:23:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.470+0000 7f47a9ffb700 1 -- 172.21.15.46:0/2085314470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a4003c00 con 0x7f47ac109860 2024-05-09T00:23:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.470+0000 7f47a9ffb700 1 -- 172.21.15.46:0/2085314470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a401b7b0 con 0x7f47ac109860 2024-05-09T00:23:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.470+0000 7f47b2af9700 1 -- 172.21.15.46:0/2085314470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ac19fed0 con 0x7f47ac109860 2024-05-09T00:23:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.470+0000 7f47b2af9700 1 -- 172.21.15.46:0/2085314470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ac1a0360 con 0x7f47ac109860 2024-05-09T00:23:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.473+0000 7f47a9ffb700 1 -- 172.21.15.46:0/2085314470 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f47a4022070 con 0x7f47ac109860 2024-05-09T00:23:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.474+0000 7f47a9ffb700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f479c0b2900 0x7f479c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.474+0000 7f47b0895700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f479c0b2900 0x7f479c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.474+0000 7f47a9ffb700 1 -- 172.21.15.46:0/2085314470 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f47a400f020 con 0x7f47ac109860 2024-05-09T00:23:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.474+0000 7f47c150fb80 1 Processor -- start 2024-05-09T00:23:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.474+0000 7f47c150fb80 1 -- start start 2024-05-09T00:23:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.474+0000 7f47b0895700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f479c0b2900 0x7f479c0b4e10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f47a0005950 tx=0x7f47a000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.474+0000 7f47c150fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abbf2d80 0x55f3abcfb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.474+0000 7f47c150fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abb78520 0x55f3abcfb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.474+0000 7f47c150fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 0x55f3abcf5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.474+0000 7f47c150fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f3abc18120 con 0x55f3abcf5480 2024-05-09T00:23:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.474+0000 7f47c150fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f3abbbda20 con 0x55f3abbf2d80 2024-05-09T00:23:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.474+0000 7f47c150fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f3abc140e0 con 0x55f3abb78520 2024-05-09T00:23:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.475+0000 7f47b1096700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 0x55f3abcf5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.475+0000 7f47b1096700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 0x55f3abcf5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48068/0 (socket says 172.21.15.46:48068) 2024-05-09T00:23:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.475+0000 7f47b1096700 1 -- 172.21.15.46:0/4201798448 learned_addr learned my addr 172.21.15.46:0/4201798448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.475+0000 7f47b1096700 1 -- 172.21.15.46:0/4201798448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abb78520 msgr2=0x55f3abcfb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.475+0000 7f47b1096700 1 --2- 172.21.15.46:0/4201798448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abb78520 0x55f3abcfb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.475+0000 7f47b1096700 1 -- 172.21.15.46:0/4201798448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abbf2d80 msgr2=0x55f3abcfb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.475+0000 7f47b1096700 1 --2- 172.21.15.46:0/4201798448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abbf2d80 0x55f3abcfb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.475+0000 7f47b1096700 1 -- 172.21.15.46:0/4201798448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3abbae580 con 0x55f3abcf5480 2024-05-09T00:23:10.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.475+0000 7f47b1096700 1 --2- 172.21.15.46:0/4201798448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 0x55f3abcf5890 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f47a40a6bf0 tx=0x7f47a400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcec2157718debb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:10.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.475+0000 7f4796ffd700 1 -- 172.21.15.46:0/4201798448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a40b3290 con 0x55f3abcf5480 2024-05-09T00:23:10.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.475+0000 7f4796ffd700 1 -- 172.21.15.46:0/4201798448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a4009cb0 con 0x55f3abcf5480 2024-05-09T00:23:10.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.475+0000 7f4796ffd700 1 -- 172.21.15.46:0/4201798448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a40e30d0 con 0x55f3abcf5480 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.476+0000 7f47c150fb80 1 -- 172.21.15.46:0/4201798448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 msgr2=0x55f3abcf5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.476+0000 7f47c150fb80 1 --2- 172.21.15.46:0/4201798448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 0x55f3abcf5890 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f47a40a6bf0 tx=0x7f47a400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.476+0000 7f47c150fb80 1 -- 172.21.15.46:0/4201798448 shutdown_connections 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.476+0000 7f47c150fb80 1 --2- 172.21.15.46:0/4201798448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 0x55f3abcf5890 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.476+0000 7f47c150fb80 1 --2- 172.21.15.46:0/4201798448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abbf2d80 0x55f3abcfb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.476+0000 7f47c150fb80 1 --2- 172.21.15.46:0/4201798448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abb78520 0x55f3abcfb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.476+0000 7f47c150fb80 1 -- 172.21.15.46:0/4201798448 >> 172.21.15.46:0/4201798448 conn(0x55f3abb7bff0 msgr2=0x55f3abcf02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.476+0000 7f47c150fb80 1 -- 172.21.15.46:0/4201798448 shutdown_connections 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.476+0000 7f47c150fb80 1 -- 172.21.15.46:0/4201798448 wait complete. 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47c150fb80 1 -- start start 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47c150fb80 1 Processor -- start 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47c150fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abb78520 0x55f3abd04b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47c150fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abbf2d80 0x55f3abd050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47c150fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 0x55f3abcfef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47c150fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f3abc18120 con 0x55f3abcf5480 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47c150fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f3abbb2ba0 con 0x55f3abb78520 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47c150fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f3abcf6030 con 0x55f3abbf2d80 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47c150fb80 10 client.0 _ll_register_callbacks cb 0x55f3aba89740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47b1096700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 0x55f3abcfef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47b0895700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abb78520 0x55f3abd04b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47b1096700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 0x55f3abcfef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48080/0 (socket says 172.21.15.46:48080) 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.477+0000 7f47b1096700 1 -- 172.21.15.46:0/2168841259 learned_addr learned my addr 172.21.15.46:0/2168841259 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f47abfff700 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abbf2d80 0x55f3abd050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f47b1096700 1 -- 172.21.15.46:0/2168841259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abbf2d80 msgr2=0x55f3abd050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f47b1096700 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abbf2d80 0x55f3abd050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f47b1096700 1 -- 172.21.15.46:0/2168841259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abb78520 msgr2=0x55f3abd04b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f47b1096700 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abb78520 0x55f3abd04b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f47b1096700 1 -- 172.21.15.46:0/2168841259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3aba93510 con 0x55f3abcf5480 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f47abfff700 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abbf2d80 0x55f3abd050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f47b1096700 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 0x55f3abcfef80 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009a20 tx=0x7f47a40b3340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf167e463f216514 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f478bfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f478bfff700 1 -- 172.21.15.46:0/2168841259 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a40cb060 con 0x55f3abcf5480 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f478bfff700 1 -- 172.21.15.46:0/2168841259 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f47a40aadd0 con 0x55f3abcf5480 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f47c150fb80 1 -- 172.21.15.46:0/2168841259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a4009cb0 con 0x55f3abcf5480 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f478bfff700 1 -- 172.21.15.46:0/2168841259 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47a40ba050 con 0x55f3abcf5480 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f47c150fb80 10 client.17346 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.478+0000 7f47c150fb80 1 -- 172.21.15.46:0/2168841259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3abd0c7a0 con 0x55f3abcf5480 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.479+0000 7f478bfff700 1 -- 172.21.15.46:0/2168841259 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a4009cb0 con 0x55f3abcf5480 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.479+0000 7f478bfff700 1 -- 172.21.15.46:0/2168841259 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f47a4011070 con 0x55f3abcf5480 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.479+0000 7f47c150fb80 10 client.17346 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:10.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.479+0000 7f47c150fb80 10 client.17346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.479+0000 7f47c150fb80 20 client.17346 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.479+0000 7f47c150fb80 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f3abd0ec70 0x55f3abd11090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.479+0000 7f47c150fb80 4 client.17346 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.479+0000 7f47c150fb80 1 -- 172.21.15.46:0/2168841259 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3abc25b00 con 0x55f3abd0ec70 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.479+0000 7f47b0895700 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f3abd0ec70 0x55f3abd11090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.480+0000 7f47b0895700 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f3abd0ec70 0x55f3abd11090 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c3ef0a45f572245 server_cookie=2386f831d605ba8c in_seq=0 out_seq=0 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.480+0000 7f478bfff700 10 client.17346 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.480+0000 7f478bfff700 1 -- 172.21.15.46:0/2168841259 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f3abc25b00 con 0x55f3abd0ec70 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.480+0000 7f478bfff700 10 client.17346 handle_command_reply: tid=0 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.480+0000 7f47c150fb80 1 client.17346 shutdown 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.481+0000 7f47c150fb80 1 -- 172.21.15.46:0/2168841259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 msgr2=0x55f3abcfef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.481+0000 7f47c150fb80 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 0x55f3abcfef80 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009a20 tx=0x7f47a40b3340 comp rx=0 tx=0).stop 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.481+0000 7f47c150fb80 1 -- 172.21.15.46:0/2168841259 shutdown_connections 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.481+0000 7f47c150fb80 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f3abd0ec70 0x55f3abd11090 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.481+0000 7f47c150fb80 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abcf5480 0x55f3abcfef80 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.481+0000 7f47c150fb80 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abb78520 0x55f3abd04b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.481+0000 7f47c150fb80 1 --2- 172.21.15.46:0/2168841259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abbf2d80 0x55f3abd050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.481+0000 7f478bfff700 0 client.17346 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.481+0000 7f47c150fb80 1 -- 172.21.15.46:0/2168841259 >> 172.21.15.46:0/2168841259 conn(0x55f3abb7bff0 msgr2=0x55f3abcea070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.481+0000 7f47c150fb80 1 -- 172.21.15.46:0/2168841259 shutdown_connections 2024-05-09T00:23:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.481+0000 7f47c150fb80 1 -- 172.21.15.46:0/2168841259 wait complete. 2024-05-09T00:23:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.481+0000 7f47c150fb80 20 client.17346 trim_cache size 0 max 16384 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47c150fb80 1 Processor -- start 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47c150fb80 1 -- start start 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47c150fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abb78520 0x55f3abd16770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47c150fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abbf2d80 0x55f3abd16cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47c150fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abcf5480 0x55f3abcea070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47c150fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1b1390 con 0x55f3abbf2d80 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47c150fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1aec20 con 0x55f3abcf5480 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47c150fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1aca50 con 0x55f3abb78520 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47abfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abbf2d80 0x55f3abd16cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47abfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abbf2d80 0x55f3abd16cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48092/0 (socket says 172.21.15.46:48092) 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47abfff700 1 -- 172.21.15.46:0/1686560360 learned_addr learned my addr 172.21.15.46:0/1686560360 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47b0895700 1 --2- 172.21.15.46:0/1686560360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abb78520 0x55f3abd16770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.502+0000 7f47b1096700 1 --2- 172.21.15.46:0/1686560360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abcf5480 0x55f3abcea070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.503+0000 7f47abfff700 1 -- 172.21.15.46:0/1686560360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abb78520 msgr2=0x55f3abd16770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:10.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.503+0000 7f47abfff700 1 --2- 172.21.15.46:0/1686560360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abb78520 0x55f3abd16770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.503+0000 7f47abfff700 1 -- 172.21.15.46:0/1686560360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abcf5480 msgr2=0x55f3abcea070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:10.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.503+0000 7f47abfff700 1 --2- 172.21.15.46:0/1686560360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abcf5480 0x55f3abcea070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.503+0000 7f47abfff700 1 -- 172.21.15.46:0/1686560360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3abceca50 con 0x55f3abbf2d80 2024-05-09T00:23:10.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.503+0000 7f47abfff700 1 --2- 172.21.15.46:0/1686560360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abbf2d80 0x55f3abd16cb0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f47ac06b540 tx=0x7f479800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5e3289e71798a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:10.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.503+0000 7f47967fc700 1 -- 172.21.15.46:0/1686560360 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798004460 con 0x55f3abbf2d80 2024-05-09T00:23:10.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.503+0000 7f47967fc700 1 -- 172.21.15.46:0/1686560360 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4798015070 con 0x55f3abbf2d80 2024-05-09T00:23:10.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.503+0000 7f47967fc700 1 -- 172.21.15.46:0/1686560360 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798005020 con 0x55f3abbf2d80 2024-05-09T00:23:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.504+0000 7f47c150fb80 1 -- 172.21.15.46:0/1686560360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abbf2d80 msgr2=0x55f3abd16cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.504+0000 7f47c150fb80 1 --2- 172.21.15.46:0/1686560360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abbf2d80 0x55f3abd16cb0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f47ac06b540 tx=0x7f479800cd60 comp rx=0 tx=0).stop 2024-05-09T00:23:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.504+0000 7f47c150fb80 1 -- 172.21.15.46:0/1686560360 shutdown_connections 2024-05-09T00:23:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.504+0000 7f47c150fb80 1 --2- 172.21.15.46:0/1686560360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abcf5480 0x55f3abcea070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.504+0000 7f47c150fb80 1 --2- 172.21.15.46:0/1686560360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abb78520 0x55f3abd16770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.504+0000 7f47c150fb80 1 --2- 172.21.15.46:0/1686560360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abbf2d80 0x55f3abd16cb0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.504+0000 7f47c150fb80 1 -- 172.21.15.46:0/1686560360 >> 172.21.15.46:0/1686560360 conn(0x55f3abb7bff0 msgr2=0x55f3abcfa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.504+0000 7f47c150fb80 1 -- 172.21.15.46:0/1686560360 shutdown_connections 2024-05-09T00:23:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.504+0000 7f47c150fb80 1 -- 172.21.15.46:0/1686560360 wait complete. 2024-05-09T00:23:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.504+0000 7f47c150fb80 1 -- start start 2024-05-09T00:23:10.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.504+0000 7f47c150fb80 1 Processor -- start 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47c150fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abb78520 0x55f3abd06040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47c150fb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abbf2d80 0x55f3abd06580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47c150fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abcf5480 0x55f3abd7d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47c150fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1ad0e0 con 0x55f3abb78520 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47c150fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1ae550 con 0x55f3abcf5480 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47c150fb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac1ad7b0 con 0x55f3abbf2d80 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47c150fb80 10 client.0 _ll_register_callbacks cb 0x55f3aba89740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47b0895700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abb78520 0x55f3abd06040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47b0895700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abb78520 0x55f3abd06040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48098/0 (socket says 172.21.15.46:48098) 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47b0895700 1 -- 172.21.15.46:0/1948868666 learned_addr learned my addr 172.21.15.46:0/1948868666 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47abfff700 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abbf2d80 0x55f3abd06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47b1096700 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abcf5480 0x55f3abd7d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47b0895700 1 -- 172.21.15.46:0/1948868666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abbf2d80 msgr2=0x55f3abd06580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47b0895700 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abbf2d80 0x55f3abd06580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47b0895700 1 -- 172.21.15.46:0/1948868666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abcf5480 msgr2=0x55f3abd7d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47b0895700 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abcf5480 0x55f3abd7d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47b0895700 1 -- 172.21.15.46:0/1948868666 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3abcfbb70 con 0x55f3abb78520 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47abfff700 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abbf2d80 0x55f3abd06580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.505+0000 7f47b1096700 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abcf5480 0x55f3abd7d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f47b0895700 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abb78520 0x55f3abd06040 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x55f3abd11710 tx=0x7f47a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94711fc0cbf55499 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f478bfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f478bfff700 1 -- 172.21.15.46:0/1948868666 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0016070 con 0x55f3abb78520 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f478bfff700 1 -- 172.21.15.46:0/1948868666 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f47a0004a30 con 0x55f3abb78520 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f47c150fb80 1 -- 172.21.15.46:0/1948868666 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4798015070 con 0x55f3abb78520 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f478bfff700 1 -- 172.21.15.46:0/1948868666 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47a0004c30 con 0x55f3abb78520 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f47c150fb80 10 client.17352 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f47c150fb80 1 -- 172.21.15.46:0/1948868666 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3abd7ec90 con 0x55f3abb78520 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f478bfff700 1 -- 172.21.15.46:0/1948868666 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4798015070 con 0x55f3abb78520 2024-05-09T00:23:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f478bfff700 1 -- 172.21.15.46:0/1948868666 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f47a002f070 con 0x55f3abb78520 2024-05-09T00:23:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f47c150fb80 10 client.17352 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f47c150fb80 10 client.17352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f47c150fb80 20 client.17352 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f47c150fb80 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f3abd7f400 0x55f3abd81820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f47c150fb80 4 client.17352 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:23:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.506+0000 7f47c150fb80 1 -- 172.21.15.46:0/1948868666 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f3abc1fba0 con 0x55f3abd7f400 2024-05-09T00:23:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.507+0000 7f47abfff700 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f3abd7f400 0x55f3abd81820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.508+0000 7f47abfff700 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f3abd7f400 0x55f3abd81820 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bab57018d0d388e server_cookie=c31c05debd1de670 in_seq=0 out_seq=0 2024-05-09T00:23:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.508+0000 7f478bfff700 10 client.17352 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f478bfff700 1 -- 172.21.15.46:0/1948868666 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f3abc1fba0 con 0x55f3abd7f400 2024-05-09T00:23:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f478bfff700 10 client.17352 handle_command_reply: tid=0 2024-05-09T00:23:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f47c150fb80 1 client.17352 shutdown 2024-05-09T00:23:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f47c150fb80 1 -- 172.21.15.46:0/1948868666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abb78520 msgr2=0x55f3abd06040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f47c150fb80 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abb78520 0x55f3abd06040 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x55f3abd11710 tx=0x7f47a0004750 comp rx=0 tx=0).stop 2024-05-09T00:23:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f47c150fb80 1 -- 172.21.15.46:0/1948868666 shutdown_connections 2024-05-09T00:23:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f47c150fb80 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55f3abd7f400 0x55f3abd81820 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f47c150fb80 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f3abcf5480 0x55f3abd7d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f47c150fb80 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3abb78520 0x55f3abd06040 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f47c150fb80 1 --2- 172.21.15.46:0/1948868666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f3abbf2d80 0x55f3abd06580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f478bfff700 0 client.17352 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f47c150fb80 1 -- 172.21.15.46:0/1948868666 >> 172.21.15.46:0/1948868666 conn(0x55f3abb7bff0 msgr2=0x55f3abcea070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f47c150fb80 1 -- 172.21.15.46:0/1948868666 shutdown_connections 2024-05-09T00:23:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f47c150fb80 1 -- 172.21.15.46:0/1948868666 wait complete. 2024-05-09T00:23:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.509+0000 7f47c150fb80 20 client.17352 trim_cache size 0 max 16384 2024-05-09T00:23:10.513 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:23:10.513 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:23:10.513 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "4350312b-cc10-431b-a63d-3db307906854", 2024-05-09T00:23:10.513 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:23:10.513 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:23:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.510+0000 7f47b2af9700 1 -- 172.21.15.46:0/2085314470 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f479c0b2900 msgr2=0x7f479c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.510+0000 7f47b2af9700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f479c0b2900 0x7f479c0b4e10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f47a0005950 tx=0x7f47a000a300 comp rx=0 tx=0).stop 2024-05-09T00:23:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.510+0000 7f47b2af9700 1 -- 172.21.15.46:0/2085314470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47ac109860 msgr2=0x7f47ac19f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.510+0000 7f47b2af9700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47ac109860 0x7f47ac19f650 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009fd0 tx=0x7f47a4007a20 comp rx=0 tx=0).stop 2024-05-09T00:23:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.510+0000 7f47b2af9700 1 -- 172.21.15.46:0/2085314470 shutdown_connections 2024-05-09T00:23:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.510+0000 7f47b2af9700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f479c0b2900 0x7f479c0b4e10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.510+0000 7f47b2af9700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47ac109860 0x7f47ac19f650 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.510+0000 7f47b2af9700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ac1045e0 0x7f47ac19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.510+0000 7f47b2af9700 1 --2- 172.21.15.46:0/2085314470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47ac108f10 0x7f47ac19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.510+0000 7f47b2af9700 1 -- 172.21.15.46:0/2085314470 >> 172.21.15.46:0/2085314470 conn(0x7f47ac0ffe70 msgr2=0x7f47ac106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.510+0000 7f47b2af9700 1 -- 172.21.15.46:0/2085314470 shutdown_connections 2024-05-09T00:23:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:10.510+0000 7f47b2af9700 1 -- 172.21.15.46:0/2085314470 wait complete. 2024-05-09T00:23:11.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:10 smithi046 ceph-mon[30151]: pgmap v730: 97 pgs: 97 active+clean; 7.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:23:11.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:10 smithi169 ceph-mon[34777]: pgmap v730: 97 pgs: 97 active+clean; 7.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:23:11.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:10 smithi142 ceph-mon[34533]: pgmap v730: 97 pgs: 97 active+clean; 7.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:23:11.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:23:11.572 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:11.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:11 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:23:11.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:11 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:23:11.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:11 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:23:11.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:11 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:23:12.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:11 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:23:12.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:11 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:23:12.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:11 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:23:12.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:11 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:23:12.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:11 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:23:12.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:11 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:23:12.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:11 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:23:12.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:11 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:23:12.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:12 smithi046 ceph-mon[30151]: pgmap v731: 97 pgs: 97 active+clean; 7.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:12.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:12 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:23:13.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:12 smithi169 ceph-mon[34777]: pgmap v731: 97 pgs: 97 active+clean; 7.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:13.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:12 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:23:13.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:12 smithi142 ceph-mon[34533]: pgmap v731: 97 pgs: 97 active+clean; 7.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:13.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:12 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:23:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.487+0000 7f1a0295c700 1 -- 172.21.15.46:0/4236476944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19fc108740 msgr2=0x7f19fc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.487+0000 7f1a0295c700 1 --2- 172.21.15.46:0/4236476944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19fc108740 0x7f19fc1100a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f19f400b0d0 tx=0x7f19f400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.487+0000 7f1a0295c700 1 -- 172.21.15.46:0/4236476944 shutdown_connections 2024-05-09T00:23:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.487+0000 7f1a0295c700 1 --2- 172.21.15.46:0/4236476944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19fc108740 0x7f19fc1100a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.487+0000 7f1a0295c700 1 --2- 172.21.15.46:0/4236476944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc1034c0 0x7f19fc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.487+0000 7f1a0295c700 1 --2- 172.21.15.46:0/4236476944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19fc107df0 0x7f19fc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.487+0000 7f1a0295c700 1 -- 172.21.15.46:0/4236476944 >> 172.21.15.46:0/4236476944 conn(0x7f19fc0fedc0 msgr2=0x7f19fc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.487+0000 7f1a0295c700 1 -- 172.21.15.46:0/4236476944 shutdown_connections 2024-05-09T00:23:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.487+0000 7f1a0295c700 1 -- 172.21.15.46:0/4236476944 wait complete. 2024-05-09T00:23:13.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.488+0000 7f1a0295c700 1 Processor -- start 2024-05-09T00:23:13.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.488+0000 7f1a0295c700 1 -- start start 2024-05-09T00:23:13.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.488+0000 7f1a0295c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc1034c0 0x7f19fc19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.488+0000 7f1a0295c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19fc107df0 0x7f19fc19c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.488+0000 7f1a0295c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19fc108740 0x7f19fc19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.488+0000 7f1a0295c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc113370 con 0x7f19fc108740 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.488+0000 7f1a0295c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1131f0 con 0x7f19fc1034c0 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.488+0000 7f1a0295c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1134f0 con 0x7f19fc107df0 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.488+0000 7f1a00ef9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19fc108740 0x7f19fc19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.488+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc1034c0 0x7f19fc19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc1034c0 0x7f19fc19bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:35578/0 (socket says 172.21.15.46:35578) 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f19fbfff700 1 -- 172.21.15.46:0/184379662 learned_addr learned my addr 172.21.15.46:0/184379662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f1a00ef9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19fc108740 0x7f19fc19c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48118/0 (socket says 172.21.15.46:48118) 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f19f3fff700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19fc107df0 0x7f19fc19c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f1a00ef9700 1 -- 172.21.15.46:0/184379662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19fc107df0 msgr2=0x7f19fc19c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f1a00ef9700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19fc107df0 0x7f19fc19c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f1a00ef9700 1 -- 172.21.15.46:0/184379662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc1034c0 msgr2=0x7f19fc19bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f1a00ef9700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc1034c0 0x7f19fc19bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f1a00ef9700 1 -- 172.21.15.46:0/184379662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19fc1a6f50 con 0x7f19fc108740 2024-05-09T00:23:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f1a00ef9700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19fc108740 0x7f19fc19c730 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009fd0 tx=0x7f19f4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f19f9ffb700 1 -- 172.21.15.46:0/184379662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4021020 con 0x7f19fc108740 2024-05-09T00:23:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f1a0295c700 1 -- 172.21.15.46:0/184379662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19fc1a71e0 con 0x7f19fc108740 2024-05-09T00:23:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f19f9ffb700 1 -- 172.21.15.46:0/184379662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19f4007e40 con 0x7f19fc108740 2024-05-09T00:23:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f19f9ffb700 1 -- 172.21.15.46:0/184379662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4005050 con 0x7f19fc108740 2024-05-09T00:23:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.489+0000 7f1a0295c700 1 -- 172.21.15.46:0/184379662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19fc1a7690 con 0x7f19fc108740 2024-05-09T00:23:13.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.492+0000 7f19f9ffb700 1 -- 172.21.15.46:0/184379662 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f19f4024020 con 0x7f19fc108740 2024-05-09T00:23:13.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.493+0000 7f19f9ffb700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f19e80b2880 0x7f19e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.493+0000 7f1a11372b80 1 Processor -- start 2024-05-09T00:23:13.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.493+0000 7f1a11372b80 1 -- start start 2024-05-09T00:23:13.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.493+0000 7f19fbfff700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f19e80b2880 0x7f19e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.493+0000 7f19f9ffb700 1 -- 172.21.15.46:0/184379662 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f19f400f020 con 0x7f19fc108740 2024-05-09T00:23:13.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.493+0000 7f1a11372b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810db0d160 0x55810dba4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.493+0000 7f1a11372b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810da234a0 0x55810dba06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.493+0000 7f1a11372b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810dba0ca0 0x55810dba1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f1a11372b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55810dac2ff0 con 0x55810da234a0 2024-05-09T00:23:13.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f1a11372b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55810da617f0 con 0x55810db0d160 2024-05-09T00:23:13.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f1a11372b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55810dabefb0 con 0x55810dba0ca0 2024-05-09T00:23:13.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19fbfff700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f19e80b2880 0x7f19e80b4d90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f19ec005950 tx=0x7f19ec00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:13.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810da234a0 0x55810dba06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810da234a0 0x55810dba06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48130/0 (socket says 172.21.15.46:48130) 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19f3fff700 1 -- 172.21.15.46:0/1174657007 learned_addr learned my addr 172.21.15.46:0/1174657007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f1a00ef9700 1 --2- 172.21.15.46:0/1174657007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810dba0ca0 0x55810dba1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19fbfff700 1 --2- 172.21.15.46:0/1174657007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810db0d160 0x55810dba4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19f3fff700 1 -- 172.21.15.46:0/1174657007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810dba0ca0 msgr2=0x55810dba1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19f3fff700 1 --2- 172.21.15.46:0/1174657007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810dba0ca0 0x55810dba1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19f3fff700 1 -- 172.21.15.46:0/1174657007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810db0d160 msgr2=0x55810dba4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19f3fff700 1 --2- 172.21.15.46:0/1174657007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810db0d160 0x55810dba4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19f3fff700 1 -- 172.21.15.46:0/1174657007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55810da59370 con 0x55810da234a0 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19fbfff700 1 --2- 172.21.15.46:0/1174657007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810db0d160 0x55810dba4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19f3fff700 1 --2- 172.21.15.46:0/1174657007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810da234a0 0x55810dba06a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f19e400ac60 tx=0x7f19e400af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcd55e7f0535cf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.494+0000 7f19f27fc700 1 -- 172.21.15.46:0/1174657007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e400fde0 con 0x55810da234a0 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.495+0000 7f19f27fc700 1 -- 172.21.15.46:0/1174657007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19e40044e0 con 0x55810da234a0 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.495+0000 7f19f27fc700 1 -- 172.21.15.46:0/1174657007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4018690 con 0x55810da234a0 2024-05-09T00:23:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.495+0000 7f1a11372b80 1 -- 172.21.15.46:0/1174657007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810da234a0 msgr2=0x55810dba06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:13.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.495+0000 7f1a11372b80 1 --2- 172.21.15.46:0/1174657007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810da234a0 0x55810dba06a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f19e400ac60 tx=0x7f19e400af70 comp rx=0 tx=0).stop 2024-05-09T00:23:13.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.495+0000 7f1a11372b80 1 -- 172.21.15.46:0/1174657007 shutdown_connections 2024-05-09T00:23:13.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.495+0000 7f1a11372b80 1 --2- 172.21.15.46:0/1174657007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810dba0ca0 0x55810dba1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.495+0000 7f1a11372b80 1 --2- 172.21.15.46:0/1174657007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810db0d160 0x55810dba4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.495+0000 7f1a11372b80 1 --2- 172.21.15.46:0/1174657007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810da234a0 0x55810dba06a0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.495+0000 7f1a11372b80 1 -- 172.21.15.46:0/1174657007 >> 172.21.15.46:0/1174657007 conn(0x55810da836a0 msgr2=0x55810db9b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:13.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.496+0000 7f1a11372b80 1 -- 172.21.15.46:0/1174657007 shutdown_connections 2024-05-09T00:23:13.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.496+0000 7f1a11372b80 1 -- 172.21.15.46:0/1174657007 wait complete. 2024-05-09T00:23:13.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.496+0000 7f1a11372b80 1 -- start start 2024-05-09T00:23:13.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.496+0000 7f1a11372b80 1 Processor -- start 2024-05-09T00:23:13.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f1a11372b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810da234a0 0x55810dbb22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f1a11372b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 0x55810dbb2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f1a11372b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810dba0ca0 0x55810dbb2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f1a11372b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55810d93eba0 con 0x55810db0d160 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f1a11372b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55810da7d150 con 0x55810da234a0 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f1a11372b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55810dbb5870 con 0x55810dba0ca0 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f1a11372b80 10 client.0 _ll_register_callbacks cb 0x55810d9346c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 0x55810dbb2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 0x55810dbb2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48132/0 (socket says 172.21.15.46:48132) 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19f3fff700 1 -- 172.21.15.46:0/1474967324 learned_addr learned my addr 172.21.15.46:0/1474967324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f1a00ef9700 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810dba0ca0 0x55810dbb2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19fbfff700 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810da234a0 0x55810dbb22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19f3fff700 1 -- 172.21.15.46:0/1474967324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810dba0ca0 msgr2=0x55810dbb2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19f3fff700 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810dba0ca0 0x55810dbb2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19f3fff700 1 -- 172.21.15.46:0/1474967324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810da234a0 msgr2=0x55810dbb22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19f3fff700 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810da234a0 0x55810dbb22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19f3fff700 1 -- 172.21.15.46:0/1474967324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55810d93e490 con 0x55810db0d160 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19f3fff700 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 0x55810dbb2830 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f19e401a040 tx=0x7f19e400ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ad2d2f1031d52f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19db7fe700 1 -- 172.21.15.46:0/1474967324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4026020 con 0x55810db0d160 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f19db7fe700 1 -- 172.21.15.46:0/1474967324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f19e4004750 con 0x55810db0d160 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.497+0000 7f1a11372b80 1 -- 172.21.15.46:0/1474967324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e40044e0 con 0x55810db0d160 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.498+0000 7f19db7fe700 1 -- 172.21.15.46:0/1474967324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19e4004300 con 0x55810db0d160 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.498+0000 7f1a11372b80 10 client.17364 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.498+0000 7f1a11372b80 1 -- 172.21.15.46:0/1474967324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55810dbc0340 con 0x55810db0d160 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.498+0000 7f19db7fe700 1 -- 172.21.15.46:0/1474967324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19e40044e0 con 0x55810db0d160 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.498+0000 7f19db7fe700 1 -- 172.21.15.46:0/1474967324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f19e40055e0 con 0x55810db0d160 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.498+0000 7f1a11372b80 10 client.17364 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.498+0000 7f1a11372b80 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.498+0000 7f1a11372b80 20 client.17364 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.498+0000 7f1a11372b80 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55810dbc2810 0x55810dbc4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.498+0000 7f1a11372b80 4 client.17364 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.498+0000 7f1a11372b80 1 -- 172.21.15.46:0/1474967324 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55810dad09d0 con 0x55810dbc2810 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.499+0000 7f19fbfff700 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55810dbc2810 0x55810dbc4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.499+0000 7f19db7fe700 10 client.17364 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.499+0000 7f19fbfff700 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55810dbc2810 0x55810dbc4c30 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc00d2a1640ecee server_cookie=fc355cb835e4125 in_seq=0 out_seq=0 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f19db7fe700 1 -- 172.21.15.46:0/1474967324 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55810dad09d0 con 0x55810dbc2810 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f19db7fe700 10 client.17364 handle_command_reply: tid=0 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f1a11372b80 1 client.17364 shutdown 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f1a11372b80 1 -- 172.21.15.46:0/1474967324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 msgr2=0x55810dbb2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f1a11372b80 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 0x55810dbb2830 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f19e401a040 tx=0x7f19e400ff40 comp rx=0 tx=0).stop 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f1a11372b80 1 -- 172.21.15.46:0/1474967324 shutdown_connections 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f1a11372b80 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55810dbc2810 0x55810dbc4c30 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f1a11372b80 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810dba0ca0 0x55810dbb2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f1a11372b80 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810da234a0 0x55810dbb22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f19db7fe700 0 client.17364 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f1a11372b80 1 --2- 172.21.15.46:0/1474967324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 0x55810dbb2830 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f1a11372b80 1 -- 172.21.15.46:0/1474967324 >> 172.21.15.46:0/1474967324 conn(0x55810da836a0 msgr2=0x55810db9b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f1a11372b80 1 -- 172.21.15.46:0/1474967324 shutdown_connections 2024-05-09T00:23:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f1a11372b80 1 -- 172.21.15.46:0/1474967324 wait complete. 2024-05-09T00:23:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.500+0000 7f1a11372b80 20 client.17364 trim_cache size 0 max 16384 2024-05-09T00:23:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.521+0000 7f1a11372b80 1 Processor -- start 2024-05-09T00:23:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.521+0000 7f1a11372b80 1 -- start start 2024-05-09T00:23:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.521+0000 7f1a11372b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810da234a0 0x55810db9fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.521+0000 7f1a11372b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 0x55810dbb2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.521+0000 7f1a11372b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810dba0ca0 0x55810dbad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.521+0000 7f1a11372b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1af1f0 con 0x55810db0d160 2024-05-09T00:23:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.521+0000 7f1a11372b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1aca80 con 0x55810dba0ca0 2024-05-09T00:23:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.521+0000 7f1a11372b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1aa8b0 con 0x55810da234a0 2024-05-09T00:23:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.521+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 0x55810dbb2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.521+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 0x55810dbb2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48146/0 (socket says 172.21.15.46:48146) 2024-05-09T00:23:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.522+0000 7f19f3fff700 1 -- 172.21.15.46:0/2743878661 learned_addr learned my addr 172.21.15.46:0/2743878661 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.522+0000 7f19fbfff700 1 --2- 172.21.15.46:0/2743878661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810da234a0 0x55810db9fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.522+0000 7f19f3fff700 1 -- 172.21.15.46:0/2743878661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810da234a0 msgr2=0x55810db9fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.522+0000 7f19f3fff700 1 --2- 172.21.15.46:0/2743878661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810da234a0 0x55810db9fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.522+0000 7f19f3fff700 1 -- 172.21.15.46:0/2743878661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810dba0ca0 msgr2=0x55810dbad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.522+0000 7f19f3fff700 1 --2- 172.21.15.46:0/2743878661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810dba0ca0 0x55810dbad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.522+0000 7f19f3fff700 1 -- 172.21.15.46:0/2743878661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55810dba4080 con 0x55810db0d160 2024-05-09T00:23:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.522+0000 7f19f3fff700 1 --2- 172.21.15.46:0/2743878661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 0x55810dbb2bb0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f19e4000c00 tx=0x7f19e4003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de69453428e0b5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.522+0000 7f19f1ffb700 1 -- 172.21.15.46:0/2743878661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e403b070 con 0x55810db0d160 2024-05-09T00:23:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.522+0000 7f19f1ffb700 1 -- 172.21.15.46:0/2743878661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19e4022750 con 0x55810db0d160 2024-05-09T00:23:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.522+0000 7f19f1ffb700 1 -- 172.21.15.46:0/2743878661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4026040 con 0x55810db0d160 2024-05-09T00:23:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.523+0000 7f1a11372b80 1 -- 172.21.15.46:0/2743878661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 msgr2=0x55810dbb2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.523+0000 7f1a11372b80 1 --2- 172.21.15.46:0/2743878661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 0x55810dbb2bb0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f19e4000c00 tx=0x7f19e4003bf0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.523+0000 7f1a11372b80 1 -- 172.21.15.46:0/2743878661 shutdown_connections 2024-05-09T00:23:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.523+0000 7f1a11372b80 1 --2- 172.21.15.46:0/2743878661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810dba0ca0 0x55810dbad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.523+0000 7f1a11372b80 1 --2- 172.21.15.46:0/2743878661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810da234a0 0x55810db9fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.523+0000 7f1a11372b80 1 --2- 172.21.15.46:0/2743878661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810db0d160 0x55810dbb2bb0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.523+0000 7f1a11372b80 1 -- 172.21.15.46:0/2743878661 >> 172.21.15.46:0/2743878661 conn(0x55810da836a0 msgr2=0x55810db9f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.523+0000 7f1a11372b80 1 -- 172.21.15.46:0/2743878661 shutdown_connections 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.523+0000 7f1a11372b80 1 -- 172.21.15.46:0/2743878661 wait complete. 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.523+0000 7f1a11372b80 1 -- start start 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f1a11372b80 1 Processor -- start 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f1a11372b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810da234a0 0x55810dbc5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f1a11372b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810db0d160 0x55810dbc6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f1a11372b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810dba0ca0 0x55810dbc6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f1a11372b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1aaf40 con 0x55810dba0ca0 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f1a11372b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1ac3b0 con 0x55810da234a0 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f1a11372b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1ab610 con 0x55810db0d160 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f1a11372b80 10 client.0 _ll_register_callbacks cb 0x55810d9346c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f1a00ef9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810dba0ca0 0x55810dbc6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f1a00ef9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810dba0ca0 0x55810dbc6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48158/0 (socket says 172.21.15.46:48158) 2024-05-09T00:23:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810db0d160 0x55810dbc6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f1a00ef9700 1 -- 172.21.15.46:0/328410669 learned_addr learned my addr 172.21.15.46:0/328410669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.524+0000 7f19fbfff700 1 --2- 172.21.15.46:0/328410669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810da234a0 0x55810dbc5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f1a00ef9700 1 -- 172.21.15.46:0/328410669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810db0d160 msgr2=0x55810dbc6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f1a00ef9700 1 --2- 172.21.15.46:0/328410669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810db0d160 0x55810dbc6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f1a00ef9700 1 -- 172.21.15.46:0/328410669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810da234a0 msgr2=0x55810dbc5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f1a00ef9700 1 --2- 172.21.15.46:0/328410669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810da234a0 0x55810dbc5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f1a00ef9700 1 -- 172.21.15.46:0/328410669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55810dc2bf50 con 0x55810dba0ca0 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f1a00ef9700 1 --2- 172.21.15.46:0/328410669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810dba0ca0 0x55810dbc6570 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f19f40a7c30 tx=0x7f19f400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce09b16e049bf7a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f19db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f19db7fe700 1 -- 172.21.15.46:0/328410669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f40b30d0 con 0x55810dba0ca0 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f19db7fe700 1 -- 172.21.15.46:0/328410669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f19f40b10c0 con 0x55810dba0ca0 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f1a11372b80 1 -- 172.21.15.46:0/328410669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e4022750 con 0x55810dba0ca0 2024-05-09T00:23:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f19db7fe700 1 -- 172.21.15.46:0/328410669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19f40b12c0 con 0x55810dba0ca0 2024-05-09T00:23:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f1a11372b80 10 client.17370 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f1a11372b80 1 -- 172.21.15.46:0/328410669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55810dc2d220 con 0x55810dba0ca0 2024-05-09T00:23:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f19db7fe700 1 -- 172.21.15.46:0/328410669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19e4022750 con 0x55810dba0ca0 2024-05-09T00:23:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.525+0000 7f19db7fe700 1 -- 172.21.15.46:0/328410669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f19f402e060 con 0x55810dba0ca0 2024-05-09T00:23:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.526+0000 7f1a11372b80 10 client.17370 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.526+0000 7f1a11372b80 10 client.17370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.526+0000 7f1a11372b80 20 client.17370 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.526+0000 7f1a11372b80 1 --2- 172.21.15.46:0/328410669 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55810dc2d880 0x55810dc2fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.526+0000 7f1a11372b80 4 client.17370 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:23:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.526+0000 7f1a11372b80 1 -- 172.21.15.46:0/328410669 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55810dacaa70 con 0x55810dc2d880 2024-05-09T00:23:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.526+0000 7f19fbfff700 1 --2- 172.21.15.46:0/328410669 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55810dc2d880 0x55810dc2fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.526+0000 7f19db7fe700 10 client.17370 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.526+0000 7f19fbfff700 1 --2- 172.21.15.46:0/328410669 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55810dc2d880 0x55810dc2fca0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a8e749972da9a8a server_cookie=7af5e8cf613940da in_seq=0 out_seq=0 2024-05-09T00:23:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f19db7fe700 1 -- 172.21.15.46:0/328410669 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55810dacaa70 con 0x55810dc2d880 2024-05-09T00:23:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f19db7fe700 10 client.17370 handle_command_reply: tid=0 2024-05-09T00:23:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f1a11372b80 1 client.17370 shutdown 2024-05-09T00:23:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f1a11372b80 1 -- 172.21.15.46:0/328410669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810dba0ca0 msgr2=0x55810dbc6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f1a11372b80 1 --2- 172.21.15.46:0/328410669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810dba0ca0 0x55810dbc6570 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f19f40a7c30 tx=0x7f19f400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f1a11372b80 1 -- 172.21.15.46:0/328410669 shutdown_connections 2024-05-09T00:23:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f1a11372b80 1 --2- 172.21.15.46:0/328410669 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55810dc2d880 0x55810dc2fca0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f1a11372b80 1 --2- 172.21.15.46:0/328410669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55810dba0ca0 0x55810dbc6570 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f1a11372b80 1 --2- 172.21.15.46:0/328410669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55810da234a0 0x55810dbc5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f1a11372b80 1 --2- 172.21.15.46:0/328410669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55810db0d160 0x55810dbc6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f19db7fe700 0 client.17370 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:13.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f1a11372b80 1 -- 172.21.15.46:0/328410669 >> 172.21.15.46:0/328410669 conn(0x55810da836a0 msgr2=0x55810dbad2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:13.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f1a11372b80 1 -- 172.21.15.46:0/328410669 shutdown_connections 2024-05-09T00:23:13.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f1a11372b80 1 -- 172.21.15.46:0/328410669 wait complete. 2024-05-09T00:23:13.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.527+0000 7f1a11372b80 20 client.17370 trim_cache size 0 max 16384 2024-05-09T00:23:13.532 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:23:13.532 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:23:13.532 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:23:13.533 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:23:13.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.528+0000 7f1a0295c700 1 -- 172.21.15.46:0/184379662 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f19e80b2880 msgr2=0x7f19e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:13.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.528+0000 7f1a0295c700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f19e80b2880 0x7f19e80b4d90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f19ec005950 tx=0x7f19ec00b500 comp rx=0 tx=0).stop 2024-05-09T00:23:13.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.528+0000 7f1a0295c700 1 -- 172.21.15.46:0/184379662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19fc108740 msgr2=0x7f19fc19c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.528+0000 7f1a0295c700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19fc108740 0x7f19fc19c730 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009fd0 tx=0x7f19f4007a00 comp rx=0 tx=0).stop 2024-05-09T00:23:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.528+0000 7f1a0295c700 1 -- 172.21.15.46:0/184379662 shutdown_connections 2024-05-09T00:23:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.528+0000 7f1a0295c700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f19e80b2880 0x7f19e80b4d90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.528+0000 7f1a0295c700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19fc108740 0x7f19fc19c730 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.528+0000 7f1a0295c700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc1034c0 0x7f19fc19bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.528+0000 7f1a0295c700 1 --2- 172.21.15.46:0/184379662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19fc107df0 0x7f19fc19c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.528+0000 7f1a0295c700 1 -- 172.21.15.46:0/184379662 >> 172.21.15.46:0/184379662 conn(0x7f19fc0fedc0 msgr2=0x7f19fc107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.528+0000 7f1a0295c700 1 -- 172.21.15.46:0/184379662 shutdown_connections 2024-05-09T00:23:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:13.528+0000 7f1a0295c700 1 -- 172.21.15.46:0/184379662 wait complete. 2024-05-09T00:23:14.388 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:23:14.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:23:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:15.022 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:14 smithi046 ceph-mon[30151]: pgmap v732: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:23:15.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:14 smithi169 ceph-mon[34777]: pgmap v732: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:23:15.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:14 smithi142 ceph-mon[34533]: pgmap v732: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:23:16.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.487+0000 7f37492a0700 1 -- 172.21.15.46:0/3347790570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37441045e0 msgr2=0x7f37441049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:16.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.487+0000 7f37492a0700 1 --2- 172.21.15.46:0/3347790570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37441045e0 0x7f37441049f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f37340099c0 tx=0x7f3734009cd0 comp rx=0 tx=0).stop 2024-05-09T00:23:16.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.487+0000 7f37492a0700 1 -- 172.21.15.46:0/3347790570 shutdown_connections 2024-05-09T00:23:16.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.487+0000 7f37492a0700 1 --2- 172.21.15.46:0/3347790570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3744109860 0x7f37441111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:16.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.487+0000 7f37492a0700 1 --2- 172.21.15.46:0/3347790570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37441045e0 0x7f37441049f0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:16.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.487+0000 7f37492a0700 1 --2- 172.21.15.46:0/3347790570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3744108f10 0x7f3744109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:16.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.487+0000 7f37492a0700 1 -- 172.21.15.46:0/3347790570 >> 172.21.15.46:0/3347790570 conn(0x7f37440ffef0 msgr2=0x7f3744102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:16.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.487+0000 7f37492a0700 1 -- 172.21.15.46:0/3347790570 shutdown_connections 2024-05-09T00:23:16.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.487+0000 7f37492a0700 1 -- 172.21.15.46:0/3347790570 wait complete. 2024-05-09T00:23:16.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.488+0000 7f37492a0700 1 Processor -- start 2024-05-09T00:23:16.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.488+0000 7f37492a0700 1 -- start start 2024-05-09T00:23:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.488+0000 7f37492a0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37441045e0 0x7f3744076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.488+0000 7f37492a0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3744108f10 0x7f3744077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.488+0000 7f37492a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3744109860 0x7f3744077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.488+0000 7f37492a0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3744114490 con 0x7f3744109860 2024-05-09T00:23:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.488+0000 7f37492a0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3744114310 con 0x7f3744108f10 2024-05-09T00:23:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.488+0000 7f37492a0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3744114610 con 0x7f37441045e0 2024-05-09T00:23:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f37437fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3744109860 0x7f3744077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f37437fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3744109860 0x7f3744077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52400/0 (socket says 172.21.15.46:52400) 2024-05-09T00:23:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f37437fe700 1 -- 172.21.15.46:0/518620234 learned_addr learned my addr 172.21.15.46:0/518620234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f37427fc700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3744108f10 0x7f3744077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f3742ffd700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37441045e0 0x7f3744076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f37437fe700 1 -- 172.21.15.46:0/518620234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37441045e0 msgr2=0x7f3744076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f37437fe700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37441045e0 0x7f3744076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f37437fe700 1 -- 172.21.15.46:0/518620234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3744108f10 msgr2=0x7f3744077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f37437fe700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3744108f10 0x7f3744077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:16.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f37437fe700 1 -- 172.21.15.46:0/518620234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37441a91a0 con 0x7f3744109860 2024-05-09T00:23:16.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f37437fe700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3744109860 0x7f3744077650 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f373c00ef30 tx=0x7f373c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:16.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f372bfff700 1 -- 172.21.15.46:0/518620234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c0092e0 con 0x7f3744109860 2024-05-09T00:23:16.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f37492a0700 1 -- 172.21.15.46:0/518620234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37441a9430 con 0x7f3744109860 2024-05-09T00:23:16.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f372bfff700 1 -- 172.21.15.46:0/518620234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373c004bb0 con 0x7f3744109860 2024-05-09T00:23:16.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f372bfff700 1 -- 172.21.15.46:0/518620234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c004020 con 0x7f3744109860 2024-05-09T00:23:16.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.489+0000 7f37492a0700 1 -- 172.21.15.46:0/518620234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37441a9920 con 0x7f3744109860 2024-05-09T00:23:16.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.490+0000 7f37492a0700 1 -- 172.21.15.46:0/518620234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37440512c0 con 0x7f3744109860 2024-05-09T00:23:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.492+0000 7f372bfff700 1 -- 172.21.15.46:0/518620234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f373c01b030 con 0x7f3744109860 2024-05-09T00:23:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.493+0000 7f372bfff700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f372c0b2880 0x7f372c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.493+0000 7f3742ffd700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f372c0b2880 0x7f372c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.493+0000 7f372bfff700 1 -- 172.21.15.46:0/518620234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f373c0e26c0 con 0x7f3744109860 2024-05-09T00:23:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.494+0000 7f3742ffd700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f372c0b2880 0x7f372c0b4d90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3734000c00 tx=0x7f373401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.497+0000 7f372bfff700 1 -- 172.21.15.46:0/518620234 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f373c022020 con 0x7f3744109860 2024-05-09T00:23:16.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.757+0000 7f37492a0700 1 -- 172.21.15.46:0/518620234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f374402e7b0 con 0x7f3744109860 2024-05-09T00:23:16.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.758+0000 7f372bfff700 1 -- 172.21.15.46:0/518620234 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f373c010040 con 0x7f3744109860 2024-05-09T00:23:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:23:16.758 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:23:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.760+0000 7f37492a0700 1 -- 172.21.15.46:0/518620234 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f372c0b2880 msgr2=0x7f372c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.760+0000 7f37492a0700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f372c0b2880 0x7f372c0b4d90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3734000c00 tx=0x7f373401a040 comp rx=0 tx=0).stop 2024-05-09T00:23:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.760+0000 7f37492a0700 1 -- 172.21.15.46:0/518620234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3744109860 msgr2=0x7f3744077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.760+0000 7f37492a0700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3744109860 0x7f3744077650 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f373c00ef30 tx=0x7f373c00c3a0 comp rx=0 tx=0).stop 2024-05-09T00:23:16.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.760+0000 7f37492a0700 1 -- 172.21.15.46:0/518620234 shutdown_connections 2024-05-09T00:23:16.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.760+0000 7f37492a0700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f372c0b2880 0x7f372c0b4d90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:16.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.760+0000 7f37492a0700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3744109860 0x7f3744077650 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:16.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.760+0000 7f37492a0700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37441045e0 0x7f3744076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:16.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.760+0000 7f37492a0700 1 --2- 172.21.15.46:0/518620234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3744108f10 0x7f3744077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:16.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.760+0000 7f37492a0700 1 -- 172.21.15.46:0/518620234 >> 172.21.15.46:0/518620234 conn(0x7f37440ffef0 msgr2=0x7f37441081b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:16.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.761+0000 7f37492a0700 1 -- 172.21.15.46:0/518620234 shutdown_connections 2024-05-09T00:23:16.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:16.761+0000 7f37492a0700 1 -- 172.21.15.46:0/518620234 wait complete. 2024-05-09T00:23:16.761 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:23:17.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:16 smithi046 ceph-mon[30151]: pgmap v733: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:23:17.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:16 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/518620234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:23:17.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:16 smithi169 ceph-mon[34777]: pgmap v733: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:23:17.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:16 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/518620234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:23:17.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:16 smithi142 ceph-mon[34533]: pgmap v733: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:23:17.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:16 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/518620234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:23:17.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:23:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:19.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:18 smithi046 ceph-mon[30151]: pgmap v734: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:23:19.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:18 smithi169 ceph-mon[34777]: pgmap v734: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:23:19.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:18 smithi142 ceph-mon[34533]: pgmap v734: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-05-09T00:23:19.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.634+0000 7fcf6493c700 1 -- 172.21.15.46:0/4130489434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf60108f00 msgr2=0x7fcf60109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:19.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.634+0000 7fcf6493c700 1 --2- 172.21.15.46:0/4130489434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf60108f00 0x7fcf60109310 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fcf540099c0 tx=0x7fcf54009cd0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.634+0000 7fcf6493c700 1 -- 172.21.15.46:0/4130489434 shutdown_connections 2024-05-09T00:23:19.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.634+0000 7fcf6493c700 1 --2- 172.21.15.46:0/4130489434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf60109850 0x7fcf601111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.634+0000 7fcf6493c700 1 --2- 172.21.15.46:0/4130489434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf601045d0 0x7fcf601049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.634+0000 7fcf6493c700 1 --2- 172.21.15.46:0/4130489434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf60108f00 0x7fcf60109310 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.634+0000 7fcf6493c700 1 -- 172.21.15.46:0/4130489434 >> 172.21.15.46:0/4130489434 conn(0x7fcf600ffea0 msgr2=0x7fcf601022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:19.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.634+0000 7fcf6493c700 1 -- 172.21.15.46:0/4130489434 shutdown_connections 2024-05-09T00:23:19.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.634+0000 7fcf6493c700 1 -- 172.21.15.46:0/4130489434 wait complete. 2024-05-09T00:23:19.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.635+0000 7fcf6493c700 1 Processor -- start 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.635+0000 7fcf6493c700 1 -- start start 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.635+0000 7fcf6493c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf601045d0 0x7fcf60076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.635+0000 7fcf6493c700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60108f00 0x7fcf60077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.635+0000 7fcf6493c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf60109850 0x7fcf60077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.635+0000 7fcf6493c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf60114460 con 0x7fcf601045d0 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.635+0000 7fcf6493c700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601142e0 con 0x7fcf60108f00 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.635+0000 7fcf6493c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601145e0 con 0x7fcf60109850 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.635+0000 7fcf5e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf601045d0 0x7fcf60076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.635+0000 7fcf5e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf601045d0 0x7fcf60076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52412/0 (socket says 172.21.15.46:52412) 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.635+0000 7fcf5e59c700 1 -- 172.21.15.46:0/3244987984 learned_addr learned my addr 172.21.15.46:0/3244987984 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.635+0000 7fcf5dd9b700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60108f00 0x7fcf60077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.636+0000 7fcf5ed9d700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf60109850 0x7fcf60077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.636+0000 7fcf5e59c700 1 -- 172.21.15.46:0/3244987984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf60109850 msgr2=0x7fcf60077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.636+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf60109850 0x7fcf60077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.636+0000 7fcf5e59c700 1 -- 172.21.15.46:0/3244987984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60108f00 msgr2=0x7fcf60077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.636+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60108f00 0x7fcf60077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.636+0000 7fcf5e59c700 1 -- 172.21.15.46:0/3244987984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf601a9130 con 0x7fcf601045d0 2024-05-09T00:23:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.636+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf601045d0 0x7fcf60076bd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000c9b0 tx=0x7fcf5000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.636+0000 7fcf4f7fe700 1 -- 172.21.15.46:0/3244987984 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50007b00 con 0x7fcf601045d0 2024-05-09T00:23:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.636+0000 7fcf4f7fe700 1 -- 172.21.15.46:0/3244987984 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf50007ca0 con 0x7fcf601045d0 2024-05-09T00:23:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.636+0000 7fcf6493c700 1 -- 172.21.15.46:0/3244987984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf601a9420 con 0x7fcf601045d0 2024-05-09T00:23:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.636+0000 7fcf4f7fe700 1 -- 172.21.15.46:0/3244987984 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50018730 con 0x7fcf601045d0 2024-05-09T00:23:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.636+0000 7fcf6493c700 1 -- 172.21.15.46:0/3244987984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf601a9980 con 0x7fcf601045d0 2024-05-09T00:23:19.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.639+0000 7fcf4f7fe700 1 -- 172.21.15.46:0/3244987984 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcf5000f450 con 0x7fcf601045d0 2024-05-09T00:23:19.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.640+0000 7fcf4f7fe700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcf480b2900 0x7fcf480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.640+0000 7fcf5dd9b700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcf480b2900 0x7fcf480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.640+0000 7fcf4f7fe700 1 -- 172.21.15.46:0/3244987984 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fcf500e2270 con 0x7fcf601045d0 2024-05-09T00:23:19.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.640+0000 7fcf5dd9b700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcf480b2900 0x7fcf480b4e10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcf54000c00 tx=0x7fcf5401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:19.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf73352b80 1 Processor -- start 2024-05-09T00:23:19.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf73352b80 1 -- start start 2024-05-09T00:23:19.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf73352b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 0x55bd194a7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf73352b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd193244a0 0x55bd194a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf73352b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd194a1400 0x55bd194a1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf73352b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bd193c40a0 con 0x55bd1939ed00 2024-05-09T00:23:19.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf73352b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bd193699a0 con 0x55bd193244a0 2024-05-09T00:23:19.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf73352b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bd193c0060 con 0x55bd194a1400 2024-05-09T00:23:19.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf5e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 0x55bd194a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf5e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 0x55bd194a7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52426/0 (socket says 172.21.15.46:52426) 2024-05-09T00:23:19.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf5e59c700 1 -- 172.21.15.46:0/558309568 learned_addr learned my addr 172.21.15.46:0/558309568 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:19.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf5dd9b700 1 --2- 172.21.15.46:0/558309568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd193244a0 0x55bd194a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.641+0000 7fcf5ed9d700 1 --2- 172.21.15.46:0/558309568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd194a1400 0x55bd194a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.642+0000 7fcf5e59c700 1 -- 172.21.15.46:0/558309568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd194a1400 msgr2=0x55bd194a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:19.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.642+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/558309568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd194a1400 0x55bd194a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.642+0000 7fcf5e59c700 1 -- 172.21.15.46:0/558309568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd193244a0 msgr2=0x55bd194a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:19.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.642+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/558309568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd193244a0 0x55bd194a7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.642+0000 7fcf5e59c700 1 -- 172.21.15.46:0/558309568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd1935a500 con 0x55bd1939ed00 2024-05-09T00:23:19.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.642+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/558309568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 0x55bd194a7240 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fcf500a5910 tx=0x7fcf500b2180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a073f6e038dcae86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:19.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.642+0000 7fcf4cff9700 1 -- 172.21.15.46:0/558309568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500a9890 con 0x55bd1939ed00 2024-05-09T00:23:19.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.642+0000 7fcf4cff9700 1 -- 172.21.15.46:0/558309568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf500adc40 con 0x55bd1939ed00 2024-05-09T00:23:19.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.642+0000 7fcf4cff9700 1 -- 172.21.15.46:0/558309568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500a9af0 con 0x55bd1939ed00 2024-05-09T00:23:19.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.643+0000 7fcf73352b80 1 -- 172.21.15.46:0/558309568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 msgr2=0x55bd194a7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:19.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.643+0000 7fcf73352b80 1 --2- 172.21.15.46:0/558309568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 0x55bd194a7240 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fcf500a5910 tx=0x7fcf500b2180 comp rx=0 tx=0).stop 2024-05-09T00:23:19.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.643+0000 7fcf73352b80 1 -- 172.21.15.46:0/558309568 shutdown_connections 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.643+0000 7fcf73352b80 1 --2- 172.21.15.46:0/558309568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd194a1400 0x55bd194a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.643+0000 7fcf73352b80 1 --2- 172.21.15.46:0/558309568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 0x55bd194a7240 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.643+0000 7fcf73352b80 1 --2- 172.21.15.46:0/558309568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd193244a0 0x55bd194a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.643+0000 7fcf73352b80 1 -- 172.21.15.46:0/558309568 >> 172.21.15.46:0/558309568 conn(0x55bd19327f70 msgr2=0x55bd1949c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.643+0000 7fcf73352b80 1 -- 172.21.15.46:0/558309568 shutdown_connections 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.643+0000 7fcf73352b80 1 -- 172.21.15.46:0/558309568 wait complete. 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf73352b80 1 -- start start 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf73352b80 1 Processor -- start 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf73352b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd194ab050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf73352b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 0x55bd194ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf73352b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd194a1400 0x55bd194abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf73352b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bd194a2160 con 0x55bd1939ed00 2024-05-09T00:23:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf73352b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bd1935eb20 con 0x55bd194a1400 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf73352b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bd194ae020 con 0x55bd193244a0 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf73352b80 10 client.0 _ll_register_callbacks cb 0x55bd192356c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf5dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 0x55bd194ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf5dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 0x55bd194ab590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52438/0 (socket says 172.21.15.46:52438) 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf5dd9b700 1 -- 172.21.15.46:0/3702202890 learned_addr learned my addr 172.21.15.46:0/3702202890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd194ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.644+0000 7fcf5ed9d700 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd194a1400 0x55bd194abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf5dd9b700 1 -- 172.21.15.46:0/3702202890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 msgr2=0x55bd194ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf5dd9b700 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd194ab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf5dd9b700 1 -- 172.21.15.46:0/3702202890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd194a1400 msgr2=0x55bd194abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf5dd9b700 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd194a1400 0x55bd194abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf5dd9b700 1 -- 172.21.15.46:0/3702202890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd1923f490 con 0x55bd1939ed00 2024-05-09T00:23:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf5ed9d700 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd194a1400 0x55bd194abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf5dd9b700 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 0x55bd194ab590 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fcf54015000 tx=0x7fcf54009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e3b3b1a0c88143 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf45ffb700 1 -- 172.21.15.46:0/3702202890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5402e850 con 0x55bd1939ed00 2024-05-09T00:23:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf45ffb700 1 -- 172.21.15.46:0/3702202890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fcf5402e9f0 con 0x55bd1939ed00 2024-05-09T00:23:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf73352b80 1 -- 172.21.15.46:0/3702202890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf500adc40 con 0x55bd1939ed00 2024-05-09T00:23:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf45ffb700 1 -- 172.21.15.46:0/3702202890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf5402ebf0 con 0x55bd1939ed00 2024-05-09T00:23:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf73352b80 10 client.17388 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf73352b80 1 -- 172.21.15.46:0/3702202890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd194b8a70 con 0x55bd1939ed00 2024-05-09T00:23:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.645+0000 7fcf45ffb700 1 -- 172.21.15.46:0/3702202890 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf500adc40 con 0x55bd1939ed00 2024-05-09T00:23:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.646+0000 7fcf45ffb700 1 -- 172.21.15.46:0/3702202890 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcf54037400 con 0x55bd1939ed00 2024-05-09T00:23:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.646+0000 7fcf73352b80 10 client.17388 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.646+0000 7fcf73352b80 10 client.17388 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:23:19.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.646+0000 7fcf73352b80 20 client.17388 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:19.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.646+0000 7fcf73352b80 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bd194bb100 0x55bd194bd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.646+0000 7fcf73352b80 4 client.17388 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:19.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.646+0000 7fcf73352b80 1 -- 172.21.15.46:0/3702202890 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd193d1a80 con 0x55bd194bb100 2024-05-09T00:23:19.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.646+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bd194bb100 0x55bd194bd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.647+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bd194bb100 0x55bd194bd550 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3750ffd070525fcf server_cookie=7a93318140e359e9 in_seq=0 out_seq=0 2024-05-09T00:23:19.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.647+0000 7fcf45ffb700 10 client.17388 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:23:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.647+0000 7fcf45ffb700 1 -- 172.21.15.46:0/3702202890 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bd193d1a80 con 0x55bd194bb100 2024-05-09T00:23:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.647+0000 7fcf45ffb700 10 client.17388 handle_command_reply: tid=0 2024-05-09T00:23:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf73352b80 1 client.17388 shutdown 2024-05-09T00:23:19.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf73352b80 1 -- 172.21.15.46:0/3702202890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 msgr2=0x55bd194ab590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf73352b80 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 0x55bd194ab590 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fcf54015000 tx=0x7fcf54009310 comp rx=0 tx=0).stop 2024-05-09T00:23:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf73352b80 1 -- 172.21.15.46:0/3702202890 shutdown_connections 2024-05-09T00:23:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf73352b80 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bd194bb100 0x55bd194bd550 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf73352b80 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd194a1400 0x55bd194abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf73352b80 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd194ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf45ffb700 0 client.17388 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:23:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf73352b80 1 --2- 172.21.15.46:0/3702202890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd1939ed00 0x55bd194ab590 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf73352b80 1 -- 172.21.15.46:0/3702202890 >> 172.21.15.46:0/3702202890 conn(0x55bd19327f70 msgr2=0x55bd19495ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf73352b80 1 -- 172.21.15.46:0/3702202890 shutdown_connections 2024-05-09T00:23:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf73352b80 1 -- 172.21.15.46:0/3702202890 wait complete. 2024-05-09T00:23:19.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.648+0000 7fcf73352b80 20 client.17388 trim_cache size 0 max 16384 2024-05-09T00:23:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.663+0000 7fcf73352b80 1 Processor -- start 2024-05-09T00:23:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.663+0000 7fcf73352b80 1 -- start start 2024-05-09T00:23:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.663+0000 7fcf73352b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd19496a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.663+0000 7fcf73352b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd1939ed00 0x55bd194ab910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.663+0000 7fcf73352b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 0x55bd194c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.663+0000 7fcf73352b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601b3570 con 0x55bd194a1400 2024-05-09T00:23:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.663+0000 7fcf73352b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601b0e00 con 0x55bd1939ed00 2024-05-09T00:23:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.663+0000 7fcf73352b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601aec30 con 0x55bd193244a0 2024-05-09T00:23:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.663+0000 7fcf5ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 0x55bd194c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.663+0000 7fcf5ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 0x55bd194c2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52444/0 (socket says 172.21.15.46:52444) 2024-05-09T00:23:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.663+0000 7fcf5ed9d700 1 -- 172.21.15.46:0/114118258 learned_addr learned my addr 172.21.15.46:0/114118258 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.664+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/114118258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd19496a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.664+0000 7fcf5dd9b700 1 --2- 172.21.15.46:0/114118258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd1939ed00 0x55bd194ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.664+0000 7fcf5ed9d700 1 -- 172.21.15.46:0/114118258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 msgr2=0x55bd19496a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.664+0000 7fcf5ed9d700 1 --2- 172.21.15.46:0/114118258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd19496a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.664+0000 7fcf5ed9d700 1 -- 172.21.15.46:0/114118258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd1939ed00 msgr2=0x55bd194ab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.664+0000 7fcf5ed9d700 1 --2- 172.21.15.46:0/114118258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd1939ed00 0x55bd194ab910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.664+0000 7fcf5ed9d700 1 -- 172.21.15.46:0/114118258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd194ad8f0 con 0x55bd194a1400 2024-05-09T00:23:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.664+0000 7fcf5dd9b700 1 --2- 172.21.15.46:0/114118258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd1939ed00 0x55bd194ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.664+0000 7fcf5ed9d700 1 --2- 172.21.15.46:0/114118258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 0x55bd194c2cd0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fcf58008710 tx=0x7fcf58007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1eaea7ba3e8c47b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.664+0000 7fcf47fff700 1 -- 172.21.15.46:0/114118258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5801b030 con 0x55bd194a1400 2024-05-09T00:23:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.664+0000 7fcf47fff700 1 -- 172.21.15.46:0/114118258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf58017670 con 0x55bd194a1400 2024-05-09T00:23:19.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.664+0000 7fcf47fff700 1 -- 172.21.15.46:0/114118258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58004280 con 0x55bd194a1400 2024-05-09T00:23:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.665+0000 7fcf73352b80 1 -- 172.21.15.46:0/114118258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 msgr2=0x55bd194c2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.665+0000 7fcf73352b80 1 --2- 172.21.15.46:0/114118258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 0x55bd194c2cd0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fcf58008710 tx=0x7fcf58007910 comp rx=0 tx=0).stop 2024-05-09T00:23:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.665+0000 7fcf73352b80 1 -- 172.21.15.46:0/114118258 shutdown_connections 2024-05-09T00:23:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.665+0000 7fcf73352b80 1 --2- 172.21.15.46:0/114118258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 0x55bd194c2cd0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.665+0000 7fcf73352b80 1 --2- 172.21.15.46:0/114118258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd19496a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.665+0000 7fcf73352b80 1 --2- 172.21.15.46:0/114118258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd1939ed00 0x55bd194ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.665+0000 7fcf73352b80 1 -- 172.21.15.46:0/114118258 >> 172.21.15.46:0/114118258 conn(0x55bd19327f70 msgr2=0x55bd194a61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.665+0000 7fcf73352b80 1 -- 172.21.15.46:0/114118258 shutdown_connections 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.665+0000 7fcf73352b80 1 -- 172.21.15.46:0/114118258 wait complete. 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf73352b80 1 -- start start 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf73352b80 1 Processor -- start 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf73352b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd194b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf73352b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd1939ed00 0x55bd194aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf73352b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 0x55bd194af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf73352b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601af2c0 con 0x55bd194a1400 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf73352b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601b0730 con 0x55bd1939ed00 2024-05-09T00:23:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf73352b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601af990 con 0x55bd193244a0 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf73352b80 10 client.0 _ll_register_callbacks cb 0x55bd194aa4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf5ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 0x55bd194af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf5ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 0x55bd194af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52458/0 (socket says 172.21.15.46:52458) 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf5e59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd194b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf5ed9d700 1 -- 172.21.15.46:0/1129472075 learned_addr learned my addr 172.21.15.46:0/1129472075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf5dd9b700 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd1939ed00 0x55bd194aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf5ed9d700 1 -- 172.21.15.46:0/1129472075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 msgr2=0x55bd194b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf5ed9d700 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd194b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf5ed9d700 1 -- 172.21.15.46:0/1129472075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd1939ed00 msgr2=0x55bd194aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.666+0000 7fcf5ed9d700 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd1939ed00 0x55bd194aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.667+0000 7fcf5ed9d700 1 -- 172.21.15.46:0/1129472075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd194af870 con 0x55bd194a1400 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.667+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd194b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.667+0000 7fcf5ed9d700 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 0x55bd194af0d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fcf58008d00 tx=0x7fcf58004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db37668eb78243b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:19.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.667+0000 7fcf45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.667+0000 7fcf45ffb700 1 -- 172.21.15.46:0/1129472075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58004020 con 0x55bd194a1400 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.667+0000 7fcf45ffb700 1 -- 172.21.15.46:0/1129472075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fcf580041c0 con 0x55bd194a1400 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.667+0000 7fcf73352b80 1 -- 172.21.15.46:0/1129472075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf58017670 con 0x55bd194a1400 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.667+0000 7fcf45ffb700 1 -- 172.21.15.46:0/1129472075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf580043c0 con 0x55bd194a1400 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.667+0000 7fcf73352b80 10 client.17394 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.667+0000 7fcf73352b80 1 -- 172.21.15.46:0/1129472075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd195257a0 con 0x55bd194a1400 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.667+0000 7fcf45ffb700 1 -- 172.21.15.46:0/1129472075 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf58017670 con 0x55bd194a1400 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.668+0000 7fcf45ffb700 1 -- 172.21.15.46:0/1129472075 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcf58021a60 con 0x55bd194a1400 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.668+0000 7fcf73352b80 10 client.17394 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.668+0000 7fcf73352b80 10 client.17394 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.668+0000 7fcf73352b80 20 client.17394 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.668+0000 7fcf73352b80 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bd19525d90 0x55bd195281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.668+0000 7fcf73352b80 4 client.17394 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:23:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.668+0000 7fcf73352b80 1 -- 172.21.15.46:0/1129472075 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd193cbb20 con 0x55bd19525d90 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.668+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bd19525d90 0x55bd195281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.668+0000 7fcf5e59c700 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bd19525d90 0x55bd195281b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ab40e2b5b4ec54e server_cookie=ce03f1f791f8fbda in_seq=0 out_seq=0 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.668+0000 7fcf45ffb700 10 client.17394 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf45ffb700 1 -- 172.21.15.46:0/1129472075 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd193cbb20 con 0x55bd19525d90 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf45ffb700 10 client.17394 handle_command_reply: tid=0 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf73352b80 1 client.17394 shutdown 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf73352b80 1 -- 172.21.15.46:0/1129472075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 msgr2=0x55bd194af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf73352b80 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 0x55bd194af0d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fcf58008d00 tx=0x7fcf58004ce0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf73352b80 1 -- 172.21.15.46:0/1129472075 shutdown_connections 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf73352b80 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55bd19525d90 0x55bd195281b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf73352b80 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd194a1400 0x55bd194af0d0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf73352b80 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd193244a0 0x55bd194b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf73352b80 1 --2- 172.21.15.46:0/1129472075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd1939ed00 0x55bd194aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf73352b80 1 -- 172.21.15.46:0/1129472075 >> 172.21.15.46:0/1129472075 conn(0x55bd19327f70 msgr2=0x55bd194a61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:19.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf45ffb700 0 client.17394 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:23:19.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf73352b80 1 -- 172.21.15.46:0/1129472075 shutdown_connections 2024-05-09T00:23:19.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf73352b80 1 -- 172.21.15.46:0/1129472075 wait complete. 2024-05-09T00:23:19.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.669+0000 7fcf73352b80 20 client.17394 trim_cache size 0 max 16384 2024-05-09T00:23:19.674 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:23:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.670+0000 7fcf6493c700 1 -- 172.21.15.46:0/3244987984 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcf480b2900 msgr2=0x7fcf480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.670+0000 7fcf6493c700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcf480b2900 0x7fcf480b4e10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcf54000c00 tx=0x7fcf5401a040 comp rx=0 tx=0).stop 2024-05-09T00:23:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.670+0000 7fcf6493c700 1 -- 172.21.15.46:0/3244987984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf601045d0 msgr2=0x7fcf60076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.670+0000 7fcf6493c700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf601045d0 0x7fcf60076bd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000c9b0 tx=0x7fcf5000cd70 comp rx=0 tx=0).stop 2024-05-09T00:23:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.670+0000 7fcf6493c700 1 -- 172.21.15.46:0/3244987984 shutdown_connections 2024-05-09T00:23:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.670+0000 7fcf6493c700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fcf480b2900 0x7fcf480b4e10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.670+0000 7fcf6493c700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf60109850 0x7fcf60077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.670+0000 7fcf6493c700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf601045d0 0x7fcf60076bd0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.670+0000 7fcf6493c700 1 --2- 172.21.15.46:0/3244987984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf60108f00 0x7fcf60077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.670+0000 7fcf6493c700 1 -- 172.21.15.46:0/3244987984 >> 172.21.15.46:0/3244987984 conn(0x7fcf600ffea0 msgr2=0x7fcf6010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.670+0000 7fcf6493c700 1 -- 172.21.15.46:0/3244987984 shutdown_connections 2024-05-09T00:23:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:19.670+0000 7fcf6493c700 1 -- 172.21.15.46:0/3244987984 wait complete. 2024-05-09T00:23:20.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:23:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:21.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:20 smithi046 ceph-mon[30151]: pgmap v735: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:23:21.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:20 smithi169 ceph-mon[34777]: pgmap v735: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:23:21.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:20 smithi142 ceph-mon[34533]: pgmap v735: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:23:22.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.533+0000 7ff977b04700 1 -- 172.21.15.46:0/3841047892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9701034c0 msgr2=0x7ff9701038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:22.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.533+0000 7ff977b04700 1 --2- 172.21.15.46:0/3841047892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9701034c0 0x7ff9701038d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0099c0 tx=0x7ff96c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.533+0000 7ff977b04700 1 -- 172.21.15.46:0/3841047892 shutdown_connections 2024-05-09T00:23:22.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.533+0000 7ff977b04700 1 --2- 172.21.15.46:0/3841047892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff970108740 0x7ff9701100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.534+0000 7ff977b04700 1 --2- 172.21.15.46:0/3841047892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9701034c0 0x7ff9701038d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.534+0000 7ff977b04700 1 --2- 172.21.15.46:0/3841047892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff970107df0 0x7ff970108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.534+0000 7ff977b04700 1 -- 172.21.15.46:0/3841047892 >> 172.21.15.46:0/3841047892 conn(0x7ff9700fedf0 msgr2=0x7ff970101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:22.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.534+0000 7ff977b04700 1 -- 172.21.15.46:0/3841047892 shutdown_connections 2024-05-09T00:23:22.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.534+0000 7ff977b04700 1 -- 172.21.15.46:0/3841047892 wait complete. 2024-05-09T00:23:22.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.534+0000 7ff977b04700 1 Processor -- start 2024-05-09T00:23:22.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff977b04700 1 -- start start 2024-05-09T00:23:22.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff977b04700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9701034c0 0x7ff97019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff977b04700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff970107df0 0x7ff97019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff977b04700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff970108740 0x7ff97019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff977b04700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701134b0 con 0x7ff970108740 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff977b04700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff970113330 con 0x7ff9701034c0 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff977b04700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff970113630 con 0x7ff970107df0 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff9760a1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff970108740 0x7ff97019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff9760a1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff970108740 0x7ff97019c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52472/0 (socket says 172.21.15.46:52472) 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff9760a1700 1 -- 172.21.15.46:0/2053790262 learned_addr learned my addr 172.21.15.46:0/2053790262 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff9760a1700 1 -- 172.21.15.46:0/2053790262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff970107df0 msgr2=0x7ff97019c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff9760a1700 1 --2- 172.21.15.46:0/2053790262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff970107df0 0x7ff97019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff9760a1700 1 -- 172.21.15.46:0/2053790262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9701034c0 msgr2=0x7ff97019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff9760a1700 1 --2- 172.21.15.46:0/2053790262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9701034c0 0x7ff97019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.535+0000 7ff9760a1700 1 -- 172.21.15.46:0/2053790262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff970199b80 con 0x7ff970108740 2024-05-09T00:23:22.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.536+0000 7ff9760a1700 1 --2- 172.21.15.46:0/2053790262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff970108740 0x7ff97019c830 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff96800b970 tx=0x7ff96800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:22.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.536+0000 7ff966ffd700 1 -- 172.21.15.46:0/2053790262 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968019070 con 0x7ff970108740 2024-05-09T00:23:22.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.536+0000 7ff966ffd700 1 -- 172.21.15.46:0/2053790262 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff968014440 con 0x7ff970108740 2024-05-09T00:23:22.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.536+0000 7ff977b04700 1 -- 172.21.15.46:0/2053790262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff970199e70 con 0x7ff970108740 2024-05-09T00:23:22.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.536+0000 7ff966ffd700 1 -- 172.21.15.46:0/2053790262 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96800ce60 con 0x7ff970108740 2024-05-09T00:23:22.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.536+0000 7ff977b04700 1 -- 172.21.15.46:0/2053790262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97019a360 con 0x7ff970108740 2024-05-09T00:23:22.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.539+0000 7ff966ffd700 1 -- 172.21.15.46:0/2053790262 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff96801c070 con 0x7ff970108740 2024-05-09T00:23:22.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.540+0000 7ff966ffd700 1 --2- 172.21.15.46:0/2053790262 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff95c0b2880 0x7ff95c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.540+0000 7ff9758a0700 1 --2- 172.21.15.46:0/2053790262 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff95c0b2880 0x7ff95c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.540+0000 7ff966ffd700 1 -- 172.21.15.46:0/2053790262 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff9680e16d0 con 0x7ff970108740 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.540+0000 7ff98651ab80 1 Processor -- start 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.540+0000 7ff98651ab80 1 -- start start 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.540+0000 7ff9758a0700 1 --2- 172.21.15.46:0/2053790262 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff95c0b2880 0x7ff95c0b4d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff96c003800 tx=0x7ff96c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff98651ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc10450d00 0x55fc10559240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff98651ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 0x55fc10559780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff98651ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc10553400 0x55fc10553810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff98651ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fc104760a0 con 0x55fc103d64a0 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff98651ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fc1041b9a0 con 0x55fc10450d00 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff98651ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fc10472060 con 0x55fc10553400 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff97509f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 0x55fc10559780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff97509f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 0x55fc10559780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52482/0 (socket says 172.21.15.46:52482) 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff97509f700 1 -- 172.21.15.46:0/3407067982 learned_addr learned my addr 172.21.15.46:0/3407067982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff9760a1700 1 --2- 172.21.15.46:0/3407067982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc10553400 0x55fc10553810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff9758a0700 1 --2- 172.21.15.46:0/3407067982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc10450d00 0x55fc10559240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff97509f700 1 -- 172.21.15.46:0/3407067982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc10553400 msgr2=0x55fc10553810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff97509f700 1 --2- 172.21.15.46:0/3407067982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc10553400 0x55fc10553810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff97509f700 1 -- 172.21.15.46:0/3407067982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc10450d00 msgr2=0x55fc10559240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff97509f700 1 --2- 172.21.15.46:0/3407067982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc10450d00 0x55fc10559240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.541+0000 7ff97509f700 1 -- 172.21.15.46:0/3407067982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1040c500 con 0x55fc103d64a0 2024-05-09T00:23:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.542+0000 7ff97509f700 1 --2- 172.21.15.46:0/3407067982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 0x55fc10559780 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff960009820 tx=0x7ff960009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380112933cd195e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.542+0000 7ff95bfff700 1 -- 172.21.15.46:0/3407067982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960015020 con 0x55fc103d64a0 2024-05-09T00:23:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.542+0000 7ff95bfff700 1 -- 172.21.15.46:0/3407067982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff960011ce0 con 0x55fc103d64a0 2024-05-09T00:23:22.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.542+0000 7ff95bfff700 1 -- 172.21.15.46:0/3407067982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960010ce0 con 0x55fc103d64a0 2024-05-09T00:23:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.543+0000 7ff98651ab80 1 -- 172.21.15.46:0/3407067982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 msgr2=0x55fc10559780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.543+0000 7ff98651ab80 1 --2- 172.21.15.46:0/3407067982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 0x55fc10559780 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff960009820 tx=0x7ff960009b30 comp rx=0 tx=0).stop 2024-05-09T00:23:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.543+0000 7ff98651ab80 1 -- 172.21.15.46:0/3407067982 shutdown_connections 2024-05-09T00:23:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.543+0000 7ff98651ab80 1 --2- 172.21.15.46:0/3407067982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc10553400 0x55fc10553810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.543+0000 7ff98651ab80 1 --2- 172.21.15.46:0/3407067982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc10450d00 0x55fc10559240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.543+0000 7ff98651ab80 1 --2- 172.21.15.46:0/3407067982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 0x55fc10559780 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.543+0000 7ff98651ab80 1 -- 172.21.15.46:0/3407067982 >> 172.21.15.46:0/3407067982 conn(0x55fc103d9f70 msgr2=0x55fc1054e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.543+0000 7ff98651ab80 1 -- 172.21.15.46:0/3407067982 shutdown_connections 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.543+0000 7ff98651ab80 1 -- 172.21.15.46:0/3407067982 wait complete. 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.543+0000 7ff98651ab80 1 -- start start 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.543+0000 7ff98651ab80 1 Processor -- start 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff98651ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 0x55fc1055d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff98651ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc10450d00 0x55fc1055d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff98651ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc10553400 0x55fc1055dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff98651ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fc10554160 con 0x55fc103d64a0 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff98651ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fc10410b20 con 0x55fc10450d00 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff98651ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fc10560020 con 0x55fc10553400 2024-05-09T00:23:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff98651ab80 10 client.0 _ll_register_callbacks cb 0x55fc102e76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff9758a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 0x55fc1055d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff9758a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 0x55fc1055d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52490/0 (socket says 172.21.15.46:52490) 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff9758a0700 1 -- 172.21.15.46:0/2651388747 learned_addr learned my addr 172.21.15.46:0/2651388747 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff9760a1700 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc10553400 0x55fc1055dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff97509f700 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc10450d00 0x55fc1055d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff9758a0700 1 -- 172.21.15.46:0/2651388747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc10553400 msgr2=0x55fc1055dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff9758a0700 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc10553400 0x55fc1055dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff9758a0700 1 -- 172.21.15.46:0/2651388747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc10450d00 msgr2=0x55fc1055d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff9758a0700 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc10450d00 0x55fc1055d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff9758a0700 1 -- 172.21.15.46:0/2651388747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc102f1490 con 0x55fc103d64a0 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.544+0000 7ff9760a1700 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc10553400 0x55fc1055dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.545+0000 7ff9758a0700 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 0x55fc1055d050 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00bb20 tx=0x7ff96c00b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3f1dcc4a804e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:22.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.545+0000 7ff9597fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.545+0000 7ff9597fa700 1 -- 172.21.15.46:0/2651388747 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c02f430 con 0x55fc103d64a0 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.545+0000 7ff9597fa700 1 -- 172.21.15.46:0/2651388747 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff96c024420 con 0x55fc103d64a0 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.545+0000 7ff98651ab80 1 -- 172.21.15.46:0/2651388747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff960011ce0 con 0x55fc103d64a0 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.545+0000 7ff9597fa700 1 -- 172.21.15.46:0/2651388747 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff96c024620 con 0x55fc103d64a0 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.545+0000 7ff98651ab80 10 client.17406 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.545+0000 7ff98651ab80 1 -- 172.21.15.46:0/2651388747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc1056aa70 con 0x55fc103d64a0 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.545+0000 7ff9597fa700 1 -- 172.21.15.46:0/2651388747 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff960011ce0 con 0x55fc103d64a0 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.545+0000 7ff9597fa700 1 -- 172.21.15.46:0/2651388747 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff96c02f930 con 0x55fc103d64a0 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.546+0000 7ff98651ab80 10 client.17406 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.546+0000 7ff98651ab80 10 client.17406 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.546+0000 7ff98651ab80 20 client.17406 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.546+0000 7ff98651ab80 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55fc1056d100 0x55fc1056f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.546+0000 7ff98651ab80 4 client.17406 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:22.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.546+0000 7ff98651ab80 1 -- 172.21.15.46:0/2651388747 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc10483a80 con 0x55fc1056d100 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.546+0000 7ff97509f700 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55fc1056d100 0x55fc1056f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.546+0000 7ff97509f700 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55fc1056d100 0x55fc1056f520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=646734d7d5e13c06 server_cookie=187c64c93b889984 in_seq=0 out_seq=0 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.546+0000 7ff9597fa700 10 client.17406 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.547+0000 7ff9597fa700 1 -- 172.21.15.46:0/2651388747 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc10483a80 con 0x55fc1056d100 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.547+0000 7ff9597fa700 10 client.17406 handle_command_reply: tid=0 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.547+0000 7ff98651ab80 1 client.17406 shutdown 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.548+0000 7ff98651ab80 1 -- 172.21.15.46:0/2651388747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 msgr2=0x55fc1055d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.548+0000 7ff98651ab80 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 0x55fc1055d050 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00bb20 tx=0x7ff96c00b540 comp rx=0 tx=0).stop 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.548+0000 7ff98651ab80 1 -- 172.21.15.46:0/2651388747 shutdown_connections 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.548+0000 7ff98651ab80 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55fc1056d100 0x55fc1056f520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.548+0000 7ff98651ab80 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc10553400 0x55fc1055dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.548+0000 7ff98651ab80 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc103d64a0 0x55fc1055d050 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.548+0000 7ff9597fa700 0 client.17406 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.548+0000 7ff98651ab80 1 --2- 172.21.15.46:0/2651388747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc10450d00 0x55fc1055d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.548+0000 7ff98651ab80 1 -- 172.21.15.46:0/2651388747 >> 172.21.15.46:0/2651388747 conn(0x55fc103d9f70 msgr2=0x55fc10547ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.548+0000 7ff98651ab80 1 -- 172.21.15.46:0/2651388747 shutdown_connections 2024-05-09T00:23:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.548+0000 7ff98651ab80 1 -- 172.21.15.46:0/2651388747 wait complete. 2024-05-09T00:23:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.548+0000 7ff98651ab80 20 client.17406 trim_cache size 0 max 16384 2024-05-09T00:23:22.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.563+0000 7ff98651ab80 1 Processor -- start 2024-05-09T00:23:22.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.563+0000 7ff98651ab80 1 -- start start 2024-05-09T00:23:22.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff98651ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc103d9f70 0x55fc10558850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff98651ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc103d64a0 0x55fc1055d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff98651ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 0x55fc10547ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff98651ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701b3740 con 0x55fc10450d00 2024-05-09T00:23:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff98651ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701b0fd0 con 0x55fc103d9f70 2024-05-09T00:23:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff98651ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701aee00 con 0x55fc103d64a0 2024-05-09T00:23:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff9760a1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 0x55fc10547ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff9760a1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 0x55fc10547ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52498/0 (socket says 172.21.15.46:52498) 2024-05-09T00:23:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff9760a1700 1 -- 172.21.15.46:0/1118038141 learned_addr learned my addr 172.21.15.46:0/1118038141 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff97509f700 1 --2- 172.21.15.46:0/1118038141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc103d64a0 0x55fc1055d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff9758a0700 1 --2- 172.21.15.46:0/1118038141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc103d9f70 0x55fc10558850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff9760a1700 1 -- 172.21.15.46:0/1118038141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc103d64a0 msgr2=0x55fc1055d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff9760a1700 1 --2- 172.21.15.46:0/1118038141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc103d64a0 0x55fc1055d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff9760a1700 1 -- 172.21.15.46:0/1118038141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc103d9f70 msgr2=0x55fc10558850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff9760a1700 1 --2- 172.21.15.46:0/1118038141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc103d9f70 0x55fc10558850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff9760a1700 1 -- 172.21.15.46:0/1118038141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1054a9d0 con 0x55fc10450d00 2024-05-09T00:23:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff9758a0700 1 --2- 172.21.15.46:0/1118038141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc103d9f70 0x55fc10558850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:23:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff9760a1700 1 --2- 172.21.15.46:0/1118038141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 0x55fc10547ff0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff9680a5f20 tx=0x7ff96800e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868d1145149fa281 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.564+0000 7ff95b7fe700 1 -- 172.21.15.46:0/1118038141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9680a8cb0 con 0x55fc10450d00 2024-05-09T00:23:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.565+0000 7ff95b7fe700 1 -- 172.21.15.46:0/1118038141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9680ad060 con 0x55fc10450d00 2024-05-09T00:23:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.565+0000 7ff95b7fe700 1 -- 172.21.15.46:0/1118038141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9680a7f10 con 0x55fc10450d00 2024-05-09T00:23:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.565+0000 7ff98651ab80 1 -- 172.21.15.46:0/1118038141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 msgr2=0x55fc10547ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.565+0000 7ff98651ab80 1 --2- 172.21.15.46:0/1118038141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 0x55fc10547ff0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff9680a5f20 tx=0x7ff96800e040 comp rx=0 tx=0).stop 2024-05-09T00:23:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.565+0000 7ff98651ab80 1 -- 172.21.15.46:0/1118038141 shutdown_connections 2024-05-09T00:23:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.565+0000 7ff98651ab80 1 --2- 172.21.15.46:0/1118038141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 0x55fc10547ff0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.565+0000 7ff98651ab80 1 --2- 172.21.15.46:0/1118038141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc103d9f70 0x55fc10558850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.565+0000 7ff98651ab80 1 --2- 172.21.15.46:0/1118038141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc103d64a0 0x55fc1055d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.565+0000 7ff98651ab80 1 -- 172.21.15.46:0/1118038141 >> 172.21.15.46:0/1118038141 conn(0x55fc10553ae0 msgr2=0x55fc10558140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.565+0000 7ff98651ab80 1 -- 172.21.15.46:0/1118038141 shutdown_connections 2024-05-09T00:23:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.566+0000 7ff98651ab80 1 -- 172.21.15.46:0/1118038141 wait complete. 2024-05-09T00:23:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.566+0000 7ff98651ab80 1 -- start start 2024-05-09T00:23:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.566+0000 7ff98651ab80 1 Processor -- start 2024-05-09T00:23:22.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.566+0000 7ff98651ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc103d64a0 0x55fc105656f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.566+0000 7ff98651ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc103d9f70 0x55fc1055c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.566+0000 7ff98651ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 0x55fc1055ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff98651ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701af490 con 0x55fc10450d00 2024-05-09T00:23:22.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff98651ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701b0900 con 0x55fc103d64a0 2024-05-09T00:23:22.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff98651ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701afb60 con 0x55fc103d9f70 2024-05-09T00:23:22.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff98651ab80 10 client.0 _ll_register_callbacks cb 0x55fc102e76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:22.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9760a1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 0x55fc1055ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9760a1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 0x55fc1055ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52512/0 (socket says 172.21.15.46:52512) 2024-05-09T00:23:22.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9760a1700 1 -- 172.21.15.46:0/2530093251 learned_addr learned my addr 172.21.15.46:0/2530093251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:22.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff97509f700 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc103d9f70 0x55fc1055c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9758a0700 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc103d64a0 0x55fc105656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9760a1700 1 -- 172.21.15.46:0/2530093251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc103d9f70 msgr2=0x55fc1055c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:22.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9760a1700 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc103d9f70 0x55fc1055c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9760a1700 1 -- 172.21.15.46:0/2530093251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc103d64a0 msgr2=0x55fc105656f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9760a1700 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc103d64a0 0x55fc105656f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9760a1700 1 -- 172.21.15.46:0/2530093251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc10559c70 con 0x55fc10450d00 2024-05-09T00:23:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9758a0700 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc103d64a0 0x55fc105656f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9760a1700 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 0x55fc1055ce40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7ff96800b940 tx=0x7ff968004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32164e01e45e4941 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9597fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9597fa700 1 -- 172.21.15.46:0/2530093251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9680a8cb0 con 0x55fc10450d00 2024-05-09T00:23:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff9597fa700 1 -- 172.21.15.46:0/2530093251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7ff9680b1890 con 0x55fc10450d00 2024-05-09T00:23:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.567+0000 7ff98651ab80 1 -- 172.21.15.46:0/2530093251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9680ad060 con 0x55fc10450d00 2024-05-09T00:23:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.568+0000 7ff9597fa700 1 -- 172.21.15.46:0/2530093251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff96800c9b0 con 0x55fc10450d00 2024-05-09T00:23:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.568+0000 7ff98651ab80 10 client.17412 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.568+0000 7ff98651ab80 1 -- 172.21.15.46:0/2530093251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc105d75b0 con 0x55fc10450d00 2024-05-09T00:23:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.568+0000 7ff9597fa700 1 -- 172.21.15.46:0/2530093251 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9680ad060 con 0x55fc10450d00 2024-05-09T00:23:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.568+0000 7ff9597fa700 1 -- 172.21.15.46:0/2530093251 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff9680a7a10 con 0x55fc10450d00 2024-05-09T00:23:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.568+0000 7ff98651ab80 10 client.17412 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.568+0000 7ff98651ab80 10 client.17412 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:23:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.568+0000 7ff98651ab80 20 client.17412 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.568+0000 7ff98651ab80 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55fc10553400 0x55fc105d9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.568+0000 7ff98651ab80 4 client.17412 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:23:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.568+0000 7ff98651ab80 1 -- 172.21.15.46:0/2530093251 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc1047db20 con 0x55fc10553400 2024-05-09T00:23:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.568+0000 7ff9758a0700 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55fc10553400 0x55fc105d9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff9597fa700 10 client.17412 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:23:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff9758a0700 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55fc10553400 0x55fc105d9bc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7aa40adbe4fb7da3 server_cookie=b40f95b5061fab35 in_seq=0 out_seq=0 2024-05-09T00:23:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff9597fa700 1 -- 172.21.15.46:0/2530093251 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc1047db20 con 0x55fc10553400 2024-05-09T00:23:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff9597fa700 10 client.17412 handle_command_reply: tid=0 2024-05-09T00:23:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff98651ab80 1 client.17412 shutdown 2024-05-09T00:23:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff98651ab80 1 -- 172.21.15.46:0/2530093251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 msgr2=0x55fc1055ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff98651ab80 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 0x55fc1055ce40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7ff96800b940 tx=0x7ff968004ad0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff98651ab80 1 -- 172.21.15.46:0/2530093251 shutdown_connections 2024-05-09T00:23:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff98651ab80 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55fc10553400 0x55fc105d9bc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff98651ab80 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc10450d00 0x55fc1055ce40 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff98651ab80 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fc103d64a0 0x55fc105656f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff98651ab80 1 --2- 172.21.15.46:0/2530093251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc103d9f70 0x55fc1055c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff9597fa700 0 client.17412 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:23:22.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff98651ab80 1 -- 172.21.15.46:0/2530093251 >> 172.21.15.46:0/2530093251 conn(0x55fc10553ae0 msgr2=0x55fc10547ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:22.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff98651ab80 1 -- 172.21.15.46:0/2530093251 shutdown_connections 2024-05-09T00:23:22.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff98651ab80 1 -- 172.21.15.46:0/2530093251 wait complete. 2024-05-09T00:23:22.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.569+0000 7ff98651ab80 20 client.17412 trim_cache size 0 max 16384 2024-05-09T00:23:22.587 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:23:22.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.570+0000 7ff977b04700 1 -- 172.21.15.46:0/2053790262 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff95c0b2880 msgr2=0x7ff95c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.570+0000 7ff977b04700 1 --2- 172.21.15.46:0/2053790262 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff95c0b2880 0x7ff95c0b4d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff96c003800 tx=0x7ff96c009f10 comp rx=0 tx=0).stop 2024-05-09T00:23:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.570+0000 7ff977b04700 1 -- 172.21.15.46:0/2053790262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff970108740 msgr2=0x7ff97019c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.570+0000 7ff977b04700 1 --2- 172.21.15.46:0/2053790262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff970108740 0x7ff97019c830 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff96800b970 tx=0x7ff96800bd30 comp rx=0 tx=0).stop 2024-05-09T00:23:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.570+0000 7ff977b04700 1 -- 172.21.15.46:0/2053790262 shutdown_connections 2024-05-09T00:23:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.570+0000 7ff977b04700 1 --2- 172.21.15.46:0/2053790262 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7ff95c0b2880 0x7ff95c0b4d90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.570+0000 7ff977b04700 1 --2- 172.21.15.46:0/2053790262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff970108740 0x7ff97019c830 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.570+0000 7ff977b04700 1 --2- 172.21.15.46:0/2053790262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9701034c0 0x7ff97019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.570+0000 7ff977b04700 1 --2- 172.21.15.46:0/2053790262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff970107df0 0x7ff97019c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:22.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.570+0000 7ff977b04700 1 -- 172.21.15.46:0/2053790262 >> 172.21.15.46:0/2053790262 conn(0x7ff9700fedf0 msgr2=0x7ff9701096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.570+0000 7ff977b04700 1 -- 172.21.15.46:0/2053790262 shutdown_connections 2024-05-09T00:23:22.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:22.570+0000 7ff977b04700 1 -- 172.21.15.46:0/2053790262 wait complete. 2024-05-09T00:23:22.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:22 smithi046 ceph-mon[30151]: pgmap v736: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:23.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:22 smithi169 ceph-mon[34777]: pgmap v736: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:23.194 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:22 smithi142 ceph-mon[34533]: pgmap v736: 97 pgs: 97 active+clean; 7.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:23.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:23:23.628 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:25.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:24 smithi046 ceph-mon[30151]: pgmap v737: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:23:25.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:24 smithi169 ceph-mon[34777]: pgmap v737: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:23:25.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:24 smithi142 ceph-mon[34533]: pgmap v737: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:23:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.698+0000 7fef8a245700 1 -- 172.21.15.46:0/2749767109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef84108740 msgr2=0x7fef841100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.698+0000 7fef8a245700 1 --2- 172.21.15.46:0/2749767109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef84108740 0x7fef841100a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00b0d0 tx=0x7fef7c00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.698+0000 7fef8a245700 1 -- 172.21.15.46:0/2749767109 shutdown_connections 2024-05-09T00:23:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.698+0000 7fef8a245700 1 --2- 172.21.15.46:0/2749767109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef84108740 0x7fef841100a0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.698+0000 7fef8a245700 1 --2- 172.21.15.46:0/2749767109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef841034c0 0x7fef841038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.698+0000 7fef8a245700 1 --2- 172.21.15.46:0/2749767109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef84107df0 0x7fef84108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.698+0000 7fef8a245700 1 -- 172.21.15.46:0/2749767109 >> 172.21.15.46:0/2749767109 conn(0x7fef840fedc0 msgr2=0x7fef841011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.698+0000 7fef8a245700 1 -- 172.21.15.46:0/2749767109 shutdown_connections 2024-05-09T00:23:25.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.698+0000 7fef8a245700 1 -- 172.21.15.46:0/2749767109 wait complete. 2024-05-09T00:23:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.699+0000 7fef8a245700 1 Processor -- start 2024-05-09T00:23:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.699+0000 7fef8a245700 1 -- start start 2024-05-09T00:23:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.699+0000 7fef8a245700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef841034c0 0x7fef8419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.699+0000 7fef8a245700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef84107df0 0x7fef8419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.699+0000 7fef8a245700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef84108740 0x7fef8419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.699+0000 7fef8a245700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef84114470 con 0x7fef841034c0 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.699+0000 7fef8a245700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef841142f0 con 0x7fef84108740 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.699+0000 7fef8a245700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef841145f0 con 0x7fef84107df0 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.699+0000 7fef837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef841034c0 0x7fef8419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.699+0000 7fef837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef841034c0 0x7fef8419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52530/0 (socket says 172.21.15.46:52530) 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.699+0000 7fef837fe700 1 -- 172.21.15.46:0/2341689375 learned_addr learned my addr 172.21.15.46:0/2341689375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef7bfff700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef84107df0 0x7fef8419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef83fff700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef84108740 0x7fef8419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef837fe700 1 -- 172.21.15.46:0/2341689375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef84107df0 msgr2=0x7fef8419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef837fe700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef84107df0 0x7fef8419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef837fe700 1 -- 172.21.15.46:0/2341689375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef84108740 msgr2=0x7fef8419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef837fe700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef84108740 0x7fef8419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef837fe700 1 -- 172.21.15.46:0/2341689375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8419fc40 con 0x7fef841034c0 2024-05-09T00:23:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef83fff700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef84108740 0x7fef8419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef837fe700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef841034c0 0x7fef8419ae20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fef7400d950 tx=0x7fef7400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef817fa700 1 -- 172.21.15.46:0/2341689375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74016070 con 0x7fef841034c0 2024-05-09T00:23:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef8a245700 1 -- 172.21.15.46:0/2341689375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8419ff30 con 0x7fef841034c0 2024-05-09T00:23:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef817fa700 1 -- 172.21.15.46:0/2341689375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef74009df0 con 0x7fef841034c0 2024-05-09T00:23:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef817fa700 1 -- 172.21.15.46:0/2341689375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7400f3c0 con 0x7fef841034c0 2024-05-09T00:23:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.700+0000 7fef8a245700 1 -- 172.21.15.46:0/2341689375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef841a7900 con 0x7fef841034c0 2024-05-09T00:23:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.704+0000 7fef817fa700 1 -- 172.21.15.46:0/2341689375 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fef74019070 con 0x7fef841034c0 2024-05-09T00:23:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef98c5bb80 1 Processor -- start 2024-05-09T00:23:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef98c5bb80 1 -- start start 2024-05-09T00:23:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef817fa700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef700b2880 0x7fef700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef98c5bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd332b160 0x563dd33c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef98c5bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef817fa700 1 -- 172.21.15.46:0/2341689375 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fef740e1e50 con 0x7fef841034c0 2024-05-09T00:23:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef98c5bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd33c7400 0x563dd33c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef83fff700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef700b2880 0x7fef700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef98c5bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563dd32e0ff0 con 0x563dd32414a0 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef98c5bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563dd327f7f0 con 0x563dd332b160 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef98c5bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563dd32dcfb0 con 0x563dd33c7400 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef7bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef7bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33c32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52534/0 (socket says 172.21.15.46:52534) 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.705+0000 7fef7bfff700 1 -- 172.21.15.46:0/2581829612 learned_addr learned my addr 172.21.15.46:0/2581829612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef837fe700 1 --2- 172.21.15.46:0/2581829612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd332b160 0x563dd33c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef83fff700 1 --2- 172.21.15.46:0/2581829612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd33c7400 0x563dd33c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef7bfff700 1 -- 172.21.15.46:0/2581829612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd33c7400 msgr2=0x563dd33c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef7bfff700 1 --2- 172.21.15.46:0/2581829612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd33c7400 0x563dd33c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef7bfff700 1 -- 172.21.15.46:0/2581829612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd332b160 msgr2=0x563dd33c2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef7bfff700 1 --2- 172.21.15.46:0/2581829612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd332b160 0x563dd33c2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef7bfff700 1 -- 172.21.15.46:0/2581829612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dd3277370 con 0x563dd32414a0 2024-05-09T00:23:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef83fff700 1 --2- 172.21.15.46:0/2581829612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd33c7400 0x563dd33c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:23:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef83fff700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef700b2880 0x7fef700b4d90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00d010 tx=0x7fef7c009d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef7bfff700 1 --2- 172.21.15.46:0/2581829612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33c32f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00ac60 tx=0x7fef6c00af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0a0f3cecb76aa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef79ffb700 1 -- 172.21.15.46:0/2581829612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c00fe90 con 0x563dd32414a0 2024-05-09T00:23:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef79ffb700 1 -- 172.21.15.46:0/2581829612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef6c0044e0 con 0x563dd32414a0 2024-05-09T00:23:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.706+0000 7fef79ffb700 1 -- 172.21.15.46:0/2581829612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c018690 con 0x563dd32414a0 2024-05-09T00:23:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.707+0000 7fef98c5bb80 1 -- 172.21.15.46:0/2581829612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 msgr2=0x563dd33c32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.707+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/2581829612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33c32f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00ac60 tx=0x7fef6c00af70 comp rx=0 tx=0).stop 2024-05-09T00:23:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.707+0000 7fef98c5bb80 1 -- 172.21.15.46:0/2581829612 shutdown_connections 2024-05-09T00:23:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.707+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/2581829612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd33c7400 0x563dd33c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.707+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/2581829612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd332b160 0x563dd33c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.707+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/2581829612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33c32f0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.707+0000 7fef98c5bb80 1 -- 172.21.15.46:0/2581829612 >> 172.21.15.46:0/2581829612 conn(0x563dd32a16a0 msgr2=0x563dd33b9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.707+0000 7fef98c5bb80 1 -- 172.21.15.46:0/2581829612 shutdown_connections 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.707+0000 7fef98c5bb80 1 -- 172.21.15.46:0/2581829612 wait complete. 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.708+0000 7fef98c5bb80 1 -- start start 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.708+0000 7fef98c5bb80 1 Processor -- start 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.708+0000 7fef98c5bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.708+0000 7fef98c5bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd332b160 0x563dd33ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.708+0000 7fef98c5bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd33c7400 0x563dd33cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.708+0000 7fef98c5bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563dd315cba0 con 0x563dd32414a0 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.708+0000 7fef98c5bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563dd329b150 con 0x563dd33c7400 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.708+0000 7fef98c5bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563dd33c81b0 con 0x563dd332b160 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.708+0000 7fef98c5bb80 10 client.0 _ll_register_callbacks cb 0x563dd31526c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.708+0000 7fef837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.708+0000 7fef837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52548/0 (socket says 172.21.15.46:52548) 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.708+0000 7fef837fe700 1 -- 172.21.15.46:0/1693587791 learned_addr learned my addr 172.21.15.46:0/1693587791 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef83fff700 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd33c7400 0x563dd33cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef7bfff700 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd332b160 0x563dd33ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef837fe700 1 -- 172.21.15.46:0/1693587791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd332b160 msgr2=0x563dd33ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef837fe700 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd332b160 0x563dd33ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef837fe700 1 -- 172.21.15.46:0/1693587791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd33c7400 msgr2=0x563dd33cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef837fe700 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd33c7400 0x563dd33cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef837fe700 1 -- 172.21.15.46:0/1693587791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dd315c490 con 0x563dd32414a0 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef837fe700 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33cc5b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fef7400b2f0 tx=0x7fef740a9030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb5e9678d08d1ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef62ffd700 1 -- 172.21.15.46:0/1693587791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74020040 con 0x563dd32414a0 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef98c5bb80 1 -- 172.21.15.46:0/1693587791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef6c0044e0 con 0x563dd32414a0 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef62ffd700 1 -- 172.21.15.46:0/1693587791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fef740b1e00 con 0x563dd32414a0 2024-05-09T00:23:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef62ffd700 1 -- 172.21.15.46:0/1693587791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef740ad7e0 con 0x563dd32414a0 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef98c5bb80 10 client.17424 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.709+0000 7fef98c5bb80 1 -- 172.21.15.46:0/1693587791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563dd33d9e10 con 0x563dd32414a0 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.710+0000 7fef62ffd700 1 -- 172.21.15.46:0/1693587791 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef6c0044e0 con 0x563dd32414a0 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.710+0000 7fef62ffd700 1 -- 172.21.15.46:0/1693587791 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fef74009940 con 0x563dd32414a0 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.710+0000 7fef98c5bb80 10 client.17424 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.710+0000 7fef98c5bb80 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.710+0000 7fef98c5bb80 20 client.17424 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.710+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dd33dc450 0x563dd33de870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.710+0000 7fef98c5bb80 4 client.17424 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.710+0000 7fef98c5bb80 1 -- 172.21.15.46:0/1693587791 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dd32ee9d0 con 0x563dd33dc450 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.710+0000 7fef7bfff700 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dd33dc450 0x563dd33de870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.711+0000 7fef7bfff700 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dd33dc450 0x563dd33de870 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d01b230a0c1267 server_cookie=3de6523c40a97dc8 in_seq=0 out_seq=0 2024-05-09T00:23:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.711+0000 7fef62ffd700 10 client.17424 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.711+0000 7fef62ffd700 1 -- 172.21.15.46:0/1693587791 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563dd32ee9d0 con 0x563dd33dc450 2024-05-09T00:23:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.711+0000 7fef62ffd700 10 client.17424 handle_command_reply: tid=0 2024-05-09T00:23:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.711+0000 7fef98c5bb80 1 client.17424 shutdown 2024-05-09T00:23:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.712+0000 7fef98c5bb80 1 -- 172.21.15.46:0/1693587791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 msgr2=0x563dd33cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.712+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33cc5b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fef7400b2f0 tx=0x7fef740a9030 comp rx=0 tx=0).stop 2024-05-09T00:23:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.712+0000 7fef98c5bb80 1 -- 172.21.15.46:0/1693587791 shutdown_connections 2024-05-09T00:23:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.712+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dd33dc450 0x563dd33de870 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.712+0000 7fef62ffd700 0 client.17424 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.712+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd33c7400 0x563dd33cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.712+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33cc5b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.712+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/1693587791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd332b160 0x563dd33ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.712+0000 7fef98c5bb80 1 -- 172.21.15.46:0/1693587791 >> 172.21.15.46:0/1693587791 conn(0x563dd32a16a0 msgr2=0x563dd33b91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.712+0000 7fef98c5bb80 1 -- 172.21.15.46:0/1693587791 shutdown_connections 2024-05-09T00:23:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.712+0000 7fef98c5bb80 1 -- 172.21.15.46:0/1693587791 wait complete. 2024-05-09T00:23:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.712+0000 7fef98c5bb80 20 client.17424 trim_cache size 0 max 16384 2024-05-09T00:23:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.727+0000 7fef98c5bb80 1 Processor -- start 2024-05-09T00:23:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.727+0000 7fef98c5bb80 1 -- start start 2024-05-09T00:23:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.727+0000 7fef98c5bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.727+0000 7fef98c5bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd332b160 0x563dd33d8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.727+0000 7fef98c5bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd33c7400 0x563dd33ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.727+0000 7fef98c5bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef841af230 con 0x563dd32414a0 2024-05-09T00:23:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.727+0000 7fef98c5bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef841acac0 con 0x563dd33c7400 2024-05-09T00:23:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.727+0000 7fef98c5bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef841aa8f0 con 0x563dd332b160 2024-05-09T00:23:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.727+0000 7fef837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.727+0000 7fef837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52562/0 (socket says 172.21.15.46:52562) 2024-05-09T00:23:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.727+0000 7fef837fe700 1 -- 172.21.15.46:0/2140348787 learned_addr learned my addr 172.21.15.46:0/2140348787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.728+0000 7fef83fff700 1 --2- 172.21.15.46:0/2140348787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd33c7400 0x563dd33ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.728+0000 7fef7bfff700 1 --2- 172.21.15.46:0/2140348787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd332b160 0x563dd33d8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.728+0000 7fef837fe700 1 -- 172.21.15.46:0/2140348787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd332b160 msgr2=0x563dd33d8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.728+0000 7fef837fe700 1 --2- 172.21.15.46:0/2140348787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd332b160 0x563dd33d8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.728+0000 7fef837fe700 1 -- 172.21.15.46:0/2140348787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd33c7400 msgr2=0x563dd33ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.728+0000 7fef837fe700 1 --2- 172.21.15.46:0/2140348787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd33c7400 0x563dd33ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.728+0000 7fef837fe700 1 -- 172.21.15.46:0/2140348787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dd33b2f60 con 0x563dd32414a0 2024-05-09T00:23:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.728+0000 7fef837fe700 1 --2- 172.21.15.46:0/2140348787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33cce70 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fef740a93c0 tx=0x7fef740a90a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27a90ef2f9bec88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.728+0000 7fef797fa700 1 -- 172.21.15.46:0/2140348787 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef740c7070 con 0x563dd32414a0 2024-05-09T00:23:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.728+0000 7fef797fa700 1 -- 172.21.15.46:0/2140348787 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef740b2290 con 0x563dd32414a0 2024-05-09T00:23:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.728+0000 7fef797fa700 1 -- 172.21.15.46:0/2140348787 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74020070 con 0x563dd32414a0 2024-05-09T00:23:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.729+0000 7fef98c5bb80 1 -- 172.21.15.46:0/2140348787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 msgr2=0x563dd33cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.729+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/2140348787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33cce70 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fef740a93c0 tx=0x7fef740a90a0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.729+0000 7fef98c5bb80 1 -- 172.21.15.46:0/2140348787 shutdown_connections 2024-05-09T00:23:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.729+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/2140348787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd33c7400 0x563dd33ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.729+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/2140348787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd32414a0 0x563dd33cce70 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.729+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/2140348787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd332b160 0x563dd33d8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.729+0000 7fef98c5bb80 1 -- 172.21.15.46:0/2140348787 >> 172.21.15.46:0/2140348787 conn(0x563dd32a16a0 msgr2=0x563dd33cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.729+0000 7fef98c5bb80 1 -- 172.21.15.46:0/2140348787 shutdown_connections 2024-05-09T00:23:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.729+0000 7fef98c5bb80 1 -- 172.21.15.46:0/2140348787 wait complete. 2024-05-09T00:23:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.729+0000 7fef98c5bb80 1 -- start start 2024-05-09T00:23:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.729+0000 7fef98c5bb80 1 Processor -- start 2024-05-09T00:23:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef98c5bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd32414a0 0x563dd33c9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef98c5bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd332b160 0x563dd33c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef98c5bb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd33c7400 0x563dd33c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef98c5bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef841aaf80 con 0x563dd332b160 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef98c5bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef841ac3f0 con 0x563dd32414a0 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef98c5bb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef841ab650 con 0x563dd33c7400 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef98c5bb80 10 client.0 _ll_register_callbacks cb 0x563dd31526c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef7bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd332b160 0x563dd33c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef7bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd332b160 0x563dd33c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52564/0 (socket says 172.21.15.46:52564) 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef7bfff700 1 -- 172.21.15.46:0/3171411825 learned_addr learned my addr 172.21.15.46:0/3171411825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef837fe700 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd32414a0 0x563dd33c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef83fff700 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd33c7400 0x563dd33c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef7bfff700 1 -- 172.21.15.46:0/3171411825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd33c7400 msgr2=0x563dd33c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef7bfff700 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd33c7400 0x563dd33c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef7bfff700 1 -- 172.21.15.46:0/3171411825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd32414a0 msgr2=0x563dd33c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef7bfff700 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd32414a0 0x563dd33c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.730+0000 7fef7bfff700 1 -- 172.21.15.46:0/3171411825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dd33d5210 con 0x563dd332b160 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef7bfff700 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd332b160 0x563dd33c9860 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x563dd33bac40 tx=0x7fef6c00a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f657f17236bbfc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef62ffd700 1 -- 172.21.15.46:0/3171411825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c005610 con 0x563dd332b160 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef62ffd700 1 -- 172.21.15.46:0/3171411825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fef6c004710 con 0x563dd332b160 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef98c5bb80 1 -- 172.21.15.46:0/3171411825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef740b2290 con 0x563dd332b160 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef62ffd700 1 -- 172.21.15.46:0/3171411825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef6c03a050 con 0x563dd332b160 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef98c5bb80 10 client.17430 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef98c5bb80 1 -- 172.21.15.46:0/3171411825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563dd3446c20 con 0x563dd332b160 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef62ffd700 1 -- 172.21.15.46:0/3171411825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef740b2290 con 0x563dd332b160 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef62ffd700 1 -- 172.21.15.46:0/3171411825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fef6c02ace0 con 0x563dd332b160 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef98c5bb80 10 client.17430 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef98c5bb80 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef98c5bb80 20 client.17430 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dd3447080 0x563dd34494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef98c5bb80 4 client.17430 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.731+0000 7fef98c5bb80 1 -- 172.21.15.46:0/3171411825 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563dd32e8a70 con 0x563dd3447080 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.732+0000 7fef837fe700 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dd3447080 0x563dd34494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.732+0000 7fef62ffd700 10 client.17430 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.732+0000 7fef837fe700 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dd3447080 0x563dd34494a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=248364924a7270f4 server_cookie=403918adc8233320 in_seq=0 out_seq=0 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.732+0000 7fef62ffd700 1 -- 172.21.15.46:0/3171411825 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563dd32e8a70 con 0x563dd3447080 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.732+0000 7fef62ffd700 10 client.17430 handle_command_reply: tid=0 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.732+0000 7fef98c5bb80 1 client.17430 shutdown 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef98c5bb80 1 -- 172.21.15.46:0/3171411825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd332b160 msgr2=0x563dd33c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd332b160 0x563dd33c9860 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x563dd33bac40 tx=0x7fef6c00a730 comp rx=0 tx=0).stop 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef98c5bb80 1 -- 172.21.15.46:0/3171411825 shutdown_connections 2024-05-09T00:23:25.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x563dd3447080 0x563dd34494a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dd33c7400 0x563dd33c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dd32414a0 0x563dd33c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef98c5bb80 1 --2- 172.21.15.46:0/3171411825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563dd332b160 0x563dd33c9860 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef98c5bb80 1 -- 172.21.15.46:0/3171411825 >> 172.21.15.46:0/3171411825 conn(0x563dd32a16a0 msgr2=0x563dd33b88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef62ffd700 0 client.17430 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef98c5bb80 1 -- 172.21.15.46:0/3171411825 shutdown_connections 2024-05-09T00:23:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef98c5bb80 1 -- 172.21.15.46:0/3171411825 wait complete. 2024-05-09T00:23:25.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef98c5bb80 20 client.17430 trim_cache size 0 max 16384 2024-05-09T00:23:25.739 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:23:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef8a245700 1 -- 172.21.15.46:0/2341689375 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef700b2880 msgr2=0x7fef700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef8a245700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef700b2880 0x7fef700b4d90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00d010 tx=0x7fef7c009d90 comp rx=0 tx=0).stop 2024-05-09T00:23:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef8a245700 1 -- 172.21.15.46:0/2341689375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef841034c0 msgr2=0x7fef8419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef8a245700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef841034c0 0x7fef8419ae20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fef7400d950 tx=0x7fef7400dc60 comp rx=0 tx=0).stop 2024-05-09T00:23:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef8a245700 1 -- 172.21.15.46:0/2341689375 shutdown_connections 2024-05-09T00:23:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef8a245700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fef700b2880 0x7fef700b4d90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef8a245700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef84108740 0x7fef8419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef8a245700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef841034c0 0x7fef8419ae20 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef8a245700 1 --2- 172.21.15.46:0/2341689375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef84107df0 0x7fef8419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:25.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.733+0000 7fef8a245700 1 -- 172.21.15.46:0/2341689375 >> 172.21.15.46:0/2341689375 conn(0x7fef840fedc0 msgr2=0x7fef84107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:25.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.734+0000 7fef8a245700 1 -- 172.21.15.46:0/2341689375 shutdown_connections 2024-05-09T00:23:25.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:25.734+0000 7fef8a245700 1 -- 172.21.15.46:0/2341689375 wait complete. 2024-05-09T00:23:27.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:26 smithi046 ceph-mon[30151]: pgmap v738: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-05-09T00:23:27.109 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:26 smithi169 ceph-mon[34777]: pgmap v738: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-05-09T00:23:27.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:26 smithi142 ceph-mon[34533]: pgmap v738: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-05-09T00:23:27.513 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T00:23:27.513 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T00:23:27.777 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:29.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:28 smithi046 ceph-mon[30151]: pgmap v739: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:23:29.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:28 smithi169 ceph-mon[34777]: pgmap v739: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:23:29.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:28 smithi142 ceph-mon[34533]: pgmap v739: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:23:29.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.631+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3367511212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d88108f10 msgr2=0x7f4d88109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:29.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.631+0000 7f4d8e7e8700 1 --2- 172.21.15.46:0/3367511212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d88108f10 0x7f4d88109320 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4d700099c0 tx=0x7f4d70009cd0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.631+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3367511212 shutdown_connections 2024-05-09T00:23:29.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.631+0000 7f4d8e7e8700 1 --2- 172.21.15.46:0/3367511212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d88109860 0x7f4d881111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.631+0000 7f4d8e7e8700 1 --2- 172.21.15.46:0/3367511212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d881045e0 0x7f4d881049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.631+0000 7f4d8e7e8700 1 --2- 172.21.15.46:0/3367511212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d88108f10 0x7f4d88109320 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.631+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3367511212 >> 172.21.15.46:0/3367511212 conn(0x7f4d880ffe70 msgr2=0x7f4d881022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.631+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3367511212 shutdown_connections 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.631+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3367511212 wait complete. 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.632+0000 7f4d8e7e8700 1 Processor -- start 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.632+0000 7f4d8e7e8700 1 -- start start 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.632+0000 7f4d8e7e8700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d881045e0 0x7f4d88075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.632+0000 7f4d8e7e8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d88108f10 0x7f4d88076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.632+0000 7f4d8e7e8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d88109860 0x7f4d88076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.632+0000 7f4d8e7e8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881144e0 con 0x7f4d88108f10 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.632+0000 7f4d8e7e8700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88114360 con 0x7f4d881045e0 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.632+0000 7f4d8e7e8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88114660 con 0x7f4d88109860 2024-05-09T00:23:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.632+0000 7f4d7ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d88108f10 0x7f4d88076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d7ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d88108f10 0x7f4d88076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37168/0 (socket says 172.21.15.46:37168) 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d7ffff700 1 -- 172.21.15.46:0/3245505155 learned_addr learned my addr 172.21.15.46:0/3245505155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d87fff700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d881045e0 0x7f4d88075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d8cd85700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d88109860 0x7f4d88076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d7ffff700 1 -- 172.21.15.46:0/3245505155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d88109860 msgr2=0x7f4d88076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d88109860 0x7f4d88076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d7ffff700 1 -- 172.21.15.46:0/3245505155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d881045e0 msgr2=0x7f4d88075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d881045e0 0x7f4d88075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d7ffff700 1 -- 172.21.15.46:0/3245505155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d881a9250 con 0x7f4d88108f10 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d8cd85700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d88109860 0x7f4d88076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d88108f10 0x7f4d88076000 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4d70006010 tx=0x7f4d700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d85ffb700 1 -- 172.21.15.46:0/3245505155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70027070 con 0x7f4d88108f10 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d85ffb700 1 -- 172.21.15.46:0/3245505155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d70003f20 con 0x7f4d88108f10 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3245505155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d881a94e0 con 0x7f4d88108f10 2024-05-09T00:23:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d85ffb700 1 -- 172.21.15.46:0/3245505155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70022cd0 con 0x7f4d88108f10 2024-05-09T00:23:29.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.633+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3245505155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d881a9930 con 0x7f4d88108f10 2024-05-09T00:23:29.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.636+0000 7f4d85ffb700 1 -- 172.21.15.46:0/3245505155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4d7002a070 con 0x7f4d88108f10 2024-05-09T00:23:29.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.637+0000 7f4d85ffb700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4d740b2880 0x7f4d740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.637+0000 7f4d87fff700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4d740b2880 0x7f4d740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.637+0000 7f4d85ffb700 1 -- 172.21.15.46:0/3245505155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4d700e3110 con 0x7f4d88108f10 2024-05-09T00:23:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.637+0000 7f4d9d1feb80 1 Processor -- start 2024-05-09T00:23:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.637+0000 7f4d9d1feb80 1 -- start start 2024-05-09T00:23:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.637+0000 7f4d87fff700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4d740b2880 0x7f4d740b4d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009a10 tx=0x7f4d78009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.637+0000 7f4d9d1feb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.637+0000 7f4d9d1feb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd2401520 0x557dd2584800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.637+0000 7f4d9d1feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd257e480 0x557dd257e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d9d1feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557dd24a1120 con 0x557dd257e480 2024-05-09T00:23:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d9d1feb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557dd2446a20 con 0x557dd247bd80 2024-05-09T00:23:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d9d1feb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557dd249d0e0 con 0x557dd2401520 2024-05-09T00:23:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d8cd85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd257e480 0x557dd257e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d8cd85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd257e480 0x557dd257e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37174/0 (socket says 172.21.15.46:37174) 2024-05-09T00:23:29.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d8cd85700 1 -- 172.21.15.46:0/4045412639 learned_addr learned my addr 172.21.15.46:0/4045412639 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:29.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d87fff700 1 --2- 172.21.15.46:0/4045412639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/4045412639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd2401520 0x557dd2584800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d8cd85700 1 -- 172.21.15.46:0/4045412639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd2401520 msgr2=0x557dd2584800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:29.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d8cd85700 1 --2- 172.21.15.46:0/4045412639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd2401520 0x557dd2584800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d8cd85700 1 -- 172.21.15.46:0/4045412639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 msgr2=0x557dd25842c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d8cd85700 1 --2- 172.21.15.46:0/4045412639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25842c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d8cd85700 1 -- 172.21.15.46:0/4045412639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dd2437580 con 0x557dd257e480 2024-05-09T00:23:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d87fff700 1 --2- 172.21.15.46:0/4045412639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25842c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:23:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d8cd85700 1 --2- 172.21.15.46:0/4045412639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd257e480 0x557dd257e890 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009e60 tx=0x7f4d800077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e82cf5361e9623d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.638+0000 7f4d7e7fc700 1 -- 172.21.15.46:0/4045412639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80014610 con 0x557dd257e480 2024-05-09T00:23:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.639+0000 7f4d7e7fc700 1 -- 172.21.15.46:0/4045412639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d80019070 con 0x557dd257e480 2024-05-09T00:23:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.639+0000 7f4d7e7fc700 1 -- 172.21.15.46:0/4045412639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80013940 con 0x557dd257e480 2024-05-09T00:23:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.639+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/4045412639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd257e480 msgr2=0x557dd257e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.639+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/4045412639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd257e480 0x557dd257e890 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009e60 tx=0x7f4d800077e0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.639+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/4045412639 shutdown_connections 2024-05-09T00:23:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.639+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/4045412639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd257e480 0x557dd257e890 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.639+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/4045412639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25842c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.639+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/4045412639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd2401520 0x557dd2584800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.639+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/4045412639 >> 172.21.15.46:0/4045412639 conn(0x557dd2404ff0 msgr2=0x557dd25792a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.639+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/4045412639 shutdown_connections 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.639+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/4045412639 wait complete. 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.640+0000 7f4d9d1feb80 1 -- start start 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.640+0000 7f4d9d1feb80 1 Processor -- start 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.640+0000 7f4d9d1feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd2588190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.640+0000 7f4d9d1feb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25886d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.640+0000 7f4d9d1feb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 0x557dd2588c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.640+0000 7f4d9d1feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557dd257f1e0 con 0x557dd2401520 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.640+0000 7f4d9d1feb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557dd243bba0 con 0x557dd247bd80 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.640+0000 7f4d9d1feb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557dd258b0a0 con 0x557dd257e480 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d9d1feb80 10 client.0 _ll_register_callbacks cb 0x557dd2312740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d87fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd2588190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d87fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd2588190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37178/0 (socket says 172.21.15.46:37178) 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d87fff700 1 -- 172.21.15.46:0/508177684 learned_addr learned my addr 172.21.15.46:0/508177684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25886d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d8cd85700 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 0x557dd2588c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d87fff700 1 -- 172.21.15.46:0/508177684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 msgr2=0x557dd2588c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d87fff700 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 0x557dd2588c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d87fff700 1 -- 172.21.15.46:0/508177684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 msgr2=0x557dd25886d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d87fff700 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25886d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d87fff700 1 -- 172.21.15.46:0/508177684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dd231c510 con 0x557dd2401520 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d87fff700 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd2588190 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4d780154f0 tx=0x7f4d78004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56becef837125ce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d677fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d677fe700 1 -- 172.21.15.46:0/508177684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78025070 con 0x557dd2401520 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/508177684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d80019070 con 0x557dd2401520 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.641+0000 7f4d677fe700 1 -- 172.21.15.46:0/508177684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4d78015d20 con 0x557dd2401520 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.642+0000 7f4d677fe700 1 -- 172.21.15.46:0/508177684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d7801b040 con 0x557dd2401520 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.642+0000 7f4d9d1feb80 10 client.17442 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.642+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/508177684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557dd2595c10 con 0x557dd2401520 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.642+0000 7f4d677fe700 1 -- 172.21.15.46:0/508177684 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d80019070 con 0x557dd2401520 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.642+0000 7f4d677fe700 1 -- 172.21.15.46:0/508177684 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4d78006750 con 0x557dd2401520 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.642+0000 7f4d9d1feb80 10 client.17442 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.642+0000 7f4d9d1feb80 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.642+0000 7f4d9d1feb80 20 client.17442 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.642+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557dd25981f0 0x557dd259a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.642+0000 7f4d9d1feb80 4 client.17442 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.642+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/508177684 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dd24aeb00 con 0x557dd25981f0 2024-05-09T00:23:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.643+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557dd25981f0 0x557dd259a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.643+0000 7f4d677fe700 10 client.17442 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.643+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557dd25981f0 0x557dd259a610 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb07acd4ddacb7ca server_cookie=32f751a0ca43b31e in_seq=0 out_seq=0 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d677fe700 1 -- 172.21.15.46:0/508177684 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557dd24aeb00 con 0x557dd25981f0 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d677fe700 10 client.17442 handle_command_reply: tid=0 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d9d1feb80 1 client.17442 shutdown 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/508177684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 msgr2=0x557dd2588190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd2588190 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4d780154f0 tx=0x7f4d78004860 comp rx=0 tx=0).stop 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/508177684 shutdown_connections 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557dd25981f0 0x557dd259a610 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 0x557dd2588c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd2588190 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/508177684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/508177684 >> 172.21.15.46:0/508177684 conn(0x557dd2404ff0 msgr2=0x557dd2573070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d677fe700 0 client.17442 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:29.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/508177684 shutdown_connections 2024-05-09T00:23:29.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/508177684 wait complete. 2024-05-09T00:23:29.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.644+0000 7f4d9d1feb80 20 client.17442 trim_cache size 0 max 16384 2024-05-09T00:23:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.666+0000 7f4d9d1feb80 1 Processor -- start 2024-05-09T00:23:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.666+0000 7f4d9d1feb80 1 -- start start 2024-05-09T00:23:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.666+0000 7f4d9d1feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd257dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.666+0000 7f4d9d1feb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd2573930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.666+0000 7f4d9d1feb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 0x557dd2582460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.666+0000 7f4d9d1feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881b3690 con 0x557dd2401520 2024-05-09T00:23:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.666+0000 7f4d9d1feb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881b0f20 con 0x557dd247bd80 2024-05-09T00:23:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.666+0000 7f4d9d1feb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881aed50 con 0x557dd257e480 2024-05-09T00:23:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.666+0000 7f4d87fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd257dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.666+0000 7f4d87fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd257dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37190/0 (socket says 172.21.15.46:37190) 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.666+0000 7f4d87fff700 1 -- 172.21.15.46:0/2447992068 learned_addr learned my addr 172.21.15.46:0/2447992068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.667+0000 7f4d8cd85700 1 --2- 172.21.15.46:0/2447992068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 0x557dd2582460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.667+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/2447992068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd2573930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.667+0000 7f4d87fff700 1 -- 172.21.15.46:0/2447992068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 msgr2=0x557dd2582460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.667+0000 7f4d87fff700 1 --2- 172.21.15.46:0/2447992068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 0x557dd2582460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.667+0000 7f4d87fff700 1 -- 172.21.15.46:0/2447992068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 msgr2=0x557dd2573930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.667+0000 7f4d87fff700 1 --2- 172.21.15.46:0/2447992068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd2573930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.667+0000 7f4d87fff700 1 -- 172.21.15.46:0/2447992068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dd2575a50 con 0x557dd2401520 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.667+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/2447992068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd2573930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.667+0000 7f4d87fff700 1 --2- 172.21.15.46:0/2447992068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd257dea0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e530 tx=0x7f4d78015960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143067309acbfbe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.667+0000 7f4d7dffb700 1 -- 172.21.15.46:0/2447992068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7803a070 con 0x557dd2401520 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.667+0000 7f4d7dffb700 1 -- 172.21.15.46:0/2447992068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d78021900 con 0x557dd2401520 2024-05-09T00:23:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.667+0000 7f4d7dffb700 1 -- 172.21.15.46:0/2447992068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78025040 con 0x557dd2401520 2024-05-09T00:23:29.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.668+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/2447992068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 msgr2=0x557dd257dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:29.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.668+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/2447992068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd257dea0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e530 tx=0x7f4d78015960 comp rx=0 tx=0).stop 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.668+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/2447992068 shutdown_connections 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.668+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/2447992068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 0x557dd2582460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.668+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/2447992068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd257dea0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.668+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/2447992068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd2573930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.668+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/2447992068 >> 172.21.15.46:0/2447992068 conn(0x557dd2404ff0 msgr2=0x557dd2581d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.668+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/2447992068 shutdown_connections 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.668+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/2447992068 wait complete. 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d9d1feb80 1 -- start start 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d9d1feb80 1 Processor -- start 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d9d1feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd25f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d9d1feb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d9d1feb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 0x557dd25fb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d9d1feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881af3e0 con 0x557dd2401520 2024-05-09T00:23:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d9d1feb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881b0850 con 0x557dd247bd80 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d9d1feb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881afab0 con 0x557dd257e480 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d9d1feb80 10 client.0 _ll_register_callbacks cb 0x557dd2312740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d87fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd25f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d87fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd25f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37192/0 (socket says 172.21.15.46:37192) 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d87fff700 1 -- 172.21.15.46:0/3781321529 learned_addr learned my addr 172.21.15.46:0/3781321529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.669+0000 7f4d8cd85700 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 0x557dd25fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d87fff700 1 -- 172.21.15.46:0/3781321529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 msgr2=0x557dd25fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d87fff700 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 0x557dd25fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d87fff700 1 -- 172.21.15.46:0/3781321529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 msgr2=0x557dd25f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d87fff700 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d87fff700 1 -- 172.21.15.46:0/3781321529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dd257a140 con 0x557dd2401520 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d87fff700 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd25f6b40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f4d78021600 tx=0x7f4d78005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c942039af4f0babc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d677fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d677fe700 1 -- 172.21.15.46:0/3781321529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7803a050 con 0x557dd2401520 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d677fe700 1 -- 172.21.15.46:0/3781321529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4d78006230 con 0x557dd2401520 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/3781321529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d78021900 con 0x557dd2401520 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d677fe700 1 -- 172.21.15.46:0/3781321529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d78025040 con 0x557dd2401520 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d9d1feb80 10 client.17448 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/3781321529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557dd2603010 con 0x557dd2401520 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.670+0000 7f4d677fe700 1 -- 172.21.15.46:0/3781321529 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d78021900 con 0x557dd2401520 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.671+0000 7f4d677fe700 1 -- 172.21.15.46:0/3781321529 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4d78046030 con 0x557dd2401520 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.671+0000 7f4d9d1feb80 10 client.17448 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.671+0000 7f4d9d1feb80 10 client.17448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.671+0000 7f4d9d1feb80 20 client.17448 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.671+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557dd2603600 0x557dd2605a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.671+0000 7f4d9d1feb80 4 client.17448 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.671+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/3781321529 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557dd24a8ba0 con 0x557dd2603600 2024-05-09T00:23:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.671+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557dd2603600 0x557dd2605a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:29.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.671+0000 7f4d677fe700 10 client.17448 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:29.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.671+0000 7f4d7ffff700 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557dd2603600 0x557dd2605a20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=136cd734cf96a9e6 server_cookie=dfcda9ee6f6ba07f in_seq=0 out_seq=0 2024-05-09T00:23:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.672+0000 7f4d677fe700 1 -- 172.21.15.46:0/3781321529 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557dd24a8ba0 con 0x557dd2603600 2024-05-09T00:23:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.672+0000 7f4d677fe700 10 client.17448 handle_command_reply: tid=0 2024-05-09T00:23:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.672+0000 7f4d9d1feb80 1 client.17448 shutdown 2024-05-09T00:23:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/3781321529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 msgr2=0x557dd25f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd25f6b40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f4d78021600 tx=0x7f4d78005da0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/3781321529 shutdown_connections 2024-05-09T00:23:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x557dd2603600 0x557dd2605a20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d677fe700 0 client.17448 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557dd257e480 0x557dd25fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dd2401520 0x557dd25f6b40 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d9d1feb80 1 --2- 172.21.15.46:0/3781321529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557dd247bd80 0x557dd25f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/3781321529 >> 172.21.15.46:0/3781321529 conn(0x557dd2404ff0 msgr2=0x557dd257dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/3781321529 shutdown_connections 2024-05-09T00:23:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d9d1feb80 1 -- 172.21.15.46:0/3781321529 wait complete. 2024-05-09T00:23:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d9d1feb80 20 client.17448 trim_cache size 0 max 16384 2024-05-09T00:23:29.678 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:23:29.678 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T00:23:29.678 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "c5f55ea7-61af-436b-b0bf-73d238c899b2", 2024-05-09T00:23:29.678 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T00:23:29.678 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:23:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3245505155 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4d740b2880 msgr2=0x7f4d740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.673+0000 7f4d8e7e8700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4d740b2880 0x7f4d740b4d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009a10 tx=0x7f4d78009190 comp rx=0 tx=0).stop 2024-05-09T00:23:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.674+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3245505155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d88108f10 msgr2=0x7f4d88076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.674+0000 7f4d8e7e8700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d88108f10 0x7f4d88076000 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4d70006010 tx=0x7f4d700047b0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.674+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3245505155 shutdown_connections 2024-05-09T00:23:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.674+0000 7f4d8e7e8700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f4d740b2880 0x7f4d740b4d90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.674+0000 7f4d8e7e8700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d88109860 0x7f4d88076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.674+0000 7f4d8e7e8700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d881045e0 0x7f4d88075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.674+0000 7f4d8e7e8700 1 --2- 172.21.15.46:0/3245505155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d88108f10 0x7f4d88076000 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.674+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3245505155 >> 172.21.15.46:0/3245505155 conn(0x7f4d880ffe70 msgr2=0x7f4d881086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.674+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3245505155 shutdown_connections 2024-05-09T00:23:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:29.674+0000 7f4d8e7e8700 1 -- 172.21.15.46:0/3245505155 wait complete. 2024-05-09T00:23:30.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T00:23:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:31.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:30 smithi046 ceph-mon[30151]: pgmap v740: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:23:31.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:30 smithi046 ceph-mon[30151]: scrub queued for path: / 2024-05-09T00:23:31.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:30 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:23:31.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:30 smithi046 ceph-mon[30151]: scrub summary: active paths [/] 2024-05-09T00:23:31.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:30 smithi046 ceph-mon[30151]: scrub summary: idle+waiting paths [/] 2024-05-09T00:23:31.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:30 smithi169 ceph-mon[34777]: pgmap v740: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:23:31.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:30 smithi169 ceph-mon[34777]: scrub queued for path: / 2024-05-09T00:23:31.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:30 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:23:31.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:30 smithi169 ceph-mon[34777]: scrub summary: active paths [/] 2024-05-09T00:23:31.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:30 smithi169 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-05-09T00:23:31.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:30 smithi142 ceph-mon[34533]: pgmap v740: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-09T00:23:31.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:30 smithi142 ceph-mon[34533]: scrub queued for path: / 2024-05-09T00:23:31.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:30 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:23:31.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:30 smithi142 ceph-mon[34533]: scrub summary: active paths [/] 2024-05-09T00:23:31.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:30 smithi142 ceph-mon[34533]: scrub summary: idle+waiting paths [/] 2024-05-09T00:23:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.644+0000 7f422730a700 1 -- 172.21.15.46:0/2792012380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4220109850 msgr2=0x7f42201111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.644+0000 7f422730a700 1 --2- 172.21.15.46:0/2792012380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4220109850 0x7f42201111b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f421800b0d0 tx=0x7f421800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.644+0000 7f422730a700 1 -- 172.21.15.46:0/2792012380 shutdown_connections 2024-05-09T00:23:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.644+0000 7f422730a700 1 --2- 172.21.15.46:0/2792012380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4220109850 0x7f42201111b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.644+0000 7f422730a700 1 --2- 172.21.15.46:0/2792012380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42201045d0 0x7f42201049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.644+0000 7f422730a700 1 --2- 172.21.15.46:0/2792012380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4220108f00 0x7f4220109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.644+0000 7f422730a700 1 -- 172.21.15.46:0/2792012380 >> 172.21.15.46:0/2792012380 conn(0x7f42200ffea0 msgr2=0x7f42201022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.644+0000 7f422730a700 1 -- 172.21.15.46:0/2792012380 shutdown_connections 2024-05-09T00:23:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.644+0000 7f422730a700 1 -- 172.21.15.46:0/2792012380 wait complete. 2024-05-09T00:23:32.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.644+0000 7f422730a700 1 Processor -- start 2024-05-09T00:23:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f422730a700 1 -- start start 2024-05-09T00:23:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f422730a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42201045d0 0x7f422019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f422730a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4220108f00 0x7f422019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f422730a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4220109850 0x7f422019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f422730a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4220114510 con 0x7f4220109850 2024-05-09T00:23:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f422730a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4220114390 con 0x7f42201045d0 2024-05-09T00:23:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f422730a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4220114690 con 0x7f4220108f00 2024-05-09T00:23:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f42258a7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4220109850 0x7f422019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f42250a6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42201045d0 0x7f422019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f42248a5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4220108f00 0x7f422019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f42250a6700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42201045d0 0x7f422019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.46:35030/0 (socket says 172.21.15.46:35030) 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f42250a6700 1 -- 172.21.15.46:0/3116451681 learned_addr learned my addr 172.21.15.46:0/3116451681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f42258a7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4220109850 0x7f422019f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37222/0 (socket says 172.21.15.46:37222) 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f42258a7700 1 -- 172.21.15.46:0/3116451681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4220108f00 msgr2=0x7f422019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.645+0000 7f42258a7700 1 --2- 172.21.15.46:0/3116451681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4220108f00 0x7f422019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.646+0000 7f42258a7700 1 -- 172.21.15.46:0/3116451681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42201045d0 msgr2=0x7f422019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.646+0000 7f42258a7700 1 --2- 172.21.15.46:0/3116451681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42201045d0 0x7f422019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.646+0000 7f42258a7700 1 -- 172.21.15.46:0/3116451681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422019fcf0 con 0x7f4220109850 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.646+0000 7f42258a7700 1 --2- 172.21.15.46:0/3116451681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4220109850 0x7f422019f650 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f4218009fd0 tx=0x7f4218007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.646+0000 7f42167fc700 1 -- 172.21.15.46:0/3116451681 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218021070 con 0x7f4220109850 2024-05-09T00:23:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.646+0000 7f422730a700 1 -- 172.21.15.46:0/3116451681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422019ff80 con 0x7f4220109850 2024-05-09T00:23:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.646+0000 7f42167fc700 1 -- 172.21.15.46:0/3116451681 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4218007e20 con 0x7f4220109850 2024-05-09T00:23:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.646+0000 7f42167fc700 1 -- 172.21.15.46:0/3116451681 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421801b930 con 0x7f4220109850 2024-05-09T00:23:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.646+0000 7f422730a700 1 -- 172.21.15.46:0/3116451681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4220076b00 con 0x7f4220109850 2024-05-09T00:23:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.649+0000 7f42167fc700 1 -- 172.21.15.46:0/3116451681 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4218024070 con 0x7f4220109850 2024-05-09T00:23:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.650+0000 7f42167fc700 1 --2- 172.21.15.46:0/3116451681 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f420c0b6c90 0x7f420c0b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.650+0000 7f42250a6700 1 --2- 172.21.15.46:0/3116451681 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f420c0b6c90 0x7f420c0b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.650+0000 7f42167fc700 1 -- 172.21.15.46:0/3116451681 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f421800f020 con 0x7f4220109850 2024-05-09T00:23:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f4235d20b80 1 Processor -- start 2024-05-09T00:23:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f42250a6700 1 --2- 172.21.15.46:0/3116451681 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f420c0b6c90 0x7f420c0b91a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f421c005950 tx=0x7f421c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f4235d20b80 1 -- start start 2024-05-09T00:23:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f4235d20b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684e68160 0x55a684f014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f4235d20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 0x55a684f01a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f4235d20b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684efb6a0 0x55a684efbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f4235d20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a684e1dff0 con 0x55a684d7e4a0 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f4235d20b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a684dbc7f0 con 0x55a684e68160 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f4235d20b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a684e19fb0 con 0x55a684efb6a0 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f42248a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 0x55a684f01a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f42248a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 0x55a684f01a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37230/0 (socket says 172.21.15.46:37230) 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f42248a5700 1 -- 172.21.15.46:0/3171263731 learned_addr learned my addr 172.21.15.46:0/3171263731 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f42250a6700 1 --2- 172.21.15.46:0/3171263731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684e68160 0x55a684f014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.651+0000 7f42258a7700 1 --2- 172.21.15.46:0/3171263731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684efb6a0 0x55a684efbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.652+0000 7f42248a5700 1 -- 172.21.15.46:0/3171263731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684efb6a0 msgr2=0x55a684efbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.652+0000 7f42248a5700 1 --2- 172.21.15.46:0/3171263731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684efb6a0 0x55a684efbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.652+0000 7f42248a5700 1 -- 172.21.15.46:0/3171263731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684e68160 msgr2=0x55a684f014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.652+0000 7f42248a5700 1 --2- 172.21.15.46:0/3171263731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684e68160 0x55a684f014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.652+0000 7f42248a5700 1 -- 172.21.15.46:0/3171263731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a684db4370 con 0x55a684d7e4a0 2024-05-09T00:23:32.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.652+0000 7f42248a5700 1 --2- 172.21.15.46:0/3171263731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 0x55a684f01a30 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4210009a80 tx=0x7f4210009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1e1e4a68364933 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:32.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.652+0000 7f420b7fe700 1 -- 172.21.15.46:0/3171263731 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210017070 con 0x55a684d7e4a0 2024-05-09T00:23:32.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.652+0000 7f420b7fe700 1 -- 172.21.15.46:0/3171263731 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f421000ed80 con 0x55a684d7e4a0 2024-05-09T00:23:32.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.652+0000 7f420b7fe700 1 -- 172.21.15.46:0/3171263731 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210013d70 con 0x55a684d7e4a0 2024-05-09T00:23:32.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.653+0000 7f4235d20b80 1 -- 172.21.15.46:0/3171263731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 msgr2=0x55a684f01a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:32.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.653+0000 7f4235d20b80 1 --2- 172.21.15.46:0/3171263731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 0x55a684f01a30 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4210009a80 tx=0x7f4210009d90 comp rx=0 tx=0).stop 2024-05-09T00:23:32.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.653+0000 7f4235d20b80 1 -- 172.21.15.46:0/3171263731 shutdown_connections 2024-05-09T00:23:32.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.653+0000 7f4235d20b80 1 --2- 172.21.15.46:0/3171263731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684efb6a0 0x55a684efbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.653+0000 7f4235d20b80 1 --2- 172.21.15.46:0/3171263731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684e68160 0x55a684f014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.653+0000 7f4235d20b80 1 --2- 172.21.15.46:0/3171263731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 0x55a684f01a30 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.653+0000 7f4235d20b80 1 -- 172.21.15.46:0/3171263731 >> 172.21.15.46:0/3171263731 conn(0x55a684dde6a0 msgr2=0x55a684ef6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.653+0000 7f4235d20b80 1 -- 172.21.15.46:0/3171263731 shutdown_connections 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.653+0000 7f4235d20b80 1 -- 172.21.15.46:0/3171263731 wait complete. 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.653+0000 7f4235d20b80 1 -- start start 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f4235d20b80 1 Processor -- start 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f4235d20b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684d7e4a0 0x55a684f051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f4235d20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684e68160 0x55a684f05720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f4235d20b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684efb6a0 0x55a684f05c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f4235d20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a684c99ba0 con 0x55a684e68160 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f4235d20b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a684dd8150 con 0x55a684efb6a0 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f4235d20b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a684f081d0 con 0x55a684d7e4a0 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f4235d20b80 10 client.0 _ll_register_callbacks cb 0x55a684c8f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f42248a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684e68160 0x55a684f05720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f42248a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684e68160 0x55a684f05720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37236/0 (socket says 172.21.15.46:37236) 2024-05-09T00:23:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f42248a5700 1 -- 172.21.15.46:0/1442303623 learned_addr learned my addr 172.21.15.46:0/1442303623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f42250a6700 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684d7e4a0 0x55a684f051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.654+0000 7f42258a7700 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684efb6a0 0x55a684f05c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f42248a5700 1 -- 172.21.15.46:0/1442303623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684d7e4a0 msgr2=0x55a684f051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f42248a5700 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684d7e4a0 0x55a684f051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f42248a5700 1 -- 172.21.15.46:0/1442303623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684efb6a0 msgr2=0x55a684f05c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f42248a5700 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684efb6a0 0x55a684f05c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f42248a5700 1 -- 172.21.15.46:0/1442303623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a684c99490 con 0x55a684e68160 2024-05-09T00:23:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f42248a5700 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684e68160 0x55a684f05720 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4210019000 tx=0x7f421000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df37f5d42921717 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f4208ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f4208ff9700 1 -- 172.21.15.46:0/1442303623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210029060 con 0x55a684e68160 2024-05-09T00:23:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f4208ff9700 1 -- 172.21.15.46:0/1442303623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f4210013b60 con 0x55a684e68160 2024-05-09T00:23:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f4235d20b80 1 -- 172.21.15.46:0/1442303623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f421000ed80 con 0x55a684e68160 2024-05-09T00:23:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f4208ff9700 1 -- 172.21.15.46:0/1442303623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4210017040 con 0x55a684e68160 2024-05-09T00:23:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f4235d20b80 10 client.17460 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f4235d20b80 1 -- 172.21.15.46:0/1442303623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a684f12bc0 con 0x55a684e68160 2024-05-09T00:23:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.655+0000 7f4208ff9700 1 -- 172.21.15.46:0/1442303623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f421000ed80 con 0x55a684e68160 2024-05-09T00:23:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.656+0000 7f4208ff9700 1 -- 172.21.15.46:0/1442303623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4210006410 con 0x55a684e68160 2024-05-09T00:23:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.656+0000 7f4235d20b80 10 client.17460 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.656+0000 7f4235d20b80 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.656+0000 7f4235d20b80 20 client.17460 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.656+0000 7f4235d20b80 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a684f15090 0x55a684f174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.656+0000 7f4235d20b80 4 client.17460 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.656+0000 7f4235d20b80 1 -- 172.21.15.46:0/1442303623 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a684e2b9d0 con 0x55a684f15090 2024-05-09T00:23:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.656+0000 7f42250a6700 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a684f15090 0x55a684f174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.657+0000 7f42250a6700 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a684f15090 0x55a684f174b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f97654eba9ad223e server_cookie=d9b08f194646d9a8 in_seq=0 out_seq=0 2024-05-09T00:23:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.657+0000 7f4208ff9700 10 client.17460 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.657+0000 7f4208ff9700 1 -- 172.21.15.46:0/1442303623 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a684e2b9d0 con 0x55a684f15090 2024-05-09T00:23:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.657+0000 7f4208ff9700 10 client.17460 handle_command_reply: tid=0 2024-05-09T00:23:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.657+0000 7f4235d20b80 1 client.17460 shutdown 2024-05-09T00:23:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.658+0000 7f4235d20b80 1 -- 172.21.15.46:0/1442303623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684e68160 msgr2=0x55a684f05720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.658+0000 7f4235d20b80 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684e68160 0x55a684f05720 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4210019000 tx=0x7f421000eb80 comp rx=0 tx=0).stop 2024-05-09T00:23:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.658+0000 7f4235d20b80 1 -- 172.21.15.46:0/1442303623 shutdown_connections 2024-05-09T00:23:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.658+0000 7f4235d20b80 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a684f15090 0x55a684f174b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.658+0000 7f4235d20b80 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684efb6a0 0x55a684f05c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.658+0000 7f4235d20b80 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684d7e4a0 0x55a684f051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.658+0000 7f4235d20b80 1 --2- 172.21.15.46:0/1442303623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684e68160 0x55a684f05720 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.658+0000 7f4208ff9700 0 client.17460 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.658+0000 7f4235d20b80 1 -- 172.21.15.46:0/1442303623 >> 172.21.15.46:0/1442303623 conn(0x55a684dde6a0 msgr2=0x55a684ef61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.658+0000 7f4235d20b80 1 -- 172.21.15.46:0/1442303623 shutdown_connections 2024-05-09T00:23:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.658+0000 7f4235d20b80 1 -- 172.21.15.46:0/1442303623 wait complete. 2024-05-09T00:23:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.658+0000 7f4235d20b80 20 client.17460 trim_cache size 0 max 16384 2024-05-09T00:23:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.678+0000 7f4235d20b80 1 Processor -- start 2024-05-09T00:23:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.678+0000 7f4235d20b80 1 -- start start 2024-05-09T00:23:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.678+0000 7f4235d20b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684d7e4a0 0x55a684f00a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.678+0000 7f4235d20b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684e68160 0x55a684f05aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.678+0000 7f4235d20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684efb6a0 0x55a684f10fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f4235d20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42201af1c0 con 0x55a684efb6a0 2024-05-09T00:23:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f4235d20b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f42201aca50 con 0x55a684d7e4a0 2024-05-09T00:23:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f4235d20b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42201aa8c0 con 0x55a684e68160 2024-05-09T00:23:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f42258a7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684efb6a0 0x55a684f10fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f42258a7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684efb6a0 0x55a684f10fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37246/0 (socket says 172.21.15.46:37246) 2024-05-09T00:23:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f42258a7700 1 -- 172.21.15.46:0/4256209790 learned_addr learned my addr 172.21.15.46:0/4256209790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f42248a5700 1 --2- 172.21.15.46:0/4256209790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684e68160 0x55a684f05aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f42250a6700 1 --2- 172.21.15.46:0/4256209790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684d7e4a0 0x55a684f00a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f42258a7700 1 -- 172.21.15.46:0/4256209790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684e68160 msgr2=0x55a684f05aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f42258a7700 1 --2- 172.21.15.46:0/4256209790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684e68160 0x55a684f05aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f42258a7700 1 -- 172.21.15.46:0/4256209790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684d7e4a0 msgr2=0x55a684f00a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:32.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f42258a7700 1 --2- 172.21.15.46:0/4256209790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684d7e4a0 0x55a684f00a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f42258a7700 1 -- 172.21.15.46:0/4256209790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a684ef7380 con 0x55a684efb6a0 2024-05-09T00:23:32.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f42250a6700 1 --2- 172.21.15.46:0/4256209790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684d7e4a0 0x55a684f00a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:32.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f42258a7700 1 --2- 172.21.15.46:0/4256209790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684efb6a0 0x55a684f10fd0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f42180a6f20 tx=0x7f421801df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dc9ab292ef70d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:32.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f420affd700 1 -- 172.21.15.46:0/4256209790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42180b3c70 con 0x55a684efb6a0 2024-05-09T00:23:32.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f420affd700 1 -- 172.21.15.46:0/4256209790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42180a7490 con 0x55a684efb6a0 2024-05-09T00:23:32.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.679+0000 7f420affd700 1 -- 172.21.15.46:0/4256209790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42180b37e0 con 0x55a684efb6a0 2024-05-09T00:23:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.680+0000 7f4235d20b80 1 -- 172.21.15.46:0/4256209790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684efb6a0 msgr2=0x55a684f10fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.680+0000 7f4235d20b80 1 --2- 172.21.15.46:0/4256209790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684efb6a0 0x55a684f10fd0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f42180a6f20 tx=0x7f421801df30 comp rx=0 tx=0).stop 2024-05-09T00:23:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.680+0000 7f4235d20b80 1 -- 172.21.15.46:0/4256209790 shutdown_connections 2024-05-09T00:23:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.680+0000 7f4235d20b80 1 --2- 172.21.15.46:0/4256209790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684efb6a0 0x55a684f10fd0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.680+0000 7f4235d20b80 1 --2- 172.21.15.46:0/4256209790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684d7e4a0 0x55a684f00a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.680+0000 7f4235d20b80 1 --2- 172.21.15.46:0/4256209790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684e68160 0x55a684f05aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.680+0000 7f4235d20b80 1 -- 172.21.15.46:0/4256209790 >> 172.21.15.46:0/4256209790 conn(0x55a684dde6a0 msgr2=0x55a684f00320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.680+0000 7f4235d20b80 1 -- 172.21.15.46:0/4256209790 shutdown_connections 2024-05-09T00:23:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.680+0000 7f4235d20b80 1 -- 172.21.15.46:0/4256209790 wait complete. 2024-05-09T00:23:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f4235d20b80 1 -- start start 2024-05-09T00:23:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f4235d20b80 1 Processor -- start 2024-05-09T00:23:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f4235d20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 0x55a684f0d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f4235d20b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684e68160 0x55a684f1c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f4235d20b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684efb6a0 0x55a684f1c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f4235d20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42201aaf10 con 0x55a684d7e4a0 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f4235d20b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f42201ac380 con 0x55a684efb6a0 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f4235d20b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42201ab5e0 con 0x55a684e68160 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f4235d20b80 10 client.0 _ll_register_callbacks cb 0x55a684c8f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f42250a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 0x55a684f0d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f42250a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 0x55a684f0d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37260/0 (socket says 172.21.15.46:37260) 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f42250a6700 1 -- 172.21.15.46:0/2731002327 learned_addr learned my addr 172.21.15.46:0/2731002327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.681+0000 7f42248a5700 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684e68160 0x55a684f1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f42258a7700 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684efb6a0 0x55a684f1c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f42250a6700 1 -- 172.21.15.46:0/2731002327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684e68160 msgr2=0x55a684f1c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f42250a6700 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684e68160 0x55a684f1c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f42250a6700 1 -- 172.21.15.46:0/2731002327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684efb6a0 msgr2=0x55a684f1c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f42250a6700 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684efb6a0 0x55a684f1c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f42250a6700 1 -- 172.21.15.46:0/2731002327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a684f1ce70 con 0x55a684d7e4a0 2024-05-09T00:23:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f42258a7700 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684efb6a0 0x55a684f1c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f42250a6700 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 0x55a684f0d960 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f421c00fa30 tx=0x7f421c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c710422a5c91fad1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f4208ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f4208ff9700 1 -- 172.21.15.46:0/2731002327 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c02f060 con 0x55a684d7e4a0 2024-05-09T00:23:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f4208ff9700 1 -- 172.21.15.46:0/2731002327 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f421c02a640 con 0x55a684d7e4a0 2024-05-09T00:23:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f4235d20b80 1 -- 172.21.15.46:0/2731002327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42180a7490 con 0x55a684d7e4a0 2024-05-09T00:23:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f4208ff9700 1 -- 172.21.15.46:0/2731002327 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f421c016070 con 0x55a684d7e4a0 2024-05-09T00:23:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f4235d20b80 10 client.17466 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.682+0000 7f4235d20b80 1 -- 172.21.15.46:0/2731002327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a684f7f690 con 0x55a684d7e4a0 2024-05-09T00:23:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.683+0000 7f4208ff9700 1 -- 172.21.15.46:0/2731002327 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42180a7490 con 0x55a684d7e4a0 2024-05-09T00:23:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.683+0000 7f4208ff9700 1 -- 172.21.15.46:0/2731002327 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f421c034070 con 0x55a684d7e4a0 2024-05-09T00:23:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.683+0000 7f4235d20b80 10 client.17466 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.683+0000 7f4235d20b80 10 client.17466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.683+0000 7f4235d20b80 20 client.17466 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.683+0000 7f4235d20b80 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a684f7fd50 0x55a684f82170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.683+0000 7f4235d20b80 4 client.17466 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T00:23:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.683+0000 7f4235d20b80 1 -- 172.21.15.46:0/2731002327 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a684e25a70 con 0x55a684f7fd50 2024-05-09T00:23:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.683+0000 7f42248a5700 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a684f7fd50 0x55a684f82170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f42248a5700 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a684f7fd50 0x55a684f82170 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c1e527f24fc7c72 server_cookie=4a1c1a934dba3a1b in_seq=0 out_seq=0 2024-05-09T00:23:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4208ff9700 10 client.17466 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4208ff9700 1 -- 172.21.15.46:0/2731002327 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a684e25a70 con 0x55a684f7fd50 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4208ff9700 10 client.17466 handle_command_reply: tid=0 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4235d20b80 1 client.17466 shutdown 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4235d20b80 1 -- 172.21.15.46:0/2731002327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 msgr2=0x55a684f0d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4235d20b80 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 0x55a684f0d960 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f421c00fa30 tx=0x7f421c0047c0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4235d20b80 1 -- 172.21.15.46:0/2731002327 shutdown_connections 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4235d20b80 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55a684f7fd50 0x55a684f82170 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4235d20b80 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a684efb6a0 0x55a684f1c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4235d20b80 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a684d7e4a0 0x55a684f0d960 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4235d20b80 1 --2- 172.21.15.46:0/2731002327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a684e68160 0x55a684f1c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4235d20b80 1 -- 172.21.15.46:0/2731002327 >> 172.21.15.46:0/2731002327 conn(0x55a684dde6a0 msgr2=0x55a684efee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4208ff9700 0 client.17466 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4235d20b80 1 -- 172.21.15.46:0/2731002327 shutdown_connections 2024-05-09T00:23:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4235d20b80 1 -- 172.21.15.46:0/2731002327 wait complete. 2024-05-09T00:23:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.684+0000 7f4235d20b80 20 client.17466 trim_cache size 0 max 16384 2024-05-09T00:23:32.692 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:23:32.692 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T00:23:32.692 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T00:23:32.692 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:23:32.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.685+0000 7f422730a700 1 -- 172.21.15.46:0/3116451681 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f420c0b6c90 msgr2=0x7f420c0b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:32.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.685+0000 7f422730a700 1 --2- 172.21.15.46:0/3116451681 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f420c0b6c90 0x7f420c0b91a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f421c005950 tx=0x7f421c00a3b0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.685+0000 7f422730a700 1 -- 172.21.15.46:0/3116451681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4220109850 msgr2=0x7f422019f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:32.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.685+0000 7f422730a700 1 --2- 172.21.15.46:0/3116451681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4220109850 0x7f422019f650 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f4218009fd0 tx=0x7f4218007a20 comp rx=0 tx=0).stop 2024-05-09T00:23:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.685+0000 7f422730a700 1 -- 172.21.15.46:0/3116451681 shutdown_connections 2024-05-09T00:23:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.685+0000 7f422730a700 1 --2- 172.21.15.46:0/3116451681 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f420c0b6c90 0x7f420c0b91a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.685+0000 7f422730a700 1 --2- 172.21.15.46:0/3116451681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4220109850 0x7f422019f650 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.685+0000 7f422730a700 1 --2- 172.21.15.46:0/3116451681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42201045d0 0x7f422019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.685+0000 7f422730a700 1 --2- 172.21.15.46:0/3116451681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4220108f00 0x7f422019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.685+0000 7f422730a700 1 -- 172.21.15.46:0/3116451681 >> 172.21.15.46:0/3116451681 conn(0x7f42200ffea0 msgr2=0x7f4220108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.685+0000 7f422730a700 1 -- 172.21.15.46:0/3116451681 shutdown_connections 2024-05-09T00:23:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:32.685+0000 7f422730a700 1 -- 172.21.15.46:0/3116451681 wait complete. 2024-05-09T00:23:33.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:32 smithi046 ceph-mon[30151]: pgmap v741: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:33.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:32 smithi046 ceph-mon[30151]: scrub summary: idle 2024-05-09T00:23:33.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:32 smithi169 ceph-mon[34777]: pgmap v741: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:33.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:32 smithi169 ceph-mon[34777]: scrub summary: idle 2024-05-09T00:23:33.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:32 smithi142 ceph-mon[34533]: pgmap v741: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:33.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:32 smithi142 ceph-mon[34533]: scrub summary: idle 2024-05-09T00:23:33.458 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T00:23:33.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:23:33.709 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:34.813 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:34 smithi046 ceph-mon[30151]: pgmap v742: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:23:35.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:34 smithi169 ceph-mon[34777]: pgmap v742: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:23:35.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:34 smithi142 ceph-mon[34533]: pgmap v742: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-05-09T00:23:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.413+0000 7f7e96835700 1 -- 172.21.15.46:0/44877158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e90108f00 msgr2=0x7f7e90109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.413+0000 7f7e96835700 1 --2- 172.21.15.46:0/44877158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e90108f00 0x7f7e90109310 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7e780099c0 tx=0x7f7e78009cd0 comp rx=0 tx=0).stop 2024-05-09T00:23:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.413+0000 7f7e96835700 1 -- 172.21.15.46:0/44877158 shutdown_connections 2024-05-09T00:23:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.413+0000 7f7e96835700 1 --2- 172.21.15.46:0/44877158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e90109850 0x7f7e901111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.413+0000 7f7e96835700 1 --2- 172.21.15.46:0/44877158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e901045d0 0x7f7e901049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.413+0000 7f7e96835700 1 --2- 172.21.15.46:0/44877158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e90108f00 0x7f7e90109310 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.413+0000 7f7e96835700 1 -- 172.21.15.46:0/44877158 >> 172.21.15.46:0/44877158 conn(0x7f7e900ffea0 msgr2=0x7f7e901022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.413+0000 7f7e96835700 1 -- 172.21.15.46:0/44877158 shutdown_connections 2024-05-09T00:23:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.413+0000 7f7e96835700 1 -- 172.21.15.46:0/44877158 wait complete. 2024-05-09T00:23:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.414+0000 7f7e96835700 1 Processor -- start 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.414+0000 7f7e96835700 1 -- start start 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.414+0000 7f7e96835700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e901045d0 0x7f7e90076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.414+0000 7f7e96835700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e90108f00 0x7f7e90077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.414+0000 7f7e96835700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e90109850 0x7f7e90077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.414+0000 7f7e96835700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e90114460 con 0x7f7e90109850 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.414+0000 7f7e96835700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901142e0 con 0x7f7e901045d0 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.414+0000 7f7e96835700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e901145e0 con 0x7f7e90108f00 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.414+0000 7f7e94dd2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e90109850 0x7f7e90077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.414+0000 7f7e94dd2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e90109850 0x7f7e90077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37288/0 (socket says 172.21.15.46:37288) 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e94dd2700 1 -- 172.21.15.46:0/1447795016 learned_addr learned my addr 172.21.15.46:0/1447795016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e87fff700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e90108f00 0x7f7e90077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e8ffff700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e901045d0 0x7f7e90076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e94dd2700 1 -- 172.21.15.46:0/1447795016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e90108f00 msgr2=0x7f7e90077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e94dd2700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e90108f00 0x7f7e90077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e94dd2700 1 -- 172.21.15.46:0/1447795016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e901045d0 msgr2=0x7f7e90076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e94dd2700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e901045d0 0x7f7e90076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e94dd2700 1 -- 172.21.15.46:0/1447795016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e901a90c0 con 0x7f7e90109850 2024-05-09T00:23:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e94dd2700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e90109850 0x7f7e90077650 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7e88007ee0 tx=0x7f7e8800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e8dffb700 1 -- 172.21.15.46:0/1447795016 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88004bb0 con 0x7f7e90109850 2024-05-09T00:23:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e8dffb700 1 -- 172.21.15.46:0/1447795016 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e88004d50 con 0x7f7e90109850 2024-05-09T00:23:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e96835700 1 -- 172.21.15.46:0/1447795016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e901a93b0 con 0x7f7e90109850 2024-05-09T00:23:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e8dffb700 1 -- 172.21.15.46:0/1447795016 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88005920 con 0x7f7e90109850 2024-05-09T00:23:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.415+0000 7f7e96835700 1 -- 172.21.15.46:0/1447795016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e901a9870 con 0x7f7e90109850 2024-05-09T00:23:35.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.416+0000 7f7e96835700 1 -- 172.21.15.46:0/1447795016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e900512c0 con 0x7f7e90109850 2024-05-09T00:23:35.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.418+0000 7f7e8dffb700 1 -- 172.21.15.46:0/1447795016 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7e8801a020 con 0x7f7e90109850 2024-05-09T00:23:35.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.419+0000 7f7e8dffb700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e7c0b2880 0x7f7e7c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:35.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.419+0000 7f7e8ffff700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e7c0b2880 0x7f7e7c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.419+0000 7f7e8dffb700 1 -- 172.21.15.46:0/1447795016 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f7e880e1ec0 con 0x7f7e90109850 2024-05-09T00:23:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.419+0000 7f7e8ffff700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e7c0b2880 0x7f7e7c0b4d90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000ac50 tx=0x7f7e8000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.423+0000 7f7e8dffb700 1 -- 172.21.15.46:0/1447795016 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e88010070 con 0x7f7e90109850 2024-05-09T00:23:35.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.683+0000 7f7e96835700 1 -- 172.21.15.46:0/1447795016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e9002e7b0 con 0x7f7e90109850 2024-05-09T00:23:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.684+0000 7f7e8dffb700 1 -- 172.21.15.46:0/1447795016 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f7e880a9ba0 con 0x7f7e90109850 2024-05-09T00:23:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:23:35.685 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:23:35.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.687+0000 7f7e96835700 1 -- 172.21.15.46:0/1447795016 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e7c0b2880 msgr2=0x7f7e7c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:35.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.687+0000 7f7e96835700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e7c0b2880 0x7f7e7c0b4d90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000ac50 tx=0x7f7e8000a380 comp rx=0 tx=0).stop 2024-05-09T00:23:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.687+0000 7f7e96835700 1 -- 172.21.15.46:0/1447795016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e90109850 msgr2=0x7f7e90077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.687+0000 7f7e96835700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e90109850 0x7f7e90077650 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7e88007ee0 tx=0x7f7e8800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:23:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.687+0000 7f7e96835700 1 -- 172.21.15.46:0/1447795016 shutdown_connections 2024-05-09T00:23:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.687+0000 7f7e96835700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f7e7c0b2880 0x7f7e7c0b4d90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:35.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.687+0000 7f7e96835700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e90109850 0x7f7e90077650 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:35.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.687+0000 7f7e96835700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e901045d0 0x7f7e90076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:35.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.687+0000 7f7e96835700 1 --2- 172.21.15.46:0/1447795016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e90108f00 0x7f7e90077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:35.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.687+0000 7f7e96835700 1 -- 172.21.15.46:0/1447795016 >> 172.21.15.46:0/1447795016 conn(0x7f7e900ffea0 msgr2=0x7f7e9010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:35.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.687+0000 7f7e96835700 1 -- 172.21.15.46:0/1447795016 shutdown_connections 2024-05-09T00:23:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:35.687+0000 7f7e96835700 1 -- 172.21.15.46:0/1447795016 wait complete. 2024-05-09T00:23:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:23:36.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:35 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1447795016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:23:36.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:35 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1447795016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:23:36.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:35 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1447795016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:23:36.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T00:23:36.673 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:37.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:36 smithi046 ceph-mon[30151]: pgmap v743: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:23:37.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:36 smithi169 ceph-mon[34777]: pgmap v743: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:23:37.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:36 smithi142 ceph-mon[34533]: pgmap v743: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-09T00:23:38.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.463+0000 7fd31d528700 1 -- 172.21.15.46:0/3621828781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd318107df0 msgr2=0x7fd318108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:38.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.463+0000 7fd31d528700 1 --2- 172.21.15.46:0/3621828781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd318107df0 0x7fd318108200 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0099c0 tx=0x7fd30c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.463+0000 7fd31d528700 1 -- 172.21.15.46:0/3621828781 shutdown_connections 2024-05-09T00:23:38.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.463+0000 7fd31d528700 1 --2- 172.21.15.46:0/3621828781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd318108740 0x7fd3181100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.463+0000 7fd31d528700 1 --2- 172.21.15.46:0/3621828781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3181034c0 0x7fd3181038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.463+0000 7fd31d528700 1 --2- 172.21.15.46:0/3621828781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd318107df0 0x7fd318108200 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.463+0000 7fd31d528700 1 -- 172.21.15.46:0/3621828781 >> 172.21.15.46:0/3621828781 conn(0x7fd3180fedc0 msgr2=0x7fd3181011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:38.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.463+0000 7fd31d528700 1 -- 172.21.15.46:0/3621828781 shutdown_connections 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.463+0000 7fd31d528700 1 -- 172.21.15.46:0/3621828781 wait complete. 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.464+0000 7fd31d528700 1 Processor -- start 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.464+0000 7fd31d528700 1 -- start start 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.464+0000 7fd31d528700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3181034c0 0x7fd31819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.464+0000 7fd31d528700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd318107df0 0x7fd31819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.464+0000 7fd31d528700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd318108740 0x7fd31819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.464+0000 7fd31d528700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd318113350 con 0x7fd318108740 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.464+0000 7fd31d528700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd3181131d0 con 0x7fd3181034c0 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.464+0000 7fd31d528700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd3181134d0 con 0x7fd318107df0 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.464+0000 7fd3177fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd318108740 0x7fd31819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.464+0000 7fd3177fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd318108740 0x7fd31819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48472/0 (socket says 172.21.15.46:48472) 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.464+0000 7fd3177fe700 1 -- 172.21.15.46:0/1435957052 learned_addr learned my addr 172.21.15.46:0/1435957052 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.464+0000 7fd316ffd700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3181034c0 0x7fd31819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.465+0000 7fd3167fc700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd318107df0 0x7fd31819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.465+0000 7fd3177fe700 1 -- 172.21.15.46:0/1435957052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd318107df0 msgr2=0x7fd31819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.465+0000 7fd3177fe700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd318107df0 0x7fd31819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.465+0000 7fd3177fe700 1 -- 172.21.15.46:0/1435957052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3181034c0 msgr2=0x7fd31819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.465+0000 7fd3177fe700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3181034c0 0x7fd31819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.465+0000 7fd3177fe700 1 -- 172.21.15.46:0/1435957052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31819fce0 con 0x7fd318108740 2024-05-09T00:23:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.465+0000 7fd3177fe700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd318108740 0x7fd31819f640 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd31000d980 tx=0x7fd31000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.465+0000 7fd2fffff700 1 -- 172.21.15.46:0/1435957052 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310017070 con 0x7fd318108740 2024-05-09T00:23:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.465+0000 7fd2fffff700 1 -- 172.21.15.46:0/1435957052 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd310005020 con 0x7fd318108740 2024-05-09T00:23:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.465+0000 7fd31d528700 1 -- 172.21.15.46:0/1435957052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31819ffd0 con 0x7fd318108740 2024-05-09T00:23:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.465+0000 7fd2fffff700 1 -- 172.21.15.46:0/1435957052 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31000fa10 con 0x7fd318108740 2024-05-09T00:23:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.465+0000 7fd31d528700 1 -- 172.21.15.46:0/1435957052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3181a78f0 con 0x7fd318108740 2024-05-09T00:23:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.468+0000 7fd2fffff700 1 -- 172.21.15.46:0/1435957052 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd31000fbb0 con 0x7fd318108740 2024-05-09T00:23:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.469+0000 7fd2fffff700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd3000b2880 0x7fd3000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.469+0000 7fd32bf3eb80 1 Processor -- start 2024-05-09T00:23:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.469+0000 7fd316ffd700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd3000b2880 0x7fd3000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.469+0000 7fd2fffff700 1 -- 172.21.15.46:0/1435957052 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd3100e2010 con 0x7fd318108740 2024-05-09T00:23:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.469+0000 7fd32bf3eb80 1 -- start start 2024-05-09T00:23:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.469+0000 7fd32bf3eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c476160 0x55ca8c50d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.469+0000 7fd32bf3eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 0x55ca8c5096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.469+0000 7fd32bf3eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c509c80 0x55ca8c50a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.469+0000 7fd32bf3eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ca8c42bff0 con 0x55ca8c38c4a0 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.469+0000 7fd32bf3eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ca8c3ca7f0 con 0x55ca8c509c80 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.469+0000 7fd32bf3eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ca8c427fb0 con 0x55ca8c476160 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd316ffd700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd3000b2880 0x7fd3000b4d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd3080060b0 tx=0x7fd30800c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd3167fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 0x55ca8c5096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd3167fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 0x55ca8c5096a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48476/0 (socket says 172.21.15.46:48476) 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd3167fc700 1 -- 172.21.15.46:0/666208442 learned_addr learned my addr 172.21.15.46:0/666208442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd316ffd700 1 --2- 172.21.15.46:0/666208442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c476160 0x55ca8c50d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd3177fe700 1 --2- 172.21.15.46:0/666208442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c509c80 0x55ca8c50a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd3167fc700 1 -- 172.21.15.46:0/666208442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c476160 msgr2=0x55ca8c50d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd3167fc700 1 --2- 172.21.15.46:0/666208442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c476160 0x55ca8c50d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd3167fc700 1 -- 172.21.15.46:0/666208442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c509c80 msgr2=0x55ca8c50a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:38.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd3167fc700 1 --2- 172.21.15.46:0/666208442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c509c80 0x55ca8c50a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd3167fc700 1 -- 172.21.15.46:0/666208442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8c3c2370 con 0x55ca8c38c4a0 2024-05-09T00:23:38.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd316ffd700 1 --2- 172.21.15.46:0/666208442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c476160 0x55ca8c50d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:23:38.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd3177fe700 1 --2- 172.21.15.46:0/666208442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c509c80 0x55ca8c50a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:23:38.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd3167fc700 1 --2- 172.21.15.46:0/666208442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 0x55ca8c5096a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0099c0 tx=0x7fd30c009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd444cb74916b24 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:38.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.470+0000 7fd2fd7fa700 1 -- 172.21.15.46:0/666208442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c003ae0 con 0x55ca8c38c4a0 2024-05-09T00:23:38.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.471+0000 7fd2fd7fa700 1 -- 172.21.15.46:0/666208442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd30c005580 con 0x55ca8c38c4a0 2024-05-09T00:23:38.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.471+0000 7fd2fd7fa700 1 -- 172.21.15.46:0/666208442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c00bcd0 con 0x55ca8c38c4a0 2024-05-09T00:23:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.471+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/666208442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 msgr2=0x55ca8c5096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.471+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/666208442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 0x55ca8c5096a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0099c0 tx=0x7fd30c009710 comp rx=0 tx=0).stop 2024-05-09T00:23:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.471+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/666208442 shutdown_connections 2024-05-09T00:23:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.471+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/666208442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c509c80 0x55ca8c50a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.471+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/666208442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c476160 0x55ca8c50d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.471+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/666208442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 0x55ca8c5096a0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.471+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/666208442 >> 172.21.15.46:0/666208442 conn(0x55ca8c3ec6a0 msgr2=0x55ca8c504400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.471+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/666208442 shutdown_connections 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.472+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/666208442 wait complete. 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.472+0000 7fd32bf3eb80 1 -- start start 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.472+0000 7fd32bf3eb80 1 Processor -- start 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.472+0000 7fd32bf3eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c38c4a0 0x55ca8c51b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.472+0000 7fd32bf3eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c476160 0x55ca8c51b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.472+0000 7fd32bf3eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 0x55ca8c51be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.472+0000 7fd32bf3eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ca8c2a7ba0 con 0x55ca8c509c80 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.472+0000 7fd32bf3eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ca8c3e6150 con 0x55ca8c476160 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.472+0000 7fd32bf3eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ca8c51e790 con 0x55ca8c38c4a0 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.472+0000 7fd32bf3eb80 10 client.0 _ll_register_callbacks cb 0x55ca8c29d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd3177fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 0x55ca8c51be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd3177fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 0x55ca8c51be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48488/0 (socket says 172.21.15.46:48488) 2024-05-09T00:23:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd3177fe700 1 -- 172.21.15.46:0/1526551175 learned_addr learned my addr 172.21.15.46:0/1526551175 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd3167fc700 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c476160 0x55ca8c51b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd316ffd700 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c38c4a0 0x55ca8c51b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd3177fe700 1 -- 172.21.15.46:0/1526551175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c38c4a0 msgr2=0x55ca8c51b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd3177fe700 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c38c4a0 0x55ca8c51b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd3177fe700 1 -- 172.21.15.46:0/1526551175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c476160 msgr2=0x55ca8c51b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd3177fe700 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c476160 0x55ca8c51b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd3177fe700 1 -- 172.21.15.46:0/1526551175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8c2a7490 con 0x55ca8c509c80 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd3177fe700 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 0x55ca8c51be80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd31000d950 tx=0x7fd310005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9240fadd87611ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd2faff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd2faff5700 1 -- 172.21.15.46:0/1526551175 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310060150 con 0x55ca8c509c80 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd2faff5700 1 -- 172.21.15.46:0/1526551175 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd31005d830 con 0x55ca8c509c80 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1526551175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd30c005580 con 0x55ca8c509c80 2024-05-09T00:23:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd2faff5700 1 -- 172.21.15.46:0/1526551175 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd31005da30 con 0x55ca8c509c80 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd32bf3eb80 10 client.17484 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.473+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1526551175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca8c529260 con 0x55ca8c509c80 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.474+0000 7fd2faff5700 1 -- 172.21.15.46:0/1526551175 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd30c005580 con 0x55ca8c509c80 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.474+0000 7fd2faff5700 1 -- 172.21.15.46:0/1526551175 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd3100b2090 con 0x55ca8c509c80 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.474+0000 7fd32bf3eb80 10 client.17484 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.474+0000 7fd32bf3eb80 10 client.17484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.474+0000 7fd32bf3eb80 20 client.17484 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.474+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca8c52b7f0 0x55ca8c52dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.474+0000 7fd32bf3eb80 4 client.17484 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.474+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1526551175 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca8c4399d0 con 0x55ca8c52b7f0 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.474+0000 7fd316ffd700 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca8c52b7f0 0x55ca8c52dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.475+0000 7fd316ffd700 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca8c52b7f0 0x55ca8c52dc10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=756f1e404e9b80d9 server_cookie=f17da5d11a30704 in_seq=0 out_seq=0 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.475+0000 7fd2faff5700 10 client.17484 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:23:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd2faff5700 1 -- 172.21.15.46:0/1526551175 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca8c4399d0 con 0x55ca8c52b7f0 2024-05-09T00:23:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd2faff5700 10 client.17484 handle_command_reply: tid=0 2024-05-09T00:23:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd32bf3eb80 1 client.17484 shutdown 2024-05-09T00:23:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1526551175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 msgr2=0x55ca8c51be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 0x55ca8c51be80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd31000d950 tx=0x7fd310005700 comp rx=0 tx=0).stop 2024-05-09T00:23:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1526551175 shutdown_connections 2024-05-09T00:23:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca8c52b7f0 0x55ca8c52dc10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 0x55ca8c51be80 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c38c4a0 0x55ca8c51b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/1526551175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c476160 0x55ca8c51b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd2faff5700 0 client.17484 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:23:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1526551175 >> 172.21.15.46:0/1526551175 conn(0x55ca8c3ec6a0 msgr2=0x55ca8c5041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1526551175 shutdown_connections 2024-05-09T00:23:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1526551175 wait complete. 2024-05-09T00:23:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.476+0000 7fd32bf3eb80 20 client.17484 trim_cache size 0 max 16384 2024-05-09T00:23:38.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.491+0000 7fd32bf3eb80 1 Processor -- start 2024-05-09T00:23:38.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.491+0000 7fd32bf3eb80 1 -- start start 2024-05-09T00:23:38.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.491+0000 7fd32bf3eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c38c4a0 0x55ca8c51bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.491+0000 7fd32bf3eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c476160 0x55ca8c5169d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.491+0000 7fd32bf3eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 0x55ca8c5276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.491+0000 7fd32bf3eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3181af280 con 0x55ca8c509c80 2024-05-09T00:23:38.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.491+0000 7fd32bf3eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd3181acb10 con 0x55ca8c38c4a0 2024-05-09T00:23:38.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.491+0000 7fd32bf3eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd3181aa940 con 0x55ca8c476160 2024-05-09T00:23:38.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd3177fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 0x55ca8c5276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd3177fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 0x55ca8c5276d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48496/0 (socket says 172.21.15.46:48496) 2024-05-09T00:23:38.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd3177fe700 1 -- 172.21.15.46:0/357052628 learned_addr learned my addr 172.21.15.46:0/357052628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd3167fc700 1 --2- 172.21.15.46:0/357052628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c476160 0x55ca8c5169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd316ffd700 1 --2- 172.21.15.46:0/357052628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c38c4a0 0x55ca8c51bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd3177fe700 1 -- 172.21.15.46:0/357052628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c476160 msgr2=0x55ca8c5169d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd3177fe700 1 --2- 172.21.15.46:0/357052628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c476160 0x55ca8c5169d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd3177fe700 1 -- 172.21.15.46:0/357052628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c38c4a0 msgr2=0x55ca8c51bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd3177fe700 1 --2- 172.21.15.46:0/357052628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c38c4a0 0x55ca8c51bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd3177fe700 1 -- 172.21.15.46:0/357052628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8c516f10 con 0x55ca8c509c80 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd3177fe700 1 --2- 172.21.15.46:0/357052628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 0x55ca8c5276d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd31005d760 tx=0x7fd310091050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b945dd8e4f306b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd2fcff9700 1 -- 172.21.15.46:0/357052628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31005d7e0 con 0x55ca8c509c80 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd2fcff9700 1 -- 172.21.15.46:0/357052628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd31008fd40 con 0x55ca8c509c80 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.492+0000 7fd2fcff9700 1 -- 172.21.15.46:0/357052628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3100a9af0 con 0x55ca8c509c80 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.493+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/357052628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 msgr2=0x55ca8c5276d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:38.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.493+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/357052628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 0x55ca8c5276d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd31005d760 tx=0x7fd310091050 comp rx=0 tx=0).stop 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.493+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/357052628 shutdown_connections 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.493+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/357052628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c509c80 0x55ca8c5276d0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.493+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/357052628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c38c4a0 0x55ca8c51bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.493+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/357052628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c476160 0x55ca8c5169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.493+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/357052628 >> 172.21.15.46:0/357052628 conn(0x55ca8c3ec6a0 msgr2=0x55ca8c5162c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.493+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/357052628 shutdown_connections 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.493+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/357052628 wait complete. 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd32bf3eb80 1 -- start start 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd32bf3eb80 1 Processor -- start 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd32bf3eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 0x55ca8c5240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd32bf3eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c476160 0x55ca8c51f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd32bf3eb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c509c80 0x55ca8c51fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd32bf3eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3181aafd0 con 0x55ca8c38c4a0 2024-05-09T00:23:38.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd32bf3eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd3181ac440 con 0x55ca8c476160 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd32bf3eb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd3181ab6a0 con 0x55ca8c509c80 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd32bf3eb80 10 client.0 _ll_register_callbacks cb 0x55ca8c29d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd316ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 0x55ca8c5240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd316ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 0x55ca8c5240b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48504/0 (socket says 172.21.15.46:48504) 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd316ffd700 1 -- 172.21.15.46:0/1093740695 learned_addr learned my addr 172.21.15.46:0/1093740695 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd3167fc700 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c476160 0x55ca8c51f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.494+0000 7fd3177fe700 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c509c80 0x55ca8c51fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd316ffd700 1 -- 172.21.15.46:0/1093740695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c509c80 msgr2=0x55ca8c51fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd316ffd700 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c509c80 0x55ca8c51fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd316ffd700 1 -- 172.21.15.46:0/1093740695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c476160 msgr2=0x55ca8c51f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd316ffd700 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c476160 0x55ca8c51f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd316ffd700 1 -- 172.21.15.46:0/1093740695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8c5201b0 con 0x55ca8c38c4a0 2024-05-09T00:23:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd316ffd700 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 0x55ca8c5240b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd308007fd0 tx=0x7fd308004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1dd4adec74ad729 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd2faff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd2faff5700 1 -- 172.21.15.46:0/1093740695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308029400 con 0x55ca8c38c4a0 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd2faff5700 1 -- 172.21.15.46:0/1093740695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fd3080295a0 con 0x55ca8c38c4a0 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1093740695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd31008fd40 con 0x55ca8c38c4a0 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd2faff5700 1 -- 172.21.15.46:0/1093740695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd3080297a0 con 0x55ca8c38c4a0 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd32bf3eb80 10 client.17490 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1093740695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca8c595f90 con 0x55ca8c38c4a0 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.495+0000 7fd2faff5700 1 -- 172.21.15.46:0/1093740695 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd31008fd40 con 0x55ca8c38c4a0 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.496+0000 7fd2faff5700 1 -- 172.21.15.46:0/1093740695 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd308030070 con 0x55ca8c38c4a0 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.496+0000 7fd32bf3eb80 10 client.17490 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.496+0000 7fd32bf3eb80 10 client.17490 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.496+0000 7fd32bf3eb80 20 client.17490 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.496+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca8c596650 0x55ca8c598a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.496+0000 7fd32bf3eb80 4 client.17490 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:23:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.496+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1093740695 --> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca8c433a70 con 0x55ca8c596650 2024-05-09T00:23:38.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.496+0000 7fd3167fc700 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca8c596650 0x55ca8c598a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:38.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.496+0000 7fd2faff5700 10 client.17490 ms_handle_connect on v2:172.21.15.169:6836/1024289476 2024-05-09T00:23:38.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.496+0000 7fd3167fc700 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca8c596650 0x55ca8c598a70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=115c8f18e1fb7bb0 server_cookie=62cf7834ba7dcda in_seq=0 out_seq=0 2024-05-09T00:23:38.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd2faff5700 1 -- 172.21.15.46:0/1093740695 <== mds.2 v2:172.21.15.169:6836/1024289476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca8c433a70 con 0x55ca8c596650 2024-05-09T00:23:38.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd2faff5700 10 client.17490 handle_command_reply: tid=0 2024-05-09T00:23:38.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd32bf3eb80 1 client.17490 shutdown 2024-05-09T00:23:38.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1093740695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 msgr2=0x55ca8c5240b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:38.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 0x55ca8c5240b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd308007fd0 tx=0x7fd308004b40 comp rx=0 tx=0).stop 2024-05-09T00:23:38.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1093740695 shutdown_connections 2024-05-09T00:23:38.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.169:6836/1024289476,v1:172.21.15.169:6837/1024289476] conn(0x55ca8c596650 0x55ca8c598a70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ca8c509c80 0x55ca8c51fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca8c38c4a0 0x55ca8c5240b0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd32bf3eb80 1 --2- 172.21.15.46:0/1093740695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca8c476160 0x55ca8c51f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1093740695 >> 172.21.15.46:0/1093740695 conn(0x55ca8c3ec6a0 msgr2=0x55ca8c519970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:38.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd2faff5700 0 client.17490 ms_handle_reset on v2:172.21.15.169:6836/1024289476 2024-05-09T00:23:38.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1093740695 shutdown_connections 2024-05-09T00:23:38.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd32bf3eb80 1 -- 172.21.15.46:0/1093740695 wait complete. 2024-05-09T00:23:38.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.497+0000 7fd32bf3eb80 20 client.17490 trim_cache size 0 max 16384 2024-05-09T00:23:38.504 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:23:38.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.498+0000 7fd31d528700 1 -- 172.21.15.46:0/1435957052 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd3000b2880 msgr2=0x7fd3000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:38.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.498+0000 7fd31d528700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd3000b2880 0x7fd3000b4d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd3080060b0 tx=0x7fd30800c040 comp rx=0 tx=0).stop 2024-05-09T00:23:38.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.498+0000 7fd31d528700 1 -- 172.21.15.46:0/1435957052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd318108740 msgr2=0x7fd31819f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:38.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.498+0000 7fd31d528700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd318108740 0x7fd31819f640 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd31000d980 tx=0x7fd31000dc90 comp rx=0 tx=0).stop 2024-05-09T00:23:38.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.498+0000 7fd31d528700 1 -- 172.21.15.46:0/1435957052 shutdown_connections 2024-05-09T00:23:38.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.498+0000 7fd31d528700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fd3000b2880 0x7fd3000b4d90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.498+0000 7fd31d528700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd318108740 0x7fd31819f640 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.498+0000 7fd31d528700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3181034c0 0x7fd31819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.498+0000 7fd31d528700 1 --2- 172.21.15.46:0/1435957052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd318107df0 0x7fd31819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:38.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.498+0000 7fd31d528700 1 -- 172.21.15.46:0/1435957052 >> 172.21.15.46:0/1435957052 conn(0x7fd3180fedc0 msgr2=0x7fd31806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:38.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.498+0000 7fd31d528700 1 -- 172.21.15.46:0/1435957052 shutdown_connections 2024-05-09T00:23:38.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:38.498+0000 7fd31d528700 1 -- 172.21.15.46:0/1435957052 wait complete. 2024-05-09T00:23:39.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:38 smithi046 ceph-mon[30151]: pgmap v744: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:23:39.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:38 smithi169 ceph-mon[34777]: pgmap v744: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:23:39.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:38 smithi142 ceph-mon[34533]: pgmap v744: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-09T00:23:39.239 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T00:23:39.494 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:40.989 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:40 smithi046 ceph-mon[30151]: pgmap v745: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:23:41.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:40 smithi169 ceph-mon[34777]: pgmap v745: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:23:41.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:40 smithi142 ceph-mon[34533]: pgmap v745: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-09T00:23:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.243+0000 7f312acad700 1 -- 172.21.15.46:0/3239874707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31241034e0 msgr2=0x7f31241038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.243+0000 7f312acad700 1 --2- 172.21.15.46:0/3239874707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31241034e0 0x7f31241038f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f31180099c0 tx=0x7f3118009cd0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.243+0000 7f312acad700 1 -- 172.21.15.46:0/3239874707 shutdown_connections 2024-05-09T00:23:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.243+0000 7f312acad700 1 --2- 172.21.15.46:0/3239874707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3124108760 0x7f31241100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.243+0000 7f312acad700 1 --2- 172.21.15.46:0/3239874707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31241034e0 0x7f31241038f0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.243+0000 7f312acad700 1 --2- 172.21.15.46:0/3239874707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3124107e10 0x7f3124108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.243+0000 7f312acad700 1 -- 172.21.15.46:0/3239874707 >> 172.21.15.46:0/3239874707 conn(0x7f31240fef30 msgr2=0x7f3124101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.243+0000 7f312acad700 1 -- 172.21.15.46:0/3239874707 shutdown_connections 2024-05-09T00:23:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.243+0000 7f312acad700 1 -- 172.21.15.46:0/3239874707 wait complete. 2024-05-09T00:23:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.244+0000 7f312acad700 1 Processor -- start 2024-05-09T00:23:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.244+0000 7f312acad700 1 -- start start 2024-05-09T00:23:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.244+0000 7f312acad700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31241034e0 0x7f31240ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.244+0000 7f312acad700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3124107e10 0x7f31240ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.244+0000 7f312acad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3124108760 0x7f31240ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.244+0000 7f312acad700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3124113390 con 0x7f3124108760 2024-05-09T00:23:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.244+0000 7f312acad700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3124113210 con 0x7f3124107e10 2024-05-09T00:23:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.244+0000 7f312acad700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3124113510 con 0x7f31241034e0 2024-05-09T00:23:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f312924a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3124108760 0x7f31240ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f312924a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3124108760 0x7f31240ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48526/0 (socket says 172.21.15.46:48526) 2024-05-09T00:23:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f312924a700 1 -- 172.21.15.46:0/3961083788 learned_addr learned my addr 172.21.15.46:0/3961083788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f3123fff700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3124107e10 0x7f31240ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f3128a49700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31241034e0 0x7f31240ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f312924a700 1 -- 172.21.15.46:0/3961083788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31241034e0 msgr2=0x7f31240ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:41.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f312924a700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31241034e0 0x7f31240ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f312924a700 1 -- 172.21.15.46:0/3961083788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3124107e10 msgr2=0x7f31240ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:41.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f312924a700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3124107e10 0x7f31240ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f312924a700 1 -- 172.21.15.46:0/3961083788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31241a90e0 con 0x7f3124108760 2024-05-09T00:23:41.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f3123fff700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3124107e10 0x7f31240ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T00:23:41.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f312924a700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3124108760 0x7f31240ffe90 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f311c00ef30 tx=0x7f311c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:41.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.245+0000 7f3121ffb700 1 -- 172.21.15.46:0/3961083788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c0092e0 con 0x7f3124108760 2024-05-09T00:23:41.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.246+0000 7f312acad700 1 -- 172.21.15.46:0/3961083788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31241a9370 con 0x7f3124108760 2024-05-09T00:23:41.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.246+0000 7f3121ffb700 1 -- 172.21.15.46:0/3961083788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f311c004bb0 con 0x7f3124108760 2024-05-09T00:23:41.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.246+0000 7f3121ffb700 1 -- 172.21.15.46:0/3961083788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c004020 con 0x7f3124108760 2024-05-09T00:23:41.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.246+0000 7f312acad700 1 -- 172.21.15.46:0/3961083788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31241a98b0 con 0x7f3124108760 2024-05-09T00:23:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.248+0000 7f3121ffb700 1 -- 172.21.15.46:0/3961083788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f311c01b030 con 0x7f3124108760 2024-05-09T00:23:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.249+0000 7f3121ffb700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f31140b2900 0x7f31140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.249+0000 7f3128a49700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f31140b2900 0x7f31140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.249+0000 7f3121ffb700 1 -- 172.21.15.46:0/3961083788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f311c0e1bc0 con 0x7f3124108760 2024-05-09T00:23:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.250+0000 7f3128a49700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f31140b2900 0x7f31140b4e10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3118003760 tx=0x7f311801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.250+0000 7f31396c3b80 1 Processor -- start 2024-05-09T00:23:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.250+0000 7f31396c3b80 1 -- start start 2024-05-09T00:23:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.250+0000 7f31396c3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 0x55f510d17240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.250+0000 7f31396c3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510b944a0 0x55f510d17780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.250+0000 7f31396c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510d11400 0x55f510d11810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.250+0000 7f31396c3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f510c340a0 con 0x55f510c0ed00 2024-05-09T00:23:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.250+0000 7f31396c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f510bd99a0 con 0x55f510d11400 2024-05-09T00:23:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.250+0000 7f31396c3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f510c30060 con 0x55f510b944a0 2024-05-09T00:23:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f3128a49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 0x55f510d17240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f3128a49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 0x55f510d17240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48538/0 (socket says 172.21.15.46:48538) 2024-05-09T00:23:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f3128a49700 1 -- 172.21.15.46:0/3435156679 learned_addr learned my addr 172.21.15.46:0/3435156679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f312924a700 1 --2- 172.21.15.46:0/3435156679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510d11400 0x55f510d11810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f3123fff700 1 --2- 172.21.15.46:0/3435156679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510b944a0 0x55f510d17780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f3128a49700 1 -- 172.21.15.46:0/3435156679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510b944a0 msgr2=0x55f510d17780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f3128a49700 1 --2- 172.21.15.46:0/3435156679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510b944a0 0x55f510d17780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f3128a49700 1 -- 172.21.15.46:0/3435156679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510d11400 msgr2=0x55f510d11810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f3128a49700 1 --2- 172.21.15.46:0/3435156679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510d11400 0x55f510d11810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f3128a49700 1 -- 172.21.15.46:0/3435156679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f510bca500 con 0x55f510c0ed00 2024-05-09T00:23:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f3123fff700 1 --2- 172.21.15.46:0/3435156679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510b944a0 0x55f510d17780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:23:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f3128a49700 1 --2- 172.21.15.46:0/3435156679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 0x55f510d17240 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3118015000 tx=0x7f3118009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54e4652d6a048b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f310effd700 1 -- 172.21.15.46:0/3435156679 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118003b40 con 0x55f510c0ed00 2024-05-09T00:23:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f310effd700 1 -- 172.21.15.46:0/3435156679 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3118003ce0 con 0x55f510c0ed00 2024-05-09T00:23:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.251+0000 7f310effd700 1 -- 172.21.15.46:0/3435156679 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118030af0 con 0x55f510c0ed00 2024-05-09T00:23:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.252+0000 7f31396c3b80 1 -- 172.21.15.46:0/3435156679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 msgr2=0x55f510d17240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.252+0000 7f31396c3b80 1 --2- 172.21.15.46:0/3435156679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 0x55f510d17240 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3118015000 tx=0x7f3118009cd0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.252+0000 7f31396c3b80 1 -- 172.21.15.46:0/3435156679 shutdown_connections 2024-05-09T00:23:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.252+0000 7f31396c3b80 1 --2- 172.21.15.46:0/3435156679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510d11400 0x55f510d11810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.252+0000 7f31396c3b80 1 --2- 172.21.15.46:0/3435156679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 0x55f510d17240 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.252+0000 7f31396c3b80 1 --2- 172.21.15.46:0/3435156679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510b944a0 0x55f510d17780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.252+0000 7f31396c3b80 1 -- 172.21.15.46:0/3435156679 >> 172.21.15.46:0/3435156679 conn(0x55f510b97f70 msgr2=0x55f510d0c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.252+0000 7f31396c3b80 1 -- 172.21.15.46:0/3435156679 shutdown_connections 2024-05-09T00:23:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.252+0000 7f31396c3b80 1 -- 172.21.15.46:0/3435156679 wait complete. 2024-05-09T00:23:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.253+0000 7f31396c3b80 1 -- start start 2024-05-09T00:23:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.253+0000 7f31396c3b80 1 Processor -- start 2024-05-09T00:23:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.253+0000 7f31396c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510b944a0 0x55f510d1b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.253+0000 7f31396c3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510c0ed00 0x55f510d1b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.253+0000 7f31396c3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510d11400 0x55f510d1bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.253+0000 7f31396c3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f510d12100 con 0x55f510d11400 2024-05-09T00:23:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.253+0000 7f31396c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f510bceb20 con 0x55f510b944a0 2024-05-09T00:23:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.253+0000 7f31396c3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f510d271d0 con 0x55f510c0ed00 2024-05-09T00:23:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.253+0000 7f31396c3b80 10 client.0 _ll_register_callbacks cb 0x55f510aa56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f312924a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510d11400 0x55f510d1bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f312924a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510d11400 0x55f510d1bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48554/0 (socket says 172.21.15.46:48554) 2024-05-09T00:23:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f312924a700 1 -- 172.21.15.46:0/2585128035 learned_addr learned my addr 172.21.15.46:0/2585128035 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f3128a49700 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510b944a0 0x55f510d1b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f3123fff700 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510c0ed00 0x55f510d1b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f312924a700 1 -- 172.21.15.46:0/2585128035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510c0ed00 msgr2=0x55f510d1b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f312924a700 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510c0ed00 0x55f510d1b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f312924a700 1 -- 172.21.15.46:0/2585128035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510b944a0 msgr2=0x55f510d1b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f312924a700 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510b944a0 0x55f510d1b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f312924a700 1 -- 172.21.15.46:0/2585128035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f510aaf490 con 0x55f510d11400 2024-05-09T00:23:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f312924a700 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510d11400 0x55f510d1bad0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f311c009c10 tx=0x7f311c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838643d7b989353f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:41.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f3103fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f3103fff700 1 -- 172.21.15.46:0/2585128035 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c0a9270 con 0x55f510d11400 2024-05-09T00:23:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f3103fff700 1 -- 172.21.15.46:0/2585128035 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f311c0b1dc0 con 0x55f510d11400 2024-05-09T00:23:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f31396c3b80 1 -- 172.21.15.46:0/2585128035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3118003ce0 con 0x55f510d11400 2024-05-09T00:23:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f3103fff700 1 -- 172.21.15.46:0/2585128035 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f311c0ad620 con 0x55f510d11400 2024-05-09T00:23:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f31396c3b80 10 client.17502 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.254+0000 7f31396c3b80 1 -- 172.21.15.46:0/2585128035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f510d28b00 con 0x55f510d11400 2024-05-09T00:23:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.255+0000 7f3103fff700 1 -- 172.21.15.46:0/2585128035 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3118003ce0 con 0x55f510d11400 2024-05-09T00:23:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.255+0000 7f3103fff700 1 -- 172.21.15.46:0/2585128035 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f311c0dfb80 con 0x55f510d11400 2024-05-09T00:23:41.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.255+0000 7f31396c3b80 10 client.17502 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.255+0000 7f31396c3b80 10 client.17502 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:23:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.255+0000 7f31396c3b80 20 client.17502 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.255+0000 7f31396c3b80 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f510d2b0e0 0x55f510d2d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.255+0000 7f31396c3b80 4 client.17502 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.255+0000 7f31396c3b80 1 -- 172.21.15.46:0/2585128035 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f510c41a80 con 0x55f510d2b0e0 2024-05-09T00:23:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.255+0000 7f3128a49700 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f510d2b0e0 0x55f510d2d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.256+0000 7f3128a49700 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f510d2b0e0 0x55f510d2d520 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c85ca7fb54e7519 server_cookie=891b4786977ed88f in_seq=0 out_seq=0 2024-05-09T00:23:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.256+0000 7f3103fff700 10 client.17502 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:23:41.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.256+0000 7f3103fff700 1 -- 172.21.15.46:0/2585128035 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f510c41a80 con 0x55f510d2b0e0 2024-05-09T00:23:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.256+0000 7f3103fff700 10 client.17502 handle_command_reply: tid=0 2024-05-09T00:23:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.256+0000 7f31396c3b80 1 client.17502 shutdown 2024-05-09T00:23:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.257+0000 7f31396c3b80 1 -- 172.21.15.46:0/2585128035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510d11400 msgr2=0x55f510d1bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.257+0000 7f31396c3b80 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510d11400 0x55f510d1bad0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f311c009c10 tx=0x7f311c004f90 comp rx=0 tx=0).stop 2024-05-09T00:23:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.257+0000 7f31396c3b80 1 -- 172.21.15.46:0/2585128035 shutdown_connections 2024-05-09T00:23:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.257+0000 7f31396c3b80 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f510d2b0e0 0x55f510d2d520 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.257+0000 7f31396c3b80 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510d11400 0x55f510d1bad0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.257+0000 7f31396c3b80 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510b944a0 0x55f510d1b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.257+0000 7f31396c3b80 1 --2- 172.21.15.46:0/2585128035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510c0ed00 0x55f510d1b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.257+0000 7f31396c3b80 1 -- 172.21.15.46:0/2585128035 >> 172.21.15.46:0/2585128035 conn(0x55f510b97f70 msgr2=0x55f510d05ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.257+0000 7f3103fff700 0 client.17502 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:23:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.257+0000 7f31396c3b80 1 -- 172.21.15.46:0/2585128035 shutdown_connections 2024-05-09T00:23:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.257+0000 7f31396c3b80 1 -- 172.21.15.46:0/2585128035 wait complete. 2024-05-09T00:23:41.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.257+0000 7f31396c3b80 20 client.17502 trim_cache size 0 max 16384 2024-05-09T00:23:41.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.272+0000 7f31396c3b80 1 Processor -- start 2024-05-09T00:23:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.272+0000 7f31396c3b80 1 -- start start 2024-05-09T00:23:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.272+0000 7f31396c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510b944a0 0x55f510d1b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.272+0000 7f31396c3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 0x55f510d32ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.272+0000 7f31396c3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510d11400 0x55f510d33220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.272+0000 7f31396c3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f31241b3540 con 0x55f510c0ed00 2024-05-09T00:23:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.272+0000 7f31396c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f31241b0dd0 con 0x55f510b944a0 2024-05-09T00:23:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.272+0000 7f31396c3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f31241aec00 con 0x55f510d11400 2024-05-09T00:23:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.272+0000 7f3123fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 0x55f510d32ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f3123fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 0x55f510d32ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48568/0 (socket says 172.21.15.46:48568) 2024-05-09T00:23:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f3123fff700 1 -- 172.21.15.46:0/1003966443 learned_addr learned my addr 172.21.15.46:0/1003966443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f3128a49700 1 --2- 172.21.15.46:0/1003966443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510b944a0 0x55f510d1b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f312924a700 1 --2- 172.21.15.46:0/1003966443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510d11400 0x55f510d33220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f3123fff700 1 -- 172.21.15.46:0/1003966443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510d11400 msgr2=0x55f510d33220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f3123fff700 1 --2- 172.21.15.46:0/1003966443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510d11400 0x55f510d33220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f3123fff700 1 -- 172.21.15.46:0/1003966443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510b944a0 msgr2=0x55f510d1b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f3123fff700 1 --2- 172.21.15.46:0/1003966443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510b944a0 0x55f510d1b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f3123fff700 1 -- 172.21.15.46:0/1003966443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f510d0cf30 con 0x55f510c0ed00 2024-05-09T00:23:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f3128a49700 1 --2- 172.21.15.46:0/1003966443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510b944a0 0x55f510d1b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f312924a700 1 --2- 172.21.15.46:0/1003966443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510d11400 0x55f510d33220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:23:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f3123fff700 1 --2- 172.21.15.46:0/1003966443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 0x55f510d32ce0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3124076ab0 tx=0x7f3110013840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2d843abbd546b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f310e7fc700 1 -- 172.21.15.46:0/1003966443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110004a30 con 0x55f510c0ed00 2024-05-09T00:23:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f310e7fc700 1 -- 172.21.15.46:0/1003966443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31100054d0 con 0x55f510c0ed00 2024-05-09T00:23:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.273+0000 7f310e7fc700 1 -- 172.21.15.46:0/1003966443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311000fb80 con 0x55f510c0ed00 2024-05-09T00:23:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.274+0000 7f31396c3b80 1 -- 172.21.15.46:0/1003966443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 msgr2=0x55f510d32ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.274+0000 7f31396c3b80 1 --2- 172.21.15.46:0/1003966443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 0x55f510d32ce0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3124076ab0 tx=0x7f3110013840 comp rx=0 tx=0).stop 2024-05-09T00:23:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.274+0000 7f31396c3b80 1 -- 172.21.15.46:0/1003966443 shutdown_connections 2024-05-09T00:23:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.274+0000 7f31396c3b80 1 --2- 172.21.15.46:0/1003966443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510d11400 0x55f510d33220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.274+0000 7f31396c3b80 1 --2- 172.21.15.46:0/1003966443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510b944a0 0x55f510d1b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.274+0000 7f31396c3b80 1 --2- 172.21.15.46:0/1003966443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510c0ed00 0x55f510d32ce0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.274+0000 7f31396c3b80 1 -- 172.21.15.46:0/1003966443 >> 172.21.15.46:0/1003966443 conn(0x55f510b97f70 msgr2=0x55f510d1b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.274+0000 7f31396c3b80 1 -- 172.21.15.46:0/1003966443 shutdown_connections 2024-05-09T00:23:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.274+0000 7f31396c3b80 1 -- 172.21.15.46:0/1003966443 wait complete. 2024-05-09T00:23:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.274+0000 7f31396c3b80 1 -- start start 2024-05-09T00:23:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f31396c3b80 1 Processor -- start 2024-05-09T00:23:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f31396c3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510b944a0 0x55f510d1e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f31396c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510c0ed00 0x55f510d1e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f31396c3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510d11400 0x55f510d93880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f31396c3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f31241af290 con 0x55f510b944a0 2024-05-09T00:23:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f31396c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f31241b0700 con 0x55f510c0ed00 2024-05-09T00:23:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f31396c3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f31241af960 con 0x55f510d11400 2024-05-09T00:23:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f31396c3b80 10 client.0 _ll_register_callbacks cb 0x55f510aa56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f3128a49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510b944a0 0x55f510d1e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f3128a49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510b944a0 0x55f510d1e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48572/0 (socket says 172.21.15.46:48572) 2024-05-09T00:23:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f3128a49700 1 -- 172.21.15.46:0/1746220477 learned_addr learned my addr 172.21.15.46:0/1746220477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f312924a700 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510d11400 0x55f510d93880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.275+0000 7f3123fff700 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510c0ed00 0x55f510d1e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f3128a49700 1 -- 172.21.15.46:0/1746220477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510d11400 msgr2=0x55f510d93880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f3128a49700 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510d11400 0x55f510d93880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f3128a49700 1 -- 172.21.15.46:0/1746220477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510c0ed00 msgr2=0x55f510d1e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f3128a49700 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510c0ed00 0x55f510d1e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f3128a49700 1 -- 172.21.15.46:0/1746220477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f510d93dc0 con 0x55f510b944a0 2024-05-09T00:23:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f3123fff700 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510c0ed00 0x55f510d1e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f3128a49700 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510b944a0 0x55f510d1e490 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3118030620 tx=0x7f31180305b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba54d7d146f96361 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f3103fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f3103fff700 1 -- 172.21.15.46:0/1746220477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118042060 con 0x55f510b944a0 2024-05-09T00:23:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f3103fff700 1 -- 172.21.15.46:0/1746220477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f3118003bd0 con 0x55f510b944a0 2024-05-09T00:23:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f31396c3b80 1 -- 172.21.15.46:0/1746220477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31100054d0 con 0x55f510b944a0 2024-05-09T00:23:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f3103fff700 1 -- 172.21.15.46:0/1746220477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f311803c480 con 0x55f510b944a0 2024-05-09T00:23:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f31396c3b80 10 client.17508 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f31396c3b80 1 -- 172.21.15.46:0/1746220477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f510d94f60 con 0x55f510b944a0 2024-05-09T00:23:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.276+0000 7f3103fff700 1 -- 172.21.15.46:0/1746220477 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31100054d0 con 0x55f510b944a0 2024-05-09T00:23:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.277+0000 7f3103fff700 1 -- 172.21.15.46:0/1746220477 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3118047070 con 0x55f510b944a0 2024-05-09T00:23:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.277+0000 7f31396c3b80 10 client.17508 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.277+0000 7f31396c3b80 10 client.17508 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-05-09T00:23:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.277+0000 7f31396c3b80 20 client.17508 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.277+0000 7f31396c3b80 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f510d951f0 0x55f510d97610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.277+0000 7f31396c3b80 4 client.17508 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:23:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.277+0000 7f31396c3b80 1 -- 172.21.15.46:0/1746220477 --> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f510c3bb20 con 0x55f510d951f0 2024-05-09T00:23:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.277+0000 7f3123fff700 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f510d951f0 0x55f510d97610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f3123fff700 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f510d951f0 0x55f510d97610 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3373cae9a04edc12 server_cookie=176f8fc6dcd2be37 in_seq=0 out_seq=0 2024-05-09T00:23:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f3103fff700 10 client.17508 ms_handle_connect on v2:172.21.15.142:6838/3645430666 2024-05-09T00:23:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f3103fff700 1 -- 172.21.15.46:0/1746220477 <== mds.1 v2:172.21.15.142:6838/3645430666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f510c3bb20 con 0x55f510d951f0 2024-05-09T00:23:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f3103fff700 10 client.17508 handle_command_reply: tid=0 2024-05-09T00:23:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f31396c3b80 1 client.17508 shutdown 2024-05-09T00:23:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f31396c3b80 1 -- 172.21.15.46:0/1746220477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510b944a0 msgr2=0x55f510d1e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f31396c3b80 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510b944a0 0x55f510d1e490 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3118030620 tx=0x7f31180305b0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f31396c3b80 1 -- 172.21.15.46:0/1746220477 shutdown_connections 2024-05-09T00:23:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f31396c3b80 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.142:6838/3645430666,v1:172.21.15.142:6839/3645430666] conn(0x55f510d951f0 0x55f510d97610 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f31396c3b80 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f510d11400 0x55f510d93880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f31396c3b80 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f510b944a0 0x55f510d1e490 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f31396c3b80 1 --2- 172.21.15.46:0/1746220477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f510c0ed00 0x55f510d1e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f31396c3b80 1 -- 172.21.15.46:0/1746220477 >> 172.21.15.46:0/1746220477 conn(0x55f510b97f70 msgr2=0x55f510d1afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f3103fff700 0 client.17508 ms_handle_reset on v2:172.21.15.142:6838/3645430666 2024-05-09T00:23:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f31396c3b80 1 -- 172.21.15.46:0/1746220477 shutdown_connections 2024-05-09T00:23:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f31396c3b80 1 -- 172.21.15.46:0/1746220477 wait complete. 2024-05-09T00:23:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.278+0000 7f31396c3b80 20 client.17508 trim_cache size 0 max 16384 2024-05-09T00:23:41.291 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:23:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.279+0000 7f312acad700 1 -- 172.21.15.46:0/3961083788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f31140b2900 msgr2=0x7f31140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.279+0000 7f312acad700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f31140b2900 0x7f31140b4e10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3118003760 tx=0x7f311801a040 comp rx=0 tx=0).stop 2024-05-09T00:23:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.279+0000 7f312acad700 1 -- 172.21.15.46:0/3961083788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3124108760 msgr2=0x7f31240ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.279+0000 7f312acad700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3124108760 0x7f31240ffe90 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f311c00ef30 tx=0x7f311c00c3a0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.279+0000 7f312acad700 1 -- 172.21.15.46:0/3961083788 shutdown_connections 2024-05-09T00:23:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.279+0000 7f312acad700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f31140b2900 0x7f31140b4e10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.279+0000 7f312acad700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3124108760 0x7f31240ffe90 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.279+0000 7f312acad700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31241034e0 0x7f31240ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.279+0000 7f312acad700 1 --2- 172.21.15.46:0/3961083788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3124107e10 0x7f31240ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.279+0000 7f312acad700 1 -- 172.21.15.46:0/3961083788 >> 172.21.15.46:0/3961083788 conn(0x7f31240fef30 msgr2=0x7f312406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.279+0000 7f312acad700 1 -- 172.21.15.46:0/3961083788 shutdown_connections 2024-05-09T00:23:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:41.279+0000 7f312acad700 1 -- 172.21.15.46:0/3961083788 wait complete. 2024-05-09T00:23:42.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T00:23:42.284 INFO:tasks.workunit.client.0.smithi046.stdout:561/600: open fd = 3 2024-05-09T00:23:42.284 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.284 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:562/600: open fd = 3 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:563/600: open fd = 3 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:564/600: open fd = 3 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:565/600: open fd = 3 2024-05-09T00:23:42.285 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:566/600: open fd = 3 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:567/600: open fd = 3 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:568/600: open fd = 3 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.286 INFO:tasks.workunit.client.0.smithi046.stdout:569/600: open fd = 3 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:570/600: open fd = 3 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:571/600: open fd = 3 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:572/600: open fd = 3 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.287 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:573/600: open fd = 3 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:574/600: open fd = 3 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:575/600: open fd = 3 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:576/600: open fd = 3 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.288 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:577/600: open fd = 3 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:578/600: open fd = 3 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:579/600: open fd = 3 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:580/600: open fd = 3 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.289 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.290 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.290 INFO:tasks.workunit.client.0.smithi046.stdout:581/600: open fd = 3 2024-05-09T00:23:42.290 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.290 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.290 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.290 INFO:tasks.workunit.client.0.smithi046.stdout:582/600: open fd = 3 2024-05-09T00:23:42.290 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.290 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.290 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.290 INFO:tasks.workunit.client.0.smithi046.stdout:583/600: open fd = 3 2024-05-09T00:23:42.290 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.290 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:584/600: open fd = 3 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:585/600: open fd = 3 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:586/600: open fd = 3 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.291 INFO:tasks.workunit.client.0.smithi046.stdout:587/600: open fd = 3 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:588/600: open fd = 3 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:589/600: open fd = 3 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:590/600: open fd = 3 2024-05-09T00:23:42.292 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:591/600: open fd = 3 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:592/600: open fd = 3 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:593/600: open fd = 3 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.293 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:594/600: open fd = 3 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:595/600: open fd = 3 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:596/600: open fd = 3 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.294 INFO:tasks.workunit.client.0.smithi046.stdout:597/600: open fd = 3 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:598/600: open fd = 3 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:599/600: open fd = 3 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:600/600: open fd = 3 2024-05-09T00:23:42.295 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.296 INFO:tasks.workunit.client.0.smithi046.stdout:write ret = 32 2024-05-09T00:23:42.296 INFO:tasks.workunit.client.0.smithi046.stdout:pread ret = 64 2024-05-09T00:23:42.296 INFO:tasks.workunit.client.0.smithi046.stderr:+ echo OK 2024-05-09T00:23:42.297 INFO:tasks.workunit.client.0.smithi046.stdout:OK 2024-05-09T00:23:42.297 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T00:23:42.298 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T00:23:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:42.341 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-05-09T00:23:42.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T00:23:42.681 DEBUG:teuthology.parallel:result is None 2024-05-09T00:23:42.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T00:23:42.719 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T00:23:42.719 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-09T00:23:42.734 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-09T00:23:42.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:42 smithi046 ceph-mon[30151]: pgmap v746: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:43.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:42 smithi169 ceph-mon[34777]: pgmap v746: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:43.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:42 smithi142 ceph-mon[34533]: pgmap v746: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-09T00:23:44.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.062+0000 7fe087d2b700 1 -- 172.21.15.46:0/592270162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe080104650 msgr2=0x7fe080104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:44.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.062+0000 7fe087d2b700 1 --2- 172.21.15.46:0/592270162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe080104650 0x7fe080104a60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0099c0 tx=0x7fe07c009cd0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.062+0000 7fe087d2b700 1 -- 172.21.15.46:0/592270162 shutdown_connections 2024-05-09T00:23:44.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.062+0000 7fe087d2b700 1 --2- 172.21.15.46:0/592270162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe080109880 0x7fe0801111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.062+0000 7fe087d2b700 1 --2- 172.21.15.46:0/592270162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe080104650 0x7fe080104a60 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.062+0000 7fe087d2b700 1 --2- 172.21.15.46:0/592270162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe080108f30 0x7fe080109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.062+0000 7fe087d2b700 1 -- 172.21.15.46:0/592270162 >> 172.21.15.46:0/592270162 conn(0x7fe080079f00 msgr2=0x7fe08007a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:44.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.062+0000 7fe087d2b700 1 -- 172.21.15.46:0/592270162 shutdown_connections 2024-05-09T00:23:44.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.062+0000 7fe087d2b700 1 -- 172.21.15.46:0/592270162 wait complete. 2024-05-09T00:23:44.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe087d2b700 1 Processor -- start 2024-05-09T00:23:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe087d2b700 1 -- start start 2024-05-09T00:23:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe087d2b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe080104650 0x7fe08019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe087d2b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe080108f30 0x7fe08019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe087d2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe080109880 0x7fe08019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe087d2b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801144d0 con 0x7fe080109880 2024-05-09T00:23:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe087d2b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe080114350 con 0x7fe080108f30 2024-05-09T00:23:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe087d2b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe080114650 con 0x7fe080104650 2024-05-09T00:23:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe0862c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe080109880 0x7fe08019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe0862c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe080109880 0x7fe08019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48586/0 (socket says 172.21.15.46:48586) 2024-05-09T00:23:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe0862c8700 1 -- 172.21.15.46:0/2208603836 learned_addr learned my addr 172.21.15.46:0/2208603836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe085ac7700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe080104650 0x7fe08019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.063+0000 7fe0852c6700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe080108f30 0x7fe08019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.064+0000 7fe0862c8700 1 -- 172.21.15.46:0/2208603836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe080104650 msgr2=0x7fe08019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.064+0000 7fe0862c8700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe080104650 0x7fe08019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.064+0000 7fe0862c8700 1 -- 172.21.15.46:0/2208603836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe080108f30 msgr2=0x7fe08019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.064+0000 7fe0862c8700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe080108f30 0x7fe08019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.064+0000 7fe0862c8700 1 -- 172.21.15.46:0/2208603836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08019fd50 con 0x7fe080109880 2024-05-09T00:23:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.064+0000 7fe0862c8700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe080109880 0x7fe08019f6b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fe078007ee0 tx=0x7fe07800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.064+0000 7fe076ffd700 1 -- 172.21.15.46:0/2208603836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078004bb0 con 0x7fe080109880 2024-05-09T00:23:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.064+0000 7fe076ffd700 1 -- 172.21.15.46:0/2208603836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe078004d50 con 0x7fe080109880 2024-05-09T00:23:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.064+0000 7fe087d2b700 1 -- 172.21.15.46:0/2208603836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0801a0040 con 0x7fe080109880 2024-05-09T00:23:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.064+0000 7fe076ffd700 1 -- 172.21.15.46:0/2208603836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078005920 con 0x7fe080109880 2024-05-09T00:23:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.064+0000 7fe087d2b700 1 -- 172.21.15.46:0/2208603836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0801a7960 con 0x7fe080109880 2024-05-09T00:23:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.067+0000 7fe076ffd700 1 -- 172.21.15.46:0/2208603836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe07801a020 con 0x7fe080109880 2024-05-09T00:23:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.068+0000 7fe076ffd700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe06c0b2900 0x7fe06c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.068+0000 7fe096741b80 1 Processor -- start 2024-05-09T00:23:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.068+0000 7fe085ac7700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe06c0b2900 0x7fe06c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.068+0000 7fe076ffd700 1 -- 172.21.15.46:0/2208603836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe0780e1850 con 0x7fe080109880 2024-05-09T00:23:44.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.068+0000 7fe096741b80 1 -- start start 2024-05-09T00:23:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.068+0000 7fe096741b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de67a05160 0x55de67a9e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.068+0000 7fe096741b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de6791b4a0 0x55de67a9ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.068+0000 7fe096741b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a986a0 0x55de67a98ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe096741b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55de679baff0 con 0x55de6791b4a0 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe096741b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55de679597f0 con 0x55de67a986a0 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe096741b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55de679b6fb0 con 0x55de67a05160 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe085ac7700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe06c0b2900 0x7fe06c0b4e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe07c000c00 tx=0x7fe07c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe0852c6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de6791b4a0 0x55de67a9ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe0852c6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de6791b4a0 0x55de67a9ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48602/0 (socket says 172.21.15.46:48602) 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe0852c6700 1 -- 172.21.15.46:0/3849967773 learned_addr learned my addr 172.21.15.46:0/3849967773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe085ac7700 1 --2- 172.21.15.46:0/3849967773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de67a05160 0x55de67a9e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe0862c8700 1 --2- 172.21.15.46:0/3849967773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a986a0 0x55de67a98ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe0852c6700 1 -- 172.21.15.46:0/3849967773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de67a05160 msgr2=0x55de67a9e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe0852c6700 1 --2- 172.21.15.46:0/3849967773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de67a05160 0x55de67a9e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe0852c6700 1 -- 172.21.15.46:0/3849967773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a986a0 msgr2=0x55de67a98ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe0852c6700 1 --2- 172.21.15.46:0/3849967773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a986a0 0x55de67a98ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe0852c6700 1 -- 172.21.15.46:0/3849967773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de67951370 con 0x55de6791b4a0 2024-05-09T00:23:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe0852c6700 1 --2- 172.21.15.46:0/3849967773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de6791b4a0 0x55de67a9ea30 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe070009cf0 tx=0x7fe070011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2886e274be4c27 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe06bfff700 1 -- 172.21.15.46:0/3849967773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe070011d80 con 0x55de6791b4a0 2024-05-09T00:23:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe06bfff700 1 -- 172.21.15.46:0/3849967773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe070004500 con 0x55de6791b4a0 2024-05-09T00:23:44.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.069+0000 7fe06bfff700 1 -- 172.21.15.46:0/3849967773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe070019640 con 0x55de6791b4a0 2024-05-09T00:23:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.070+0000 7fe096741b80 1 -- 172.21.15.46:0/3849967773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de6791b4a0 msgr2=0x55de67a9ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.070+0000 7fe096741b80 1 --2- 172.21.15.46:0/3849967773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de6791b4a0 0x55de67a9ea30 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe070009cf0 tx=0x7fe070011440 comp rx=0 tx=0).stop 2024-05-09T00:23:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.070+0000 7fe096741b80 1 -- 172.21.15.46:0/3849967773 shutdown_connections 2024-05-09T00:23:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.070+0000 7fe096741b80 1 --2- 172.21.15.46:0/3849967773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a986a0 0x55de67a98ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.070+0000 7fe096741b80 1 --2- 172.21.15.46:0/3849967773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de67a05160 0x55de67a9e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.070+0000 7fe096741b80 1 --2- 172.21.15.46:0/3849967773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de6791b4a0 0x55de67a9ea30 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.070+0000 7fe096741b80 1 -- 172.21.15.46:0/3849967773 >> 172.21.15.46:0/3849967773 conn(0x55de6797b6a0 msgr2=0x55de67a93400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.071+0000 7fe096741b80 1 -- 172.21.15.46:0/3849967773 shutdown_connections 2024-05-09T00:23:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.071+0000 7fe096741b80 1 -- 172.21.15.46:0/3849967773 wait complete. 2024-05-09T00:23:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.071+0000 7fe096741b80 1 -- start start 2024-05-09T00:23:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.071+0000 7fe096741b80 1 Processor -- start 2024-05-09T00:23:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe096741b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de6791b4a0 0x55de67aa2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe096741b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de67a05160 0x55de67aa26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe096741b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 0x55de67aa2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe096741b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55de67836ba0 con 0x55de67a986a0 2024-05-09T00:23:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe096741b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55de67975150 con 0x55de6791b4a0 2024-05-09T00:23:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe096741b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55de67aa95e0 con 0x55de67a05160 2024-05-09T00:23:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe096741b80 10 client.0 _ll_register_callbacks cb 0x55de6782c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0862c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 0x55de67aa2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0862c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 0x55de67aa2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48614/0 (socket says 172.21.15.46:48614) 2024-05-09T00:23:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0862c8700 1 -- 172.21.15.46:0/2773139222 learned_addr learned my addr 172.21.15.46:0/2773139222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0852c6700 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de67a05160 0x55de67aa26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe085ac7700 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de6791b4a0 0x55de67aa2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0862c8700 1 -- 172.21.15.46:0/2773139222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de67a05160 msgr2=0x55de67aa26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0862c8700 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de67a05160 0x55de67aa26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0862c8700 1 -- 172.21.15.46:0/2773139222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de6791b4a0 msgr2=0x55de67aa2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:44.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0862c8700 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de6791b4a0 0x55de67aa2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0862c8700 1 -- 172.21.15.46:0/2773139222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de67836490 con 0x55de67a986a0 2024-05-09T00:23:44.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe085ac7700 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de6791b4a0 0x55de67aa2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:44.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0862c8700 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 0x55de67aa2bf0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe07800d700 tx=0x7fe078005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4ff0d74d5d563e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:44.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0697fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0697fa700 1 -- 172.21.15.46:0/2773139222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780a9530 con 0x55de67a986a0 2024-05-09T00:23:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.072+0000 7fe0697fa700 1 -- 172.21.15.46:0/2773139222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe0780b1a10 con 0x55de67a986a0 2024-05-09T00:23:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.073+0000 7fe096741b80 1 -- 172.21.15.46:0/2773139222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe070004500 con 0x55de67a986a0 2024-05-09T00:23:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.073+0000 7fe0697fa700 1 -- 172.21.15.46:0/2773139222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe0780c8050 con 0x55de67a986a0 2024-05-09T00:23:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.073+0000 7fe096741b80 10 client.17520 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.073+0000 7fe096741b80 1 -- 172.21.15.46:0/2773139222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55de67ab3fb0 con 0x55de67a986a0 2024-05-09T00:23:44.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.073+0000 7fe0697fa700 1 -- 172.21.15.46:0/2773139222 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe070004500 con 0x55de67a986a0 2024-05-09T00:23:44.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.073+0000 7fe0697fa700 1 -- 172.21.15.46:0/2773139222 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe0780a9090 con 0x55de67a986a0 2024-05-09T00:23:44.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.073+0000 7fe096741b80 10 client.17520 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:44.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.073+0000 7fe096741b80 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.073+0000 7fe096741b80 20 client.17520 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.073+0000 7fe096741b80 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55de67ab6540 0x55de67ab8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.073+0000 7fe096741b80 4 client.17520 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T00:23:44.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.073+0000 7fe096741b80 1 -- 172.21.15.46:0/2773139222 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de679c89d0 con 0x55de67ab6540 2024-05-09T00:23:44.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.074+0000 7fe085ac7700 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55de67ab6540 0x55de67ab8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.075+0000 7fe0697fa700 10 client.17520 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:44.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.075+0000 7fe085ac7700 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55de67ab6540 0x55de67ab8960 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b920951732113e server_cookie=cb46b9c47384a14d in_seq=0 out_seq=0 2024-05-09T00:23:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe0697fa700 1 -- 172.21.15.46:0/2773139222 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de679c89d0 con 0x55de67ab6540 2024-05-09T00:23:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe0697fa700 10 client.17520 handle_command_reply: tid=0 2024-05-09T00:23:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe096741b80 1 client.17520 shutdown 2024-05-09T00:23:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe096741b80 1 -- 172.21.15.46:0/2773139222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 msgr2=0x55de67aa2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe096741b80 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 0x55de67aa2bf0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe07800d700 tx=0x7fe078005290 comp rx=0 tx=0).stop 2024-05-09T00:23:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe096741b80 1 -- 172.21.15.46:0/2773139222 shutdown_connections 2024-05-09T00:23:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe096741b80 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55de67ab6540 0x55de67ab8960 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe096741b80 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 0x55de67aa2bf0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe096741b80 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de6791b4a0 0x55de67aa2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe096741b80 1 --2- 172.21.15.46:0/2773139222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de67a05160 0x55de67aa26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe0697fa700 0 client.17520 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:44.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe096741b80 1 -- 172.21.15.46:0/2773139222 >> 172.21.15.46:0/2773139222 conn(0x55de6797b6a0 msgr2=0x55de67a931c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:44.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe096741b80 1 -- 172.21.15.46:0/2773139222 shutdown_connections 2024-05-09T00:23:44.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe096741b80 1 -- 172.21.15.46:0/2773139222 wait complete. 2024-05-09T00:23:44.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.076+0000 7fe096741b80 20 client.17520 trim_cache size 0 max 16384 2024-05-09T00:23:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.091+0000 7fe096741b80 1 Processor -- start 2024-05-09T00:23:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.091+0000 7fe096741b80 1 -- start start 2024-05-09T00:23:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.091+0000 7fe096741b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de6791b4a0 0x55de67a9d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.091+0000 7fe096741b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a05160 0x55de67ab2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.091+0000 7fe096741b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a986a0 0x55de67abe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.091+0000 7fe096741b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801af340 con 0x55de67a05160 2024-05-09T00:23:44.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.091+0000 7fe096741b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801acbd0 con 0x55de67a986a0 2024-05-09T00:23:44.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.091+0000 7fe096741b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801aaa40 con 0x55de6791b4a0 2024-05-09T00:23:44.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe0852c6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a05160 0x55de67ab2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe0852c6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a05160 0x55de67ab2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48618/0 (socket says 172.21.15.46:48618) 2024-05-09T00:23:44.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe0852c6700 1 -- 172.21.15.46:0/662654371 learned_addr learned my addr 172.21.15.46:0/662654371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe085ac7700 1 --2- 172.21.15.46:0/662654371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de6791b4a0 0x55de67a9d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe0862c8700 1 --2- 172.21.15.46:0/662654371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a986a0 0x55de67abe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe0852c6700 1 -- 172.21.15.46:0/662654371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de6791b4a0 msgr2=0x55de67a9d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe0852c6700 1 --2- 172.21.15.46:0/662654371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de6791b4a0 0x55de67a9d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe0852c6700 1 -- 172.21.15.46:0/662654371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a986a0 msgr2=0x55de67abe0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe0852c6700 1 --2- 172.21.15.46:0/662654371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a986a0 0x55de67abe0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe0852c6700 1 -- 172.21.15.46:0/662654371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de67a9e050 con 0x55de67a05160 2024-05-09T00:23:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe0852c6700 1 --2- 172.21.15.46:0/662654371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a05160 0x55de67ab2420 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe0700097a0 tx=0x7fe070005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f066e67fa5b5a815 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe06b7fe700 1 -- 172.21.15.46:0/662654371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe070029020 con 0x55de67a05160 2024-05-09T00:23:44.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe06b7fe700 1 -- 172.21.15.46:0/662654371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe070013de0 con 0x55de67a05160 2024-05-09T00:23:44.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.092+0000 7fe06b7fe700 1 -- 172.21.15.46:0/662654371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0700036a0 con 0x55de67a05160 2024-05-09T00:23:44.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.093+0000 7fe096741b80 1 -- 172.21.15.46:0/662654371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a05160 msgr2=0x55de67ab2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:44.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.093+0000 7fe096741b80 1 --2- 172.21.15.46:0/662654371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a05160 0x55de67ab2420 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe0700097a0 tx=0x7fe070005de0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.093+0000 7fe096741b80 1 -- 172.21.15.46:0/662654371 shutdown_connections 2024-05-09T00:23:44.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.093+0000 7fe096741b80 1 --2- 172.21.15.46:0/662654371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a986a0 0x55de67abe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.093+0000 7fe096741b80 1 --2- 172.21.15.46:0/662654371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de6791b4a0 0x55de67a9d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.093+0000 7fe096741b80 1 --2- 172.21.15.46:0/662654371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a05160 0x55de67ab2420 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.093+0000 7fe096741b80 1 -- 172.21.15.46:0/662654371 >> 172.21.15.46:0/662654371 conn(0x55de6797b6a0 msgr2=0x55de67a9d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:44.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.093+0000 7fe096741b80 1 -- 172.21.15.46:0/662654371 shutdown_connections 2024-05-09T00:23:44.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.093+0000 7fe096741b80 1 -- 172.21.15.46:0/662654371 wait complete. 2024-05-09T00:23:44.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe096741b80 1 -- start start 2024-05-09T00:23:44.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe096741b80 1 Processor -- start 2024-05-09T00:23:44.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe096741b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de6791b4a0 0x55de67aaea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe096741b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a05160 0x55de67aabdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe096741b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 0x55de67aac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe096741b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801ab090 con 0x55de67a986a0 2024-05-09T00:23:44.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe096741b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801ac500 con 0x55de67a05160 2024-05-09T00:23:44.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe096741b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801ab760 con 0x55de6791b4a0 2024-05-09T00:23:44.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe096741b80 10 client.0 _ll_register_callbacks cb 0x55de6782c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T00:23:44.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe0862c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 0x55de67aac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe0862c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 0x55de67aac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48628/0 (socket says 172.21.15.46:48628) 2024-05-09T00:23:44.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe0862c8700 1 -- 172.21.15.46:0/3753364976 learned_addr learned my addr 172.21.15.46:0/3753364976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:44.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.094+0000 7fe085ac7700 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de6791b4a0 0x55de67aaea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe0852c6700 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a05160 0x55de67aabdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe0862c8700 1 -- 172.21.15.46:0/3753364976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de6791b4a0 msgr2=0x55de67aaea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:44.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe0862c8700 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de6791b4a0 0x55de67aaea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe0862c8700 1 -- 172.21.15.46:0/3753364976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a05160 msgr2=0x55de67aabdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T00:23:44.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe0862c8700 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a05160 0x55de67aabdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe0862c8700 1 -- 172.21.15.46:0/3753364976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de67aac830 con 0x55de67a986a0 2024-05-09T00:23:44.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe0862c8700 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 0x55de67aac2f0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fe07806eb60 tx=0x7fe0780051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6c65c72f6ae0fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:44.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe0697fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T00:23:44.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe0697fa700 1 -- 172.21.15.46:0/3753364976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780a9530 con 0x55de67a986a0 2024-05-09T00:23:44.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe0697fa700 1 -- 172.21.15.46:0/3753364976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fe0780a5420 con 0x55de67a986a0 2024-05-09T00:23:44.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe096741b80 1 -- 172.21.15.46:0/3753364976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe070013de0 con 0x55de67a986a0 2024-05-09T00:23:44.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe0697fa700 1 -- 172.21.15.46:0/3753364976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe07806ef90 con 0x55de67a986a0 2024-05-09T00:23:44.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe096741b80 10 client.17526 fetch_fsmap learned FSMap version 26 2024-05-09T00:23:44.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.095+0000 7fe096741b80 1 -- 172.21.15.46:0/3753364976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55de67b20890 con 0x55de67a986a0 2024-05-09T00:23:44.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.096+0000 7fe0697fa700 1 -- 172.21.15.46:0/3753364976 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe070013de0 con 0x55de67a986a0 2024-05-09T00:23:44.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.096+0000 7fe0697fa700 1 -- 172.21.15.46:0/3753364976 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe07808d020 con 0x55de67a986a0 2024-05-09T00:23:44.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.096+0000 7fe096741b80 10 client.17526 fetch_fsmap finished waiting for FSMap version 26 2024-05-09T00:23:44.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.096+0000 7fe096741b80 10 client.17526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-05-09T00:23:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.096+0000 7fe096741b80 20 client.17526 populate_metadata read hostname 'smithi046' 2024-05-09T00:23:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.096+0000 7fe096741b80 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55de67b20f30 0x55de67b23350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.096+0000 7fe096741b80 4 client.17526 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T00:23:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.096+0000 7fe096741b80 1 -- 172.21.15.46:0/3753364976 --> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de679c2a70 con 0x55de67b20f30 2024-05-09T00:23:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.096+0000 7fe085ac7700 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55de67b20f30 0x55de67b23350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe085ac7700 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55de67b20f30 0x55de67b23350 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4541aac79f60c1ce server_cookie=8c6a1abe625d9224 in_seq=0 out_seq=0 2024-05-09T00:23:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe0697fa700 10 client.17526 ms_handle_connect on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe0697fa700 1 -- 172.21.15.46:0/3753364976 <== mds.0 v2:172.21.15.169:6838/1664161265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de679c2a70 con 0x55de67b20f30 2024-05-09T00:23:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe0697fa700 10 client.17526 handle_command_reply: tid=0 2024-05-09T00:23:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe096741b80 1 client.17526 shutdown 2024-05-09T00:23:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe096741b80 1 -- 172.21.15.46:0/3753364976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 msgr2=0x55de67aac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe096741b80 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 0x55de67aac2f0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fe07806eb60 tx=0x7fe0780051f0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe096741b80 1 -- 172.21.15.46:0/3753364976 shutdown_connections 2024-05-09T00:23:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe096741b80 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.169:6838/1664161265,v1:172.21.15.169:6839/1664161265] conn(0x55de67b20f30 0x55de67b23350 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe096741b80 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de67a986a0 0x55de67aac2f0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe096741b80 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55de6791b4a0 0x55de67aaea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe096741b80 1 --2- 172.21.15.46:0/3753364976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de67a05160 0x55de67aabdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe0697fa700 0 client.17526 ms_handle_reset on v2:172.21.15.169:6838/1664161265 2024-05-09T00:23:44.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe096741b80 1 -- 172.21.15.46:0/3753364976 >> 172.21.15.46:0/3753364976 conn(0x55de6797b6a0 msgr2=0x55de67a9be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:44.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe096741b80 1 -- 172.21.15.46:0/3753364976 shutdown_connections 2024-05-09T00:23:44.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe096741b80 1 -- 172.21.15.46:0/3753364976 wait complete. 2024-05-09T00:23:44.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.097+0000 7fe096741b80 20 client.17526 trim_cache size 0 max 16384 2024-05-09T00:23:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T00:23:44.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.098+0000 7fe087d2b700 1 -- 172.21.15.46:0/2208603836 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe06c0b2900 msgr2=0x7fe06c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.098+0000 7fe087d2b700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe06c0b2900 0x7fe06c0b4e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe07c000c00 tx=0x7fe07c01a040 comp rx=0 tx=0).stop 2024-05-09T00:23:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.098+0000 7fe087d2b700 1 -- 172.21.15.46:0/2208603836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe080109880 msgr2=0x7fe08019f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.098+0000 7fe087d2b700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe080109880 0x7fe08019f6b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fe078007ee0 tx=0x7fe07800d3b0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.098+0000 7fe087d2b700 1 -- 172.21.15.46:0/2208603836 shutdown_connections 2024-05-09T00:23:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.098+0000 7fe087d2b700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fe06c0b2900 0x7fe06c0b4e10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.098+0000 7fe087d2b700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe080109880 0x7fe08019f6b0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.098+0000 7fe087d2b700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe080104650 0x7fe08019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.098+0000 7fe087d2b700 1 --2- 172.21.15.46:0/2208603836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe080108f30 0x7fe08019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.098+0000 7fe087d2b700 1 -- 172.21.15.46:0/2208603836 >> 172.21.15.46:0/2208603836 conn(0x7fe080079f00 msgr2=0x7fe080102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.098+0000 7fe087d2b700 1 -- 172.21.15.46:0/2208603836 shutdown_connections 2024-05-09T00:23:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:44.098+0000 7fe087d2b700 1 -- 172.21.15.46:0/2208603836 wait complete. 2024-05-09T00:23:44.803 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-09T00:23:44.804 INFO:tasks.fwd_scrub:done joining 2024-05-09T00:23:44.804 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T00:23:44.933 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T00:23:44.959 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:44 smithi046 ceph-mon[30151]: pgmap v747: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:23:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.055+0000 7f719182b640 1 -- 172.21.15.46:0/1971879860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c102380 msgr2=0x7f718c10bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.055+0000 7f719182b640 1 --2- 172.21.15.46:0/1971879860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c102380 0x7f718c10bde0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7184009ca0 tx=0x7f718402a8c0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.056+0000 7f719182b640 1 -- 172.21.15.46:0/1971879860 shutdown_connections 2024-05-09T00:23:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.056+0000 7f719182b640 1 --2- 172.21.15.46:0/1971879860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c102380 0x7f718c10bde0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.056+0000 7f719182b640 1 --2- 172.21.15.46:0/1971879860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f718c101880 0x7f718c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.056+0000 7f719182b640 1 --2- 172.21.15.46:0/1971879860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f718c1051d0 0x7f718c1055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.056+0000 7f719182b640 1 -- 172.21.15.46:0/1971879860 >> 172.21.15.46:0/1971879860 conn(0x7f718c0fd4e0 msgr2=0x7f718c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.056+0000 7f719182b640 1 -- 172.21.15.46:0/1971879860 shutdown_connections 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.056+0000 7f719182b640 1 -- 172.21.15.46:0/1971879860 wait complete. 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.056+0000 7f719182b640 1 Processor -- start 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.056+0000 7f719182b640 1 -- start start 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f719182b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c101880 0x7f718c104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f719182b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f718c102380 0x7f718c102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f719182b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f718c1051d0 0x7f718c102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f719182b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f718c10f760 con 0x7f718c101880 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f719182b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f718c10f5e0 con 0x7f718c102380 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f719182b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f718c10f8e0 con 0x7f718c1051d0 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f718affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c101880 0x7f718c104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f718affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c101880 0x7f718c104380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48650/0 (socket says 172.21.15.46:48650) 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f718affd640 1 -- 172.21.15.46:0/3031312939 learned_addr learned my addr 172.21.15.46:0/3031312939 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f7183fff640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f718c102380 0x7f718c102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f718b7fe640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f718c1051d0 0x7f718c102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f718affd640 1 -- 172.21.15.46:0/3031312939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f718c1051d0 msgr2=0x7f718c102f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f718affd640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f718c1051d0 0x7f718c102f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f718affd640 1 -- 172.21.15.46:0/3031312939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f718c102380 msgr2=0x7f718c102a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f718affd640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f718c102380 0x7f718c102a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f718affd640 1 -- 172.21.15.46:0/3031312939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c103690 con 0x7f718c101880 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f718b7fe640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f718c1051d0 0x7f718c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f7183fff640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f718c102380 0x7f718c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f718affd640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c101880 0x7f718c104380 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f717c00c970 tx=0x7f717c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f7190829640 1 -- 172.21.15.46:0/3031312939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c004220 con 0x7f718c101880 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f7190829640 1 -- 172.21.15.46:0/3031312939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f717c0043c0 con 0x7f718c101880 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f7190829640 1 -- 172.21.15.46:0/3031312939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c004a00 con 0x7f718c101880 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f719182b640 1 -- 172.21.15.46:0/3031312939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c1a43a0 con 0x7f718c101880 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.057+0000 7f719182b640 1 -- 172.21.15.46:0/3031312939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c1a4860 con 0x7f718c101880 2024-05-09T00:23:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.058+0000 7f719182b640 1 -- 172.21.15.46:0/3031312939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c10ab70 con 0x7f718c101880 2024-05-09T00:23:45.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.060+0000 7f7190829640 1 -- 172.21.15.46:0/3031312939 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f717c0079b0 con 0x7f718c101880 2024-05-09T00:23:45.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.061+0000 7f7190829640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71780b2870 0x7f71780b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:45.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.061+0000 7f7183fff640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71780b2870 0x7f71780b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:45.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.061+0000 7f7190829640 1 -- 172.21.15.46:0/3031312939 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f717c0e2400 con 0x7f718c101880 2024-05-09T00:23:45.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.061+0000 7f7183fff640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71780b2870 0x7f71780b4d80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7174005d00 tx=0x7f7174005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:45.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.065+0000 7f7190829640 1 -- 172.21.15.46:0/3031312939 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f717c014030 con 0x7f718c101880 2024-05-09T00:23:45.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:44 smithi169 ceph-mon[34777]: pgmap v747: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:23:45.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:44 smithi142 ceph-mon[34533]: pgmap v747: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-09T00:23:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.261+0000 7f719182b640 1 -- 172.21.15.46:0/3031312939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f718c02de00 con 0x7f718c101880 2024-05-09T00:23:45.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.262+0000 7f7190829640 1 -- 172.21.15.46:0/3031312939 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22694 (secure 0 0 0) 0x7f718c02de00 con 0x7f718c101880 2024-05-09T00:23:45.263 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:23:45.264 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","created":"2024-05-09T00:00:14.356524+0000","modified":"2024-05-09T00:09:10.458026+0000","last_up_change":"2024-05-09T00:07:23.332796+0000","last_in_change":"2024-05-09T00:07:03.238539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:03:33.278910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:08:21.014416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:08:24.922968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"740d6388-5c1f-45eb-93b0-7583bfa5a225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6803","nonce":3727342125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6805","nonce":3727342125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6809","nonce":3727342125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6807","nonce":3727342125}]},"public_addr":"172.21.15.46:6803/3727342125","cluster_addr":"172.21.15.46:6805/3727342125","heartbeat_back_addr":"172.21.15.46:6809/3727342125","heartbeat_front_addr":"172.21.15.46:6807/3727342125","state":["exists","up"]},{"osd":1,"uuid":"85fbf3ae-0be8-4504-84d6-66fae0d2169d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6801","nonce":3156042469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6803","nonce":3156042469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6807","nonce":3156042469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6805","nonce":3156042469}]},"public_addr":"172.21.15.142:6801/3156042469","cluster_addr":"172.21.15.142:6803/3156042469","heartbeat_back_addr":"172.21.15.142:6807/3156042469","heartbeat_front_addr":"172.21.15.142:6805/3156042469","state":["exists","up"]},{"osd":2,"uuid":"05c070e7-83ad-4949-9834-4ce51ac0a2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6801","nonce":2680673064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6803","nonce":2680673064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6807","nonce":2680673064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6805","nonce":2680673064}]},"public_addr":"172.21.15.169:6801/2680673064","cluster_addr":"172.21.15.169:6803/2680673064","heartbeat_back_addr":"172.21.15.169:6807/2680673064","heartbeat_front_addr":"172.21.15.169:6805/2680673064","state":["exists","up"]},{"osd":3,"uuid":"09d7bbe1-58b6-4407-bef2-920f179dfb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6811","nonce":1158536261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6813","nonce":1158536261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6817","nonce":1158536261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6815","nonce":1158536261}]},"public_addr":"172.21.15.46:6811/1158536261","cluster_addr":"172.21.15.46:6813/1158536261","heartbeat_back_addr":"172.21.15.46:6817/1158536261","heartbeat_front_addr":"172.21.15.46:6815/1158536261","state":["exists","up"]},{"osd":4,"uuid":"66c9644b-6b0b-4fe7-aa63-b9d87bff7153","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6809","nonce":2299616865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6811","nonce":2299616865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6815","nonce":2299616865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6813","nonce":2299616865}]},"public_addr":"172.21.15.142:6809/2299616865","cluster_addr":"172.21.15.142:6811/2299616865","heartbeat_back_addr":"172.21.15.142:6815/2299616865","heartbeat_front_addr":"172.21.15.142:6813/2299616865","state":["exists","up"]},{"osd":5,"uuid":"96a72e01-1c5f-418e-bb9b-b790d9b3059b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6809","nonce":445044713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6811","nonce":445044713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6815","nonce":445044713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6813","nonce":445044713}]},"public_addr":"172.21.15.169:6809/445044713","cluster_addr":"172.21.15.169:6811/445044713","heartbeat_back_addr":"172.21.15.169:6815/445044713","heartbeat_front_addr":"172.21.15.169:6813/445044713","state":["exists","up"]},{"osd":6,"uuid":"6b459542-0e56-4369-b542-e36c46a18f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6819","nonce":1394035482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6821","nonce":1394035482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6825","nonce":1394035482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6823","nonce":1394035482}]},"public_addr":"172.21.15.46:6819/1394035482","cluster_addr":"172.21.15.46:6821/1394035482","heartbeat_back_addr":"172.21.15.46:6825/1394035482","heartbeat_front_addr":"172.21.15.46:6823/1394035482","state":["exists","up"]},{"osd":7,"uuid":"786b4517-4cbd-4fa3-b5a0-39cccf839893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6817","nonce":596805607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6819","nonce":596805607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6823","nonce":596805607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6821","nonce":596805607}]},"public_addr":"172.21.15.142:6817/596805607","cluster_addr":"172.21.15.142:6819/596805607","heartbeat_back_addr":"172.21.15.142:6823/596805607","heartbeat_front_addr":"172.21.15.142:6821/596805607","state":["exists","up"]},{"osd":8,"uuid":"b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6817","nonce":2650652376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6819","nonce":2650652376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6823","nonce":2650652376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6821","nonce":2650652376}]},"public_addr":"172.21.15.169:6817/2650652376","cluster_addr":"172.21.15.169:6819/2650652376","heartbeat_back_addr":"172.21.15.169:6823/2650652376","heartbeat_front_addr":"172.21.15.169:6821/2650652376","state":["exists","up"]},{"osd":9,"uuid":"32df0dab-5875-4beb-b733-60a915c94a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6827","nonce":374861248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6829","nonce":374861248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6833","nonce":374861248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6831","nonce":374861248}]},"public_addr":"172.21.15.46:6827/374861248","cluster_addr":"172.21.15.46:6829/374861248","heartbeat_back_addr":"172.21.15.46:6833/374861248","heartbeat_front_addr":"172.21.15.46:6831/374861248","state":["exists","up"]},{"osd":10,"uuid":"380a4f47-e528-4258-8836-32eac0e6e630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6825","nonce":1364891982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6827","nonce":1364891982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6831","nonce":1364891982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6829","nonce":1364891982}]},"public_addr":"172.21.15.142:6825/1364891982","cluster_addr":"172.21.15.142:6827/1364891982","heartbeat_back_addr":"172.21.15.142:6831/1364891982","heartbeat_front_addr":"172.21.15.142:6829/1364891982","state":["exists","up"]},{"osd":11,"uuid":"56b04b15-6fcd-497a-aa66-19da9624b656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6825","nonce":4262478976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6827","nonce":4262478976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6831","nonce":4262478976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6829","nonce":4262478976}]},"public_addr":"172.21.15.169:6825/4262478976","cluster_addr":"172.21.15.169:6827/4262478976","heartbeat_back_addr":"172.21.15.169:6831/4262478976","heartbeat_front_addr":"172.21.15.169:6829/4262478976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:02:42.159298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:06.053328+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:29.587137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:54.573782+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:20.276073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:45.320565+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:10.985522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:36.806137+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:02.266736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:29.788773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:55.827639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:07:21.762382+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/1072138393":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2786891364":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/7723180":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/626888490":"2024-05-10T00:01:09.191239+0000","172.21.15.46:6800/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/4236543921":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2168313126":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6800/953818590":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6801/953818590":"2024-05-10T00:00:41.833323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:23:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.264+0000 7f719182b640 1 -- 172.21.15.46:0/3031312939 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71780b2870 msgr2=0x7f71780b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.264+0000 7f719182b640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71780b2870 0x7f71780b4d80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7174005d00 tx=0x7f7174005c90 comp rx=0 tx=0).stop 2024-05-09T00:23:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.264+0000 7f719182b640 1 -- 172.21.15.46:0/3031312939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c101880 msgr2=0x7f718c104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.264+0000 7f719182b640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c101880 0x7f718c104380 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f717c00c970 tx=0x7f717c00ce30 comp rx=0 tx=0).stop 2024-05-09T00:23:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.264+0000 7f719182b640 1 -- 172.21.15.46:0/3031312939 shutdown_connections 2024-05-09T00:23:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.264+0000 7f719182b640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f71780b2870 0x7f71780b4d80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.264+0000 7f719182b640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f718c1051d0 0x7f718c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.264+0000 7f719182b640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f718c102380 0x7f718c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.264+0000 7f719182b640 1 --2- 172.21.15.46:0/3031312939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c101880 0x7f718c104380 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.264+0000 7f719182b640 1 -- 172.21.15.46:0/3031312939 >> 172.21.15.46:0/3031312939 conn(0x7f718c0fd4e0 msgr2=0x7f718c107d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.264+0000 7f719182b640 1 -- 172.21.15.46:0/3031312939 shutdown_connections 2024-05-09T00:23:45.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.264+0000 7f719182b640 1 -- 172.21.15.46:0/3031312939 wait complete. 2024-05-09T00:23:45.277 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T00:03:33.278910+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T00:08:21.014416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T00:08:24.922968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T00:23:45.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T00:23:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.435+0000 7f1e06f1f640 1 -- 172.21.15.46:0/2646877654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e00102b30 msgr2=0x7f1e00103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.435+0000 7f1e06f1f640 1 --2- 172.21.15.46:0/2646877654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e00102b30 0x7f1e00103830 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009ca0 tx=0x7f1df802a8c0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.435+0000 7f1e06f1f640 1 -- 172.21.15.46:0/2646877654 shutdown_connections 2024-05-09T00:23:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.435+0000 7f1e06f1f640 1 --2- 172.21.15.46:0/2646877654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00108a70 0x7f1e0010ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.435+0000 7f1e06f1f640 1 --2- 172.21.15.46:0/2646877654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e00103d70 0x7f1e00104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.435+0000 7f1e06f1f640 1 --2- 172.21.15.46:0/2646877654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e00102b30 0x7f1e00103830 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.435+0000 7f1e06f1f640 1 -- 172.21.15.46:0/2646877654 >> 172.21.15.46:0/2646877654 conn(0x7f1e000fdd20 msgr2=0x7f1e001001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.435+0000 7f1e06f1f640 1 -- 172.21.15.46:0/2646877654 shutdown_connections 2024-05-09T00:23:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.435+0000 7f1e06f1f640 1 -- 172.21.15.46:0/2646877654 wait complete. 2024-05-09T00:23:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.435+0000 7f1e06f1f640 1 Processor -- start 2024-05-09T00:23:45.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.435+0000 7f1e06f1f640 1 -- start start 2024-05-09T00:23:45.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e06f1f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e00102b30 0x7f1e0019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:45.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e06f1f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00103d70 0x7f1e0019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:45.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e06f1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e00108a70 0x7f1e001a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:45.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e06f1f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0010e610 con 0x7f1e00108a70 2024-05-09T00:23:45.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e06f1f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0010e490 con 0x7f1e00103d70 2024-05-09T00:23:45.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e06f1f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0010e790 con 0x7f1e00102b30 2024-05-09T00:23:45.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e054bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e00108a70 0x7f1e001a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:45.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e054bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e00108a70 0x7f1e001a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48668/0 (socket says 172.21.15.46:48668) 2024-05-09T00:23:45.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e054bc640 1 -- 172.21.15.46:0/1157670627 learned_addr learned my addr 172.21.15.46:0/1157670627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:45.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e04cbb640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e00102b30 0x7f1e0019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1dfffff640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00103d70 0x7f1e0019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e054bc640 1 -- 172.21.15.46:0/1157670627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e00102b30 msgr2=0x7f1e0019a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e054bc640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e00102b30 0x7f1e0019a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e054bc640 1 -- 172.21.15.46:0/1157670627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00103d70 msgr2=0x7f1e0019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e054bc640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00103d70 0x7f1e0019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e054bc640 1 -- 172.21.15.46:0/1157670627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e001a1ee0 con 0x7f1e00108a70 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e04cbb640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e00102b30 0x7f1e0019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e054bc640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e00108a70 0x7f1e001a1870 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1df000d950 tx=0x7f1df000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1dfdffb640 1 -- 172.21.15.46:0/1157670627 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0017030 con 0x7f1e00108a70 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1dfdffb640 1 -- 172.21.15.46:0/1157670627 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df0004020 con 0x7f1e00108a70 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1e06f1f640 1 -- 172.21.15.46:0/1157670627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e001a21d0 con 0x7f1e00108a70 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.436+0000 7f1dfdffb640 1 -- 172.21.15.46:0/1157670627 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0003b50 con 0x7f1e00108a70 2024-05-09T00:23:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.437+0000 7f1e06f1f640 1 -- 172.21.15.46:0/1157670627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e001a26d0 con 0x7f1e00108a70 2024-05-09T00:23:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.437+0000 7f1e06f1f640 1 -- 172.21.15.46:0/1157670627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e001a2990 con 0x7f1e00108a70 2024-05-09T00:23:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.439+0000 7f1dfdffb640 1 -- 172.21.15.46:0/1157670627 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1df001a070 con 0x7f1e00108a70 2024-05-09T00:23:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.440+0000 7f1dfdffb640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1de80b27f0 0x7f1de80b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.440+0000 7f1dfdffb640 1 -- 172.21.15.46:0/1157670627 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1df00e1e10 con 0x7f1e00108a70 2024-05-09T00:23:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.440+0000 7f1e04cbb640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1de80b27f0 0x7f1de80b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:45.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.440+0000 7f1e04cbb640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1de80b27f0 0x7f1de80b4d00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1df8004af0 tx=0x7f1df8000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:45.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.444+0000 7f1dfdffb640 1 -- 172.21.15.46:0/1157670627 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1df0021070 con 0x7f1e00108a70 2024-05-09T00:23:45.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.638+0000 7f1e06f1f640 1 -- 172.21.15.46:0/1157670627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e001a2b60 con 0x7f1e00108a70 2024-05-09T00:23:45.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.639+0000 7f1dfdffb640 1 -- 172.21.15.46:0/1157670627 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f1df00a9b30 con 0x7f1e00108a70 2024-05-09T00:23:45.640 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-09T00:23:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.641+0000 7f1e06f1f640 1 -- 172.21.15.46:0/1157670627 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1de80b27f0 msgr2=0x7f1de80b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.641+0000 7f1e06f1f640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1de80b27f0 0x7f1de80b4d00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1df8004af0 tx=0x7f1df8000f40 comp rx=0 tx=0).stop 2024-05-09T00:23:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.641+0000 7f1e06f1f640 1 -- 172.21.15.46:0/1157670627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e00108a70 msgr2=0x7f1e001a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.641+0000 7f1e06f1f640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e00108a70 0x7f1e001a1870 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1df000d950 tx=0x7f1df000de10 comp rx=0 tx=0).stop 2024-05-09T00:23:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.641+0000 7f1e06f1f640 1 -- 172.21.15.46:0/1157670627 shutdown_connections 2024-05-09T00:23:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.641+0000 7f1e06f1f640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1de80b27f0 0x7f1de80b4d00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.641+0000 7f1e06f1f640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e00108a70 0x7f1e001a1870 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.641+0000 7f1e06f1f640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e00103d70 0x7f1e0019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.641+0000 7f1e06f1f640 1 --2- 172.21.15.46:0/1157670627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e00102b30 0x7f1e0019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.641+0000 7f1e06f1f640 1 -- 172.21.15.46:0/1157670627 >> 172.21.15.46:0/1157670627 conn(0x7f1e000fdd20 msgr2=0x7f1e0010a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:45.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.641+0000 7f1e06f1f640 1 -- 172.21.15.46:0/1157670627 shutdown_connections 2024-05-09T00:23:45.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.641+0000 7f1e06f1f640 1 -- 172.21.15.46:0/1157670627 wait complete. 2024-05-09T00:23:45.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T00:23:45.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.771+0000 7f80d0e7e640 1 -- 172.21.15.46:0/2464180744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80cc1051b0 msgr2=0x7f80cc1055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.771+0000 7f80d0e7e640 1 --2- 172.21.15.46:0/2464180744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80cc1051b0 0x7f80cc1055c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009900 tx=0x7f80bc02a410 comp rx=0 tx=0).stop 2024-05-09T00:23:45.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.771+0000 7f80d0e7e640 1 -- 172.21.15.46:0/2464180744 shutdown_connections 2024-05-09T00:23:45.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.771+0000 7f80d0e7e640 1 --2- 172.21.15.46:0/2464180744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80cc102360 0x7f80cc10bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.771+0000 7f80d0e7e640 1 --2- 172.21.15.46:0/2464180744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80cc101860 0x7f80cc101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.771+0000 7f80d0e7e640 1 --2- 172.21.15.46:0/2464180744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80cc1051b0 0x7f80cc1055c0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.771+0000 7f80d0e7e640 1 -- 172.21.15.46:0/2464180744 >> 172.21.15.46:0/2464180744 conn(0x7f80cc0fd460 msgr2=0x7f80cc0ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:45.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.771+0000 7f80d0e7e640 1 -- 172.21.15.46:0/2464180744 shutdown_connections 2024-05-09T00:23:45.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.771+0000 7f80d0e7e640 1 -- 172.21.15.46:0/2464180744 wait complete. 2024-05-09T00:23:45.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.772+0000 7f80d0e7e640 1 Processor -- start 2024-05-09T00:23:45.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.772+0000 7f80d0e7e640 1 -- start start 2024-05-09T00:23:45.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80d0e7e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80cc101860 0x7f80cc19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:45.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80d0e7e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80cc102360 0x7f80cc19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:45.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80d0e7e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80cc1051b0 0x7f80cc1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:45.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80d0e7e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc10f720 con 0x7f80cc1051b0 2024-05-09T00:23:45.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80d0e7e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc10f5a0 con 0x7f80cc101860 2024-05-09T00:23:45.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80d0e7e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc10f8a0 con 0x7f80cc102360 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80cad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80cc1051b0 0x7f80cc1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80cad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80cc1051b0 0x7f80cc1a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48688/0 (socket says 172.21.15.46:48688) 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80cad9d640 1 -- 172.21.15.46:0/1466380989 learned_addr learned my addr 172.21.15.46:0/1466380989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80c9d9b640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80cc102360 0x7f80cc19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80ca59c640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80cc101860 0x7f80cc19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80cad9d640 1 -- 172.21.15.46:0/1466380989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80cc102360 msgr2=0x7f80cc19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80cad9d640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80cc102360 0x7f80cc19a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80cad9d640 1 -- 172.21.15.46:0/1466380989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80cc101860 msgr2=0x7f80cc19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80cad9d640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80cc101860 0x7f80cc19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80cad9d640 1 -- 172.21.15.46:0/1466380989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc1a1ec0 con 0x7f80cc1051b0 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80cad9d640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80cc1051b0 0x7f80cc1a1850 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009fd0 tx=0x7f80c400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80bb7fe640 1 -- 172.21.15.46:0/1466380989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4017070 con 0x7f80cc1051b0 2024-05-09T00:23:45.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80d0e7e640 1 -- 172.21.15.46:0/1466380989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc1a21b0 con 0x7f80cc1051b0 2024-05-09T00:23:45.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.773+0000 7f80bb7fe640 1 -- 172.21.15.46:0/1466380989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c4003e20 con 0x7f80cc1051b0 2024-05-09T00:23:45.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.774+0000 7f80bb7fe640 1 -- 172.21.15.46:0/1466380989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4010460 con 0x7f80cc1051b0 2024-05-09T00:23:45.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.774+0000 7f80d0e7e640 1 -- 172.21.15.46:0/1466380989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc1a2700 con 0x7f80cc1051b0 2024-05-09T00:23:45.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.775+0000 7f80d0e7e640 1 -- 172.21.15.46:0/1466380989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80cc108b40 con 0x7f80cc1051b0 2024-05-09T00:23:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.776+0000 7f80bb7fe640 1 -- 172.21.15.46:0/1466380989 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f80c4003970 con 0x7f80cc1051b0 2024-05-09T00:23:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.777+0000 7f80bb7fe640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f80b40b27f0 0x7f80b40b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.777+0000 7f80ca59c640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f80b40b27f0 0x7f80b40b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.777+0000 7f80bb7fe640 1 -- 172.21.15.46:0/1466380989 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f80c40e2330 con 0x7f80cc1051b0 2024-05-09T00:23:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.778+0000 7f80ca59c640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f80b40b27f0 0x7f80b40b4d00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f80bc005ab0 tx=0x7f80bc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:45.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.780+0000 7f80bb7fe640 1 -- 172.21.15.46:0/1466380989 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80c40a99c0 con 0x7f80cc1051b0 2024-05-09T00:23:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.973+0000 7f80d0e7e640 1 -- 172.21.15.46:0/1466380989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f80cc02dc30 con 0x7f80cc1051b0 2024-05-09T00:23:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.974+0000 7f80bb7fe640 1 -- 172.21.15.46:0/1466380989 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f80c40b2090 con 0x7f80cc1051b0 2024-05-09T00:23:45.975 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-09T00:23:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.976+0000 7f80d0e7e640 1 -- 172.21.15.46:0/1466380989 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f80b40b27f0 msgr2=0x7f80b40b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.976+0000 7f80d0e7e640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f80b40b27f0 0x7f80b40b4d00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f80bc005ab0 tx=0x7f80bc031400 comp rx=0 tx=0).stop 2024-05-09T00:23:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.976+0000 7f80d0e7e640 1 -- 172.21.15.46:0/1466380989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80cc1051b0 msgr2=0x7f80cc1a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.976+0000 7f80d0e7e640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80cc1051b0 0x7f80cc1a1850 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009fd0 tx=0x7f80c400b260 comp rx=0 tx=0).stop 2024-05-09T00:23:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.976+0000 7f80d0e7e640 1 -- 172.21.15.46:0/1466380989 shutdown_connections 2024-05-09T00:23:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.976+0000 7f80d0e7e640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f80b40b27f0 0x7f80b40b4d00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.976+0000 7f80d0e7e640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80cc1051b0 0x7f80cc1a1850 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.976+0000 7f80d0e7e640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80cc102360 0x7f80cc19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.976+0000 7f80d0e7e640 1 --2- 172.21.15.46:0/1466380989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80cc101860 0x7f80cc19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.976+0000 7f80d0e7e640 1 -- 172.21.15.46:0/1466380989 >> 172.21.15.46:0/1466380989 conn(0x7f80cc0fd460 msgr2=0x7f80cc105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.976+0000 7f80d0e7e640 1 -- 172.21.15.46:0/1466380989 shutdown_connections 2024-05-09T00:23:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:45.976+0000 7f80d0e7e640 1 -- 172.21.15.46:0/1466380989 wait complete. 2024-05-09T00:23:45.987 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:45 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3031312939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:23:45.987 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:45 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1157670627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:23:45.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T00:23:46.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:45 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3031312939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:23:46.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:45 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1157670627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:23:46.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.109+0000 7f99425ce640 1 -- 172.21.15.46:0/4249401406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c109b70 msgr2=0x7f993c10bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.109+0000 7f99425ce640 1 --2- 172.21.15.46:0/4249401406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c109b70 0x7f993c10bf90 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9934009ca0 tx=0x7f993402a8c0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.109+0000 7f99425ce640 1 -- 172.21.15.46:0/4249401406 shutdown_connections 2024-05-09T00:23:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.109+0000 7f99425ce640 1 --2- 172.21.15.46:0/4249401406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c109b70 0x7f993c10bf90 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.109+0000 7f99425ce640 1 --2- 172.21.15.46:0/4249401406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c104e70 0x7f993c105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.109+0000 7f99425ce640 1 --2- 172.21.15.46:0/4249401406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f993c103c30 0x7f993c104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.109+0000 7f99425ce640 1 -- 172.21.15.46:0/4249401406 >> 172.21.15.46:0/4249401406 conn(0x7f993c0fee40 msgr2=0x7f993c1012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.109+0000 7f99425ce640 1 -- 172.21.15.46:0/4249401406 shutdown_connections 2024-05-09T00:23:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.109+0000 7f99425ce640 1 -- 172.21.15.46:0/4249401406 wait complete. 2024-05-09T00:23:46.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.109+0000 7f99425ce640 1 Processor -- start 2024-05-09T00:23:46.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.109+0000 7f99425ce640 1 -- start start 2024-05-09T00:23:46.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f99425ce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f993c103c30 0x7f993c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:46.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f99425ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c104e70 0x7f993c19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:46.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f99425ce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c109b70 0x7f993c1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:46.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f99425ce640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f993c10f760 con 0x7f993c109b70 2024-05-09T00:23:46.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f99425ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f993c10f5e0 con 0x7f993c104e70 2024-05-09T00:23:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f99425ce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f993c10f8e0 con 0x7f993c103c30 2024-05-09T00:23:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f9940b6b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c109b70 0x7f993c1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f9940b6b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c109b70 0x7f993c1a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48702/0 (socket says 172.21.15.46:48702) 2024-05-09T00:23:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f9940b6b640 1 -- 172.21.15.46:0/440231279 learned_addr learned my addr 172.21.15.46:0/440231279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:46.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f993bfff640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f993c103c30 0x7f993c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:46.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f993b7fe640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c104e70 0x7f993c19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:46.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f9940b6b640 1 -- 172.21.15.46:0/440231279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f993c103c30 msgr2=0x7f993c19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:46.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f9940b6b640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f993c103c30 0x7f993c19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f9940b6b640 1 -- 172.21.15.46:0/440231279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c104e70 msgr2=0x7f993c19a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:46.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f9940b6b640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c104e70 0x7f993c19a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f9940b6b640 1 -- 172.21.15.46:0/440231279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c1a1e80 con 0x7f993c109b70 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.110+0000 7f9940b6b640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c109b70 0x7f993c1a1810 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9934005e30 tx=0x7f993402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.111+0000 7f99397fa640 1 -- 172.21.15.46:0/440231279 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934003bf0 con 0x7f993c109b70 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.111+0000 7f99397fa640 1 -- 172.21.15.46:0/440231279 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9934003d90 con 0x7f993c109b70 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.111+0000 7f99425ce640 1 -- 172.21.15.46:0/440231279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993c1a2110 con 0x7f993c109b70 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.111+0000 7f99397fa640 1 -- 172.21.15.46:0/440231279 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993402fb90 con 0x7f993c109b70 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.111+0000 7f99425ce640 1 -- 172.21.15.46:0/440231279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993c1a2660 con 0x7f993c109b70 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.112+0000 7f99425ce640 1 -- 172.21.15.46:0/440231279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f993c108980 con 0x7f993c109b70 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.113+0000 7f99397fa640 1 -- 172.21.15.46:0/440231279 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f993402fd30 con 0x7f993c109b70 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.114+0000 7f99397fa640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f99240b27f0 0x7f99240b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.115+0000 7f993bfff640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f99240b27f0 0x7f99240b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.115+0000 7f99397fa640 1 -- 172.21.15.46:0/440231279 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f9934103b90 con 0x7f993c109b70 2024-05-09T00:23:46.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.115+0000 7f993bfff640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f99240b27f0 0x7f99240b4d00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f993c104830 tx=0x7f992c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:46.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.118+0000 7f99397fa640 1 -- 172.21.15.46:0/440231279 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f99340cb8b0 con 0x7f993c109b70 2024-05-09T00:23:46.194 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:45 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3031312939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:23:46.194 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:45 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1157670627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T00:23:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.312+0000 7f99425ce640 1 -- 172.21.15.46:0/440231279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f993c0638d0 con 0x7f993c109b70 2024-05-09T00:23:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.313+0000 7f99397fa640 1 -- 172.21.15.46:0/440231279 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f99340cb1b0 con 0x7f993c109b70 2024-05-09T00:23:46.314 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-09T00:23:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.315+0000 7f99425ce640 1 -- 172.21.15.46:0/440231279 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f99240b27f0 msgr2=0x7f99240b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.315+0000 7f99425ce640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f99240b27f0 0x7f99240b4d00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f993c104830 tx=0x7f992c005f90 comp rx=0 tx=0).stop 2024-05-09T00:23:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.315+0000 7f99425ce640 1 -- 172.21.15.46:0/440231279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c109b70 msgr2=0x7f993c1a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.315+0000 7f99425ce640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c109b70 0x7f993c1a1810 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9934005e30 tx=0x7f993402f380 comp rx=0 tx=0).stop 2024-05-09T00:23:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.315+0000 7f99425ce640 1 -- 172.21.15.46:0/440231279 shutdown_connections 2024-05-09T00:23:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.315+0000 7f99425ce640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f99240b27f0 0x7f99240b4d00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.315+0000 7f99425ce640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c109b70 0x7f993c1a1810 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.315+0000 7f99425ce640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f993c104e70 0x7f993c19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.315+0000 7f99425ce640 1 --2- 172.21.15.46:0/440231279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f993c103c30 0x7f993c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.315+0000 7f99425ce640 1 -- 172.21.15.46:0/440231279 >> 172.21.15.46:0/440231279 conn(0x7f993c0fee40 msgr2=0x7f993c105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.315+0000 7f99425ce640 1 -- 172.21.15.46:0/440231279 shutdown_connections 2024-05-09T00:23:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.315+0000 7f99425ce640 1 -- 172.21.15.46:0/440231279 wait complete. 2024-05-09T00:23:46.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T00:23:46.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.442+0000 7f8dc15b2640 1 -- 172.21.15.46:0/1218557699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dbc109b80 msgr2=0x7f8dbc10bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:46.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.442+0000 7f8dc15b2640 1 --2- 172.21.15.46:0/1218557699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dbc109b80 0x7f8dbc10bfa0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009ca0 tx=0x7f8db402a8c0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.442+0000 7f8dc15b2640 1 -- 172.21.15.46:0/1218557699 shutdown_connections 2024-05-09T00:23:46.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.442+0000 7f8dc15b2640 1 --2- 172.21.15.46:0/1218557699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dbc109b80 0x7f8dbc10bfa0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.442+0000 7f8dc15b2640 1 --2- 172.21.15.46:0/1218557699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc104e80 0x7f8dbc105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.442+0000 7f8dc15b2640 1 --2- 172.21.15.46:0/1218557699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dbc103c40 0x7f8dbc104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.442+0000 7f8dc15b2640 1 -- 172.21.15.46:0/1218557699 >> 172.21.15.46:0/1218557699 conn(0x7f8dbc0fee70 msgr2=0x7f8dbc1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:46.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.443+0000 7f8dc15b2640 1 -- 172.21.15.46:0/1218557699 shutdown_connections 2024-05-09T00:23:46.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.443+0000 7f8dc15b2640 1 -- 172.21.15.46:0/1218557699 wait complete. 2024-05-09T00:23:46.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.443+0000 7f8dc15b2640 1 Processor -- start 2024-05-09T00:23:46.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.443+0000 7f8dc15b2640 1 -- start start 2024-05-09T00:23:46.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8dc15b2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dbc103c40 0x7f8dbc19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:46.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8dc15b2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dbc104e80 0x7f8dbc19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:46.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8dc15b2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc109b80 0x7f8dbc1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:46.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8dc15b2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc10f770 con 0x7f8dbc104e80 2024-05-09T00:23:46.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8dc15b2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc10f5f0 con 0x7f8dbc103c40 2024-05-09T00:23:46.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8dc15b2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc10f8f0 con 0x7f8dbc109b80 2024-05-09T00:23:46.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8db27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dbc104e80 0x7f8dbc19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:46.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8db27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dbc104e80 0x7f8dbc19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59628/0 (socket says 172.21.15.46:59628) 2024-05-09T00:23:46.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8db27fc640 1 -- 172.21.15.46:0/3990078669 learned_addr learned my addr 172.21.15.46:0/3990078669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:23:46.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8dbaffd640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dbc103c40 0x7f8dbc19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:46.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8dbb7fe640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc109b80 0x7f8dbc1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:46.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8db27fc640 1 -- 172.21.15.46:0/3990078669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc109b80 msgr2=0x7f8dbc1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:46.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8db27fc640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc109b80 0x7f8dbc1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8db27fc640 1 -- 172.21.15.46:0/3990078669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dbc103c40 msgr2=0x7f8dbc19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:46.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8db27fc640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dbc103c40 0x7f8dbc19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:46.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8db27fc640 1 -- 172.21.15.46:0/3990078669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dbc1a1e50 con 0x7f8dbc104e80 2024-05-09T00:23:46.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8dbb7fe640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc109b80 0x7f8dbc1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:23:46.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8db27fc640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dbc104e80 0x7f8dbc19a850 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8da400d770 tx=0x7f8da400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:46.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8db8ff9640 1 -- 172.21.15.46:0/3990078669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4017070 con 0x7f8dbc104e80 2024-05-09T00:23:46.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8dc15b2640 1 -- 172.21.15.46:0/3990078669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dbc1a2140 con 0x7f8dbc104e80 2024-05-09T00:23:46.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8db8ff9640 1 -- 172.21.15.46:0/3990078669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8da40054d0 con 0x7f8dbc104e80 2024-05-09T00:23:46.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8db8ff9640 1 -- 172.21.15.46:0/3990078669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4004c30 con 0x7f8dbc104e80 2024-05-09T00:23:46.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.444+0000 7f8dc15b2640 1 -- 172.21.15.46:0/3990078669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dbc1a2640 con 0x7f8dbc104e80 2024-05-09T00:23:46.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.446+0000 7f8dc15b2640 1 -- 172.21.15.46:0/3990078669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dbc108990 con 0x7f8dbc104e80 2024-05-09T00:23:46.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.447+0000 7f8db8ff9640 1 -- 172.21.15.46:0/3990078669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8da4004e10 con 0x7f8dbc104e80 2024-05-09T00:23:46.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.448+0000 7f8db8ff9640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8da80b27f0 0x7f8da80b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:23:46.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.448+0000 7f8dbaffd640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8da80b27f0 0x7f8da80b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:23:46.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.448+0000 7f8db8ff9640 1 -- 172.21.15.46:0/3990078669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8da40e1d30 con 0x7f8dbc104e80 2024-05-09T00:23:46.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.448+0000 7f8dbaffd640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8da80b27f0 0x7f8da80b4d00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc104840 tx=0x7f8dac00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:23:46.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.453+0000 7f8db8ff9640 1 -- 172.21.15.46:0/3990078669 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8da40a9a50 con 0x7f8dbc104e80 2024-05-09T00:23:46.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.699+0000 7f8dc15b2640 1 -- 172.21.15.46:0/3990078669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f8dbc0638d0 con 0x7f8dbc104e80 2024-05-09T00:23:46.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.709+0000 7f8db8ff9640 1 -- 172.21.15.46:0/3990078669 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+815638 (secure 0 0 0) 0x7f8da40a9350 con 0x7f8dbc104e80 2024-05-09T00:23:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:23:46.721 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:23:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 18, 2024-05-09T00:23:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "active_gid": 14152, 2024-05-09T00:23:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "active_name": "x", 2024-05-09T00:23:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addrs": { 2024-05-09T00:23:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-09T00:23:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-09T00:23:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6800", 2024-05-09T00:23:46.721 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 286117730 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6801", 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 286117730 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addr": "172.21.15.46:6801/286117730", 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "active_change": "2024-05-09T00:01:09.191408+0000", 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "active_mgr_features": 4540701547738038271, 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: "standbys": [ 2024-05-09T00:23:46.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 24109, 2024-05-09T00:23:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "z", 2024-05-09T00:23:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540701547738038271, 2024-05-09T00:23:46.723 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-05-09T00:23:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-05-09T00:23:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:46.726 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.727 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.729 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.730 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.731 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.732 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.733 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.734 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-05-09T00:23:46.735 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-05-09T00:23:46.736 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-05-09T00:23:46.737 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-05-09T00:23:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-05-09T00:23:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.738 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.739 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.740 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-05-09T00:23:46.741 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-05-09T00:23:46.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-05-09T00:23:46.743 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-05-09T00:23:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-05-09T00:23:46.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-05-09T00:23:46.746 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-05-09T00:23:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:46.748 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.749 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.750 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.751 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.753 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.754 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-05-09T00:23:46.756 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.757 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T00:23:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:46.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.759 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.761 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.763 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-05-09T00:23:46.764 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.765 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.766 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.768 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.769 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.771 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.772 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.773 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.774 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.775 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T00:23:46.776 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T00:23:46.777 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.778 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T00:23:46.779 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T00:23:46.780 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.781 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.782 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.783 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.784 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.785 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.786 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.787 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.788 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.789 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.790 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T00:23:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-05-09T00:23:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T00:23:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-09T00:23:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T00:23:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T00:23:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T00:23:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T00:23:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-05-09T00:23:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T00:23:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "[]", 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "{}", 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T00:23:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T00:23:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T00:23:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T00:23:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T00:23:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T00:23:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T00:23:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T00:23:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T00:23:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.880 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.881 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.882 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.884 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.885 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.887 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.888 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.889 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.890 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.891 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.892 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.893 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.894 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.895 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.896 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.897 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.898 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.899 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.900 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.902 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.903 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.904 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.905 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.906 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.907 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.908 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-05-09T00:23:46.909 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-05-09T00:23:46.910 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:46.911 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.912 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.913 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.915 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.916 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.917 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.918 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.919 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.920 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.921 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.922 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.923 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-05-09T00:23:46.924 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-05-09T00:23:46.925 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.926 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-05-09T00:23:46.927 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.928 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.928 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T00:23:46.930 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.930 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.930 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.930 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.930 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.930 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.930 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.930 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.930 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.934 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.935 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-05-09T00:23:46.936 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.937 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-05-09T00:23:46.938 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T00:23:46.939 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.940 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.941 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.942 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.943 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.945 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.946 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.949 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.950 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.952 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.953 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.955 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.956 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.957 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.958 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.959 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.960 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.961 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.962 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.963 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.964 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.965 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.966 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.967 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.968 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-05-09T00:23:46.969 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.971 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.972 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.973 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.974 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.976 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.977 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.978 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.979 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.980 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.981 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.982 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.983 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.984 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.985 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.986 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.989 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.991 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.993 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:46.995 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.996 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:46.998 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:46.999 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.000 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.001 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.003 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.004 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.005 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.006 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.010 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.011 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.013 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.014 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.015 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.016 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.017 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.018 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.019 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.020 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:47.021 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.023 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.024 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T00:23:47.025 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.026 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.027 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.028 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.029 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.030 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.031 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.033 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.035 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.036 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.037 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.038 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.038 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.038 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.038 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.038 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.038 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.038 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.038 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.038 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.038 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.038 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.040 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.041 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.042 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.043 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.046 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.047 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.048 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-05-09T00:23:47.049 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.051 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.052 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.053 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.054 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.055 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.057 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.059 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.060 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.061 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.063 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.065 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.066 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.067 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.068 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.069 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.070 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.071 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.072 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.073 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.074 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.076 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.078 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.079 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.080 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.081 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.082 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.083 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-05-09T00:23:47.084 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.085 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.086 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.087 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.088 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.089 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.091 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.092 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.094 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.095 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.100 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.101 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.102 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smb", 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.104 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.105 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.106 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.107 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.108 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.109 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: "update_orchestration": { 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "update_orchestration", 2024-05-09T00:23:47.110 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.112 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.113 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.114 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.116 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.118 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.119 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.120 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.121 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.122 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.122 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.122 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.122 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.122 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.122 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.122 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.122 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.123 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.124 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.125 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.127 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.128 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.130 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.131 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.131 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.131 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.131 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.131 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.131 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.131 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.131 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.134 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.134 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.134 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.134 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.134 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.136 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.137 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.138 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.138 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.138 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.138 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.138 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.138 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.138 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.138 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.138 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.139 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.141 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.142 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.143 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.144 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.144 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-05-09T00:23:47.144 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.144 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.144 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.144 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-05-09T00:23:47.145 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-05-09T00:23:47.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.145 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.145 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.145 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.145 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.145 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.145 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.145 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T00:23:47.145 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.146 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.146 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-05-09T00:23:47.146 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-05-09T00:23:47.146 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.146 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.146 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.146 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.146 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.146 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.146 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.147 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T00:23:47.147 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.147 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.147 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.147 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-05-09T00:23:47.147 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-05-09T00:23:47.147 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.147 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.147 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.148 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.148 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.148 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.148 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T00:23:47.148 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.148 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.148 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-05-09T00:23:47.148 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-05-09T00:23:47.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.148 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.149 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.149 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.149 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.149 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.149 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T00:23:47.149 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.149 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.149 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-05-09T00:23:47.150 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-05-09T00:23:47.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.150 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.150 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.150 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.150 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.150 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.150 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T00:23:47.150 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.150 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.151 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.151 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-05-09T00:23:47.151 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-05-09T00:23:47.151 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.151 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.151 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.151 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.151 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.151 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.151 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.151 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.152 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.153 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.154 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-05-09T00:23:47.155 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-05-09T00:23:47.156 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.156 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.156 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.156 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.156 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.156 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-05-09T00:23:47.156 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-05-09T00:23:47.156 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.156 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.156 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.157 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-05-09T00:23:47.157 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.157 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.157 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.157 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.157 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.157 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.157 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.157 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-05-09T00:23:47.157 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-05-09T00:23:47.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.158 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.158 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.158 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.158 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.158 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.158 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.158 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.158 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.159 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-05-09T00:23:47.159 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-05-09T00:23:47.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.159 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.159 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.159 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.159 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.159 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.159 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.159 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.159 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.160 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.161 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.164 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.166 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-05-09T00:23:47.167 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.169 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.170 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.171 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.172 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.173 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.174 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.175 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.176 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.178 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T00:23:47.180 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.183 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:47.185 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.187 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.188 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.190 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.191 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.192 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.193 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.194 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.196 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 24113, 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "y", 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540701547738038271, 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.197 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.199 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.200 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.201 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.202 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.203 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.204 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.205 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.207 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.208 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.209 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.210 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.211 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-05-09T00:23:47.212 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.213 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:47.214 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T00:23:47.216 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-05-09T00:23:47.217 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-05-09T00:23:47.218 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.219 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.220 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.222 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.223 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.224 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.225 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.226 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-05-09T00:23:47.227 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:47.228 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.229 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.230 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.232 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-05-09T00:23:47.232 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-05-09T00:23:47.232 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:47.232 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.232 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.232 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.232 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.232 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.232 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.232 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T00:23:47.233 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.233 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.233 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.233 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-05-09T00:23:47.233 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.233 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.234 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.234 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-05-09T00:23:47.234 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-05-09T00:23:47.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.234 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.234 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-09T00:23:47.234 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.234 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.235 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.235 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T00:23:47.235 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.235 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.235 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-05-09T00:23:47.235 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-05-09T00:23:47.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.235 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.236 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.237 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.239 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.240 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.241 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.242 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.243 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.244 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.245 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T00:23:47.246 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-05-09T00:23:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-05-09T00:23:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-05-09T00:23:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-05-09T00:23:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T00:23:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-05-09T00:23:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-09T00:23:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-05-09T00:23:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-05-09T00:23:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-05-09T00:23:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-05-09T00:23:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-05-09T00:23:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-05-09T00:23:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-05-09T00:23:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-05-09T00:23:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T00:23:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T00:23:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "[]", 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "{}", 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T00:23:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T00:23:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-05-09T00:23:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-05-09T00:23:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-05-09T00:23:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-05-09T00:23:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-05-09T00:23:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-09T00:23:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T00:23:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T00:23:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-05-09T00:23:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-05-09T00:23:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-05-09T00:23:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-05-09T00:23:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-05-09T00:23:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-05-09T00:23:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-05-09T00:23:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-05-09T00:23:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-05-09T00:23:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-05-09T00:23:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-05-09T00:23:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-09T00:23:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-05-09T00:23:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-05-09T00:23:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T00:23:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-05-09T00:23:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-05-09T00:23:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-05-09T00:23:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T00:23:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-09T00:23:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-09T00:23:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-05-09T00:23:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T00:23:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-05-09T00:23:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T00:23:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-05-09T00:23:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-05-09T00:23:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smb", 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "update_orchestration": { 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "update_orchestration", 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T00:23:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-05-09T00:23:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-05-09T00:23:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard", 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-05-09T00:23:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-05-09T00:23:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T00:23:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T00:23:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-05-09T00:23:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T00:23:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T00:23:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T00:23:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T00:23:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T00:23:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-05-09T00:23:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-05-09T00:23:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T00:23:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T00:23:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-05-09T00:23:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-05-09T00:23:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-05-09T00:23:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-05-09T00:23:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T00:23:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T00:23:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-05-09T00:23:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-05-09T00:23:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T00:23:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T00:23:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "[]", 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "{}", 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T00:23:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T00:23:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T00:23:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T00:23:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T00:23:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T00:23:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T00:23:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T00:23:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T00:23:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-05-09T00:23:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-09T00:23:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T00:23:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-05-09T00:23:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-05-09T00:23:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-05-09T00:23:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-05-09T00:23:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-05-09T00:23:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T00:23:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-05-09T00:23:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-09T00:23:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-05-09T00:23:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-05-09T00:23:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-05-09T00:23:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-05-09T00:23:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-05-09T00:23:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-05-09T00:23:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-05-09T00:23:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smb", 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "update_orchestration": { 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "update_orchestration", 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-05-09T00:23:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-05-09T00:23:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T00:23:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-05-09T00:23:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-05-09T00:23:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T00:23:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.108 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T00:23:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T00:23:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "services": { 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard": "https://172.21.15.46:8443/" 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "always_on_modules": { 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "octopus": [ 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-09T00:23:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "pacific": [ 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-09T00:23:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "quincy": [ 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-09T00:23:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "reef": [ 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "squid": [ 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-09T00:23:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "last_failure_osd_epoch": 3, 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "active_clients": [ 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-09T00:23:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3613578525 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "libcephsqlite", 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 291802359 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1678263194 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-09T00:23:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T00:23:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-09T00:23:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-09T00:23:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 429688555 2024-05-09T00:23:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T00:23:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T00:23:48.123 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:23:48.124 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:46 smithi142 ceph-mon[34533]: pgmap v748: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:23:48.124 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:46 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1466380989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:23:48.124 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:46 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/440231279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:23:48.124 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:46 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3990078669' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T00:23:48.124 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:46 smithi169 ceph-mon[34777]: pgmap v748: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:23:48.124 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:46 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1466380989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:23:48.125 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:46 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/440231279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:23:48.125 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:46 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3990078669' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T00:23:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.714+0000 7f8dc15b2640 1 -- 172.21.15.46:0/3990078669 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8da80b27f0 msgr2=0x7f8da80b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.714+0000 7f8dc15b2640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8da80b27f0 0x7f8da80b4d00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc104840 tx=0x7f8dac00b390 comp rx=0 tx=0).stop 2024-05-09T00:23:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.714+0000 7f8dc15b2640 1 -- 172.21.15.46:0/3990078669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dbc104e80 msgr2=0x7f8dbc19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:23:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.714+0000 7f8dc15b2640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dbc104e80 0x7f8dbc19a850 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8da400d770 tx=0x7f8da400dc30 comp rx=0 tx=0).stop 2024-05-09T00:23:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.714+0000 7f8dc15b2640 1 -- 172.21.15.46:0/3990078669 shutdown_connections 2024-05-09T00:23:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.714+0000 7f8dc15b2640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8da80b27f0 0x7f8da80b4d00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.714+0000 7f8dc15b2640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc109b80 0x7f8dbc1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.714+0000 7f8dc15b2640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dbc104e80 0x7f8dbc19a850 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.714+0000 7f8dc15b2640 1 --2- 172.21.15.46:0/3990078669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dbc103c40 0x7f8dbc19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:23:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.714+0000 7f8dc15b2640 1 -- 172.21.15.46:0/3990078669 >> 172.21.15.46:0/3990078669 conn(0x7f8dbc0fee70 msgr2=0x7f8dbc105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:23:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.715+0000 7f8dc15b2640 1 -- 172.21.15.46:0/3990078669 shutdown_connections 2024-05-09T00:23:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:23:46.715+0000 7f8dc15b2640 1 -- 172.21.15.46:0/3990078669 wait complete. 2024-05-09T00:23:48.173 DEBUG:tasks.check_counter:Getting stats from a 2024-05-09T00:23:48.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-09T00:23:48.174 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:46 smithi046 ceph-mon[30151]: pgmap v748: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T00:23:48.174 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:46 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1466380989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T00:23:48.174 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:46 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/440231279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T00:23:48.175 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:46 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3990078669' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T00:23:48.440 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:49.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:48 smithi169 ceph-mon[34777]: pgmap v749: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-05-09T00:23:49.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:48 smithi142 ceph-mon[34533]: pgmap v749: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-05-09T00:23:49.300 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:48 smithi046 ceph-mon[30151]: pgmap v749: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-05-09T00:23:50.224 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:23:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:23:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 4, 2024-05-09T00:23:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 4, 2024-05-09T00:23:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 2944, 2024-05-09T00:23:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 448, 2024-05-09T00:23:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-09T00:23:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-05-09T00:23:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.001839196, 2024-05-09T00:23:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000039061, 2024-05-09T00:23:50.224 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000693084, 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3, 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000798668, 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000266222 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T00:23:50.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2, 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 192, 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 320, 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 191200, 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 2, 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.016793990, 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.012432119, 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.001903635, 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:23:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 190, 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.011043316, 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000058122 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 191200 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 255, 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 252, 2024-05-09T00:23:50.227 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 524080, 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 155280, 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.040236114, 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.017022471, 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.017571611, 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.004988002, 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 251, 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.011951703, 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000047616 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:23:50.228 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 524080, 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 155280 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:23:50.229 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:23:50.230 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-05-09T00:23:50.231 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 147960, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 28, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-05-09T00:23:50.232 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:23:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-05-09T00:23:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-05-09T00:23:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-05-09T00:23:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:23:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:23:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-05-09T00:23:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-05-09T00:23:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-05-09T00:23:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-05-09T00:23:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-05-09T00:23:50.233 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 257, 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 488050, 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 257, 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-09T00:23:50.234 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-09T00:23:50.235 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 257, 2024-05-09T00:23:50.235 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 488050, 2024-05-09T00:23:50.235 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-09T00:23:50.235 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:50.235 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:50.235 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:50.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:50.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:50.235 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:23:50.924 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-09T00:23:50.924 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-09T00:23:50.924 DEBUG:tasks.check_counter:Getting stats from d 2024-05-09T00:23:50.924 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-09T00:23:51.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:50 smithi169 ceph-mon[34777]: pgmap v750: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-09T00:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:51.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:50 smithi142 ceph-mon[34533]: pgmap v750: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-09T00:23:51.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:50 smithi046 ceph-mon[30151]: pgmap v750: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2024-05-09T00:23:52.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:52 smithi046 ceph-mon[30151]: pgmap v751: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 KiB/s wr, 8 op/s 2024-05-09T00:23:52.961 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:23:52.961 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:23:52.961 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 4, 2024-05-09T00:23:52.961 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 4, 2024-05-09T00:23:52.961 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 2944, 2024-05-09T00:23:52.961 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 448, 2024-05-09T00:23:52.962 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-09T00:23:52.962 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-05-09T00:23:52.962 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.001769600, 2024-05-09T00:23:52.962 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000034788, 2024-05-09T00:23:52.962 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000605913, 2024-05-09T00:23:52.962 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:23:52.962 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:23:52.962 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3, 2024-05-09T00:23:52.962 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000818398, 2024-05-09T00:23:52.962 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000272799 2024-05-09T00:23:52.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:52.962 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 261, 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 258, 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 525712, 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 159120, 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-05-09T00:23:52.963 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.039729512, 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.016719950, 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.017394670, 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.005139335, 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 257, 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.012552433, 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000048842 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:52.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 525712, 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 159120 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2, 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 197, 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 320, 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 192080, 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 2, 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.017426311, 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.012696897, 2024-05-09T00:23:52.965 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002222356, 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 195, 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.011488036, 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000058913 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T00:23:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 192080 2024-05-09T00:23:52.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:52.967 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-05-09T00:23:52.967 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-05-09T00:23:52.967 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-05-09T00:23:52.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:52.967 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-05-09T00:23:52.967 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:23:52.967 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-05-09T00:23:52.967 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:23:52.967 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:23:52.967 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:23:52.968 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:23:52.968 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:23:52.968 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:23:52.968 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:23:52.968 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:23:52.968 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:23:52.968 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:23:52.968 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:23:52.968 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:23:52.968 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:23:52.968 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-05-09T00:23:52.969 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:23:52.969 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-05-09T00:23:52.969 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:23:52.969 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:23:52.969 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:23:52.969 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:23:52.969 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:23:52.969 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:23:52.969 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:23:52.969 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-05-09T00:23:52.969 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:23:52.969 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:23:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:23:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-05-09T00:23:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-05-09T00:23:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-05-09T00:23:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:23:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:23:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:23:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:23:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 147888, 2024-05-09T00:23:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 26, 2024-05-09T00:23:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:23:52.971 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-05-09T00:23:52.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-05-09T00:23:52.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-05-09T00:23:52.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:23:52.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-05-09T00:23:52.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:23:52.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-05-09T00:23:52.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-05-09T00:23:52.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-05-09T00:23:52.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:23:52.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:23:52.972 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-05-09T00:23:52.972 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-05-09T00:23:52.972 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-05-09T00:23:52.972 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-05-09T00:23:52.972 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-05-09T00:23:52.972 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-05-09T00:23:52.972 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-05-09T00:23:52.972 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-05-09T00:23:52.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:52.972 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:23:52.972 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-09T00:23:52.972 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 263, 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 488830, 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 263, 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 263, 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 488830, 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:52.973 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:52.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:52.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:52.974 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:23:53.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:52 smithi169 ceph-mon[34777]: pgmap v751: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 KiB/s wr, 8 op/s 2024-05-09T00:23:53.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:52 smithi142 ceph-mon[34533]: pgmap v751: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 KiB/s wr, 8 op/s 2024-05-09T00:23:53.632 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-09T00:23:53.632 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-09T00:23:53.633 DEBUG:tasks.check_counter:Getting stats from g 2024-05-09T00:23:53.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-09T00:23:53.895 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:55.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:54 smithi169 ceph-mon[34777]: pgmap v752: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.1 KiB/s wr, 10 op/s 2024-05-09T00:23:55.172 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:54 smithi046 ceph-mon[30151]: pgmap v752: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.1 KiB/s wr, 10 op/s 2024-05-09T00:23:55.194 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:54 smithi142 ceph-mon[34533]: pgmap v752: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.1 KiB/s wr, 10 op/s 2024-05-09T00:23:55.764 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:23:55.764 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.001398393, 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000288340, 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:55.765 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2, 2024-05-09T00:23:55.766 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 194, 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 320, 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 191552, 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 2, 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.016195411, 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.012222569, 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.001794823, 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 192, 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.010730092, 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000055885 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:23:55.767 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 191552 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 261, 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 256, 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 527568, 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 157008, 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-05-09T00:23:55.768 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.040218113, 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.017378703, 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.017469818, 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.004961593, 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 256, 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.012870788, 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000050276 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 527568, 2024-05-09T00:23:55.769 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 157008 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:23:55.770 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:23:55.771 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 148000, 2024-05-09T00:23:55.772 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 29, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-05-09T00:23:55.773 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 259, 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 488310, 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-09T00:23:55.774 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 259, 2024-05-09T00:23:55.775 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-09T00:23:55.775 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-09T00:23:55.775 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 259, 2024-05-09T00:23:55.775 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 488310, 2024-05-09T00:23:55.775 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-09T00:23:55.775 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:55.775 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:55.775 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:55.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:55.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:55.775 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:23:56.547 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-09T00:23:56.547 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-09T00:23:56.547 DEBUG:tasks.check_counter:Getting stats from j 2024-05-09T00:23:56.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-09T00:23:56.808 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:23:57.099 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:56 smithi046 ceph-mon[30151]: pgmap v753: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-09T00:23:57.105 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:56 smithi169 ceph-mon[34777]: pgmap v753: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-09T00:23:57.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:56 smithi142 ceph-mon[34533]: pgmap v753: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-09T00:23:58.588 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T00:23:58.588 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:23:58.588 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 267, 2024-05-09T00:23:58.588 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 262, 2024-05-09T00:23:58.588 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 529200, 2024-05-09T00:23:58.588 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 160848, 2024-05-09T00:23:58.588 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-09T00:23:58.588 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-05-09T00:23:58.588 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.041155300, 2024-05-09T00:23:58.589 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.016638233, 2024-05-09T00:23:58.589 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.017874828, 2024-05-09T00:23:58.589 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.005196626, 2024-05-09T00:23:58.589 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:23:58.589 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 262, 2024-05-09T00:23:58.589 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.012716943, 2024-05-09T00:23:58.589 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000048537 2024-05-09T00:23:58.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:58.589 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:23:58.589 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:58.589 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:58.589 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:58.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:58.590 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 529200, 2024-05-09T00:23:58.590 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 160848 2024-05-09T00:23:58.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:58.590 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:23:58.590 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2, 2024-05-09T00:23:58.590 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 199, 2024-05-09T00:23:58.590 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 320, 2024-05-09T00:23:58.590 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 192432, 2024-05-09T00:23:58.590 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-09T00:23:58.590 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 2, 2024-05-09T00:23:58.590 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.016478119, 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.012208425, 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002038694, 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 197, 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.011796758, 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000059882 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:58.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:58.592 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T00:23:58.592 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 192432 2024-05-09T00:23:58.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:58.592 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:23:58.592 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-05-09T00:23:58.592 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-05-09T00:23:58.592 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-05-09T00:23:58.592 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-05-09T00:23:58.592 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-05-09T00:23:58.592 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.000453673, 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000372782, 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:58.593 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:58.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:58.594 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T00:23:58.594 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T00:23:58.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:58.594 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-05-09T00:23:58.594 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-05-09T00:23:58.594 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-05-09T00:23:58.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:58.594 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-05-09T00:23:58.594 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:23:58.595 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-05-09T00:23:58.595 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:23:58.595 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:23:58.595 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:23:58.595 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:23:58.595 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:23:58.595 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:23:58.595 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:23:58.595 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:23:58.595 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:23:58.595 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:23:58.596 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:23:58.596 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:23:58.596 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:23:58.596 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-05-09T00:23:58.596 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:23:58.596 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-05-09T00:23:58.596 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:23:58.596 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:23:58.596 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:23:58.596 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:23:58.596 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:23:58.596 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:23:58.597 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:23:58.597 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-05-09T00:23:58.597 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:23:58.597 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:23:58.597 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:23:58.597 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-05-09T00:23:58.597 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-05-09T00:23:58.597 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-05-09T00:23:58.597 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:23:58.597 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:23:58.597 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:23:58.597 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:23:58.598 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 147977, 2024-05-09T00:23:58.598 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 29, 2024-05-09T00:23:58.598 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:23:58.598 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-05-09T00:23:58.598 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-05-09T00:23:58.598 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-05-09T00:23:58.598 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:23:58.598 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-05-09T00:23:58.598 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:23:58.598 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-05-09T00:23:58.598 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-05-09T00:23:58.599 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-05-09T00:23:58.599 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:23:58.599 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:23:58.599 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-05-09T00:23:58.599 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-05-09T00:23:58.599 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-05-09T00:23:58.599 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-05-09T00:23:58.599 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-05-09T00:23:58.600 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-05-09T00:23:58.600 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-05-09T00:23:58.600 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-05-09T00:23:58.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T00:23:58.600 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:23:58.600 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-09T00:23:58.600 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-09T00:23:58.600 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-09T00:23:58.600 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 265, 2024-05-09T00:23:58.600 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 489090, 2024-05-09T00:23:58.600 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-09T00:23:58.600 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 265, 2024-05-09T00:23:58.601 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-09T00:23:58.601 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-09T00:23:58.601 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 265, 2024-05-09T00:23:58.601 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 489090, 2024-05-09T00:23:58.601 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-09T00:23:58.601 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T00:23:58.602 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T00:23:58.602 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T00:23:58.602 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:58.602 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T00:23:58.602 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T00:23:58.922 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:23:58 smithi046 ceph-mon[30151]: pgmap v754: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-05-09T00:23:59.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:23:58 smithi169 ceph-mon[34777]: pgmap v754: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-05-09T00:23:59.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:23:58 smithi142 ceph-mon[34533]: pgmap v754: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-05-09T00:23:59.319 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-09T00:23:59.320 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-09T00:23:59.320 DEBUG:tasks.check_counter:Getting stats from b 2024-05-09T00:23:59.320 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-09T00:23:59.570 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:24:01.009 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:00 smithi142 ceph-mon[34533]: pgmap v755: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-09T00:24:01.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:00 smithi169 ceph-mon[34777]: pgmap v755: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-09T00:24:01.313 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:00 smithi046 ceph-mon[30151]: pgmap v755: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-09T00:24:01.457 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-09T00:24:01.458 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:24:01.458 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 535, 2024-05-09T00:24:01.458 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 716, 2024-05-09T00:24:01.458 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 112298, 2024-05-09T00:24:01.458 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 781134, 2024-05-09T00:24:01.458 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 55, 2024-05-09T00:24:01.458 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2024-05-09T00:24:01.458 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.095521476, 2024-05-09T00:24:01.458 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.031556739, 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.051205288, 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.006559916, 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 715, 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.053808673, 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000075256 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1147, 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000379268, 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000330 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.459 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T00:24:01.460 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T00:24:01.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.460 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:24:01.460 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 72, 2024-05-09T00:24:01.460 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 264, 2024-05-09T00:24:01.460 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 23827, 2024-05-09T00:24:01.460 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 580965, 2024-05-09T00:24:01.460 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 16, 2024-05-09T00:24:01.460 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-05-09T00:24:01.461 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.021526273, 2024-05-09T00:24:01.461 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.010081222, 2024-05-09T00:24:01.461 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.009476973, 2024-05-09T00:24:01.461 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000233432, 2024-05-09T00:24:01.461 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:01.461 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 262, 2024-05-09T00:24:01.461 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.030598855, 2024-05-09T00:24:01.461 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000116789 2024-05-09T00:24:01.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.461 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:01.461 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 114, 2024-05-09T00:24:01.461 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000026618, 2024-05-09T00:24:01.462 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000233 2024-05-09T00:24:01.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.462 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T00:24:01.462 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 507456 2024-05-09T00:24:01.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.462 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:24:01.462 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 320, 2024-05-09T00:24:01.462 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 310, 2024-05-09T00:24:01.462 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 567854, 2024-05-09T00:24:01.462 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 173367, 2024-05-09T00:24:01.462 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 8, 2024-05-09T00:24:01.463 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-05-09T00:24:01.463 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.057389251, 2024-05-09T00:24:01.463 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.021474561, 2024-05-09T00:24:01.463 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.029241365, 2024-05-09T00:24:01.463 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.007332789, 2024-05-09T00:24:01.463 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:01.463 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 309, 2024-05-09T00:24:01.463 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.031329024, 2024-05-09T00:24:01.463 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000101388 2024-05-09T00:24:01.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.464 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:01.464 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 21, 2024-05-09T00:24:01.464 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000005902, 2024-05-09T00:24:01.464 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000281 2024-05-09T00:24:01.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.464 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 564016, 2024-05-09T00:24:01.464 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 162544 2024-05-09T00:24:01.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.464 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-05-09T00:24:01.464 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-05-09T00:24:01.465 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-05-09T00:24:01.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.465 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-05-09T00:24:01.465 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-09T00:24:01.465 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-09T00:24:01.465 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 45, 2024-05-09T00:24:01.465 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.001578403, 2024-05-09T00:24:01.465 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000035075 2024-05-09T00:24:01.465 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:01.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.466 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-05-09T00:24:01.466 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-09T00:24:01.466 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-09T00:24:01.466 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2, 2024-05-09T00:24:01.467 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000006310, 2024-05-09T00:24:01.467 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000003155 2024-05-09T00:24:01.467 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:01.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.468 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-05-09T00:24:01.468 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 1, 2024-05-09T00:24:01.468 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 1, 2024-05-09T00:24:01.468 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-05-09T00:24:01.468 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-05-09T00:24:01.468 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000130827, 2024-05-09T00:24:01.468 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000130827 2024-05-09T00:24:01.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.468 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-05-09T00:24:01.468 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-05-09T00:24:01.468 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-05-09T00:24:01.468 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 11, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 10, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 0, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 10, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 10, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 1, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 1, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 1, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-05-09T00:24:01.469 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-05-09T00:24:01.470 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 5, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 298, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-05-09T00:24:01.471 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 1, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-05-09T00:24:01.472 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-05-09T00:24:01.473 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T00:24:01.474 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-05-09T00:24:01.474 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-05-09T00:24:01.474 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-05-09T00:24:01.474 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-05-09T00:24:01.474 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-05-09T00:24:01.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.474 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-05-09T00:24:01.474 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-05-09T00:24:01.474 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 3, 2024-05-09T00:24:01.474 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 3, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 2, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 2, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 2, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 4195200, 2024-05-09T00:24:01.475 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2, 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.003857151, 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001928575 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 13, 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 13, 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 0, 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 11, 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 12, 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 1, 2024-05-09T00:24:01.476 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 10, 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 10, 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 0, 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 1, 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 1, 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 123800, 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 223632 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 1, 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 1, 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 10, 2024-05-09T00:24:01.477 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.478 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.479 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000130827, 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000130827 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.481 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.482 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-05-09T00:24:01.482 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.482 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.483 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-05-09T00:24:01.484 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.484 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.484 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.484 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-05-09T00:24:01.484 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.484 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.484 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.484 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-05-09T00:24:01.485 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.485 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.485 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.485 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-05-09T00:24:01.485 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.485 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.485 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.486 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.487 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-05-09T00:24:01.488 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.489 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 1, 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-05-09T00:24:01.490 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 1, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 1, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 715, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:24:01.491 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:24:01.492 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 255507, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 90, 2024-05-09T00:24:01.493 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7624, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 160, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-05-09T00:24:01.494 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 51240, 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 200, 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 37, 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 11108, 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 33, 2024-05-09T00:24:01.495 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 37, 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 37, 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.255995392, 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006918794 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 37, 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 59 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 37, 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 0, 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 37, 2024-05-09T00:24:01.496 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 10, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 0, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 2, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 6, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 6, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-05-09T00:24:01.497 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-05-09T00:24:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 0, 2024-05-09T00:24:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20, 2024-05-09T00:24:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-05-09T00:24:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-05-09T00:24:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-05-09T00:24:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-05-09T00:24:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-05-09T00:24:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-05-09T00:24:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T00:24:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-05-09T00:24:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-05-09T00:24:01.498 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-05-09T00:24:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 80, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 2, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 9, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 14, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 5, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 14, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 0, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 1 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.500 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 0, 2024-05-09T00:24:01.501 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 923, 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 602588, 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 923, 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 923, 2024-05-09T00:24:01.502 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 602588, 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:01.503 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 37, 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 11108, 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 37, 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 11108, 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-05-09T00:24:01.504 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 37, 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 37, 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 37, 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 37, 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.505 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.506 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x5598dd382680": { 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 3, 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 896, 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 3, 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2, 2024-05-09T00:24:01.507 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 896, 2024-05-09T00:24:01.508 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:01.508 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:01.508 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:01.508 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:01.508 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:01.508 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:01.508 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-09T00:24:02.107 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:01 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:24:02.178 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:01 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:24:02.206 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-09T00:24:02.207 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-09T00:24:02.207 DEBUG:tasks.check_counter:Getting stats from e 2024-05-09T00:24:02.207 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-09T00:24:02.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:01 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T00:24:02.450 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:24:03.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:02 smithi169 ceph-mon[34777]: pgmap v756: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-09T00:24:03.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:24:03.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:24:03.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:02 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:24:03.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:02 smithi142 ceph-mon[34533]: pgmap v756: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-09T00:24:03.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:24:03.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:24:03.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:02 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:24:03.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:02 smithi046 ceph-mon[30151]: pgmap v756: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-09T00:24:03.313 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T00:24:03.313 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T00:24:03.313 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:02 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' 2024-05-09T00:24:04.222 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-09T00:24:04.223 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:24:04.223 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 930, 2024-05-09T00:24:04.223 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 930, 2024-05-09T00:24:04.223 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 247559, 2024-05-09T00:24:04.223 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 294980, 2024-05-09T00:24:04.223 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 7, 2024-05-09T00:24:04.223 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-05-09T00:24:04.223 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.092794702, 2024-05-09T00:24:04.223 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.036520913, 2024-05-09T00:24:04.223 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.022469528, 2024-05-09T00:24:04.224 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.024393497, 2024-05-09T00:24:04.224 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:04.224 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 929, 2024-05-09T00:24:04.224 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.038620539, 2024-05-09T00:24:04.224 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000041572 2024-05-09T00:24:04.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.224 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:04.224 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.224 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.224 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 837, 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1034, 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 205838, 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 685853, 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.092683359, 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.032781476, 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.019564209, 2024-05-09T00:24:04.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.030916683, 2024-05-09T00:24:04.226 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:04.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1032, 2024-05-09T00:24:04.226 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.042981251, 2024-05-09T00:24:04.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000041648 2024-05-09T00:24:04.226 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.226 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:04.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.226 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 420000 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1191, 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1186, 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 857856, 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 453826, 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 3, 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.165280450, 2024-05-09T00:24:04.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.055141672, 2024-05-09T00:24:04.228 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.052165711, 2024-05-09T00:24:04.228 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.046122727, 2024-05-09T00:24:04.228 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:04.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1185, 2024-05-09T00:24:04.228 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.070331858, 2024-05-09T00:24:04.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000059351 2024-05-09T00:24:04.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.228 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:04.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.228 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 549072, 2024-05-09T00:24:04.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 161584 2024-05-09T00:24:04.229 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.229 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-05-09T00:24:04.229 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-05-09T00:24:04.229 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-05-09T00:24:04.229 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.229 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-05-09T00:24:04.229 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4404, 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.137484946, 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000031218 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.230 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-05-09T00:24:04.231 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-05-09T00:24:04.232 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-05-09T00:24:04.232 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-05-09T00:24:04.232 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-05-09T00:24:04.232 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 0, 2024-05-09T00:24:04.232 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 0, 2024-05-09T00:24:04.232 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 0, 2024-05-09T00:24:04.232 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-05-09T00:24:04.232 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 0, 2024-05-09T00:24:04.232 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-05-09T00:24:04.232 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-05-09T00:24:04.232 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 1, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-05-09T00:24:04.233 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-05-09T00:24:04.234 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-05-09T00:24:04.234 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-05-09T00:24:04.234 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-05-09T00:24:04.234 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 0, 2024-05-09T00:24:04.234 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 0, 2024-05-09T00:24:04.234 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-05-09T00:24:04.234 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T00:24:04.234 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-05-09T00:24:04.234 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T00:24:04.234 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-05-09T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-05-09T00:24:04.236 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-05-09T00:24:04.236 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.236 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-05-09T00:24:04.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-05-09T00:24:04.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-05-09T00:24:04.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-05-09T00:24:04.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-05-09T00:24:04.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-05-09T00:24:04.236 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-05-09T00:24:04.237 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-05-09T00:24:04.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T00:24:04.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-05-09T00:24:04.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-05-09T00:24:04.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-05-09T00:24:04.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-05-09T00:24:04.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-05-09T00:24:04.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.238 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-05-09T00:24:04.238 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-05-09T00:24:04.238 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 5, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 2, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 2, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 4195732, 2024-05-09T00:24:04.239 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 4195424, 2024-05-09T00:24:04.240 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-05-09T00:24:04.240 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.240 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.240 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.240 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.240 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 5 2024-05-09T00:24:04.240 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.240 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-05-09T00:24:04.240 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 3, 2024-05-09T00:24:04.240 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 3, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 0, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 1, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 1, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 0, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 0, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 0, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 0, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 118616, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 215440 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-05-09T00:24:04.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-05-09T00:24:04.242 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-05-09T00:24:04.242 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-05-09T00:24:04.242 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-05-09T00:24:04.242 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-05-09T00:24:04.242 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-05-09T00:24:04.242 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.242 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.242 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.242 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-05-09T00:24:04.242 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.243 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-05-09T00:24:04.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.244 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.244 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-05-09T00:24:04.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.244 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.244 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-05-09T00:24:04.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.244 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-05-09T00:24:04.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.246 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.246 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-05-09T00:24:04.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.246 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.246 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-05-09T00:24:04.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.247 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.247 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.247 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.247 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-05-09T00:24:04.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.248 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.248 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-05-09T00:24:04.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.248 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.248 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-05-09T00:24:04.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.248 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.249 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-05-09T00:24:04.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.249 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.249 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-05-09T00:24:04.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.249 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.250 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.251 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.251 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-05-09T00:24:04.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.251 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.251 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.251 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-05-09T00:24:04.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.251 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.251 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.252 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-05-09T00:24:04.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.252 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.252 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-05-09T00:24:04.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.252 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.252 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-05-09T00:24:04.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.253 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.253 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.253 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-05-09T00:24:04.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.253 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.253 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.253 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.255 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.255 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-05-09T00:24:04.255 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-05-09T00:24:04.255 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-05-09T00:24:04.255 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.255 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.255 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.255 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:04.255 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.255 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-05-09T00:24:04.255 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-05-09T00:24:04.255 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 2161, 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:24:04.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-05-09T00:24:04.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-05-09T00:24:04.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 190431, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 52, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:24:04.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7624, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 160, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 8512, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 13, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-05-09T00:24:04.260 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 2681, 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 21, 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 3, 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 2681, 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2681, 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.351975206, 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000504280 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2681, 2024-05-09T00:24:04.261 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2682 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 2681, 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 2681, 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 1753, 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 926, 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-05-09T00:24:04.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-05-09T00:24:04.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-05-09T00:24:04.264 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 80, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 2, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 5, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 6, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 1, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-05-09T00:24:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T00:24:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 0, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 2954, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1028421, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 2954, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-09T00:24:04.267 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2954, 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1028421, 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-09T00:24:04.268 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 2681, 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 1433, 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2681, 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1433, 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:04.269 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 2681, 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 2681, 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2681, 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 2681, 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:04.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:04.271 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55cf652568f0": { 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-09T00:24:04.272 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-09T00:24:04.273 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-09T00:24:04.273 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:04.273 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:04.273 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:04.273 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:04.273 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:04.273 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:04.273 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-09T00:24:05.053 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-09T00:24:05.053 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-09T00:24:05.053 DEBUG:tasks.check_counter:Getting stats from h 2024-05-09T00:24:05.054 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-09T00:24:05.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:04 smithi169 ceph-mon[34777]: pgmap v757: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-09T00:24:05.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:04 smithi142 ceph-mon[34533]: pgmap v757: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-09T00:24:05.294 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:24:05.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:04 smithi046 ceph-mon[30151]: pgmap v757: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-09T00:24:07.104 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:06 smithi142 ceph-mon[34533]: pgmap v758: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T00:24:07.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:06 smithi169 ceph-mon[34777]: pgmap v758: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T00:24:07.152 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-09T00:24:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:24:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2571, 2024-05-09T00:24:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2573, 2024-05-09T00:24:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 731717, 2024-05-09T00:24:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 817585, 2024-05-09T00:24:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 6, 2024-05-09T00:24:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-05-09T00:24:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.216618221, 2024-05-09T00:24:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.069406291, 2024-05-09T00:24:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.054985109, 2024-05-09T00:24:07.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.075175718, 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2571, 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.084560919, 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000032890 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.153 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 272, 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 267, 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 549084, 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 161002, 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 3, 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.044245777, 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.016967843, 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.019833367, 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.006776171, 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 266, 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.016170515, 2024-05-09T00:24:07.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000060791 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 548528, 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 160304 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 7, 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 201, 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 3532, 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 510542, 2024-05-09T00:24:07.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 6, 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.012625248, 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.008464137, 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.002112277, 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000026510, 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 200, 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.015048764, 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000075243 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 510224 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4418, 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.118786906, 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000026887 2024-05-09T00:24:07.157 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-05-09T00:24:07.158 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 0, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 0, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 0, 2024-05-09T00:24:07.159 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 1, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-05-09T00:24:07.160 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-05-09T00:24:07.161 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-05-09T00:24:07.162 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-05-09T00:24:07.163 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-05-09T00:24:07.164 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 3, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 2, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 2, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-05-09T00:24:07.165 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-05-09T00:24:07.166 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 4195200, 2024-05-09T00:24:07.166 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 4194892, 2024-05-09T00:24:07.166 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-05-09T00:24:07.166 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.166 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.166 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.166 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 3 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 3, 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 3, 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 0, 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 1, 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 1, 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 0, 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 0, 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 0, 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 0, 2024-05-09T00:24:07.167 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 119104, 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 215440 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.168 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.169 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.170 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-05-09T00:24:07.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.172 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.172 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.172 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-05-09T00:24:07.172 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.172 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.172 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.172 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-05-09T00:24:07.172 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.173 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.173 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.173 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-05-09T00:24:07.173 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.173 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.173 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.173 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-05-09T00:24:07.174 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.174 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.174 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.174 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-05-09T00:24:07.174 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.174 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.174 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.174 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.175 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.176 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.177 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.178 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-05-09T00:24:07.179 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 1, 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-05-09T00:24:07.180 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 2164, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:24:07.181 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:24:07.182 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-05-09T00:24:07.183 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:24:07.183 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:24:07.183 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:24:07.183 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-05-09T00:24:07.183 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-05-09T00:24:07.183 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-05-09T00:24:07.183 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:24:07.183 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:24:07.183 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:24:07.183 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:24:07.183 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 181327, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 46, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7624, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 160, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-05-09T00:24:07.184 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 8512, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 13, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 2574, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 21, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 3, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 2574, 2024-05-09T00:24:07.185 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2574, 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.482972769, 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000576135 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2574, 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2575 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 2574, 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 2574, 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-05-09T00:24:07.186 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 1645, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 927, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-05-09T00:24:07.187 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-05-09T00:24:07.188 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 80, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 2, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 4, 2024-05-09T00:24:07.189 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 5, 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 1, 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-05-09T00:24:07.190 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 0, 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-09T00:24:07.191 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 2846, 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1011282, 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 2846, 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2846, 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1011282, 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-05-09T00:24:07.192 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 2574, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 901, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2574, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 901, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.193 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 2574, 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 2574, 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2574, 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 2574, 2024-05-09T00:24:07.194 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-09T00:24:07.195 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x555ecbf408f0": { 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-05-09T00:24:07.196 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:07.197 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:07.197 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-09T00:24:07.197 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-09T00:24:07.197 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-05-09T00:24:07.197 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-05-09T00:24:07.197 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:07.197 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:07.197 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:07.197 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:07.197 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:07.197 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:07.197 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-09T00:24:07.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:06 smithi046 ceph-mon[30151]: pgmap v758: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T00:24:08.083 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-09T00:24:08.083 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-09T00:24:08.083 DEBUG:tasks.check_counter:Getting stats from k 2024-05-09T00:24:08.083 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-09T00:24:08.322 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T00:24:09.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:08 smithi169 ceph-mon[34777]: pgmap v759: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T00:24:09.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:08 smithi142 ceph-mon[34533]: pgmap v759: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T00:24:09.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:08 smithi046 ceph-mon[30151]: pgmap v759: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T00:24:10.106 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 0, 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2, 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 0, 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 0, 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 0, 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.001077268, 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.000271588, 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:10.107 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2, 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 203, 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 320, 2024-05-09T00:24:10.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 193136, 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.012277464, 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.008966407, 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.001011499, 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 201, 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.012310865, 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000061248 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:10.109 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 193136 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 272, 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 267, 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 530656, 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 163712, 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 2, 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 1, 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.047945514, 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.018344254, 2024-05-09T00:24:10.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.021132450, 2024-05-09T00:24:10.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.007574889, 2024-05-09T00:24:10.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:10.111 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 267, 2024-05-09T00:24:10.111 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.017385785, 2024-05-09T00:24:10.111 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000065115 2024-05-09T00:24:10.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:10.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:10.111 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:10.111 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:10.111 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:10.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 530656, 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 163712 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 0, 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:24:10.112 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:24:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:24:10.114 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 147960, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 28, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 0, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 0, 2024-05-09T00:24:10.115 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:24:10.116 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:24:10.116 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-05-09T00:24:10.116 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-05-09T00:24:10.116 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 928, 2024-05-09T00:24:10.116 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 2, 2024-05-09T00:24:10.116 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-05-09T00:24:10.116 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-05-09T00:24:10.116 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-05-09T00:24:10.117 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-05-09T00:24:10.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-05-09T00:24:10.117 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:24:10.117 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-05-09T00:24:10.117 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-05-09T00:24:10.117 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-05-09T00:24:10.117 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 270, 2024-05-09T00:24:10.117 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 489843, 2024-05-09T00:24:10.117 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:10.117 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 270, 2024-05-09T00:24:10.118 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-05-09T00:24:10.118 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-05-09T00:24:10.118 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 270, 2024-05-09T00:24:10.118 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 489843, 2024-05-09T00:24:10.118 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-05-09T00:24:10.118 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-05-09T00:24:10.118 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-05-09T00:24:10.118 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-05-09T00:24:10.118 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:10.118 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-05-09T00:24:10.118 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-05-09T00:24:10.841 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-09T00:24:10.841 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-09T00:24:10.841 DEBUG:tasks.check_counter:Getting stats from c 2024-05-09T00:24:10.842 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-09T00:24:11.094 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:24:11.114 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:10 smithi046 ceph-mon[30151]: pgmap v760: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-05-09T00:24:11.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:10 smithi142 ceph-mon[34533]: pgmap v760: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-05-09T00:24:11.216 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:10 smithi169 ceph-mon[34777]: pgmap v760: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-05-09T00:24:13.050 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-05-09T00:24:13.050 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:24:13.050 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 50, 2024-05-09T00:24:13.050 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 53, 2024-05-09T00:24:13.050 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 13589, 2024-05-09T00:24:13.050 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 65136, 2024-05-09T00:24:13.050 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 15, 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 2, 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.020602963, 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.004222836, 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.010140550, 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000540194, 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 52, 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.016667135, 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000320521 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 45, 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000012741, 2024-05-09T00:24:13.051 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000283 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 4528, 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 1888 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 363, 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 357, 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 572173, 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 727994, 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 54, 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 6, 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.068784310, 2024-05-09T00:24:13.052 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.019810996, 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.039810444, 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.007596675, 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 354, 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.035246748, 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000099567 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 46, 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000006446, 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000140 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 561152, 2024-05-09T00:24:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 166224 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 922, 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1037, 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 218638, 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 718620, 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 8, 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.112332015, 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.056049349, 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.054096362, 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.005000959, 2024-05-09T00:24:13.054 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1037, 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.056999110, 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000054965 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1719, 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000467496, 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000271 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 425024 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-05-09T00:24:13.055 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 41, 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001806643, 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000044064 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T00:24:13.056 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000008120, 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000004060 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 5, 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 3, 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3, 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001589958, 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000529986 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-05-09T00:24:13.057 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 2, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 11, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 14, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 0, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 14, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 14, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 2, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 2, 2024-05-09T00:24:13.058 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 2, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 7, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 3, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 2, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-05-09T00:24:13.059 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 5, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 298, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-05-09T00:24:13.060 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 2, 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 3, 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 2, 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 2 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-05-09T00:24:13.061 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 3, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-05-09T00:24:13.062 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 5, 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 5, 2024-05-09T00:24:13.063 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 2, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4195732, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.008423511, 2024-05-09T00:24:13.064 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002105877 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 17, 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 17, 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 0, 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 15, 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 16, 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 1, 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 14, 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 14, 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 0, 2024-05-09T00:24:13.065 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 2, 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 3, 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 1, 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 121184, 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 223632 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 7, 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 5, 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 14, 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.066 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001352248, 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000676124 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-05-09T00:24:13.067 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-05-09T00:24:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-05-09T00:24:13.069 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000237710, 2024-05-09T00:24:13.069 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000237710 2024-05-09T00:24:13.069 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.069 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-05-09T00:24:13.069 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.069 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.069 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.069 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.069 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-05-09T00:24:13.069 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.069 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-05-09T00:24:13.070 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.071 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.071 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.071 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.071 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-05-09T00:24:13.071 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-05-09T00:24:13.072 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.073 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.074 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.075 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.076 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-05-09T00:24:13.077 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 2, 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 1, 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-05-09T00:24:13.078 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 1, 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 1, 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 741, 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:24:13.079 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:24:13.080 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:24:13.081 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 247067, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 82, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 7624, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 160, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:24:13.082 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 68437, 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 294, 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-05-09T00:24:13.083 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 42, 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 12195, 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 28, 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 42, 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 42, 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.275993989, 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.006571285 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 42, 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 66 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.084 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 42, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 0, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 42, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 0, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 10, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 0, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 4, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 7, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-05-09T00:24:13.085 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 6, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 0, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 20, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T00:24:13.086 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-05-09T00:24:13.087 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 80, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 2, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 8, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 12, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 4, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 18, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 1 2024-05-09T00:24:13.088 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-05-09T00:24:13.089 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 1331, 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 666027, 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 1331, 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T00:24:13.090 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 1331, 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 666027, 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T00:24:13.091 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 42, 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 12195, 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 42, 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 12195, 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-05-09T00:24:13.092 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 42, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 42, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 42, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 42, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.093 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-05-09T00:24:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x55d0db644680": { 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 5, 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1428, 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 5, 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T00:24:13.095 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T00:24:13.096 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 4, 2024-05-09T00:24:13.096 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1428, 2024-05-09T00:24:13.096 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:13.096 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:13.096 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:13.096 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:13.096 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:13.096 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:13.096 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-05-09T00:24:13.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:12 smithi142 ceph-mon[34533]: pgmap v761: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:24:13.313 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:12 smithi046 ceph-mon[30151]: pgmap v761: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:24:13.346 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:12 smithi169 ceph-mon[34777]: pgmap v761: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T00:24:13.907 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-09T00:24:13.907 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-09T00:24:13.907 DEBUG:tasks.check_counter:Getting stats from f 2024-05-09T00:24:13.907 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-09T00:24:14.161 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:24:15.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:14 smithi142 ceph-mon[34533]: pgmap v762: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-09T00:24:15.313 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:14 smithi046 ceph-mon[30151]: pgmap v762: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-09T00:24:15.342 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:14 smithi169 ceph-mon[34777]: pgmap v762: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-09T00:24:15.963 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-05-09T00:24:15.963 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:24:15.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 2205, 2024-05-09T00:24:15.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 2200, 2024-05-09T00:24:15.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 5313402, 2024-05-09T00:24:15.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 776768, 2024-05-09T00:24:15.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-05-09T00:24:15.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 1, 2024-05-09T00:24:15.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.249152370, 2024-05-09T00:24:15.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.079883027, 2024-05-09T00:24:15.963 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.077433905, 2024-05-09T00:24:15.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.072480743, 2024-05-09T00:24:15.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:15.964 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2199, 2024-05-09T00:24:15.964 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.092088313, 2024-05-09T00:24:15.964 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000041877 2024-05-09T00:24:15.964 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.964 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:15.964 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.964 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.964 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.964 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.965 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 549776, 2024-05-09T00:24:15.965 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 163024 2024-05-09T00:24:15.965 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.965 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:24:15.965 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 298, 2024-05-09T00:24:15.965 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 495, 2024-05-09T00:24:15.965 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 75854, 2024-05-09T00:24:15.965 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 517272, 2024-05-09T00:24:15.965 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-05-09T00:24:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-05-09T00:24:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.042689155, 2024-05-09T00:24:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.019172597, 2024-05-09T00:24:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.008150494, 2024-05-09T00:24:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.010722895, 2024-05-09T00:24:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 494, 2024-05-09T00:24:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.029373580, 2024-05-09T00:24:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000059460 2024-05-09T00:24:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 424032 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1189, 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1191, 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 3394856, 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 378109, 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.116327270, 2024-05-09T00:24:15.967 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.042235688, 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.028353227, 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.035536680, 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1189, 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.042754079, 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000035958 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.968 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 5815, 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.166675452, 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000028663 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.969 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 0, 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 0, 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.970 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 0, 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 18, 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 0, 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 18, 2024-05-09T00:24:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-05-09T00:24:15.972 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 18, 2024-05-09T00:24:15.972 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-05-09T00:24:15.972 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 0, 2024-05-09T00:24:15.972 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 0, 2024-05-09T00:24:15.972 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 2, 2024-05-09T00:24:15.972 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 0, 2024-05-09T00:24:15.972 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 0, 2024-05-09T00:24:15.972 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-05-09T00:24:15.973 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-05-09T00:24:15.973 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-05-09T00:24:15.973 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-05-09T00:24:15.973 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-05-09T00:24:15.973 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-05-09T00:24:15.973 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-05-09T00:24:15.973 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-05-09T00:24:15.973 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-05-09T00:24:15.973 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-05-09T00:24:15.974 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-05-09T00:24:15.974 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-05-09T00:24:15.974 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-05-09T00:24:15.974 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-05-09T00:24:15.974 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 4, 2024-05-09T00:24:15.974 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 234, 2024-05-09T00:24:15.974 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-05-09T00:24:15.974 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T00:24:15.975 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-05-09T00:24:15.975 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T00:24:15.975 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-05-09T00:24:15.975 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-05-09T00:24:15.975 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T00:24:15.975 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T00:24:15.975 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-05-09T00:24:15.975 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-05-09T00:24:15.975 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T00:24:15.976 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-05-09T00:24:15.976 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T00:24:15.976 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T00:24:15.976 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T00:24:15.976 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-05-09T00:24:15.976 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-05-09T00:24:15.976 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-05-09T00:24:15.976 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-05-09T00:24:15.976 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.977 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-05-09T00:24:15.977 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-05-09T00:24:15.977 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-05-09T00:24:15.977 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-05-09T00:24:15.977 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-05-09T00:24:15.977 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-05-09T00:24:15.977 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-05-09T00:24:15.977 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-05-09T00:24:15.978 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-05-09T00:24:15.978 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-05-09T00:24:15.978 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-05-09T00:24:15.978 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-05-09T00:24:15.978 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-05-09T00:24:15.978 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-05-09T00:24:15.978 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-05-09T00:24:15.978 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-05-09T00:24:15.979 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-05-09T00:24:15.979 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-05-09T00:24:15.979 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-05-09T00:24:15.979 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-05-09T00:24:15.979 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T00:24:15.979 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-05-09T00:24:15.979 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-05-09T00:24:15.979 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-05-09T00:24:15.979 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-05-09T00:24:15.980 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-05-09T00:24:15.980 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.980 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-05-09T00:24:15.980 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-05-09T00:24:15.980 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 0, 2024-05-09T00:24:15.980 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-05-09T00:24:15.980 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-05-09T00:24:15.980 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 2160, 2024-05-09T00:24:15.980 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-05-09T00:24:15.981 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-05-09T00:24:15.981 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 0, 2024-05-09T00:24:15.981 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-05-09T00:24:15.981 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-05-09T00:24:15.981 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 4, 2024-05-09T00:24:15.981 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-05-09T00:24:15.981 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-05-09T00:24:15.981 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-05-09T00:24:15.982 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-05-09T00:24:15.982 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 11486192, 2024-05-09T00:24:15.982 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 11483563, 2024-05-09T00:24:15.982 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-05-09T00:24:15.982 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.982 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.982 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.982 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.982 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 2160 2024-05-09T00:24:15.983 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.983 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-05-09T00:24:15.983 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 16, 2024-05-09T00:24:15.983 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 17, 2024-05-09T00:24:15.983 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 1, 2024-05-09T00:24:15.983 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 10, 2024-05-09T00:24:15.983 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 11, 2024-05-09T00:24:15.983 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 1, 2024-05-09T00:24:15.984 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 18, 2024-05-09T00:24:15.984 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 19, 2024-05-09T00:24:15.984 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 1, 2024-05-09T00:24:15.984 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 0, 2024-05-09T00:24:15.984 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 0, 2024-05-09T00:24:15.984 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 0, 2024-05-09T00:24:15.984 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 122852, 2024-05-09T00:24:15.984 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 215440 2024-05-09T00:24:15.984 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.985 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-05-09T00:24:15.985 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 0, 2024-05-09T00:24:15.985 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-05-09T00:24:15.985 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 0, 2024-05-09T00:24:15.985 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 0, 2024-05-09T00:24:15.985 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-05-09T00:24:15.985 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-05-09T00:24:15.985 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.985 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.986 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.986 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.986 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-05-09T00:24:15.986 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.986 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.986 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.986 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.986 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-05-09T00:24:15.987 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.987 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.987 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.987 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.987 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-05-09T00:24:15.987 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.987 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.987 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.987 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.988 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-05-09T00:24:15.988 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.988 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.988 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.988 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.988 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-05-09T00:24:15.988 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.988 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.989 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.989 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.989 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-05-09T00:24:15.989 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.989 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.989 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.989 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.989 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-05-09T00:24:15.990 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.990 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.990 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.990 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.990 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-05-09T00:24:15.990 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.990 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.990 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.990 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.990 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-05-09T00:24:15.990 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.991 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.991 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.991 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.991 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-05-09T00:24:15.991 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.991 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.991 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.991 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.991 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-05-09T00:24:15.992 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.992 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.992 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.992 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.992 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-05-09T00:24:15.992 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.992 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.992 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.992 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.992 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.993 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.994 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.995 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.996 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.997 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:15.998 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 2, 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 1, 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 0, 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 0, 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 2140, 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-05-09T00:24:15.999 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:24:16.000 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:24:16.001 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 189457, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 52, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-05-09T00:24:16.002 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 7624, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 160, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 61664, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 455, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-05-09T00:24:16.003 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 3412, 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 26, 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 4, 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 3412, 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3412, 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1.637963784, 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000480059 2024-05-09T00:24:16.004 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3412, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 3413 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 3412, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 3412, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 0, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 1852, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 0, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 1557, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 0, 2024-05-09T00:24:16.005 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 0, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 0, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 2, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 0, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-05-09T00:24:16.006 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-05-09T00:24:16.007 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-05-09T00:24:16.007 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-05-09T00:24:16.007 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-05-09T00:24:16.007 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T00:24:16.007 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-05-09T00:24:16.007 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-05-09T00:24:16.007 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-05-09T00:24:16.007 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-05-09T00:24:16.008 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-05-09T00:24:16.009 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-05-09T00:24:16.009 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 80, 2024-05-09T00:24:16.009 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-05-09T00:24:16.009 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 2, 2024-05-09T00:24:16.009 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 5, 2024-05-09T00:24:16.009 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 5, 2024-05-09T00:24:16.009 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-05-09T00:24:16.009 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-05-09T00:24:16.010 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 0, 2024-05-09T00:24:16.010 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 2, 2024-05-09T00:24:16.010 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 0 2024-05-09T00:24:16.010 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:16.010 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-05-09T00:24:16.010 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 0, 2024-05-09T00:24:16.010 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-05-09T00:24:16.010 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-05-09T00:24:16.011 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-05-09T00:24:16.011 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:16.011 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-05-09T00:24:16.011 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-05-09T00:24:16.011 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:16.011 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-05-09T00:24:16.011 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-05-09T00:24:16.011 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T00:24:16.011 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 3688, 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 8435070, 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 3688, 2024-05-09T00:24:16.012 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T00:24:16.013 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T00:24:16.013 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 3688, 2024-05-09T00:24:16.013 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 8435070, 2024-05-09T00:24:16.013 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:16.013 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:16.013 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:16.013 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:16.013 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:16.013 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:16.013 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-05-09T00:24:16.013 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:16.013 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T00:24:16.014 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:16.014 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T00:24:16.014 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T00:24:16.014 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:16.014 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:16.014 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 3412, 2024-05-09T00:24:16.014 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 7291898, 2024-05-09T00:24:16.014 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 3412, 2024-05-09T00:24:16.014 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 7291898, 2024-05-09T00:24:16.014 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:16.014 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:16.015 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 3412, 2024-05-09T00:24:16.016 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 3412, 2024-05-09T00:24:16.016 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 3412, 2024-05-09T00:24:16.016 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 3412, 2024-05-09T00:24:16.016 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:16.016 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:16.016 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:16.016 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:16.016 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:16.016 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:16.016 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-05-09T00:24:16.016 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:16.017 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:16.018 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:16.018 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:16.018 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:16.018 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x5558c41688f0": { 2024-05-09T00:24:16.018 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:16.018 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T00:24:16.019 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:16.019 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T00:24:16.019 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T00:24:16.019 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:16.019 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:16.019 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T00:24:16.019 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T00:24:16.019 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-05-09T00:24:16.019 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-05-09T00:24:16.019 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:16.020 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:16.020 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:16.020 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:16.021 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:16.021 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:16.022 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-05-09T00:24:16.878 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-09T00:24:16.878 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-09T00:24:16.878 DEBUG:tasks.check_counter:Getting stats from i 2024-05-09T00:24:16.878 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-09T00:24:17.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:16 smithi169 ceph-mon[34777]: pgmap v763: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T00:24:17.122 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:24:17.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:16 smithi142 ceph-mon[34533]: pgmap v763: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T00:24:17.313 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:16 smithi046 ceph-mon[30151]: pgmap v763: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T00:24:18.879 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-05-09T00:24:18.879 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:24:18.879 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 4003, 2024-05-09T00:24:18.879 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 3814, 2024-05-09T00:24:18.879 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1058683, 2024-05-09T00:24:18.879 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 10267951, 2024-05-09T00:24:18.879 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 163, 2024-05-09T00:24:18.879 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 6, 2024-05-09T00:24:18.880 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.415155773, 2024-05-09T00:24:18.880 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.136827510, 2024-05-09T00:24:18.880 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.171294167, 2024-05-09T00:24:18.880 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.084350745, 2024-05-09T00:24:18.880 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:18.880 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3816, 2024-05-09T00:24:18.880 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.190482793, 2024-05-09T00:24:18.880 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000049916 2024-05-09T00:24:18.880 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.880 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1264, 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000360949, 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000285 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1389, 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1451, 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 914605, 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 474706, 2024-05-09T00:24:18.881 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 7, 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 5, 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.155291160, 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.068562906, 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.086463324, 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.012522804, 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1453, 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.235352861, 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000161977 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1570, 2024-05-09T00:24:18.882 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000463549, 2024-05-09T00:24:18.883 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000295 2024-05-09T00:24:18.883 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.883 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 579248, 2024-05-09T00:24:18.883 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 193984 2024-05-09T00:24:18.883 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.883 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:24:18.883 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 2496, 2024-05-09T00:24:18.883 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 4296, 2024-05-09T00:24:18.883 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 879358, 2024-05-09T00:24:18.883 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 1896541, 2024-05-09T00:24:18.883 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 15, 2024-05-09T00:24:18.883 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 8, 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.364667964, 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.131317961, 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.168819773, 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.066547007, 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4288, 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.338605693, 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000078965 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4820, 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001402200, 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000290 2024-05-09T00:24:18.884 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 52288, 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 462816 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 5135, 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1.648759585, 2024-05-09T00:24:18.885 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000321082 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 23, 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000596750, 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000025945 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 643, 2024-05-09T00:24:18.886 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 643, 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 643, 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 3.375656132, 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.005249854 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 12, 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 26, 2024-05-09T00:24:18.887 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 0, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 26, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 25, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 1, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 1, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 2, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 684, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 675, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-05-09T00:24:18.888 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 3, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 316, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 4, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 234, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-05-09T00:24:18.889 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 316, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 204, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 23, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 181, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 112, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 1, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 611, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 614, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 599, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T00:24:18.890 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 707, 2024-05-09T00:24:18.891 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 690, 2024-05-09T00:24:18.891 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 2, 2024-05-09T00:24:18.891 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 622 2024-05-09T00:24:18.891 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.891 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-05-09T00:24:18.891 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-05-09T00:24:18.891 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-05-09T00:24:18.891 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-05-09T00:24:18.891 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-05-09T00:24:18.891 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 4, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 1, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 3, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 2, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-05-09T00:24:18.892 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 23, 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 2161, 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-05-09T00:24:18.893 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 2161, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 4, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 4, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 11486791, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-05-09T00:24:18.894 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2150, 2024-05-09T00:24:18.895 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 4.536420390, 2024-05-09T00:24:18.895 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002109962 2024-05-09T00:24:18.896 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.896 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-05-09T00:24:18.896 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.896 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-05-09T00:24:18.896 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 24, 2024-05-09T00:24:18.896 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 49, 2024-05-09T00:24:18.896 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 25, 2024-05-09T00:24:18.896 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 19, 2024-05-09T00:24:18.897 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 20, 2024-05-09T00:24:18.897 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 1, 2024-05-09T00:24:18.897 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 25, 2024-05-09T00:24:18.897 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 29, 2024-05-09T00:24:18.897 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 4, 2024-05-09T00:24:18.897 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 1, 2024-05-09T00:24:18.897 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 15, 2024-05-09T00:24:18.897 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 14, 2024-05-09T00:24:18.897 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 127448, 2024-05-09T00:24:18.897 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 231824 2024-05-09T00:24:18.897 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.897 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-05-09T00:24:18.898 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 1861, 2024-05-09T00:24:18.898 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-05-09T00:24:18.898 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 643, 2024-05-09T00:24:18.898 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 16, 2024-05-09T00:24:18.898 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-05-09T00:24:18.898 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-05-09T00:24:18.898 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 5, 2024-05-09T00:24:18.898 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.004155961, 2024-05-09T00:24:18.898 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000831192 2024-05-09T00:24:18.898 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.898 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-05-09T00:24:18.898 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 15, 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.003224485, 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000214965 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-05-09T00:24:18.899 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.900 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.901 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-05-09T00:24:18.902 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.902 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.902 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.902 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.902 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 8, 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.014887640, 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001860955 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.903 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000239248, 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000239248 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.004546452, 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002273226 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-05-09T00:24:18.904 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.002450528, 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001225264 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-05-09T00:24:18.905 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.011305908, 2024-05-09T00:24:18.906 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.005652954 2024-05-09T00:24:18.906 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.906 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-05-09T00:24:18.906 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.906 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.906 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.906 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.906 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-05-09T00:24:18.906 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.906 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.906 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 600, 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 3.308185380, 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.005513642 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.907 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 7, 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.022455174, 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.003207882 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.908 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.004205356, 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.004205356 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.909 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:18.910 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.910 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-05-09T00:24:18.910 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-05-09T00:24:18.910 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 2, 2024-05-09T00:24:18.910 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 1, 2024-05-09T00:24:18.910 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-05-09T00:24:18.910 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-05-09T00:24:18.910 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 89, 2024-05-09T00:24:18.910 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 89, 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 746, 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:24:18.911 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:24:18.912 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 303200, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 122, 2024-05-09T00:24:18.913 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 7624, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 160, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-05-09T00:24:18.914 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 102088, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 838, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 3732, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 7313595, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 37, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 3732, 2024-05-09T00:24:18.915 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3732, 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 26.735413923, 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.007163830 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3732, 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 3775 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 3732, 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 364, 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 3368, 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-05-09T00:24:18.916 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-05-09T00:24:18.917 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 0, 2024-05-09T00:24:18.917 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 11, 2024-05-09T00:24:18.917 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 25, 2024-05-09T00:24:18.917 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 2152, 2024-05-09T00:24:18.917 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 57, 2024-05-09T00:24:18.917 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-05-09T00:24:18.917 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-05-09T00:24:18.917 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-05-09T00:24:18.918 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-05-09T00:24:18.918 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 515, 2024-05-09T00:24:18.918 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-05-09T00:24:18.918 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-05-09T00:24:18.918 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-05-09T00:24:18.918 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 339, 2024-05-09T00:24:18.918 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 22, 2024-05-09T00:24:18.918 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-05-09T00:24:18.918 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-05-09T00:24:18.918 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-05-09T00:24:18.918 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-05-09T00:24:18.918 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 599, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-05-09T00:24:18.919 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 80, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 2, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 12, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 12, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-05-09T00:24:18.920 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 47, 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 8 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.921 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 1, 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 1, 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 2, 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 2, 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 7884, 2024-05-09T00:24:18.922 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 2116358, 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 7884, 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 7884, 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 2116358, 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:18.923 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T00:24:18.924 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:18.924 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T00:24:18.924 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T00:24:18.924 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:18.924 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:18.924 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 3732, 2024-05-09T00:24:18.924 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:18 smithi169 ceph-mon[34777]: pgmap v764: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 7313797, 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 3732, 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 7313797, 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 3732, 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 3732, 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 3732, 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 3732, 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:18.926 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 2, 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 202, 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 2, 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2, 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 202, 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:18.927 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.928 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.928 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.928 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:18.928 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:18.929 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x562976db04e0": { 2024-05-09T00:24:18.929 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:18.929 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-05-09T00:24:18.929 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:18.929 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 2161, 2024-05-09T00:24:18.929 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 7292487, 2024-05-09T00:24:18.929 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:18.930 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 2161, 2024-05-09T00:24:18.930 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T00:24:18.930 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T00:24:18.930 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2150, 2024-05-09T00:24:18.930 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 7292487, 2024-05-09T00:24:18.930 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:18.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:18.930 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:18.930 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:18.930 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:18.930 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:18.930 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-05-09T00:24:19.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:18 smithi142 ceph-mon[34533]: pgmap v764: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T00:24:19.314 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:18 smithi046 ceph-mon[30151]: pgmap v764: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T00:24:19.731 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-09T00:24:19.731 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-09T00:24:19.731 DEBUG:tasks.check_counter:Getting stats from l 2024-05-09T00:24:19.731 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-09T00:24:19.970 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T00:24:21.062 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:20 smithi169 ceph-mon[34777]: pgmap v765: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T00:24:21.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:20 smithi142 ceph-mon[34533]: pgmap v765: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T00:24:21.312 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:20 smithi046 ceph-mon[30151]: pgmap v765: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T00:24:21.680 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-05-09T00:24:21.681 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T00:24:21.681 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 4, 2024-05-09T00:24:21.681 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 4, 2024-05-09T00:24:21.681 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 2944, 2024-05-09T00:24:21.681 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 448, 2024-05-09T00:24:21.681 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-05-09T00:24:21.681 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 0, 2024-05-09T00:24:21.681 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.001429995, 2024-05-09T00:24:21.681 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.000037722, 2024-05-09T00:24:21.681 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.000608544, 2024-05-09T00:24:21.681 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3, 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000720137, 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000240045 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T00:24:21.682 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 2, 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 206, 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 320, 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 193664, 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 2, 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.015143994, 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.010819912, 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.001465005, 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:21.683 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 204, 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.015169990, 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000074362 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 193664 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 272, 2024-05-09T00:24:21.684 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 269, 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 528800, 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 165824, 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 2, 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 1, 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.049009102, 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.019332874, 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.019704983, 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.007700764, 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 268, 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.012553239, 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000046840 2024-05-09T00:24:21.685 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:21.686 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-05-09T00:24:21.686 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:21.686 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:21.686 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:21.686 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:21.686 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 528800, 2024-05-09T00:24:21.686 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 165824 2024-05-09T00:24:21.686 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:21.686 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-05-09T00:24:21.686 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 0, 2024-05-09T00:24:21.686 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-05-09T00:24:21.686 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T00:24:21.687 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-05-09T00:24:21.688 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 148744, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 30, 2024-05-09T00:24:21.689 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-05-09T00:24:21.690 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 928, 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 2, 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 274, 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 490363, 2024-05-09T00:24:21.691 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-05-09T00:24:21.692 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 274, 2024-05-09T00:24:21.692 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-05-09T00:24:21.692 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-05-09T00:24:21.692 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 274, 2024-05-09T00:24:21.692 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 490363, 2024-05-09T00:24:21.692 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-05-09T00:24:21.692 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-05-09T00:24:21.692 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-05-09T00:24:21.692 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-05-09T00:24:21.692 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:21.692 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-09T00:24:21.692 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-05-09T00:24:22.386 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-09T00:24:22.386 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-09T00:24:22.386 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-09T00:24:22.396 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T00:24:22.396 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T00:24:22.397 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T00:24:22.420 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T00:24:22.420 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T00:24:22.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-09T00:24:22.775 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:24:23.131 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:22 smithi046 ceph-mon[30151]: pgmap v766: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-09T00:24:23.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:22 smithi142 ceph-mon[34533]: pgmap v766: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-09T00:24:23.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:22 smithi169 ceph-mon[34777]: pgmap v766: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-09T00:24:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.848+0000 7f69fbe85700 1 -- 172.21.15.46:0/1392247205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f4109850 msgr2=0x7f69f41111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.848+0000 7f69fbe85700 1 --2- 172.21.15.46:0/1392247205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f4109850 0x7f69f41111b0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00b0d0 tx=0x7f69ec00b3e0 comp rx=0 tx=0).stop 2024-05-09T00:24:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.848+0000 7f69fbe85700 1 -- 172.21.15.46:0/1392247205 shutdown_connections 2024-05-09T00:24:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.848+0000 7f69fbe85700 1 --2- 172.21.15.46:0/1392247205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f4109850 0x7f69f41111b0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.848+0000 7f69fbe85700 1 --2- 172.21.15.46:0/1392247205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f41045d0 0x7f69f41049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.848+0000 7f69fbe85700 1 --2- 172.21.15.46:0/1392247205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69f4108f00 0x7f69f4109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.848+0000 7f69fbe85700 1 -- 172.21.15.46:0/1392247205 >> 172.21.15.46:0/1392247205 conn(0x7f69f40ffea0 msgr2=0x7f69f41022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.848+0000 7f69fbe85700 1 -- 172.21.15.46:0/1392247205 shutdown_connections 2024-05-09T00:24:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.848+0000 7f69fbe85700 1 -- 172.21.15.46:0/1392247205 wait complete. 2024-05-09T00:24:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.849+0000 7f69fbe85700 1 Processor -- start 2024-05-09T00:24:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.849+0000 7f69fbe85700 1 -- start start 2024-05-09T00:24:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.849+0000 7f69fbe85700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f41045d0 0x7f69f419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.849+0000 7f69fbe85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f4108f00 0x7f69f419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.849+0000 7f69fbe85700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69f4109850 0x7f69f419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.849+0000 7f69fbe85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4114500 con 0x7f69f4108f00 2024-05-09T00:24:24.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.849+0000 7f69fbe85700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4114380 con 0x7f69f41045d0 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.849+0000 7f69fbe85700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4114680 con 0x7f69f4109850 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.849+0000 7f69f9420700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f4108f00 0x7f69f419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.849+0000 7f69f9420700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f4108f00 0x7f69f419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58178/0 (socket says 172.21.15.46:58178) 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.849+0000 7f69f9420700 1 -- 172.21.15.46:0/3526905844 learned_addr learned my addr 172.21.15.46:0/3526905844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69fa422700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69f4109850 0x7f69f419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69f9c21700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f41045d0 0x7f69f419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69f9420700 1 -- 172.21.15.46:0/3526905844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69f4109850 msgr2=0x7f69f419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69f9420700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69f4109850 0x7f69f419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69f9420700 1 -- 172.21.15.46:0/3526905844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f41045d0 msgr2=0x7f69f419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69f9420700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f41045d0 0x7f69f419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69f9420700 1 -- 172.21.15.46:0/3526905844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f419fd40 con 0x7f69f4108f00 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69f9420700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f4108f00 0x7f69f419b410 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f69e400d980 tx=0x7f69e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69eaffd700 1 -- 172.21.15.46:0/3526905844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4017070 con 0x7f69f4108f00 2024-05-09T00:24:24.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69eaffd700 1 -- 172.21.15.46:0/3526905844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69e400fd40 con 0x7f69f4108f00 2024-05-09T00:24:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69fbe85700 1 -- 172.21.15.46:0/3526905844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f41a0030 con 0x7f69f4108f00 2024-05-09T00:24:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69eaffd700 1 -- 172.21.15.46:0/3526905844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e400e3b0 con 0x7f69f4108f00 2024-05-09T00:24:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.850+0000 7f69fbe85700 1 -- 172.21.15.46:0/3526905844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f41a7950 con 0x7f69f4108f00 2024-05-09T00:24:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.851+0000 7f69fbe85700 1 -- 172.21.15.46:0/3526905844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f40512c0 con 0x7f69f4108f00 2024-05-09T00:24:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.853+0000 7f69eaffd700 1 -- 172.21.15.46:0/3526905844 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f69e4010610 con 0x7f69f4108f00 2024-05-09T00:24:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.854+0000 7f69eaffd700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f69e00b2880 0x7f69e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.854+0000 7f69f9c21700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f69e00b2880 0x7f69e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.854+0000 7f69eaffd700 1 -- 172.21.15.46:0/3526905844 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f69e40e1c80 con 0x7f69f4108f00 2024-05-09T00:24:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.855+0000 7f69f9c21700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f69e00b2880 0x7f69e00b4d90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009a10 tx=0x7f69f0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:24.858+0000 7f69eaffd700 1 -- 172.21.15.46:0/3526905844 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69e40a99a0 con 0x7f69f4108f00 2024-05-09T00:24:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.088+0000 7f69fbe85700 1 -- 172.21.15.46:0/3526905844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f69f419c1d0 con 0x7f69f4108f00 2024-05-09T00:24:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.089+0000 7f69eaffd700 1 -- 172.21.15.46:0/3526905844 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v80) ==== 81+0+586 (secure 0 0 0) 0x7f69e40a92a0 con 0x7f69f4108f00 2024-05-09T00:24:25.091 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/1620504773 2024-05-10T00:01:09.191239+0000 2024-05-09T00:24:25.091 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1072138393 2024-05-10T00:01:09.191239+0000 2024-05-09T00:24:25.091 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2786891364 2024-05-10T00:00:41.833323+0000 2024-05-09T00:24:25.091 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/7723180 2024-05-10T00:00:41.833323+0000 2024-05-09T00:24:25.091 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/626888490 2024-05-10T00:01:09.191239+0000 2024-05-09T00:24:25.091 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/1620504773 2024-05-10T00:01:09.191239+0000 2024-05-09T00:24:25.091 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/4236543921 2024-05-10T00:01:09.191239+0000 2024-05-09T00:24:25.091 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2168313126 2024-05-10T00:00:41.833323+0000 2024-05-09T00:24:25.091 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/953818590 2024-05-10T00:00:41.833323+0000 2024-05-09T00:24:25.091 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/953818590 2024-05-10T00:00:41.833323+0000 2024-05-09T00:24:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.091+0000 7f69fbe85700 1 -- 172.21.15.46:0/3526905844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f69e00b2880 msgr2=0x7f69e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.091+0000 7f69fbe85700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f69e00b2880 0x7f69e00b4d90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009a10 tx=0x7f69f0009190 comp rx=0 tx=0).stop 2024-05-09T00:24:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.091+0000 7f69fbe85700 1 -- 172.21.15.46:0/3526905844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f4108f00 msgr2=0x7f69f419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.091+0000 7f69fbe85700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f4108f00 0x7f69f419b410 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f69e400d980 tx=0x7f69e400dc90 comp rx=0 tx=0).stop 2024-05-09T00:24:25.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.091+0000 7f69fbe85700 1 -- 172.21.15.46:0/3526905844 shutdown_connections 2024-05-09T00:24:25.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.091+0000 7f69fbe85700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f69e00b2880 0x7f69e00b4d90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:25.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.091+0000 7f69fbe85700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69f4109850 0x7f69f419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:25.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.091+0000 7f69fbe85700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69f41045d0 0x7f69f419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:25.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.091+0000 7f69fbe85700 1 --2- 172.21.15.46:0/3526905844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f4108f00 0x7f69f419b410 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:25.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.091+0000 7f69fbe85700 1 -- 172.21.15.46:0/3526905844 >> 172.21.15.46:0/3526905844 conn(0x7f69f40ffea0 msgr2=0x7f69f41086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:25.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.091+0000 7f69fbe85700 1 -- 172.21.15.46:0/3526905844 shutdown_connections 2024-05-09T00:24:25.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:25.091+0000 7f69fbe85700 1 -- 172.21.15.46:0/3526905844 wait complete. 2024-05-09T00:24:25.098 INFO:teuthology.orchestra.run.smithi046.stderr:listed 10 entries 2024-05-09T00:24:25.103 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:24 smithi046 ceph-mon[30151]: pgmap v767: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T00:24:25.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:24 smithi142 ceph-mon[34533]: pgmap v767: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T00:24:25.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:24 smithi169 ceph-mon[34777]: pgmap v767: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T00:24:25.881 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T00:24:25.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:24:25.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-09T00:24:25.928 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T00:24:25.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:24:25.928 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T00:24:25.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:24:25.973 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-05-09T00:24:25.990 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T00:24:25.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:24:25.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T00:24:26.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T00:24:26.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-05-09T00:24:26.123 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:25 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/3526905844' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T00:24:26.149 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-09T00:24:26.158 INFO:tasks.cephadm:Teardown begin 2024-05-09T00:24:26.158 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T00:24:26.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:24:26.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:25 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/3526905844' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T00:24:26.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:25 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/3526905844' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T00:24:26.441 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:24:27.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:26 smithi142 ceph-mon[34533]: pgmap v768: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-09T00:24:27.313 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:26 smithi046 ceph-mon[30151]: pgmap v768: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-09T00:24:27.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:26 smithi169 ceph-mon[34777]: pgmap v768: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-09T00:24:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.281+0000 7fa3850b3700 1 -- 172.21.15.46:0/545976049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa380108740 msgr2=0x7fa3801100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.281+0000 7fa3850b3700 1 --2- 172.21.15.46:0/545976049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa380108740 0x7fa3801100a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa37800b0d0 tx=0x7fa37800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:24:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.281+0000 7fa3850b3700 1 -- 172.21.15.46:0/545976049 shutdown_connections 2024-05-09T00:24:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.281+0000 7fa3850b3700 1 --2- 172.21.15.46:0/545976049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa380108740 0x7fa3801100a0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.281+0000 7fa3850b3700 1 --2- 172.21.15.46:0/545976049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3801034c0 0x7fa3801038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.281+0000 7fa3850b3700 1 --2- 172.21.15.46:0/545976049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa380107df0 0x7fa380108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.281+0000 7fa3850b3700 1 -- 172.21.15.46:0/545976049 >> 172.21.15.46:0/545976049 conn(0x7fa3800fedf0 msgr2=0x7fa380101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.281+0000 7fa3850b3700 1 -- 172.21.15.46:0/545976049 shutdown_connections 2024-05-09T00:24:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.281+0000 7fa3850b3700 1 -- 172.21.15.46:0/545976049 wait complete. 2024-05-09T00:24:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.282+0000 7fa3850b3700 1 Processor -- start 2024-05-09T00:24:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.282+0000 7fa3850b3700 1 -- start start 2024-05-09T00:24:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa3850b3700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3801034c0 0x7fa38019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa3850b3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa380107df0 0x7fa38019c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa3850b3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa380108740 0x7fa3801a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa3850b3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa380113370 con 0x7fa380108740 2024-05-09T00:24:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa3850b3700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801131f0 con 0x7fa3801034c0 2024-05-09T00:24:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa3850b3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801134f0 con 0x7fa380107df0 2024-05-09T00:24:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa37f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa380108740 0x7fa3801a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa37f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa380108740 0x7fa3801a6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36056/0 (socket says 172.21.15.46:36056) 2024-05-09T00:24:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa37f59e700 1 -- 172.21.15.46:0/1416933178 learned_addr learned my addr 172.21.15.46:0/1416933178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:24:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa37e59c700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa380107df0 0x7fa38019c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa37ed9d700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3801034c0 0x7fa38019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa37f59e700 1 -- 172.21.15.46:0/1416933178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa380107df0 msgr2=0x7fa38019c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa37f59e700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa380107df0 0x7fa38019c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa37f59e700 1 -- 172.21.15.46:0/1416933178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3801034c0 msgr2=0x7fa38019bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa37f59e700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3801034c0 0x7fa38019bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.283+0000 7fa37f59e700 1 -- 172.21.15.46:0/1416933178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38019ca20 con 0x7fa380108740 2024-05-09T00:24:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.284+0000 7fa37f59e700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa380108740 0x7fa3801a6cb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa378009850 tx=0x7fa378007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.284+0000 7fa367fff700 1 -- 172.21.15.46:0/1416933178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37801f070 con 0x7fa380108740 2024-05-09T00:24:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.284+0000 7fa3850b3700 1 -- 172.21.15.46:0/1416933178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3801a72b0 con 0x7fa380108740 2024-05-09T00:24:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.284+0000 7fa367fff700 1 -- 172.21.15.46:0/1416933178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa378004c50 con 0x7fa380108740 2024-05-09T00:24:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.284+0000 7fa367fff700 1 -- 172.21.15.46:0/1416933178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37801b690 con 0x7fa380108740 2024-05-09T00:24:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.284+0000 7fa3850b3700 1 -- 172.21.15.46:0/1416933178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3801a7700 con 0x7fa380108740 2024-05-09T00:24:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.285+0000 7fa3850b3700 1 -- 172.21.15.46:0/1416933178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3800512c0 con 0x7fa380108740 2024-05-09T00:24:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.287+0000 7fa367fff700 1 -- 172.21.15.46:0/1416933178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa378022070 con 0x7fa380108740 2024-05-09T00:24:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.288+0000 7fa367fff700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa3680b2880 0x7fa3680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.288+0000 7fa37ed9d700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa3680b2880 0x7fa3680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.288+0000 7fa367fff700 1 -- 172.21.15.46:0/1416933178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7fa37800f020 con 0x7fa380108740 2024-05-09T00:24:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.289+0000 7fa37ed9d700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa3680b2880 0x7fa3680b4d90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa370009c60 tx=0x7fa370009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.293+0000 7fa367fff700 1 -- 172.21.15.46:0/1416933178 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa378029020 con 0x7fa380108740 2024-05-09T00:24:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.591+0000 7fa3850b3700 1 -- 172.21.15.46:0/1416933178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa38002e7b0 con 0x7fa380108740 2024-05-09T00:24:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.592+0000 7fa367fff700 1 -- 172.21.15.46:0/1416933178 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7fa378009cb0 con 0x7fa380108740 2024-05-09T00:24:28.592 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:24:28.593 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.595+0000 7fa3850b3700 1 -- 172.21.15.46:0/1416933178 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa3680b2880 msgr2=0x7fa3680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.595+0000 7fa3850b3700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa3680b2880 0x7fa3680b4d90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa370009c60 tx=0x7fa370009260 comp rx=0 tx=0).stop 2024-05-09T00:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.595+0000 7fa3850b3700 1 -- 172.21.15.46:0/1416933178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa380108740 msgr2=0x7fa3801a6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.595+0000 7fa3850b3700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa380108740 0x7fa3801a6cb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa378009850 tx=0x7fa378007a20 comp rx=0 tx=0).stop 2024-05-09T00:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.595+0000 7fa3850b3700 1 -- 172.21.15.46:0/1416933178 shutdown_connections 2024-05-09T00:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.595+0000 7fa3850b3700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7fa3680b2880 0x7fa3680b4d90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.595+0000 7fa3850b3700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa380108740 0x7fa3801a6cb0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.595+0000 7fa3850b3700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3801034c0 0x7fa38019bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.595+0000 7fa3850b3700 1 --2- 172.21.15.46:0/1416933178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa380107df0 0x7fa38019c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.595+0000 7fa3850b3700 1 -- 172.21.15.46:0/1416933178 >> 172.21.15.46:0/1416933178 conn(0x7fa3800fedf0 msgr2=0x7fa380107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.595+0000 7fa3850b3700 1 -- 172.21.15.46:0/1416933178 shutdown_connections 2024-05-09T00:24:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:28.595+0000 7fa3850b3700 1 -- 172.21.15.46:0/1416933178 wait complete. 2024-05-09T00:24:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:24:29.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:28 smithi142 ceph-mon[34533]: pgmap v769: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T00:24:29.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:28 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1416933178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:24:29.310 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:28 smithi046 ceph-mon[30151]: pgmap v769: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T00:24:29.310 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:28 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1416933178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:24:29.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T00:24:29.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:28 smithi169 ceph-mon[34777]: pgmap v769: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T00:24:29.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:28 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1416933178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:24:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:24:31.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:30 smithi046 ceph-mon[30151]: pgmap v770: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T00:24:31.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:30 smithi142 ceph-mon[34533]: pgmap v770: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T00:24:31.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:30 smithi169 ceph-mon[34777]: pgmap v770: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T00:24:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.502+0000 7f8bdff0b700 1 -- 172.21.15.46:0/4130472762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd81034c0 msgr2=0x7f8bd81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.502+0000 7f8bdff0b700 1 --2- 172.21.15.46:0/4130472762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd81034c0 0x7f8bd81038d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400b0d0 tx=0x7f8bd400b3e0 comp rx=0 tx=0).stop 2024-05-09T00:24:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.502+0000 7f8bdff0b700 1 -- 172.21.15.46:0/4130472762 shutdown_connections 2024-05-09T00:24:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.502+0000 7f8bdff0b700 1 --2- 172.21.15.46:0/4130472762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd8108740 0x7f8bd81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.502+0000 7f8bdff0b700 1 --2- 172.21.15.46:0/4130472762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd81034c0 0x7f8bd81038d0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.502+0000 7f8bdff0b700 1 --2- 172.21.15.46:0/4130472762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd8107df0 0x7f8bd8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.502+0000 7f8bdff0b700 1 -- 172.21.15.46:0/4130472762 >> 172.21.15.46:0/4130472762 conn(0x7f8bd80fedc0 msgr2=0x7f8bd81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.502+0000 7f8bdff0b700 1 -- 172.21.15.46:0/4130472762 shutdown_connections 2024-05-09T00:24:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.502+0000 7f8bdff0b700 1 -- 172.21.15.46:0/4130472762 wait complete. 2024-05-09T00:24:31.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bdff0b700 1 Processor -- start 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bdff0b700 1 -- start start 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bdff0b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd81034c0 0x7f8bd819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bdff0b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd8107df0 0x7f8bd819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bdff0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8108740 0x7f8bd819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bdff0b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8113390 con 0x7f8bd8108740 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bdff0b700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8113210 con 0x7f8bd8107df0 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bdff0b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8113510 con 0x7f8bd81034c0 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bde4a8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8108740 0x7f8bd819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bde4a8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8108740 0x7f8bd819f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36074/0 (socket says 172.21.15.46:36074) 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bde4a8700 1 -- 172.21.15.46:0/175137893 learned_addr learned my addr 172.21.15.46:0/175137893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bddca7700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd81034c0 0x7f8bd819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.503+0000 7f8bdd4a6700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd8107df0 0x7f8bd819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:31.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.504+0000 7f8bde4a8700 1 -- 172.21.15.46:0/175137893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd81034c0 msgr2=0x7f8bd819ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.504+0000 7f8bde4a8700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd81034c0 0x7f8bd819ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.504+0000 7f8bde4a8700 1 -- 172.21.15.46:0/175137893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd8107df0 msgr2=0x7f8bd819b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.504+0000 7f8bde4a8700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd8107df0 0x7f8bd819b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.504+0000 7f8bde4a8700 1 -- 172.21.15.46:0/175137893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd819fc50 con 0x7f8bd8108740 2024-05-09T00:24:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.504+0000 7f8bde4a8700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8108740 0x7f8bd819f660 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000c900 tx=0x7f8bd000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.504+0000 7f8bceffd700 1 -- 172.21.15.46:0/175137893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00078d0 con 0x7f8bd8108740 2024-05-09T00:24:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.504+0000 7f8bceffd700 1 -- 172.21.15.46:0/175137893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd000f450 con 0x7f8bd8108740 2024-05-09T00:24:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.504+0000 7f8bdff0b700 1 -- 172.21.15.46:0/175137893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd819ff40 con 0x7f8bd8108740 2024-05-09T00:24:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.504+0000 7f8bceffd700 1 -- 172.21.15.46:0/175137893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0018620 con 0x7f8bd8108740 2024-05-09T00:24:31.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.504+0000 7f8bdff0b700 1 -- 172.21.15.46:0/175137893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd81a0430 con 0x7f8bd8108740 2024-05-09T00:24:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.505+0000 7f8bdff0b700 1 -- 172.21.15.46:0/175137893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd80512c0 con 0x7f8bd8108740 2024-05-09T00:24:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.507+0000 7f8bceffd700 1 -- 172.21.15.46:0/175137893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8bd0007a70 con 0x7f8bd8108740 2024-05-09T00:24:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.508+0000 7f8bceffd700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8bc40b2880 0x7f8bc40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.508+0000 7f8bddca7700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8bc40b2880 0x7f8bc40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.508+0000 7f8bceffd700 1 -- 172.21.15.46:0/175137893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f8bd00e2130 con 0x7f8bd8108740 2024-05-09T00:24:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.508+0000 7f8bddca7700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8bc40b2880 0x7f8bc40b4d90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009900 tx=0x7f8bd400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.511+0000 7f8bceffd700 1 -- 172.21.15.46:0/175137893 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8bd0014030 con 0x7f8bd8108740 2024-05-09T00:24:31.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.776+0000 7f8bdff0b700 1 -- 172.21.15.46:0/175137893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bd802e5f0 con 0x7f8bd8108740 2024-05-09T00:24:31.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.777+0000 7f8bceffd700 1 -- 172.21.15.46:0/175137893 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11879 (secure 0 0 0) 0x7f8bd00a9ea0 con 0x7f8bd8108740 2024-05-09T00:24:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:24:31.778 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1791470384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1791470384},{"type":"v1","addr":"172.21.15.46:6835","nonce":1791470384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4102968476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4102968476},{"type":"v1","addr":"172.21.15.46:6837","nonce":4102968476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/612832273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":612832273},{"type":"v1","addr":"172.21.15.46:6839","nonce":612832273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1387176303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1387176303},{"type":"v1","addr":"172.21.15.46:6841","nonce":1387176303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/512022880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":512022880},{"type":"v1","addr":"172.21.15.142:6833","nonce":512022880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2870369857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2870369857},{"type":"v1","addr":"172.21.15.169:6833","nonce":2870369857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T00:08:28.009815+0000","modified":"2024-05-09T00:11:50.794344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24497,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2701851507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2701851507},{"type":"v1","addr":"172.21.15.142:6835","nonce":2701851507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/3779152362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3779152362},{"type":"v1","addr":"172.21.15.142:6837","nonce":3779152362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1842121683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1842121683},{"type":"v1","addr":"172.21.15.169:6835","nonce":1842121683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.169:6837/1024289476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1024289476},{"type":"v1","addr":"172.21.15.169:6837","nonce":1024289476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3645430666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3645430666},{"type":"v1","addr":"172.21.15.142:6839","nonce":3645430666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6839/1664161265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1664161265},{"type":"v1","addr":"172.21.15.169:6839","nonce":1664161265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24487,24497,24499]},"id":1}]} 2024-05-09T00:24:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.780+0000 7f8bdff0b700 1 -- 172.21.15.46:0/175137893 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8bc40b2880 msgr2=0x7f8bc40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.780+0000 7f8bdff0b700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8bc40b2880 0x7f8bc40b4d90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009900 tx=0x7f8bd400b040 comp rx=0 tx=0).stop 2024-05-09T00:24:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.780+0000 7f8bdff0b700 1 -- 172.21.15.46:0/175137893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8108740 msgr2=0x7f8bd819f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.780+0000 7f8bdff0b700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8108740 0x7f8bd819f660 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000c900 tx=0x7f8bd000cc10 comp rx=0 tx=0).stop 2024-05-09T00:24:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.780+0000 7f8bdff0b700 1 -- 172.21.15.46:0/175137893 shutdown_connections 2024-05-09T00:24:31.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.780+0000 7f8bdff0b700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f8bc40b2880 0x7f8bc40b4d90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.780+0000 7f8bdff0b700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd8108740 0x7f8bd819f660 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.780+0000 7f8bdff0b700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bd81034c0 0x7f8bd819ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.780+0000 7f8bdff0b700 1 --2- 172.21.15.46:0/175137893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd8107df0 0x7f8bd819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.780+0000 7f8bdff0b700 1 -- 172.21.15.46:0/175137893 >> 172.21.15.46:0/175137893 conn(0x7f8bd80fedc0 msgr2=0x7f8bd81063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.780+0000 7f8bdff0b700 1 -- 172.21.15.46:0/175137893 shutdown_connections 2024-05-09T00:24:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:31.780+0000 7f8bdff0b700 1 -- 172.21.15.46:0/175137893 wait complete. 2024-05-09T00:24:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-05-09T00:24:32.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:31 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/175137893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:24:32.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:31 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/175137893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:24:32.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:31 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/175137893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T00:24:32.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T00:24:32.796 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:24:33.190 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:32 smithi046 ceph-mon[30151]: pgmap v771: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T00:24:33.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:32 smithi142 ceph-mon[34533]: pgmap v771: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T00:24:33.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:32 smithi169 ceph-mon[34777]: pgmap v771: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T00:24:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.500+0000 7f1a379c1700 1 -- 172.21.15.46:0/1792375553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a30108f20 msgr2=0x7f1a30109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.500+0000 7f1a379c1700 1 --2- 172.21.15.46:0/1792375553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a30108f20 0x7f1a30109330 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f1a200099c0 tx=0x7f1a20009cd0 comp rx=0 tx=0).stop 2024-05-09T00:24:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.500+0000 7f1a379c1700 1 -- 172.21.15.46:0/1792375553 shutdown_connections 2024-05-09T00:24:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.500+0000 7f1a379c1700 1 --2- 172.21.15.46:0/1792375553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a30109870 0x7f1a301111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.500+0000 7f1a379c1700 1 --2- 172.21.15.46:0/1792375553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a301045f0 0x7f1a30104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.500+0000 7f1a379c1700 1 --2- 172.21.15.46:0/1792375553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a30108f20 0x7f1a30109330 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.500+0000 7f1a379c1700 1 -- 172.21.15.46:0/1792375553 >> 172.21.15.46:0/1792375553 conn(0x7f1a300ffec0 msgr2=0x7f1a30102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.500+0000 7f1a379c1700 1 -- 172.21.15.46:0/1792375553 shutdown_connections 2024-05-09T00:24:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.500+0000 7f1a379c1700 1 -- 172.21.15.46:0/1792375553 wait complete. 2024-05-09T00:24:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.501+0000 7f1a379c1700 1 Processor -- start 2024-05-09T00:24:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.501+0000 7f1a379c1700 1 -- start start 2024-05-09T00:24:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.501+0000 7f1a379c1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a301045f0 0x7f1a30076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.501+0000 7f1a379c1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a30108f20 0x7f1a30077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.501+0000 7f1a379c1700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a30109870 0x7f1a30077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.501+0000 7f1a379c1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301144f0 con 0x7f1a30108f20 2024-05-09T00:24:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.501+0000 7f1a379c1700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30114370 con 0x7f1a30109870 2024-05-09T00:24:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.501+0000 7f1a379c1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30114670 con 0x7f1a301045f0 2024-05-09T00:24:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a34f5c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a30108f20 0x7f1a30077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a34f5c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a30108f20 0x7f1a30077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36094/0 (socket says 172.21.15.46:36094) 2024-05-09T00:24:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a34f5c700 1 -- 172.21.15.46:0/995273924 learned_addr learned my addr 172.21.15.46:0/995273924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:24:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a3575d700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a301045f0 0x7f1a30076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a35f5e700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a30109870 0x7f1a30077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a34f5c700 1 -- 172.21.15.46:0/995273924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a301045f0 msgr2=0x7f1a30076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a34f5c700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a301045f0 0x7f1a30076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a34f5c700 1 -- 172.21.15.46:0/995273924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a30109870 msgr2=0x7f1a30077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a34f5c700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a30109870 0x7f1a30077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a34f5c700 1 -- 172.21.15.46:0/995273924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a301a91c0 con 0x7f1a30108f20 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a3575d700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a301045f0 0x7f1a30076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a34f5c700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a30108f20 0x7f1a30077110 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009f50 tx=0x7f1a200037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a267fc700 1 -- 172.21.15.46:0/995273924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20027070 con 0x7f1a30108f20 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a267fc700 1 -- 172.21.15.46:0/995273924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a20004140 con 0x7f1a30108f20 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a379c1700 1 -- 172.21.15.46:0/995273924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a301a9450 con 0x7f1a30108f20 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.502+0000 7f1a267fc700 1 -- 172.21.15.46:0/995273924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2000f460 con 0x7f1a30108f20 2024-05-09T00:24:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.503+0000 7f1a379c1700 1 -- 172.21.15.46:0/995273924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a301a9900 con 0x7f1a30108f20 2024-05-09T00:24:34.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.504+0000 7f1a379c1700 1 -- 172.21.15.46:0/995273924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a300512c0 con 0x7f1a30108f20 2024-05-09T00:24:34.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.505+0000 7f1a267fc700 1 -- 172.21.15.46:0/995273924 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1a2002a070 con 0x7f1a30108f20 2024-05-09T00:24:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.506+0000 7f1a267fc700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1a1c0b2900 0x7f1a1c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.506+0000 7f1a3575d700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1a1c0b2900 0x7f1a1c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.506+0000 7f1a267fc700 1 -- 172.21.15.46:0/995273924 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f1a200e36c0 con 0x7f1a30108f20 2024-05-09T00:24:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.507+0000 7f1a3575d700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1a1c0b2900 0x7f1a1c0b4e10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c005fd0 tx=0x7f1a2c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:34.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.510+0000 7f1a267fc700 1 -- 172.21.15.46:0/995273924 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1a20013070 con 0x7f1a30108f20 2024-05-09T00:24:34.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.724+0000 7f1a379c1700 1 -- 172.21.15.46:0/995273924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a3002e5f0 con 0x7f1a30108f20 2024-05-09T00:24:34.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.726+0000 7f1a267fc700 1 -- 172.21.15.46:0/995273924 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22694 (secure 0 0 0) 0x7f1a200ab470 con 0x7f1a30108f20 2024-05-09T00:24:34.726 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T00:24:34.726 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"eec67860-0d96-11ef-bc97-c7b262605968","created":"2024-05-09T00:00:14.356524+0000","modified":"2024-05-09T00:09:10.458026+0000","last_up_change":"2024-05-09T00:07:23.332796+0000","last_in_change":"2024-05-09T00:07:03.238539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T00:03:33.278910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T00:08:21.014416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T00:08:24.922968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"740d6388-5c1f-45eb-93b0-7583bfa5a225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6803","nonce":3727342125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6805","nonce":3727342125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6809","nonce":3727342125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3727342125},{"type":"v1","addr":"172.21.15.46:6807","nonce":3727342125}]},"public_addr":"172.21.15.46:6803/3727342125","cluster_addr":"172.21.15.46:6805/3727342125","heartbeat_back_addr":"172.21.15.46:6809/3727342125","heartbeat_front_addr":"172.21.15.46:6807/3727342125","state":["exists","up"]},{"osd":1,"uuid":"85fbf3ae-0be8-4504-84d6-66fae0d2169d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6801","nonce":3156042469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6803","nonce":3156042469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6807","nonce":3156042469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":3156042469},{"type":"v1","addr":"172.21.15.142:6805","nonce":3156042469}]},"public_addr":"172.21.15.142:6801/3156042469","cluster_addr":"172.21.15.142:6803/3156042469","heartbeat_back_addr":"172.21.15.142:6807/3156042469","heartbeat_front_addr":"172.21.15.142:6805/3156042469","state":["exists","up"]},{"osd":2,"uuid":"05c070e7-83ad-4949-9834-4ce51ac0a2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6801","nonce":2680673064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6803","nonce":2680673064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6807","nonce":2680673064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2680673064},{"type":"v1","addr":"172.21.15.169:6805","nonce":2680673064}]},"public_addr":"172.21.15.169:6801/2680673064","cluster_addr":"172.21.15.169:6803/2680673064","heartbeat_back_addr":"172.21.15.169:6807/2680673064","heartbeat_front_addr":"172.21.15.169:6805/2680673064","state":["exists","up"]},{"osd":3,"uuid":"09d7bbe1-58b6-4407-bef2-920f179dfb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6811","nonce":1158536261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6813","nonce":1158536261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6817","nonce":1158536261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1158536261},{"type":"v1","addr":"172.21.15.46:6815","nonce":1158536261}]},"public_addr":"172.21.15.46:6811/1158536261","cluster_addr":"172.21.15.46:6813/1158536261","heartbeat_back_addr":"172.21.15.46:6817/1158536261","heartbeat_front_addr":"172.21.15.46:6815/1158536261","state":["exists","up"]},{"osd":4,"uuid":"66c9644b-6b0b-4fe7-aa63-b9d87bff7153","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6809","nonce":2299616865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6811","nonce":2299616865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6815","nonce":2299616865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2299616865},{"type":"v1","addr":"172.21.15.142:6813","nonce":2299616865}]},"public_addr":"172.21.15.142:6809/2299616865","cluster_addr":"172.21.15.142:6811/2299616865","heartbeat_back_addr":"172.21.15.142:6815/2299616865","heartbeat_front_addr":"172.21.15.142:6813/2299616865","state":["exists","up"]},{"osd":5,"uuid":"96a72e01-1c5f-418e-bb9b-b790d9b3059b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6809","nonce":445044713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6811","nonce":445044713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6815","nonce":445044713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":445044713},{"type":"v1","addr":"172.21.15.169:6813","nonce":445044713}]},"public_addr":"172.21.15.169:6809/445044713","cluster_addr":"172.21.15.169:6811/445044713","heartbeat_back_addr":"172.21.15.169:6815/445044713","heartbeat_front_addr":"172.21.15.169:6813/445044713","state":["exists","up"]},{"osd":6,"uuid":"6b459542-0e56-4369-b542-e36c46a18f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6819","nonce":1394035482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6821","nonce":1394035482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6825","nonce":1394035482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1394035482},{"type":"v1","addr":"172.21.15.46:6823","nonce":1394035482}]},"public_addr":"172.21.15.46:6819/1394035482","cluster_addr":"172.21.15.46:6821/1394035482","heartbeat_back_addr":"172.21.15.46:6825/1394035482","heartbeat_front_addr":"172.21.15.46:6823/1394035482","state":["exists","up"]},{"osd":7,"uuid":"786b4517-4cbd-4fa3-b5a0-39cccf839893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6817","nonce":596805607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6819","nonce":596805607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6823","nonce":596805607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":596805607},{"type":"v1","addr":"172.21.15.142:6821","nonce":596805607}]},"public_addr":"172.21.15.142:6817/596805607","cluster_addr":"172.21.15.142:6819/596805607","heartbeat_back_addr":"172.21.15.142:6823/596805607","heartbeat_front_addr":"172.21.15.142:6821/596805607","state":["exists","up"]},{"osd":8,"uuid":"b0fc8f1c-0be0-48c2-926d-08fcfb1e0afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6817","nonce":2650652376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6819","nonce":2650652376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6823","nonce":2650652376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":2650652376},{"type":"v1","addr":"172.21.15.169:6821","nonce":2650652376}]},"public_addr":"172.21.15.169:6817/2650652376","cluster_addr":"172.21.15.169:6819/2650652376","heartbeat_back_addr":"172.21.15.169:6823/2650652376","heartbeat_front_addr":"172.21.15.169:6821/2650652376","state":["exists","up"]},{"osd":9,"uuid":"32df0dab-5875-4beb-b733-60a915c94a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6827","nonce":374861248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6829","nonce":374861248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6833","nonce":374861248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":374861248},{"type":"v1","addr":"172.21.15.46:6831","nonce":374861248}]},"public_addr":"172.21.15.46:6827/374861248","cluster_addr":"172.21.15.46:6829/374861248","heartbeat_back_addr":"172.21.15.46:6833/374861248","heartbeat_front_addr":"172.21.15.46:6831/374861248","state":["exists","up"]},{"osd":10,"uuid":"380a4f47-e528-4258-8836-32eac0e6e630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6825","nonce":1364891982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6827","nonce":1364891982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6831","nonce":1364891982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1364891982},{"type":"v1","addr":"172.21.15.142:6829","nonce":1364891982}]},"public_addr":"172.21.15.142:6825/1364891982","cluster_addr":"172.21.15.142:6827/1364891982","heartbeat_back_addr":"172.21.15.142:6831/1364891982","heartbeat_front_addr":"172.21.15.142:6829/1364891982","state":["exists","up"]},{"osd":11,"uuid":"56b04b15-6fcd-497a-aa66-19da9624b656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6825","nonce":4262478976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6827","nonce":4262478976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6831","nonce":4262478976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":4262478976},{"type":"v1","addr":"172.21.15.169:6829","nonce":4262478976}]},"public_addr":"172.21.15.169:6825/4262478976","cluster_addr":"172.21.15.169:6827/4262478976","heartbeat_back_addr":"172.21.15.169:6831/4262478976","heartbeat_front_addr":"172.21.15.169:6829/4262478976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:02:42.159298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:06.053328+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:29.587137+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:03:54.573782+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:20.276073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:04:45.320565+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:10.985522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:05:36.806137+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:02.266736+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:29.788773+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:06:55.827639+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T00:07:21.762382+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/1072138393":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2786891364":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/7723180":"2024-05-10T00:00:41.833323+0000","172.21.15.46:0/626888490":"2024-05-10T00:01:09.191239+0000","172.21.15.46:6800/1620504773":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/4236543921":"2024-05-10T00:01:09.191239+0000","172.21.15.46:0/2168313126":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6800/953818590":"2024-05-10T00:00:41.833323+0000","172.21.15.46:6801/953818590":"2024-05-10T00:00:41.833323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T00:24:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.728+0000 7f1a379c1700 1 -- 172.21.15.46:0/995273924 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1a1c0b2900 msgr2=0x7f1a1c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.728+0000 7f1a379c1700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1a1c0b2900 0x7f1a1c0b4e10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c005fd0 tx=0x7f1a2c005e30 comp rx=0 tx=0).stop 2024-05-09T00:24:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.728+0000 7f1a379c1700 1 -- 172.21.15.46:0/995273924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a30108f20 msgr2=0x7f1a30077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.728+0000 7f1a379c1700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a30108f20 0x7f1a30077110 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009f50 tx=0x7f1a200037c0 comp rx=0 tx=0).stop 2024-05-09T00:24:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.728+0000 7f1a379c1700 1 -- 172.21.15.46:0/995273924 shutdown_connections 2024-05-09T00:24:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.728+0000 7f1a379c1700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f1a1c0b2900 0x7f1a1c0b4e10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.728+0000 7f1a379c1700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a30109870 0x7f1a30077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.728+0000 7f1a379c1700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a301045f0 0x7f1a30076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:34.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.728+0000 7f1a379c1700 1 --2- 172.21.15.46:0/995273924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a30108f20 0x7f1a30077110 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:34.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.728+0000 7f1a379c1700 1 -- 172.21.15.46:0/995273924 >> 172.21.15.46:0/995273924 conn(0x7f1a300ffec0 msgr2=0x7f1a3010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:34.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.728+0000 7f1a379c1700 1 -- 172.21.15.46:0/995273924 shutdown_connections 2024-05-09T00:24:34.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:34.728+0000 7f1a379c1700 1 -- 172.21.15.46:0/995273924 wait complete. 2024-05-09T00:24:35.124 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:34 smithi046 ceph-mon[30151]: pgmap v772: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T00:24:35.124 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:34 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/995273924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:24:35.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:34 smithi142 ceph-mon[34533]: pgmap v772: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T00:24:35.193 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:34 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/995273924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:24:35.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:34 smithi169 ceph-mon[34777]: pgmap v772: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T00:24:35.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:34 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/995273924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T00:24:35.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-09T00:24:35.832 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:24:36.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:35 smithi046 ceph-mon[30151]: pgmap v773: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T00:24:36.196 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:35 smithi142 ceph-mon[34533]: pgmap v773: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T00:24:36.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:35 smithi169 ceph-mon[34777]: pgmap v773: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T00:24:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.672+0000 7f70eecb4700 1 -- 172.21.15.46:0/2392839213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e8107df0 msgr2=0x7f70e8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.672+0000 7f70eecb4700 1 --2- 172.21.15.46:0/2392839213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e8107df0 0x7f70e8108200 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f70d800b0d0 tx=0x7f70d800b3e0 comp rx=0 tx=0).stop 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.672+0000 7f70eecb4700 1 -- 172.21.15.46:0/2392839213 shutdown_connections 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.672+0000 7f70eecb4700 1 --2- 172.21.15.46:0/2392839213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70e8108740 0x7f70e81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.672+0000 7f70eecb4700 1 --2- 172.21.15.46:0/2392839213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70e81034c0 0x7f70e81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.672+0000 7f70eecb4700 1 --2- 172.21.15.46:0/2392839213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e8107df0 0x7f70e8108200 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.672+0000 7f70eecb4700 1 -- 172.21.15.46:0/2392839213 >> 172.21.15.46:0/2392839213 conn(0x7f70e80fedc0 msgr2=0x7f70e81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.672+0000 7f70eecb4700 1 -- 172.21.15.46:0/2392839213 shutdown_connections 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.672+0000 7f70eecb4700 1 -- 172.21.15.46:0/2392839213 wait complete. 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.672+0000 7f70eecb4700 1 Processor -- start 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70eecb4700 1 -- start start 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70eecb4700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70e81034c0 0x7f70e8199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70eecb4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e8107df0 0x7f70e819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70eecb4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70e8108740 0x7f70e819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70eecb4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8113350 con 0x7f70e8107df0 2024-05-09T00:24:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70eecb4700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81131d0 con 0x7f70e81034c0 2024-05-09T00:24:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70eecb4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81134d0 con 0x7f70e8108740 2024-05-09T00:24:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e8107df0 0x7f70e819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e8107df0 0x7f70e819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38318/0 (socket says 172.21.15.46:38318) 2024-05-09T00:24:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70e7fff700 1 -- 172.21.15.46:0/2599609463 learned_addr learned my addr 172.21.15.46:0/2599609463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:24:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70ed251700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70e8108740 0x7f70e819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70eca50700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70e81034c0 0x7f70e8199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70e7fff700 1 -- 172.21.15.46:0/2599609463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70e8108740 msgr2=0x7f70e819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70e7fff700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70e8108740 0x7f70e819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70e7fff700 1 -- 172.21.15.46:0/2599609463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70e81034c0 msgr2=0x7f70e8199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70e7fff700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70e81034c0 0x7f70e8199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70e7fff700 1 -- 172.21.15.46:0/2599609463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e819ec10 con 0x7f70e8107df0 2024-05-09T00:24:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.673+0000 7f70eca50700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70e81034c0 0x7f70e8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:24:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.674+0000 7f70e7fff700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e8107df0 0x7f70e819a2e0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f70d80095c0 tx=0x7f70d8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.674+0000 7f70e5ffb700 1 -- 172.21.15.46:0/2599609463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8021020 con 0x7f70e8107df0 2024-05-09T00:24:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.674+0000 7f70eecb4700 1 -- 172.21.15.46:0/2599609463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e819eea0 con 0x7f70e8107df0 2024-05-09T00:24:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.674+0000 7f70e5ffb700 1 -- 172.21.15.46:0/2599609463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70d8004c50 con 0x7f70e8107df0 2024-05-09T00:24:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.674+0000 7f70e5ffb700 1 -- 172.21.15.46:0/2599609463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d80052b0 con 0x7f70e8107df0 2024-05-09T00:24:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.674+0000 7f70eecb4700 1 -- 172.21.15.46:0/2599609463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e81a6820 con 0x7f70e8107df0 2024-05-09T00:24:37.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.675+0000 7f70eecb4700 1 -- 172.21.15.46:0/2599609463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e8104970 con 0x7f70e8107df0 2024-05-09T00:24:37.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.676+0000 7f70e5ffb700 1 -- 172.21.15.46:0/2599609463 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f70d8024020 con 0x7f70e8107df0 2024-05-09T00:24:37.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.677+0000 7f70e5ffb700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f70d40b2880 0x7f70d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:37.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.677+0000 7f70eca50700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f70d40b2880 0x7f70d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:37.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.677+0000 7f70e5ffb700 1 -- 172.21.15.46:0/2599609463 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f70d800f020 con 0x7f70e8107df0 2024-05-09T00:24:37.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.678+0000 7f70eca50700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f70d40b2880 0x7f70d40b4d90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f70d0005950 tx=0x7f70d00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.681+0000 7f70e5ffb700 1 -- 172.21.15.46:0/2599609463 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70d80aba00 con 0x7f70e8107df0 2024-05-09T00:24:37.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.947+0000 7f70eecb4700 1 -- 172.21.15.46:0/2599609463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f70e819b0a0 con 0x7f70e8107df0 2024-05-09T00:24:37.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.949+0000 7f70e5ffb700 1 -- 172.21.15.46:0/2599609463 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f70d80ab300 con 0x7f70e8107df0 2024-05-09T00:24:37.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.953+0000 7f70eecb4700 1 -- 172.21.15.46:0/2599609463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f70d40b2880 msgr2=0x7f70d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:37.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.953+0000 7f70eecb4700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f70d40b2880 0x7f70d40b4d90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f70d0005950 tx=0x7f70d00058e0 comp rx=0 tx=0).stop 2024-05-09T00:24:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.953+0000 7f70eecb4700 1 -- 172.21.15.46:0/2599609463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e8107df0 msgr2=0x7f70e819a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.953+0000 7f70eecb4700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e8107df0 0x7f70e819a2e0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f70d80095c0 tx=0x7f70d8007a20 comp rx=0 tx=0).stop 2024-05-09T00:24:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.953+0000 7f70eecb4700 1 -- 172.21.15.46:0/2599609463 shutdown_connections 2024-05-09T00:24:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.953+0000 7f70eecb4700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f70d40b2880 0x7f70d40b4d90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.953+0000 7f70eecb4700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70e8108740 0x7f70e819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.953+0000 7f70eecb4700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70e81034c0 0x7f70e8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:37.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.953+0000 7f70eecb4700 1 --2- 172.21.15.46:0/2599609463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70e8107df0 0x7f70e819a2e0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:37.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.953+0000 7f70eecb4700 1 -- 172.21.15.46:0/2599609463 >> 172.21.15.46:0/2599609463 conn(0x7f70e80fedc0 msgr2=0x7f70e81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:37.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.953+0000 7f70eecb4700 1 -- 172.21.15.46:0/2599609463 shutdown_connections 2024-05-09T00:24:37.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:37.953+0000 7f70eecb4700 1 -- 172.21.15.46:0/2599609463 wait complete. 2024-05-09T00:24:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T00:24:38.313 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:37 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a[30147]: 2024-05-09T00:24:37.948+0000 7f4504712700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T00:24:38.356 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:24:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i[65278]: ignoring --setuser ceph since I am not root 2024-05-09T00:24:38.356 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:24:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i[65278]: ignoring --setgroup ceph since I am not root 2024-05-09T00:24:38.357 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:24:38 smithi169 ceph-mds[65282]: -- 172.21.15.169:0/961559732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559aaa8e5a40 con 0x559aaa92d180 2024-05-09T00:24:38.357 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:24:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i[65278]: starting mds.i at 2024-05-09T00:24:38.357 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:24:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f[64658]: ignoring --setuser ceph since I am not root 2024-05-09T00:24:38.357 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:24:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f[64658]: ignoring --setgroup ceph since I am not root 2024-05-09T00:24:38.357 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:24:38 smithi169 ceph-mds[64662]: -- 172.21.15.169:0/3245925459 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5585ab03da40 con 0x5585ab085180 2024-05-09T00:24:38.358 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:24:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f[64658]: starting mds.f at 2024-05-09T00:24:38.358 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:24:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c[64970]: ignoring --setuser ceph since I am not root 2024-05-09T00:24:38.358 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:24:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c[64970]: ignoring --setgroup ceph since I am not root 2024-05-09T00:24:38.358 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:24:38 smithi169 ceph-mds[64974]: -- 172.21.15.169:0/4199149846 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c88503a40 con 0x556c8854ba80 2024-05-09T00:24:38.358 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:24:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c[64970]: starting mds.c at 2024-05-09T00:24:38.443 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:24:38 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-b[64936]: ignoring --setuser ceph since I am not root 2024-05-09T00:24:38.444 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:24:38 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-b[64936]: ignoring --setgroup ceph since I am not root 2024-05-09T00:24:38.444 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:24:38 smithi142 ceph-mds[64940]: -- 172.21.15.142:0/4154544791 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55736febba40 con 0x55736ff03a80 2024-05-09T00:24:38.444 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:24:38 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-b[64936]: starting mds.b at 2024-05-09T00:24:38.444 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:24:38 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-e[64303]: ignoring --setuser ceph since I am not root 2024-05-09T00:24:38.444 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:24:38 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-e[64303]: ignoring --setgroup ceph since I am not root 2024-05-09T00:24:38.444 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:24:38 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-e[64303]: starting mds.e at 2024-05-09T00:24:38.445 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:24:38 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-h[64617]: ignoring --setuser ceph since I am not root 2024-05-09T00:24:38.445 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:24:38 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-h[64617]: ignoring --setgroup ceph since I am not root 2024-05-09T00:24:38.445 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:24:38 smithi142 ceph-mds[64621]: -- 172.21.15.142:0/1618337997 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe9aa13a40 con 0x55fe9aa5ba80 2024-05-09T00:24:38.445 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:24:38 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-h[64617]: starting mds.h at 2024-05-09T00:24:38.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T00:24:38.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:38 smithi169 ceph-mon[34777]: pgmap v774: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T00:24:38.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:38 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2599609463' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T00:24:38.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:38 smithi169 ceph-mon[34777]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T00:24:38.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:38 smithi169 ceph-mon[34777]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T00:24:38.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:38 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2599609463' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T00:24:38.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:38 smithi169 ceph-mon[34777]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T00:24:38.857 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:38 smithi169 ceph-mon[34777]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T00:24:38.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:38 smithi142 ceph-mon[34533]: pgmap v774: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T00:24:38.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:38 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2599609463' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T00:24:38.944 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:38 smithi142 ceph-mon[34533]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T00:24:38.944 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:38 smithi142 ceph-mon[34533]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T00:24:38.944 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:38 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2599609463' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T00:24:38.944 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:38 smithi142 ceph-mon[34533]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T00:24:38.944 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:38 smithi142 ceph-mon[34533]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T00:24:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:24:39.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:38 smithi046 ceph-mon[30151]: pgmap v774: 97 pgs: 97 active+clean; 7.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T00:24:39.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:38 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2599609463' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T00:24:39.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:38 smithi046 ceph-mon[30151]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T00:24:39.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:38 smithi046 ceph-mon[30151]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T00:24:39.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:38 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2599609463' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T00:24:39.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:38 smithi046 ceph-mon[30151]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T00:24:39.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:38 smithi046 ceph-mon[30151]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T00:24:40.273 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.142:6834/3683292136,v1:172.21.15.142:6837/3683292136] up:boot 2024-05-09T00:24:40.273 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.142:6835/2335595701,v1:172.21.15.142:6838/2335595701] up:boot 2024-05-09T00:24:40.273 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.142:6836/2741944261,v1:172.21.15.142:6839/2741944261] up:boot 2024-05-09T00:24:40.273 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.169:6835/2759002423,v1:172.21.15.169:6838/2759002423] up:boot 2024-05-09T00:24:40.273 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.169:6834/2151650092,v1:172.21.15.169:6837/2151650092] up:boot 2024-05-09T00:24:40.274 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: mds.? [v2:172.21.15.169:6836/1722468451,v1:172.21.15.169:6839/1722468451] up:boot 2024-05-09T00:24:40.274 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-09T00:24:40.274 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T00:24:40.274 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T00:24:40.274 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:24:40.274 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T00:24:40.274 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T00:24:40.274 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T00:24:40.274 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:39 smithi046 ceph-mon[30151]: pgmap v776: 97 pgs: 97 active+clean; 7.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T00:24:40.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.142:6834/3683292136,v1:172.21.15.142:6837/3683292136] up:boot 2024-05-09T00:24:40.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.142:6835/2335595701,v1:172.21.15.142:6838/2335595701] up:boot 2024-05-09T00:24:40.356 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.142:6836/2741944261,v1:172.21.15.142:6839/2741944261] up:boot 2024-05-09T00:24:40.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.169:6835/2759002423,v1:172.21.15.169:6838/2759002423] up:boot 2024-05-09T00:24:40.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.169:6834/2151650092,v1:172.21.15.169:6837/2151650092] up:boot 2024-05-09T00:24:40.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: mds.? [v2:172.21.15.169:6836/1722468451,v1:172.21.15.169:6839/1722468451] up:boot 2024-05-09T00:24:40.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-09T00:24:40.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T00:24:40.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T00:24:40.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:24:40.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T00:24:40.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T00:24:40.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T00:24:40.357 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:39 smithi169 ceph-mon[34777]: pgmap v776: 97 pgs: 97 active+clean; 7.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T00:24:40.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.142:6834/3683292136,v1:172.21.15.142:6837/3683292136] up:boot 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.142:6835/2335595701,v1:172.21.15.142:6838/2335595701] up:boot 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.142:6836/2741944261,v1:172.21.15.142:6839/2741944261] up:boot 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.169:6835/2759002423,v1:172.21.15.169:6838/2759002423] up:boot 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.169:6834/2151650092,v1:172.21.15.169:6837/2151650092] up:boot 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: mds.? [v2:172.21.15.169:6836/1722468451,v1:172.21.15.169:6839/1722468451] up:boot 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: from='mgr.14152 172.21.15.46:0/4112106728' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T00:24:40.444 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:39 smithi142 ceph-mon[34533]: pgmap v776: 97 pgs: 97 active+clean; 7.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T00:24:40.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.880+0000 7f0f6c95f700 1 -- 172.21.15.46:0/397693296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f68108740 msgr2=0x7f0f681100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:40.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.880+0000 7f0f6c95f700 1 --2- 172.21.15.46:0/397693296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f68108740 0x7f0f681100a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000b0d0 tx=0x7f0f6000b3e0 comp rx=0 tx=0).stop 2024-05-09T00:24:40.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.880+0000 7f0f6c95f700 1 -- 172.21.15.46:0/397693296 shutdown_connections 2024-05-09T00:24:40.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.880+0000 7f0f6c95f700 1 --2- 172.21.15.46:0/397693296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f68108740 0x7f0f681100a0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:40.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.880+0000 7f0f6c95f700 1 --2- 172.21.15.46:0/397693296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f681034c0 0x7f0f681038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:40.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.880+0000 7f0f6c95f700 1 --2- 172.21.15.46:0/397693296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f68107df0 0x7f0f68108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:40.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.880+0000 7f0f6c95f700 1 -- 172.21.15.46:0/397693296 >> 172.21.15.46:0/397693296 conn(0x7f0f680fedf0 msgr2=0x7f0f68101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:40.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.880+0000 7f0f6c95f700 1 -- 172.21.15.46:0/397693296 shutdown_connections 2024-05-09T00:24:40.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.880+0000 7f0f6c95f700 1 -- 172.21.15.46:0/397693296 wait complete. 2024-05-09T00:24:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.881+0000 7f0f6c95f700 1 Processor -- start 2024-05-09T00:24:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.881+0000 7f0f6c95f700 1 -- start start 2024-05-09T00:24:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f6c95f700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f681034c0 0x7f0f6819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f6c95f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f68107df0 0x7f0f6819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f6c95f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f68108740 0x7f0f68199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f6c95f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f68113370 con 0x7f0f68107df0 2024-05-09T00:24:40.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f6c95f700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f681131f0 con 0x7f0f681034c0 2024-05-09T00:24:40.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f6c95f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f681134f0 con 0x7f0f68108740 2024-05-09T00:24:40.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f68107df0 0x7f0f6819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:40.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f68107df0 0x7f0f6819fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38336/0 (socket says 172.21.15.46:38336) 2024-05-09T00:24:40.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f65d9b700 1 -- 172.21.15.46:0/2798771195 learned_addr learned my addr 172.21.15.46:0/2798771195 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:24:40.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f6659c700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f681034c0 0x7f0f6819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:40.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f66d9d700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f68108740 0x7f0f68199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:40.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f65d9b700 1 -- 172.21.15.46:0/2798771195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f68108740 msgr2=0x7f0f68199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:40.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f65d9b700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f68108740 0x7f0f68199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:40.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f65d9b700 1 -- 172.21.15.46:0/2798771195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f681034c0 msgr2=0x7f0f6819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:40.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f65d9b700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f681034c0 0x7f0f6819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:40.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f65d9b700 1 -- 172.21.15.46:0/2798771195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6819a160 con 0x7f0f68107df0 2024-05-09T00:24:40.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f66d9d700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f68108740 0x7f0f68199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T00:24:40.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.882+0000 7f0f65d9b700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f68107df0 0x7f0f6819fdd0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00d980 tx=0x7f0f5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:40.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.883+0000 7f0f577fe700 1 -- 172.21.15.46:0/2798771195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c017070 con 0x7f0f68107df0 2024-05-09T00:24:40.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.883+0000 7f0f6c95f700 1 -- 172.21.15.46:0/2798771195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6819a450 con 0x7f0f68107df0 2024-05-09T00:24:40.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.883+0000 7f0f577fe700 1 -- 172.21.15.46:0/2798771195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f5c009de0 con 0x7f0f68107df0 2024-05-09T00:24:40.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.883+0000 7f0f577fe700 1 -- 172.21.15.46:0/2798771195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c00f3c0 con 0x7f0f68107df0 2024-05-09T00:24:40.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.883+0000 7f0f6c95f700 1 -- 172.21.15.46:0/2798771195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f681a6720 con 0x7f0f68107df0 2024-05-09T00:24:40.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.884+0000 7f0f6c95f700 1 -- 172.21.15.46:0/2798771195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f680512c0 con 0x7f0f68107df0 2024-05-09T00:24:40.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.885+0000 7f0f577fe700 1 -- 172.21.15.46:0/2798771195 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0f5c009930 con 0x7f0f68107df0 2024-05-09T00:24:40.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.886+0000 7f0f577fe700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0f500b2900 0x7f0f500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:40.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.886+0000 7f0f6659c700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0f500b2900 0x7f0f500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:40.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.886+0000 7f0f577fe700 1 -- 172.21.15.46:0/2798771195 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f0f5c0e20c0 con 0x7f0f68107df0 2024-05-09T00:24:40.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.887+0000 7f0f6659c700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0f500b2900 0x7f0f500b4e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0f58005950 tx=0x7f0f5800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:40.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:40.891+0000 7f0f577fe700 1 -- 172.21.15.46:0/2798771195 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f5c0a9cd0 con 0x7f0f68107df0 2024-05-09T00:24:41.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.156+0000 7f0f6c95f700 1 -- 172.21.15.46:0/2798771195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0f6802e5f0 con 0x7f0f68107df0 2024-05-09T00:24:41.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.208+0000 7f0f577fe700 1 -- 172.21.15.46:0/2798771195 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f0f5c0a95d0 con 0x7f0f68107df0 2024-05-09T00:24:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.211+0000 7f0f6c95f700 1 -- 172.21.15.46:0/2798771195 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0f500b2900 msgr2=0x7f0f500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.211+0000 7f0f6c95f700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0f500b2900 0x7f0f500b4e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0f58005950 tx=0x7f0f5800a300 comp rx=0 tx=0).stop 2024-05-09T00:24:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.211+0000 7f0f6c95f700 1 -- 172.21.15.46:0/2798771195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f68107df0 msgr2=0x7f0f6819fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.211+0000 7f0f6c95f700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f68107df0 0x7f0f6819fdd0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00d980 tx=0x7f0f5c00dc90 comp rx=0 tx=0).stop 2024-05-09T00:24:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.211+0000 7f0f6c95f700 1 -- 172.21.15.46:0/2798771195 shutdown_connections 2024-05-09T00:24:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.211+0000 7f0f6c95f700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f0f500b2900 0x7f0f500b4e10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.211+0000 7f0f6c95f700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f68108740 0x7f0f68199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.211+0000 7f0f6c95f700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f681034c0 0x7f0f6819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.211+0000 7f0f6c95f700 1 --2- 172.21.15.46:0/2798771195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f68107df0 0x7f0f6819fdd0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.211+0000 7f0f6c95f700 1 -- 172.21.15.46:0/2798771195 >> 172.21.15.46:0/2798771195 conn(0x7f0f680fedf0 msgr2=0x7f0f68107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.211+0000 7f0f6c95f700 1 -- 172.21.15.46:0/2798771195 shutdown_connections 2024-05-09T00:24:41.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:41.211+0000 7f0f6c95f700 1 -- 172.21.15.46:0/2798771195 wait complete. 2024-05-09T00:24:41.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:41 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2798771195' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:24:41.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:41 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2798771195' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:24:41.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:41 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2798771195' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T00:24:41.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T00:24:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:24:42.471 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:42 smithi046 ceph-mon[30151]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T00:24:42.471 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:42 smithi046 ceph-mon[30151]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T00:24:42.471 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:42 smithi046 ceph-mon[30151]: Cluster is now healthy 2024-05-09T00:24:42.471 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:42 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/2798771195' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T00:24:42.471 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:42 smithi046 ceph-mon[30151]: fsmap 12 up:standby 2024-05-09T00:24:42.471 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:42 smithi046 ceph-mon[30151]: pgmap v777: 97 pgs: 97 active+clean; 7.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T00:24:42.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:42 smithi169 ceph-mon[34777]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T00:24:42.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:42 smithi169 ceph-mon[34777]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T00:24:42.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:42 smithi169 ceph-mon[34777]: Cluster is now healthy 2024-05-09T00:24:42.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:42 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/2798771195' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T00:24:42.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:42 smithi169 ceph-mon[34777]: fsmap 12 up:standby 2024-05-09T00:24:42.606 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:42 smithi169 ceph-mon[34777]: pgmap v777: 97 pgs: 97 active+clean; 7.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T00:24:42.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:42 smithi142 ceph-mon[34533]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T00:24:42.694 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:42 smithi142 ceph-mon[34533]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T00:24:42.694 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:42 smithi142 ceph-mon[34533]: Cluster is now healthy 2024-05-09T00:24:42.694 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:42 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/2798771195' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T00:24:42.694 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:42 smithi142 ceph-mon[34533]: fsmap 12 up:standby 2024-05-09T00:24:42.694 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:42 smithi142 ceph-mon[34533]: pgmap v777: 97 pgs: 97 active+clean; 7.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T00:24:44.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.087+0000 7f949a8e2700 1 -- 172.21.15.46:0/1454563857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94941045d0 msgr2=0x7f94941049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:44.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.087+0000 7f949a8e2700 1 --2- 172.21.15.46:0/1454563857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94941045d0 0x7f94941049e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f94840099c0 tx=0x7f9484009cd0 comp rx=0 tx=0).stop 2024-05-09T00:24:44.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.087+0000 7f949a8e2700 1 -- 172.21.15.46:0/1454563857 shutdown_connections 2024-05-09T00:24:44.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.087+0000 7f949a8e2700 1 --2- 172.21.15.46:0/1454563857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9494109850 0x7f94941111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:44.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.087+0000 7f949a8e2700 1 --2- 172.21.15.46:0/1454563857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94941045d0 0x7f94941049e0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:44.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.087+0000 7f949a8e2700 1 --2- 172.21.15.46:0/1454563857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9494108f00 0x7f9494109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:44.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.087+0000 7f949a8e2700 1 -- 172.21.15.46:0/1454563857 >> 172.21.15.46:0/1454563857 conn(0x7f94940ffea0 msgr2=0x7f94941022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:44.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.087+0000 7f949a8e2700 1 -- 172.21.15.46:0/1454563857 shutdown_connections 2024-05-09T00:24:44.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.087+0000 7f949a8e2700 1 -- 172.21.15.46:0/1454563857 wait complete. 2024-05-09T00:24:44.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.088+0000 7f949a8e2700 1 Processor -- start 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.088+0000 7f949a8e2700 1 -- start start 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.088+0000 7f949a8e2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94941045d0 0x7f9494076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.088+0000 7f949a8e2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9494108f00 0x7f9494077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.088+0000 7f949a8e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9494109850 0x7f9494077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.088+0000 7f949a8e2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9494114500 con 0x7f9494109850 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f949a8e2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9494114380 con 0x7f94941045d0 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f949a8e2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9494114680 con 0x7f9494108f00 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9498e7f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9494109850 0x7f9494077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9498e7f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9494109850 0x7f9494077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38352/0 (socket says 172.21.15.46:38352) 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9498e7f700 1 -- 172.21.15.46:0/278657452 learned_addr learned my addr 172.21.15.46:0/278657452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9493fff700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94941045d0 0x7f9494076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f948bfff700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9494108f00 0x7f9494077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9498e7f700 1 -- 172.21.15.46:0/278657452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9494108f00 msgr2=0x7f9494077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9498e7f700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9494108f00 0x7f9494077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9498e7f700 1 -- 172.21.15.46:0/278657452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94941045d0 msgr2=0x7f9494076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:44.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9498e7f700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94941045d0 0x7f9494076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:44.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9498e7f700 1 -- 172.21.15.46:0/278657452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94941a91d0 con 0x7f9494109850 2024-05-09T00:24:44.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f948bfff700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9494108f00 0x7f9494077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T00:24:44.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9498e7f700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9494109850 0x7f9494077650 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f948c00b970 tx=0x7f948c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:44.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9491ffb700 1 -- 172.21.15.46:0/278657452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c019070 con 0x7f9494109850 2024-05-09T00:24:44.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9491ffb700 1 -- 172.21.15.46:0/278657452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f948c00cde0 con 0x7f9494109850 2024-05-09T00:24:44.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f949a8e2700 1 -- 172.21.15.46:0/278657452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94941a94c0 con 0x7f9494109850 2024-05-09T00:24:44.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.089+0000 7f9491ffb700 1 -- 172.21.15.46:0/278657452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c007370 con 0x7f9494109850 2024-05-09T00:24:44.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.090+0000 7f949a8e2700 1 -- 172.21.15.46:0/278657452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94941a99b0 con 0x7f9494109850 2024-05-09T00:24:44.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.090+0000 7f949a8e2700 1 -- 172.21.15.46:0/278657452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94940512c0 con 0x7f9494109850 2024-05-09T00:24:44.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.092+0000 7f9491ffb700 1 -- 172.21.15.46:0/278657452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f948c01c070 con 0x7f9494109850 2024-05-09T00:24:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.093+0000 7f9491ffb700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94800b2880 0x7f94800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.094+0000 7f9493fff700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94800b2880 0x7f94800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.094+0000 7f9491ffb700 1 -- 172.21.15.46:0/278657452 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8498+0+0 (secure 0 0 0) 0x7f948c0e1fe0 con 0x7f9494109850 2024-05-09T00:24:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.094+0000 7f9493fff700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94800b2880 0x7f94800b4d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9484009990 tx=0x7f94840036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.097+0000 7f9491ffb700 1 -- 172.21.15.46:0/278657452 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f948c023070 con 0x7f9494109850 2024-05-09T00:24:44.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.311+0000 7f949a8e2700 1 -- 172.21.15.46:0/278657452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f949402e5f0 con 0x7f9494109850 2024-05-09T00:24:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.630+0000 7f9491ffb700 1 -- 172.21.15.46:0/278657452 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) ==== 181+0+0 (secure 0 0 0) 0x7f948c0a9b90 con 0x7f9494109850 2024-05-09T00:24:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.633+0000 7f949a8e2700 1 -- 172.21.15.46:0/278657452 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94800b2880 msgr2=0x7f94800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.633+0000 7f949a8e2700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94800b2880 0x7f94800b4d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9484009990 tx=0x7f94840036e0 comp rx=0 tx=0).stop 2024-05-09T00:24:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.633+0000 7f949a8e2700 1 -- 172.21.15.46:0/278657452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9494109850 msgr2=0x7f9494077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.633+0000 7f949a8e2700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9494109850 0x7f9494077650 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f948c00b970 tx=0x7f948c00bc80 comp rx=0 tx=0).stop 2024-05-09T00:24:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.633+0000 7f949a8e2700 1 -- 172.21.15.46:0/278657452 shutdown_connections 2024-05-09T00:24:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.633+0000 7f949a8e2700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f94800b2880 0x7f94800b4d90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.633+0000 7f949a8e2700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9494109850 0x7f9494077650 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.633+0000 7f949a8e2700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94941045d0 0x7f9494076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.633+0000 7f949a8e2700 1 --2- 172.21.15.46:0/278657452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9494108f00 0x7f9494077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.633+0000 7f949a8e2700 1 -- 172.21.15.46:0/278657452 >> 172.21.15.46:0/278657452 conn(0x7f94940ffea0 msgr2=0x7f949410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.634+0000 7f949a8e2700 1 -- 172.21.15.46:0/278657452 shutdown_connections 2024-05-09T00:24:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:44.634+0000 7f949a8e2700 1 -- 172.21.15.46:0/278657452 wait complete. 2024-05-09T00:24:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-05-09T00:24:44.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:44 smithi142 ceph-mon[34533]: pgmap v778: 97 pgs: 97 active+clean; 7.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4 op/s 2024-05-09T00:24:44.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:44 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/278657452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T00:24:44.957 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:44 smithi046 ceph-mon[30151]: pgmap v778: 97 pgs: 97 active+clean; 7.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4 op/s 2024-05-09T00:24:44.957 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:44 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/278657452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T00:24:45.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:44 smithi169 ceph-mon[34777]: pgmap v778: 97 pgs: 97 active+clean; 7.5 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4 op/s 2024-05-09T00:24:45.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:44 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/278657452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T00:24:45.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid eec67860-0d96-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T00:24:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T00:24:45.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:45 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/278657452' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T00:24:45.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:45 smithi046 ceph-mon[30151]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T00:24:45.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:45 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/278657452' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T00:24:45.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:45 smithi142 ceph-mon[34533]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T00:24:46.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:45 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/278657452' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T00:24:46.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:45 smithi169 ceph-mon[34777]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T00:24:46.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:46 smithi142 ceph-mon[34533]: pgmap v780: 65 pgs: 65 active+clean; 577 KiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:24:46.945 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:46 smithi046 ceph-mon[30151]: pgmap v780: 65 pgs: 65 active+clean; 577 KiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:24:47.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:46 smithi169 ceph-mon[34777]: pgmap v780: 65 pgs: 65 active+clean; 577 KiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:24:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.554+0000 7f68843f7700 1 -- 172.21.15.46:0/2437224330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c1045d0 msgr2=0x7f687c1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.554+0000 7f68843f7700 1 --2- 172.21.15.46:0/2437224330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c1045d0 0x7f687c1049e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f68780099c0 tx=0x7f6878009cd0 comp rx=0 tx=0).stop 2024-05-09T00:24:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.554+0000 7f68843f7700 1 -- 172.21.15.46:0/2437224330 shutdown_connections 2024-05-09T00:24:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.554+0000 7f68843f7700 1 --2- 172.21.15.46:0/2437224330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687c109850 0x7f687c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.554+0000 7f68843f7700 1 --2- 172.21.15.46:0/2437224330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c1045d0 0x7f687c1049e0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.554+0000 7f68843f7700 1 --2- 172.21.15.46:0/2437224330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f687c108f00 0x7f687c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.554+0000 7f68843f7700 1 -- 172.21.15.46:0/2437224330 >> 172.21.15.46:0/2437224330 conn(0x7f687c0ffea0 msgr2=0x7f687c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.554+0000 7f68843f7700 1 -- 172.21.15.46:0/2437224330 shutdown_connections 2024-05-09T00:24:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.554+0000 7f68843f7700 1 -- 172.21.15.46:0/2437224330 wait complete. 2024-05-09T00:24:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f68843f7700 1 Processor -- start 2024-05-09T00:24:47.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f68843f7700 1 -- start start 2024-05-09T00:24:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f68843f7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f687c1045d0 0x7f687c076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f68843f7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c108f00 0x7f687c077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f68843f7700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687c109850 0x7f687c077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f68843f7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f687c1144f0 con 0x7f687c108f00 2024-05-09T00:24:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f68843f7700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f687c114370 con 0x7f687c109850 2024-05-09T00:24:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f68843f7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f687c114670 con 0x7f687c1045d0 2024-05-09T00:24:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f6881992700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c108f00 0x7f687c077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f6881992700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c108f00 0x7f687c077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47604/0 (socket says 172.21.15.46:47604) 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f6881992700 1 -- 172.21.15.46:0/1187020463 learned_addr learned my addr 172.21.15.46:0/1187020463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f6882994700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687c109850 0x7f687c077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.555+0000 7f6882193700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f687c1045d0 0x7f687c076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.556+0000 7f6881992700 1 -- 172.21.15.46:0/1187020463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f687c1045d0 msgr2=0x7f687c076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.556+0000 7f6881992700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f687c1045d0 0x7f687c076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.556+0000 7f6881992700 1 -- 172.21.15.46:0/1187020463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687c109850 msgr2=0x7f687c077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.556+0000 7f6881992700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687c109850 0x7f687c077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.556+0000 7f6881992700 1 -- 172.21.15.46:0/1187020463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c1a9240 con 0x7f687c108f00 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.556+0000 7f6881992700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c108f00 0x7f687c077110 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f686c00b500 tx=0x7f686c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.556+0000 7f68737fe700 1 -- 172.21.15.46:0/1187020463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c015020 con 0x7f687c108f00 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.556+0000 7f68737fe700 1 -- 172.21.15.46:0/1187020463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f686c011ce0 con 0x7f687c108f00 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.556+0000 7f68843f7700 1 -- 172.21.15.46:0/1187020463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687c1a9530 con 0x7f687c108f00 2024-05-09T00:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.556+0000 7f68737fe700 1 -- 172.21.15.46:0/1187020463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c010cb0 con 0x7f687c108f00 2024-05-09T00:24:47.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.556+0000 7f68843f7700 1 -- 172.21.15.46:0/1187020463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687c1a9a70 con 0x7f687c108f00 2024-05-09T00:24:47.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.557+0000 7f68843f7700 1 -- 172.21.15.46:0/1187020463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6860005330 con 0x7f687c108f00 2024-05-09T00:24:47.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.560+0000 7f68737fe700 1 -- 172.21.15.46:0/1187020463 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f686c018020 con 0x7f687c108f00 2024-05-09T00:24:47.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.561+0000 7f68737fe700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f68680b2880 0x7f68680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T00:24:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.561+0000 7f6882193700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f68680b2880 0x7f68680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T00:24:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.561+0000 7f68737fe700 1 -- 172.21.15.46:0/1187020463 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8002+0+0 (secure 0 0 0) 0x7f686c0e1c40 con 0x7f687c108f00 2024-05-09T00:24:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.561+0000 7f6882193700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f68680b2880 0x7f68680b4d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6878009990 tx=0x7f6878000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T00:24:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.564+0000 7f68737fe700 1 -- 172.21.15.46:0/1187020463 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f686c01f020 con 0x7f687c108f00 2024-05-09T00:24:47.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:47.780+0000 7f68843f7700 1 -- 172.21.15.46:0/1187020463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6860005620 con 0x7f687c108f00 2024-05-09T00:24:48.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.645+0000 7f68737fe700 1 -- 172.21.15.46:0/1187020463 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) ==== 169+0+0 (secure 0 0 0) 0x7f686c0a99f0 con 0x7f687c108f00 2024-05-09T00:24:48.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.650+0000 7f68843f7700 1 -- 172.21.15.46:0/1187020463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f68680b2880 msgr2=0x7f68680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:48.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.650+0000 7f68843f7700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f68680b2880 0x7f68680b4d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6878009990 tx=0x7f6878000bc0 comp rx=0 tx=0).stop 2024-05-09T00:24:48.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.650+0000 7f68843f7700 1 -- 172.21.15.46:0/1187020463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c108f00 msgr2=0x7f687c077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T00:24:48.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.650+0000 7f68843f7700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c108f00 0x7f687c077110 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f686c00b500 tx=0x7f686c00b810 comp rx=0 tx=0).stop 2024-05-09T00:24:48.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.650+0000 7f68843f7700 1 -- 172.21.15.46:0/1187020463 shutdown_connections 2024-05-09T00:24:48.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.650+0000 7f68843f7700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.46:6800/286117730,v1:172.21.15.46:6801/286117730] conn(0x7f68680b2880 0x7f68680b4d90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:48.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.650+0000 7f68843f7700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f687c109850 0x7f687c077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:48.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.650+0000 7f68843f7700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f687c1045d0 0x7f687c076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:48.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.650+0000 7f68843f7700 1 --2- 172.21.15.46:0/1187020463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f687c108f00 0x7f687c077110 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T00:24:48.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.650+0000 7f68843f7700 1 -- 172.21.15.46:0/1187020463 >> 172.21.15.46:0/1187020463 conn(0x7f687c0ffea0 msgr2=0x7f687c102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T00:24:48.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.650+0000 7f68843f7700 1 -- 172.21.15.46:0/1187020463 shutdown_connections 2024-05-09T00:24:48.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T00:24:48.650+0000 7f68843f7700 1 -- 172.21.15.46:0/1187020463 wait complete. 2024-05-09T00:24:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-05-09T00:24:48.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:48 smithi142 ceph-mon[34533]: pgmap v781: 65 pgs: 65 active+clean; 577 KiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:24:48.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:48 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1187020463' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T00:24:48.983 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:48 smithi046 ceph-mon[30151]: pgmap v781: 65 pgs: 65 active+clean; 577 KiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:24:48.984 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:48 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1187020463' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T00:24:49.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:48 smithi169 ceph-mon[34777]: pgmap v781: 65 pgs: 65 active+clean; 577 KiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T00:24:49.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:48 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1187020463' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T00:24:49.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:24:49.461 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:24:49.494 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:24:49.525 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-09T00:24:49.525 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T00:24:49.543 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T00:24:49.560 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T00:24:49.581 INFO:tasks.cephadm:Stopping all daemons... 2024-05-09T00:24:49.581 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-09T00:24:49.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.a 2024-05-09T00:24:49.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:49 smithi046 systemd[1]: Stopping Ceph mon.a for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:24:49.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:49 smithi169 ceph-mon[34777]: from='client.? 172.21.15.46:0/1187020463' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T00:24:49.856 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:49 smithi169 ceph-mon[34777]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T00:24:49.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:49 smithi046 ceph-mon[30151]: from='client.? 172.21.15.46:0/1187020463' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T00:24:49.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:49 smithi046 ceph-mon[30151]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T00:24:49.946 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:49 smithi142 ceph-mon[34533]: from='client.? 172.21.15.46:0/1187020463' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T00:24:49.947 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:49 smithi142 ceph-mon[34533]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T00:24:50.313 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:49 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a[30147]: 2024-05-09T00:24:49.906+0000 7f4509f1d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:24:50.313 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:49 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a[30147]: 2024-05-09T00:24:49.906+0000 7f4509f1d700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-09T00:24:50.313 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:50 smithi046 podman[116904]: 2024-05-09 00:24:50.080338367 +0000 UTC m=+0.288783682 container died 74e04927330a46af4264e25637bb122c894dd15339cd3ccfe10f0b968faa3272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T00:24:50.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:50 smithi046 podman[116904]: 2024-05-09 00:24:50.531848157 +0000 UTC m=+0.740293514 container cleanup 74e04927330a46af4264e25637bb122c894dd15339cd3ccfe10f0b968faa3272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-09T00:24:50.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 00:24:50 smithi046 bash[116904]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-a 2024-05-09T00:24:50.965 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.a.service' 2024-05-09T00:24:51.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:24:51.018 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-09T00:24:51.018 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-09T00:24:51.018 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.b 2024-05-09T00:24:51.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:51 smithi142 systemd[1]: Stopping Ceph mon.b for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:24:51.658 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:51 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-b[34528]: 2024-05-09T00:24:51.339+0000 7f80ba035700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:24:51.658 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:51 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-b[34528]: 2024-05-09T00:24:51.339+0000 7f80ba035700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-09T00:24:51.658 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:51 smithi142 podman[69704]: 2024-05-09 00:24:51.381629842 +0000 UTC m=+0.162751853 container died 4de2356ef3de9b2893e65068ec6cd5264615116fbcae8161e9b2e4a8769f1c2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T00:24:51.943 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:51 smithi142 podman[69704]: 2024-05-09 00:24:51.757514596 +0000 UTC m=+0.538636597 container cleanup 4de2356ef3de9b2893e65068ec6cd5264615116fbcae8161e9b2e4a8769f1c2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:24:51.944 INFO:journalctl@ceph.mon.b.smithi142.stdout:May 09 00:24:51 smithi142 bash[69704]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-b 2024-05-09T00:24:52.179 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.b.service' 2024-05-09T00:24:52.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:24:52.232 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-09T00:24:52.232 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-09T00:24:52.232 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.c 2024-05-09T00:24:52.532 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:52 smithi169 systemd[1]: Stopping Ceph mon.c for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:24:52.809 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:52 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-c[34771]: 2024-05-09T00:24:52.531+0000 7fb6a0a18700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:24:52.809 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:52 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-c[34771]: 2024-05-09T00:24:52.531+0000 7fb6a0a18700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-09T00:24:52.809 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:52 smithi169 podman[70778]: 2024-05-09 00:24:52.553203388 +0000 UTC m=+0.132215929 container died c6d470b5b04dc36c48618da9811d6204bfaa854d37316834916edce1cca7f1dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-c, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-09T00:24:53.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:52 smithi169 podman[70778]: 2024-05-09 00:24:52.949649606 +0000 UTC m=+0.528662149 container cleanup c6d470b5b04dc36c48618da9811d6204bfaa854d37316834916edce1cca7f1dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-c, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-09T00:24:53.106 INFO:journalctl@ceph.mon.c.smithi169.stdout:May 09 00:24:52 smithi169 bash[70778]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mon-c 2024-05-09T00:24:53.324 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mon.c.service' 2024-05-09T00:24:53.382 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:24:53.382 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-09T00:24:53.382 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-09T00:24:53.382 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.x 2024-05-09T00:24:53.704 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:24:53 smithi046 systemd[1]: Stopping Ceph mgr.x for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:24:54.021 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:24:53 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x[30382]: 2024-05-09T00:24:53.703+0000 7f17a3008700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:24:54.021 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:24:53 smithi046 podman[117025]: 2024-05-09 00:24:53.734543978 +0000 UTC m=+0.154820841 container died 0ab4817301559982aba657317959817b69f3f40ce0365aa815183f3309723797 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-09T00:24:54.313 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:24:54 smithi046 podman[117025]: 2024-05-09 00:24:54.22142103 +0000 UTC m=+0.641697893 container cleanup 0ab4817301559982aba657317959817b69f3f40ce0365aa815183f3309723797 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T00:24:54.313 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 00:24:54 smithi046 bash[117025]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-x 2024-05-09T00:24:54.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-09T00:24:54.695 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:24:54.695 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-09T00:24:54.695 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-09T00:24:54.695 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.y 2024-05-09T00:24:55.033 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:24:54 smithi142 systemd[1]: Stopping Ceph mgr.y for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:24:55.347 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:24:55 smithi142 podman[69826]: 2024-05-09 00:24:55.033642193 +0000 UTC m=+0.146627010 container died 9dcca95c5554f4eedae026a5a0f03679a8d99291bc78e6ed53844c8dd0dd8fc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS) 2024-05-09T00:24:55.689 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:24:55 smithi142 podman[69826]: 2024-05-09 00:24:55.446587245 +0000 UTC m=+0.559572059 container cleanup 9dcca95c5554f4eedae026a5a0f03679a8d99291bc78e6ed53844c8dd0dd8fc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T00:24:55.689 INFO:journalctl@ceph.mgr.y.smithi142.stdout:May 09 00:24:55 smithi142 bash[69826]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-y 2024-05-09T00:24:55.820 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-09T00:24:55.852 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:24:55.853 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-09T00:24:55.853 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-09T00:24:55.853 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.z 2024-05-09T00:24:56.183 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:24:55 smithi169 systemd[1]: Stopping Ceph mgr.z for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:24:56.473 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:24:56 smithi169 podman[70902]: 2024-05-09 00:24:56.183857245 +0000 UTC m=+0.136691469 container died d6dd060bda59e2c576716c94f3e66c403770da198f7a013592f0793a1153275f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T00:24:56.840 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:24:56 smithi169 podman[70902]: 2024-05-09 00:24:56.555316125 +0000 UTC m=+0.508150353 container cleanup d6dd060bda59e2c576716c94f3e66c403770da198f7a013592f0793a1153275f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:24:56.840 INFO:journalctl@ceph.mgr.z.smithi169.stdout:May 09 00:24:56 smithi169 bash[70902]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mgr-z 2024-05-09T00:24:56.981 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-09T00:24:57.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:24:57.014 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-09T00:24:57.014 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-09T00:24:57.014 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.0 2024-05-09T00:24:57.313 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:24:57 smithi046 systemd[1]: Stopping Ceph osd.0 for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:24:57.813 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:24:57 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0[42715]: 2024-05-09T00:24:57.334+0000 7fc3ca48e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:24:57.813 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:24:57 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0[42715]: 2024-05-09T00:24:57.334+0000 7fc3ca48e700 -1 osd.0 83 *** Got signal Terminated *** 2024-05-09T00:24:57.813 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:24:57 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0[42715]: 2024-05-09T00:24:57.334+0000 7fc3ca48e700 -1 osd.0 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:25:03.044 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:25:02 smithi046 podman[117146]: 2024-05-09 00:25:02.747936199 +0000 UTC m=+5.534788319 container died ce03115b2dc9f3b071f4ad4babec17b517690fad7dc92bd8d3613564dfe1fe6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True) 2024-05-09T00:25:03.313 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:25:03 smithi046 podman[117146]: 2024-05-09 00:25:03.243567406 +0000 UTC m=+6.030419528 container cleanup ce03115b2dc9f3b071f4ad4babec17b517690fad7dc92bd8d3613564dfe1fe6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-09T00:25:03.313 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:25:03 smithi046 bash[117146]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0 2024-05-09T00:25:03.813 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:25:03 smithi046 podman[117164]: 2024-05-09 00:25:03.503547924 +0000 UTC m=+0.753394082 container remove ce03115b2dc9f3b071f4ad4babec17b517690fad7dc92bd8d3613564dfe1fe6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:25:04.091 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:25:03 smithi046 podman[117258]: 2024-05-09 00:25:03.724339393 +0000 UTC m=+0.018461625 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:25:04.091 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:25:03 smithi046 podman[117258]: 2024-05-09 00:25:03.871228155 +0000 UTC m=+0.165350372 container create 1746c4b487ed201a707f17251704ad141e39e9afd719e8dac71c482a80a1b002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-09T00:25:04.377 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:25:04 smithi046 podman[117258]: 2024-05-09 00:25:04.090944563 +0000 UTC m=+0.385066788 container init 1746c4b487ed201a707f17251704ad141e39e9afd719e8dac71c482a80a1b002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:25:04.377 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:25:04 smithi046 podman[117258]: 2024-05-09 00:25:04.095630088 +0000 UTC m=+0.389752309 container start 1746c4b487ed201a707f17251704ad141e39e9afd719e8dac71c482a80a1b002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-09T00:25:04.377 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:25:04 smithi046 podman[117258]: 2024-05-09 00:25:04.190829752 +0000 UTC m=+0.484951990 container attach 1746c4b487ed201a707f17251704ad141e39e9afd719e8dac71c482a80a1b002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-09T00:25:04.710 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 00:25:04 smithi046 podman[117258]: 2024-05-09 00:25:04.410320304 +0000 UTC m=+0.704442521 container died 1746c4b487ed201a707f17251704ad141e39e9afd719e8dac71c482a80a1b002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True) 2024-05-09T00:25:05.003 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.0.service' 2024-05-09T00:25:05.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:25:05.043 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-09T00:25:05.044 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-09T00:25:05.044 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.1 2024-05-09T00:25:05.385 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:05 smithi142 systemd[1]: Stopping Ceph osd.1 for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:25:05.693 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:05 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1[39140]: 2024-05-09T00:25:05.384+0000 7f5a832e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:25:05.693 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:05 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1[39140]: 2024-05-09T00:25:05.384+0000 7f5a832e3700 -1 osd.1 83 *** Got signal Terminated *** 2024-05-09T00:25:05.693 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:05 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1[39140]: 2024-05-09T00:25:05.384+0000 7f5a832e3700 -1 osd.1 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:25:11.061 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:10 smithi142 podman[69961]: 2024-05-09 00:25:10.761729878 +0000 UTC m=+5.518739394 container died 4a41407b4d74a68f3a3dcdc869e7ba2112bca268c00e264fd723263ec86252fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-05-09T00:25:11.427 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:11 smithi142 podman[69961]: 2024-05-09 00:25:11.159904516 +0000 UTC m=+5.916914040 container cleanup 4a41407b4d74a68f3a3dcdc869e7ba2112bca268c00e264fd723263ec86252fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-09T00:25:11.427 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:11 smithi142 bash[69961]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1 2024-05-09T00:25:11.693 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:11 smithi142 podman[69980]: 2024-05-09 00:25:11.427971124 +0000 UTC m=+0.655027779 container remove 4a41407b4d74a68f3a3dcdc869e7ba2112bca268c00e264fd723263ec86252fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-09T00:25:12.082 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:11 smithi142 podman[70068]: 2024-05-09 00:25:11.695659038 +0000 UTC m=+0.018467958 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:25:12.083 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:11 smithi142 podman[70068]: 2024-05-09 00:25:11.829100685 +0000 UTC m=+0.151909592 container create 76a1f1682e7c95c8f7ed3921a8f024bdb4bc8eed030786d4f7903b6bb4e51a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T00:25:12.376 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:12 smithi142 podman[70068]: 2024-05-09 00:25:12.082049397 +0000 UTC m=+0.404858304 container init 76a1f1682e7c95c8f7ed3921a8f024bdb4bc8eed030786d4f7903b6bb4e51a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-05-09T00:25:12.376 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:12 smithi142 podman[70068]: 2024-05-09 00:25:12.085436666 +0000 UTC m=+0.408245573 container start 76a1f1682e7c95c8f7ed3921a8f024bdb4bc8eed030786d4f7903b6bb4e51a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T00:25:12.377 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:12 smithi142 podman[70068]: 2024-05-09 00:25:12.181974259 +0000 UTC m=+0.504783182 container attach 76a1f1682e7c95c8f7ed3921a8f024bdb4bc8eed030786d4f7903b6bb4e51a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T00:25:12.693 INFO:journalctl@ceph.osd.1.smithi142.stdout:May 09 00:25:12 smithi142 podman[70068]: 2024-05-09 00:25:12.37707916 +0000 UTC m=+0.699888071 container died 76a1f1682e7c95c8f7ed3921a8f024bdb4bc8eed030786d4f7903b6bb4e51a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0) 2024-05-09T00:25:13.107 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.1.service' 2024-05-09T00:25:13.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:25:13.147 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-09T00:25:13.147 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-09T00:25:13.147 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.2 2024-05-09T00:25:13.451 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:13 smithi169 systemd[1]: Stopping Ceph osd.2 for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:25:13.797 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:13 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2[39953]: 2024-05-09T00:25:13.451+0000 7fbbc6ea8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:25:13.797 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:13 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2[39953]: 2024-05-09T00:25:13.451+0000 7fbbc6ea8700 -1 osd.2 83 *** Got signal Terminated *** 2024-05-09T00:25:13.797 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:13 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2[39953]: 2024-05-09T00:25:13.451+0000 7fbbc6ea8700 -1 osd.2 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:25:19.235 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:18 smithi169 podman[71027]: 2024-05-09 00:25:18.977275498 +0000 UTC m=+5.635241386 container died 3697d9ad88bed25013e2721bad0061979a5e315f8970179655d8c43f7b132095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-05-09T00:25:19.606 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:19 smithi169 podman[71027]: 2024-05-09 00:25:19.326453643 +0000 UTC m=+5.984419521 container cleanup 3697d9ad88bed25013e2721bad0061979a5e315f8970179655d8c43f7b132095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T00:25:19.606 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:19 smithi169 bash[71027]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2 2024-05-09T00:25:19.941 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:19 smithi169 podman[71057]: 2024-05-09 00:25:19.644624887 +0000 UTC m=+0.669211837 container remove 3697d9ad88bed25013e2721bad0061979a5e315f8970179655d8c43f7b132095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:25:20.206 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:19 smithi169 podman[71151]: 2024-05-09 00:25:19.844269686 +0000 UTC m=+0.018551518 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:25:20.206 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:19 smithi169 podman[71151]: 2024-05-09 00:25:19.970532886 +0000 UTC m=+0.144814757 container create 293866889f22457064838e66d3c2042446d7bf9bf574ecca6df366b7e902d91d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-09T00:25:20.206 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:20 smithi169 podman[71151]: 2024-05-09 00:25:20.206671918 +0000 UTC m=+0.380953736 container init 293866889f22457064838e66d3c2042446d7bf9bf574ecca6df366b7e902d91d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-09T00:25:20.503 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:20 smithi169 podman[71151]: 2024-05-09 00:25:20.210137173 +0000 UTC m=+0.384418992 container start 293866889f22457064838e66d3c2042446d7bf9bf574ecca6df366b7e902d91d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-09T00:25:20.503 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:20 smithi169 podman[71151]: 2024-05-09 00:25:20.306787417 +0000 UTC m=+0.481069255 container attach 293866889f22457064838e66d3c2042446d7bf9bf574ecca6df366b7e902d91d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0) 2024-05-09T00:25:20.810 INFO:journalctl@ceph.osd.2.smithi169.stdout:May 09 00:25:20 smithi169 podman[71151]: 2024-05-09 00:25:20.503942312 +0000 UTC m=+0.678224131 container died 293866889f22457064838e66d3c2042446d7bf9bf574ecca6df366b7e902d91d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T00:25:21.137 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.2.service' 2024-05-09T00:25:21.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:25:21.181 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-09T00:25:21.181 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-09T00:25:21.181 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.3 2024-05-09T00:25:21.515 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:21 smithi046 systemd[1]: Stopping Ceph osd.3 for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:25:21.814 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:21 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3[49666]: 2024-05-09T00:25:21.514+0000 7f5924edd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:25:21.814 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:21 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3[49666]: 2024-05-09T00:25:21.514+0000 7f5924edd700 -1 osd.3 83 *** Got signal Terminated *** 2024-05-09T00:25:21.814 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:21 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3[49666]: 2024-05-09T00:25:21.514+0000 7f5924edd700 -1 osd.3 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:25:23.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:23 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:23.211+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:24.606 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:24 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:24.167+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:24.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:24 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:24.228+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:25.606 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:25.131+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:25.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:25 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:25.230+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:26.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:25 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:25.811+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:26.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:26 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:26.092+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:26.563 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:26 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3[49666]: 2024-05-09T00:25:26.127+0000 7f5921493700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.184691+0000 front 2024-05-09T00:24:57.184718+0000 (oldest deadline 2024-05-09T00:25:25.985650+0000) 2024-05-09T00:25:26.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:26 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:26.251+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:26.856 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:26 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:26.579+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:27.056 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:26 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:26.764+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:27.313 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:27 smithi046 podman[117381]: 2024-05-09 00:25:27.056562008 +0000 UTC m=+5.669927695 container died 6e5f45b29ff05ce62409401a28aadc23923d172e22e05e423dacbd34e1e7fa76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T00:25:27.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:27 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:27.065+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:27.674 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:27 smithi046 podman[117381]: 2024-05-09 00:25:27.414996943 +0000 UTC m=+6.028362628 container cleanup 6e5f45b29ff05ce62409401a28aadc23923d172e22e05e423dacbd34e1e7fa76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:25:27.674 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:27 smithi046 bash[117381]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3 2024-05-09T00:25:27.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:27 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:27.293+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:27.856 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:27 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:27.577+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:28.040 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:27 smithi046 podman[117398]: 2024-05-09 00:25:27.674559867 +0000 UTC m=+0.620476968 container remove 6e5f45b29ff05ce62409401a28aadc23923d172e22e05e423dacbd34e1e7fa76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-09T00:25:28.041 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:27 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:27.748+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:28.313 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:28 smithi046 podman[117488]: 2024-05-09 00:25:27.944568635 +0000 UTC m=+0.018853592 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:25:28.314 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:28 smithi046 podman[117488]: 2024-05-09 00:25:28.075794462 +0000 UTC m=+0.150079406 container create 0f64a44b002c65c2cefc924aed7de4ac3209e0d5e2b9e5bb3efc57edb8f027fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-05-09T00:25:28.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:28 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:28.103+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:28.650 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:28 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:28.436+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:28.651 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:28 smithi046 podman[117488]: 2024-05-09 00:25:28.336660457 +0000 UTC m=+0.410945412 container init 0f64a44b002c65c2cefc924aed7de4ac3209e0d5e2b9e5bb3efc57edb8f027fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-09T00:25:28.651 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:28 smithi046 podman[117488]: 2024-05-09 00:25:28.340955378 +0000 UTC m=+0.415240329 container start 0f64a44b002c65c2cefc924aed7de4ac3209e0d5e2b9e5bb3efc57edb8f027fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-09T00:25:28.651 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:28 smithi046 podman[117488]: 2024-05-09 00:25:28.419846878 +0000 UTC m=+0.494131838 container attach 0f64a44b002c65c2cefc924aed7de4ac3209e0d5e2b9e5bb3efc57edb8f027fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T00:25:28.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:28 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:28.258+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:28.856 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:28 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:28.604+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:29.014 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:28 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:28.758+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:29.015 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 00:25:28 smithi046 podman[117488]: 2024-05-09 00:25:28.650923258 +0000 UTC m=+0.725208205 container died 0f64a44b002c65c2cefc924aed7de4ac3209e0d5e2b9e5bb3efc57edb8f027fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T00:25:29.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.3.service' 2024-05-09T00:25:29.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:29 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:29.057+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:29.390 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:25:29.390 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-09T00:25:29.390 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-09T00:25:29.391 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.4 2024-05-09T00:25:29.694 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:29 smithi142 systemd[1]: Stopping Ceph osd.4 for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:25:29.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:29 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:29.240+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:29.787 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:29 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:29.460+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:30.036 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:29 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:29.612+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:30.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:29 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:29.787+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:30.193 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:29 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4[45669]: 2024-05-09T00:25:29.712+0000 7f0c64f0a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:25:30.193 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:29 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4[45669]: 2024-05-09T00:25:29.712+0000 7f0c64f0a700 -1 osd.4 83 *** Got signal Terminated *** 2024-05-09T00:25:30.193 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:29 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4[45669]: 2024-05-09T00:25:29.712+0000 7f0c64f0a700 -1 osd.4 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:25:30.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:30 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:30.036+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:30.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:30 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:30.288+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:30.772 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:30 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:30.493+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:31.028 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:30 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:30.626+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:31.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:30 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:30.772+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:31.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:31 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:31.027+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:31.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:31 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:31.292+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:31.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:31 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:31.292+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:31.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:31 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:31.728+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:31.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:31 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:31.479+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:31.856 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:31 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:31.578+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:31.856 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:31 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:31.578+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:32.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:32 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:32.054+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:32.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:32 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:32.294+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:32.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:32 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:32.294+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:32.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:32 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:32.701+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:32.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:32 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:32.701+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:32.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:32 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:32.470+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:32.915 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:32 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:32.609+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:32.915 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:32 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:32.609+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:33.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:33 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:33.068+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:33.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:32 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:32.914+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:33.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:33 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:33.295+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:33.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:33 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:33.295+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:33.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:33 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:33.709+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:33.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:33 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:33.709+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:33.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:33 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:33.499+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:33.888 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:33 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:33.627+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:33.888 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:33 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:33.627+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:34.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:34 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:34.092+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:34.357 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:33 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:33.888+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:34.595 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:34 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4[45669]: 2024-05-09T00:25:34.414+0000 7f0c614c0700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.365688+0000 front 2024-05-09T00:25:10.365724+0000 (oldest deadline 2024-05-09T00:25:33.865927+0000) 2024-05-09T00:25:34.595 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:34 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:34.308+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:34.595 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:34 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:34.308+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:34.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:34 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:34.721+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:34.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:34 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:34.721+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:34.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:34 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:34.536+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:34.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:34 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:34.536+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:34.905 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:34 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:34.611+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:34.905 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:34 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:34.611+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:35.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:34 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:34.905+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:35.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:35 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:35.044+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:35.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:35 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:35.044+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.731956+0000 front 2024-05-09T00:25:09.731777+0000 (oldest deadline 2024-05-09T00:25:35.031726+0000) 2024-05-09T00:25:35.507 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:35 smithi142 podman[70176]: 2024-05-09 00:25:35.196630642 +0000 UTC m=+5.610157334 container died b557784b3cbae45a48646519fe9167accee187e691a056d6fa0bbaebb0646289 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:25:35.507 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:35 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:35.295+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:35.507 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:35 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:35.295+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:35.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:35 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:35.741+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:35.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:35 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:35.741+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:35.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:35 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:35.541+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:35.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:35 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:35.541+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:35.865 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:35 smithi142 podman[70176]: 2024-05-09 00:25:35.605870167 +0000 UTC m=+6.019396858 container cleanup b557784b3cbae45a48646519fe9167accee187e691a056d6fa0bbaebb0646289 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-09T00:25:35.865 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:35 smithi142 bash[70176]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4 2024-05-09T00:25:35.865 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:35 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:35.751+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:35.872 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:35 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:35.618+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:35.872 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:35 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:35.618+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:36.187 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:35 smithi142 podman[70193]: 2024-05-09 00:25:35.865521026 +0000 UTC m=+0.665202148 container remove b557784b3cbae45a48646519fe9167accee187e691a056d6fa0bbaebb0646289 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-09T00:25:36.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:35 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:35.872+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:36.357 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:36 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:36.001+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:36.357 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:36 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:36.001+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.731956+0000 front 2024-05-09T00:25:09.731777+0000 (oldest deadline 2024-05-09T00:25:35.031726+0000) 2024-05-09T00:25:36.443 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:36 smithi142 podman[70290]: 2024-05-09 00:25:36.090743407 +0000 UTC m=+0.018419892 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:25:36.443 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:36 smithi142 podman[70290]: 2024-05-09 00:25:36.199975401 +0000 UTC m=+0.127651860 container create d7eb34cce27252c8748339dcd641be4467bb2971d4e87b1ecd737d9ca218d469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:25:36.444 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:36 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:36.246+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:36.444 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:36 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:36.246+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:36.716 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:36 smithi142 podman[70290]: 2024-05-09 00:25:36.46516973 +0000 UTC m=+0.392846189 container init d7eb34cce27252c8748339dcd641be4467bb2971d4e87b1ecd737d9ca218d469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:25:36.716 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:36 smithi142 podman[70290]: 2024-05-09 00:25:36.468404474 +0000 UTC m=+0.396080942 container start d7eb34cce27252c8748339dcd641be4467bb2971d4e87b1ecd737d9ca218d469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:25:36.716 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:36 smithi142 podman[70290]: 2024-05-09 00:25:36.561409531 +0000 UTC m=+0.489086009 container attach d7eb34cce27252c8748339dcd641be4467bb2971d4e87b1ecd737d9ca218d469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS) 2024-05-09T00:25:36.913 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:36 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:36.638+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:36.913 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:36 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:36.638+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:37.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:36 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:36.706+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:37.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:36 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:36.706+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:37.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:36 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:36.577+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:37.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:36 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:36.577+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:37.139 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:36 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:36.714+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:37.139 INFO:journalctl@ceph.osd.4.smithi142.stdout:May 09 00:25:36 smithi142 podman[70290]: 2024-05-09 00:25:36.77069223 +0000 UTC m=+0.698368702 container died d7eb34cce27252c8748339dcd641be4467bb2971d4e87b1ecd737d9ca218d469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-09T00:25:37.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:36 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:36.995+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:37.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:36 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:36.995+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.731956+0000 front 2024-05-09T00:25:09.731777+0000 (oldest deadline 2024-05-09T00:25:35.031726+0000) 2024-05-09T00:25:37.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:36 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:36.913+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:37.443 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:37 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:37.260+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:37.443 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:37 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:37.260+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:37.465 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.4.service' 2024-05-09T00:25:37.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:25:37.507 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-09T00:25:37.507 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-09T00:25:37.508 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.5 2024-05-09T00:25:37.821 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:37 smithi169 systemd[1]: Stopping Ceph osd.5 for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:25:37.821 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:37 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:37.591+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:37.821 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:37 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:37.591+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:38.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:37 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:37.688+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:38.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:37 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:37.688+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:38.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:37 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:37.596+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:38.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:37 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:37.596+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:38.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:37 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:37.881+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:38.107 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:37 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:37.820+0000 7fd83b99f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:25:38.107 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:37 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:37.820+0000 7fd83b99f700 -1 osd.5 83 *** Got signal Terminated *** 2024-05-09T00:25:38.107 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:37 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:37.820+0000 7fd83b99f700 -1 osd.5 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:25:38.107 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:38.008+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:38.107 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:38.008+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.731956+0000 front 2024-05-09T00:25:09.731777+0000 (oldest deadline 2024-05-09T00:25:35.031726+0000) 2024-05-09T00:25:38.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:37 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:37.702+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:38.698 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:38 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:38.278+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:38.698 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:38 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:38.278+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:38.856 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:38.602+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:38.856 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:38.602+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:39.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:38 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:38.675+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:39.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:38 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:38.675+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:39.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:38 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:38.592+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:39.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:38 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:38.592+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:39.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:38 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:38.711+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:39.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:38 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:38.876+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:39.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:39 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:39.024+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:39.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:39 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:39.024+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.731956+0000 front 2024-05-09T00:25:09.731777+0000 (oldest deadline 2024-05-09T00:25:35.031726+0000) 2024-05-09T00:25:39.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:39 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:39.321+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:39.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:39 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:39.321+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:40.046 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:39 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:39.894+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:40.047 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:39 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:39.650+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:40.047 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:39 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:39.650+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:40.047 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:39 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:39.650+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:40.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:39 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:39.667+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:40.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:39 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:39.667+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:40.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:39 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:39.564+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:40.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:39 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:39.564+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:40.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:39 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:39.754+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:40.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:40 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:40.046+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:40.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:40 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:40.046+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.731956+0000 front 2024-05-09T00:25:09.731777+0000 (oldest deadline 2024-05-09T00:25:35.031726+0000) 2024-05-09T00:25:40.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:40 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:40.349+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:40.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:40 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:40.349+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:41.040 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:40 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:40.863+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:41.040 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:40 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:40.863+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:41.041 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:40 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:40.670+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:41.041 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:40 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:40.670+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:41.041 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:40 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:40.670+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:41.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:40.692+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:41.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:40.692+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:41.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:40.692+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.897164+0000 front 2024-05-09T00:25:15.897201+0000 (oldest deadline 2024-05-09T00:25:39.997277+0000) 2024-05-09T00:25:41.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:40.613+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:41.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:40.613+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:41.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:40.613+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:41.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:40 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:40.775+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:41.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:40 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:40.775+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:41.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:41 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:41.040+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:41.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:41 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:41.040+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.731956+0000 front 2024-05-09T00:25:09.731777+0000 (oldest deadline 2024-05-09T00:25:35.031726+0000) 2024-05-09T00:25:41.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:41 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:41.040+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.332540+0000 front 2024-05-09T00:25:17.332575+0000 (oldest deadline 2024-05-09T00:25:40.832631+0000) 2024-05-09T00:25:41.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:41 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:41.380+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:41.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:41 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:41.380+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:41.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:41 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:41.621+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:41.911 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:41 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:41.621+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:41.911 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:41 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:41.621+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:42.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:41 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:41.693+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:42.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:41 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:41.693+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:42.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:41 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:41.693+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.897164+0000 front 2024-05-09T00:25:15.897201+0000 (oldest deadline 2024-05-09T00:25:39.997277+0000) 2024-05-09T00:25:42.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:41 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:41.599+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:42.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:41 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:41.599+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:42.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:41 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:41.599+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:42.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:41 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:41.771+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:42.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:41 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:41.771+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:42.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:42 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:42.011+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.830356+0000 front 2024-05-09T00:24:59.830341+0000 (oldest deadline 2024-05-09T00:25:23.930100+0000) 2024-05-09T00:25:42.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:42 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:42.011+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.731956+0000 front 2024-05-09T00:25:09.731777+0000 (oldest deadline 2024-05-09T00:25:35.031726+0000) 2024-05-09T00:25:42.356 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:42 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5[46941]: 2024-05-09T00:25:42.011+0000 7fd837f55700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.332540+0000 front 2024-05-09T00:25:17.332575+0000 (oldest deadline 2024-05-09T00:25:40.832631+0000) 2024-05-09T00:25:42.357 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:41 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:41.910+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:42.357 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:41 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:41.910+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:42.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:42 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:42.351+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:42.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:42 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:42.351+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:42.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:42 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:42.734+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:42.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:42 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:42.734+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:42.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:42 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:42.734+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.897164+0000 front 2024-05-09T00:25:15.897201+0000 (oldest deadline 2024-05-09T00:25:39.997277+0000) 2024-05-09T00:25:42.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:42 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:42.557+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:42.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:42 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:42.557+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:42.814 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:42 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:42.557+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:43.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:42 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:42.883+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:43.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:42 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:42.883+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:43.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:42 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:42.639+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:43.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:42 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:42.639+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:43.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:42 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:42.639+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:43.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:42 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:42.735+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:43.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:42 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:42.735+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:43.463 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:43 smithi169 podman[71258]: 2024-05-09 00:25:43.165535256 +0000 UTC m=+5.466962116 container died dd8522f6cdb09284dfeb11bc4dafd17b05dac6966aa8e2949115f2dc85099ebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-05-09T00:25:43.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:43 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:43.342+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:43.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:43 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:43.342+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:43.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:43 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:43.698+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:43.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:43 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:43.698+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:43.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:43 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:43.698+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.897164+0000 front 2024-05-09T00:25:15.897201+0000 (oldest deadline 2024-05-09T00:25:39.997277+0000) 2024-05-09T00:25:43.814 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:43 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:43.512+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:43.814 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:43 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:43.512+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:43.814 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:43 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:43.512+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:43.822 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:43 smithi169 podman[71258]: 2024-05-09 00:25:43.546581015 +0000 UTC m=+5.848007866 container cleanup dd8522f6cdb09284dfeb11bc4dafd17b05dac6966aa8e2949115f2dc85099ebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T00:25:43.823 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:43 smithi169 bash[71258]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5 2024-05-09T00:25:43.823 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:43 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:43.669+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:43.823 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:43 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:43.669+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:43.823 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:43 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:43.669+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:44.106 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:43 smithi169 podman[71276]: 2024-05-09 00:25:43.823076045 +0000 UTC m=+0.654962953 container remove dd8522f6cdb09284dfeb11bc4dafd17b05dac6966aa8e2949115f2dc85099ebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:25:44.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:43 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:43.901+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:44.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:43 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:43.901+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:44.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:43 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:43.721+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:44.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:43 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:43.721+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:44.418 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:44 smithi169 podman[71369]: 2024-05-09 00:25:44.057576751 +0000 UTC m=+0.018202348 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:25:44.418 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:44 smithi169 podman[71369]: 2024-05-09 00:25:44.182434584 +0000 UTC m=+0.143060181 container create f7e7915f93bb43f0fde08d55da9124011a2927620fc323280610280543cdad24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:25:44.418 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:44 smithi169 podman[71369]: 2024-05-09 00:25:44.418337148 +0000 UTC m=+0.378962745 container init f7e7915f93bb43f0fde08d55da9124011a2927620fc323280610280543cdad24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-09T00:25:44.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:44 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:44.377+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:44.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:44 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:44.377+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:44.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:44 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:44.377+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:44.717 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:44 smithi169 podman[71369]: 2024-05-09 00:25:44.421796447 +0000 UTC m=+0.382422046 container start f7e7915f93bb43f0fde08d55da9124011a2927620fc323280610280543cdad24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-09T00:25:44.717 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:44 smithi169 podman[71369]: 2024-05-09 00:25:44.518288343 +0000 UTC m=+0.478913944 container attach f7e7915f93bb43f0fde08d55da9124011a2927620fc323280610280543cdad24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-09T00:25:44.717 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:44 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:44.659+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:44.717 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:44 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:44.659+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:44.717 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:44 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:44.659+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:44.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:44 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:44.701+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:44.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:44 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:44.701+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:44.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:44 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:44.701+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.897164+0000 front 2024-05-09T00:25:15.897201+0000 (oldest deadline 2024-05-09T00:25:39.997277+0000) 2024-05-09T00:25:44.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:44 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:44.468+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:44.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:44 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:44.468+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:44.814 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:44 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:44.468+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:45.072 INFO:journalctl@ceph.osd.5.smithi169.stdout:May 09 00:25:44 smithi169 podman[71369]: 2024-05-09 00:25:44.717530245 +0000 UTC m=+0.678155856 container died f7e7915f93bb43f0fde08d55da9124011a2927620fc323280610280543cdad24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-09T00:25:45.072 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:44 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:44.948+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:45.072 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:44 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:44.948+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:45.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:44 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:44.731+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:45.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:44 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:44.731+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:45.394 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.5.service' 2024-05-09T00:25:45.435 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:25:45.435 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-09T00:25:45.436 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-09T00:25:45.436 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.6 2024-05-09T00:25:45.677 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:45 smithi046 systemd[1]: Stopping Ceph osd.6 for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:25:45.678 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:45 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:45.423+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:45.678 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:45 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:45.423+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:45.678 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:45 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:45.423+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:45.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:45 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:45.398+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:45.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:45 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:45.398+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:45.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:45 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:45.398+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:45.950 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:45 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:45.626+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:45.950 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:45 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:45.626+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:45.950 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:45 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:45.626+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:46.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:45 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:45.677+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:46.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:45 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:45.677+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:46.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:45 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:45.677+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.897164+0000 front 2024-05-09T00:25:15.897201+0000 (oldest deadline 2024-05-09T00:25:39.997277+0000) 2024-05-09T00:25:46.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:45 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:45.768+0000 7fb23e7a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:25:46.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:45 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:45.768+0000 7fb23e7a1700 -1 osd.6 83 *** Got signal Terminated *** 2024-05-09T00:25:46.063 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:45 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:45.768+0000 7fb23e7a1700 -1 osd.6 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:25:46.167 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:45 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:45.749+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:46.167 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:45 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:45.749+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:46.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:45 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:45.950+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:46.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:45 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:45.950+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:46.443 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:46 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:46.366+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:46.443 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:46 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:46.366+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:46.444 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:46 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:46.366+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:46.695 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:46.675+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:46.695 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:46.675+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:46.695 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:46.675+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.897164+0000 front 2024-05-09T00:25:15.897201+0000 (oldest deadline 2024-05-09T00:25:39.997277+0000) 2024-05-09T00:25:46.696 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:46.444+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:46.696 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:46.444+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:46.696 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:46.444+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:46.904 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:46 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:46.632+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:46.904 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:46 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:46.632+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:46.904 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:46 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:46.632+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:47.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:46 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:46.777+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:47.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:46 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:46.777+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:47.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:46 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:46.904+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:47.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:46 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:46.904+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:47.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:47 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:47.321+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:47.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:47 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:47.321+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:47.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:47 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:47.321+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:47.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:47 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:47.693+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:47.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:47 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:47.693+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:47.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:47 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:47.693+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.897164+0000 front 2024-05-09T00:25:15.897201+0000 (oldest deadline 2024-05-09T00:25:39.997277+0000) 2024-05-09T00:25:47.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:47 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:47.473+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:47.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:47 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:47.473+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:47.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:47 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:47.473+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:48.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:47 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:47.871+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:48.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:47 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:47.871+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:48.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:47 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:47.871+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:25:48.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:47 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:47.678+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:48.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:47 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:47.678+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:48.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:47 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:47.678+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:48.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:47 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:47.762+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:48.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:47 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:47.762+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:48.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:48 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:48.314+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:48.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:48 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:48.314+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:48.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:48 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:48.314+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:48.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:48 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:48.470+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:48.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:48 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:48.470+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:48.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:48 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:48.470+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:48.814 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:48 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:48.671+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:48.814 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:48 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:48.671+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:48.814 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:48 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:48.671+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.897164+0000 front 2024-05-09T00:25:15.897201+0000 (oldest deadline 2024-05-09T00:25:39.997277+0000) 2024-05-09T00:25:48.814 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:48 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:48.671+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.298198+0000 front 2024-05-09T00:25:22.298219+0000 (oldest deadline 2024-05-09T00:25:48.198238+0000) 2024-05-09T00:25:49.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:48 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:48.884+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:49.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:48 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:48.884+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:49.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:48 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:48.884+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:25:49.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:48 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:48.663+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:49.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:48 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:48.663+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:49.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:48 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:48.663+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:49.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:48 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:48.793+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:49.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:48 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:48.793+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:49.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:49 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:49.337+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:49.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:49 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:49.337+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:49.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:49 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:49.337+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:49.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:49 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:49.475+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:49.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:49 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:49.475+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:49.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:49 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:49.475+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:49.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:49 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:49.649+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:49.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:49 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:49.649+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:49.814 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:49 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:49.649+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.897164+0000 front 2024-05-09T00:25:15.897201+0000 (oldest deadline 2024-05-09T00:25:39.997277+0000) 2024-05-09T00:25:49.814 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:49 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:49.649+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.298198+0000 front 2024-05-09T00:25:22.298219+0000 (oldest deadline 2024-05-09T00:25:48.198238+0000) 2024-05-09T00:25:49.891 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:49 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:49.620+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:49.891 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:49 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:49.620+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:49.891 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:49 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:49.620+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:50.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:49 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:49.810+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:50.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:49 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:49.810+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:50.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:49 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:49.810+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.682431+0000 front 2024-05-09T00:25:23.682166+0000 (oldest deadline 2024-05-09T00:25:49.482527+0000) 2024-05-09T00:25:50.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:49 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:49.891+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:50.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:49 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:49.891+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:50.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:49 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:49.891+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:25:50.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:50 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:50.289+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:50.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:50 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:50.289+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:50.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:50 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:50.289+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:50.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:50 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:50.500+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:50.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:50 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:50.500+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:50.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:50 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:50.500+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:50.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:50 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:50.687+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.894836+0000 front 2024-05-09T00:25:01.894901+0000 (oldest deadline 2024-05-09T00:25:25.395158+0000) 2024-05-09T00:25:50.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:50 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:50.687+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:09.496436+0000 front 2024-05-09T00:25:09.496682+0000 (oldest deadline 2024-05-09T00:25:32.396401+0000) 2024-05-09T00:25:50.814 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:50 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:50.687+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.897164+0000 front 2024-05-09T00:25:15.897201+0000 (oldest deadline 2024-05-09T00:25:39.997277+0000) 2024-05-09T00:25:50.814 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:50 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6[56354]: 2024-05-09T00:25:50.687+0000 7fb236d4f700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.298198+0000 front 2024-05-09T00:25:22.298219+0000 (oldest deadline 2024-05-09T00:25:48.198238+0000) 2024-05-09T00:25:50.937 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:50 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:50.669+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:50.938 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:50 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:50.669+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:50.938 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:50 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:50.669+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:51.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:50 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:50.805+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:51.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:50 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:50.805+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:51.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:50 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:50.805+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.682431+0000 front 2024-05-09T00:25:23.682166+0000 (oldest deadline 2024-05-09T00:25:49.482527+0000) 2024-05-09T00:25:51.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:50 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:50.937+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:51.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:50 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:50.937+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:51.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:50 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:50.937+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:25:51.522 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:51 smithi046 podman[117592]: 2024-05-09 00:25:51.223880009 +0000 UTC m=+5.584835677 container died 11ad0e87abef8910978ad23fa0bd6a9f48eeaba25cd1af7b2047dd39ef60c439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T00:25:51.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:51 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:51.276+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:51.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:51 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:51.276+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:51.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:51 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:51.276+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:51.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:51 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:51.276+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:25:51.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:51 smithi046 podman[117592]: 2024-05-09 00:25:51.735179433 +0000 UTC m=+6.096135152 container cleanup 11ad0e87abef8910978ad23fa0bd6a9f48eeaba25cd1af7b2047dd39ef60c439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-09T00:25:51.813 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:51 smithi046 bash[117592]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6 2024-05-09T00:25:51.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:51 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:51.522+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:51.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:51 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:51.522+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:51.814 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:51 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:51.522+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:52.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:51 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:51.703+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:52.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:51 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:51.703+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:52.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:51 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:51.703+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:52.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:51 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:51.892+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:52.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:51 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:51.892+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:52.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:51 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:51.892+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:25:52.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:51 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:51.802+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:52.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:51 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:51.802+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:52.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:51 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:51.802+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.682431+0000 front 2024-05-09T00:25:23.682166+0000 (oldest deadline 2024-05-09T00:25:49.482527+0000) 2024-05-09T00:25:52.313 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:51 smithi046 podman[117623]: 2024-05-09 00:25:51.961638898 +0000 UTC m=+0.730552432 container remove 11ad0e87abef8910978ad23fa0bd6a9f48eeaba25cd1af7b2047dd39ef60c439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-09T00:25:52.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:52 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:52.304+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:52.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:52 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:52.304+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:52.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:52 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:52.304+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:52.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:52 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:52.304+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:25:52.698 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:52 smithi046 podman[117716]: 2024-05-09 00:25:52.321611413 +0000 UTC m=+0.018292831 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:25:52.698 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:52 smithi046 podman[117716]: 2024-05-09 00:25:52.470748713 +0000 UTC m=+0.167430113 container create e507f5d0738f2ef7d4cdb3c3c3f7c85d05e160debc6f004d703453f050073a01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T00:25:52.698 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:52 smithi046 podman[117716]: 2024-05-09 00:25:52.698342281 +0000 UTC m=+0.395023685 container init e507f5d0738f2ef7d4cdb3c3c3f7c85d05e160debc6f004d703453f050073a01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:25:52.698 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:52 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:52.559+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:52.698 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:52 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:52.559+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:52.698 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:52 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:52.559+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:52.698 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:52 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:52.559+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:25:53.021 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:52 smithi046 podman[117716]: 2024-05-09 00:25:52.702107586 +0000 UTC m=+0.398788987 container start e507f5d0738f2ef7d4cdb3c3c3f7c85d05e160debc6f004d703453f050073a01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-09T00:25:53.021 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:52 smithi046 podman[117716]: 2024-05-09 00:25:52.781660114 +0000 UTC m=+0.478341532 container attach e507f5d0738f2ef7d4cdb3c3c3f7c85d05e160debc6f004d703453f050073a01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:25:53.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:52 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:52.678+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:53.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:52 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:52.678+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:53.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:52 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:52.678+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:53.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:52 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:52.678+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:25:53.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:52 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:52.869+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:53.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:52 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:52.869+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:53.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:52 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:52.869+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:25:53.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:52 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:52.810+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:53.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:52 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:52.810+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:53.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:52 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:52.810+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.682431+0000 front 2024-05-09T00:25:23.682166+0000 (oldest deadline 2024-05-09T00:25:49.482527+0000) 2024-05-09T00:25:53.313 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 00:25:53 smithi046 podman[117716]: 2024-05-09 00:25:53.021925781 +0000 UTC m=+0.718607172 container died e507f5d0738f2ef7d4cdb3c3c3f7c85d05e160debc6f004d703453f050073a01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-09T00:25:53.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:53 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:53.276+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:53.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:53 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:53.276+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:53.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:53 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:53.276+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:53.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:53 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:53.276+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:25:53.810 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:53 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:53.567+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:53.810 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:53 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:53.567+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:53.810 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:53 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:53.567+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:53.810 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:53 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:53.567+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:25:53.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.6.service' 2024-05-09T00:25:53.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:25:53.874 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-09T00:25:53.874 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-09T00:25:53.874 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.7 2024-05-09T00:25:53.919 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:53 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:53.639+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:53.919 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:53 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:53.639+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:53.919 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:53 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:53.639+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:53.919 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:53 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:53.639+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:25:54.184 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:53 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:53.798+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:54.184 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:53 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:53.798+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:54.184 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:53 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:53.798+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.682431+0000 front 2024-05-09T00:25:23.682166+0000 (oldest deadline 2024-05-09T00:25:49.482527+0000) 2024-05-09T00:25:54.184 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:53 smithi142 systemd[1]: Stopping Ceph osd.7 for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:25:54.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:53 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:53.918+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:54.357 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:53 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:53.918+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:54.357 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:53 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:53.918+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:25:54.443 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:54 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:54.183+0000 7f7d49d81700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:25:54.444 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:54 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:54.183+0000 7f7d49d81700 -1 osd.7 83 *** Got signal Terminated *** 2024-05-09T00:25:54.444 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:54 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:54.183+0000 7f7d49d81700 -1 osd.7 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:25:54.444 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:54 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:54.310+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:54.444 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:54 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:54.310+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:54.444 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:54 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:54.310+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:54.444 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:54 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:54.310+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:25:55.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:54 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:54.614+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:55.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:54 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:54.614+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:55.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:54 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:54.614+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:55.064 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:54 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:54.614+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:25:55.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:54 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:54.673+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:55.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:54 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:54.673+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:55.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:54 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:54.673+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:55.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:54 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:54.673+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:25:55.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:54 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:54.915+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:55.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:54 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:54.915+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:55.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:54 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:54.915+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:25:55.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:54 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:54.792+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:55.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:54 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:54.792+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:55.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:54 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:54.792+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.682431+0000 front 2024-05-09T00:25:23.682166+0000 (oldest deadline 2024-05-09T00:25:49.482527+0000) 2024-05-09T00:25:55.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:55 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:55.319+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:55.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:55 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:55.319+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:55.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:55 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:55.319+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:55.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:55 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:55.319+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:25:56.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:55 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:55.621+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:56.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:55 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:55.621+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:56.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:55 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:55.621+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:56.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:55 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:55.621+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:25:56.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:55 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:55.943+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:56.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:55 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:55.943+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:56.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:55 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:55.943+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:25:56.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:55 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:55.697+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:56.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:55 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:55.697+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:56.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:55 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:55.697+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:56.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:55 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:55.697+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:25:56.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:55 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:55.811+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:56.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:55 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:55.811+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:56.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:55 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:55.811+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.682431+0000 front 2024-05-09T00:25:23.682166+0000 (oldest deadline 2024-05-09T00:25:49.482527+0000) 2024-05-09T00:25:56.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:55 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:55.811+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.383720+0000 front 2024-05-09T00:25:32.383753+0000 (oldest deadline 2024-05-09T00:25:55.283889+0000) 2024-05-09T00:25:56.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:56 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:56.332+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:56.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:56 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:56.332+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:56.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:56 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:56.332+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:56.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:56 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:56.332+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:25:57.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:56 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:56.623+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:57.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:56 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:56.623+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:57.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:56 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:56.623+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:57.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:56 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:56.623+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:25:57.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:56 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:56.746+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:57.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:56 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:56.746+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:57.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:56 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:56.746+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:57.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:56 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:56.746+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:25:57.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:56 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:56.907+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:57.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:56 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:56.907+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:57.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:56 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:56.907+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:25:57.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:56 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:56.907+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.283089+0000 front 2024-05-09T00:25:32.283045+0000 (oldest deadline 2024-05-09T00:25:56.883326+0000) 2024-05-09T00:25:57.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:56 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:56.794+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:57.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:56 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:56.794+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:57.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:56 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:56.794+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.682431+0000 front 2024-05-09T00:25:23.682166+0000 (oldest deadline 2024-05-09T00:25:49.482527+0000) 2024-05-09T00:25:57.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:56 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:56.794+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.383720+0000 front 2024-05-09T00:25:32.383753+0000 (oldest deadline 2024-05-09T00:25:55.283889+0000) 2024-05-09T00:25:57.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:57 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:57.348+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:57.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:57 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:57.348+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:57.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:57 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:57.348+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:57.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:57 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:57.348+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:25:57.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:57 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:57.348+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:25:57.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:57 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:57.612+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:57.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:57 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:57.612+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:57.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:57 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:57.612+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:57.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:57 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:57.612+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:25:57.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:57 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:57.612+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:25:58.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:57 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:57.877+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:58.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:57 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:57.877+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:58.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:57 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:57.877+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:25:58.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:57 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:57.877+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.283089+0000 front 2024-05-09T00:25:32.283045+0000 (oldest deadline 2024-05-09T00:25:56.883326+0000) 2024-05-09T00:25:58.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:57 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:57.784+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:58.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:57 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:57.784+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:58.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:57 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:57.784+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:58.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:57 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:57.784+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:25:58.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:57 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:57.784+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:25:58.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:57 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:57.773+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:58.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:57 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:57.773+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:58.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:57 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:57.773+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.682431+0000 front 2024-05-09T00:25:23.682166+0000 (oldest deadline 2024-05-09T00:25:49.482527+0000) 2024-05-09T00:25:58.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:57 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:57.773+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.383720+0000 front 2024-05-09T00:25:32.383753+0000 (oldest deadline 2024-05-09T00:25:55.283889+0000) 2024-05-09T00:25:58.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:58 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:58.364+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:58.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:58 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:58.364+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:58.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:58 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:58.364+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:58.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:58 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:58.364+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:25:58.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:58 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:58.364+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:25:59.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:58 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:58.619+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:25:59.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:58 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:58.619+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:25:59.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:58 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:58.619+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:25:59.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:58 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:58.619+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:25:59.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:58 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:58.619+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:25:59.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:58 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:58.837+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:25:59.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:58 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:58.837+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:25:59.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:58 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:58.837+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:25:59.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:58 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:58.837+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.283089+0000 front 2024-05-09T00:25:32.283045+0000 (oldest deadline 2024-05-09T00:25:56.883326+0000) 2024-05-09T00:25:59.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:58 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:58.808+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:25:59.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:58 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:58.808+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:25:59.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:58 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:58.808+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:25:59.108 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:58 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:58.808+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:25:59.108 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:58 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:58.808+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:25:59.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:58 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:58.762+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:10.279770+0000 front 2024-05-09T00:25:10.279880+0000 (oldest deadline 2024-05-09T00:25:35.579838+0000) 2024-05-09T00:25:59.194 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:58 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:58.762+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:17.280821+0000 front 2024-05-09T00:25:17.280989+0000 (oldest deadline 2024-05-09T00:25:40.180864+0000) 2024-05-09T00:25:59.195 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:58 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:58.762+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.682431+0000 front 2024-05-09T00:25:23.682166+0000 (oldest deadline 2024-05-09T00:25:49.482527+0000) 2024-05-09T00:25:59.195 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:58 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7[52501]: 2024-05-09T00:25:58.762+0000 7f7d46337700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.383720+0000 front 2024-05-09T00:25:32.383753+0000 (oldest deadline 2024-05-09T00:25:55.283889+0000) 2024-05-09T00:25:59.693 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:59 smithi142 podman[70397]: 2024-05-09 00:25:59.554652524 +0000 UTC m=+5.487557713 container died 16b0dc942852f40f1e3be50698636aed0845abe516cbc4b4af41f0c327895ab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-09T00:25:59.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:59 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:59.362+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:25:59.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:59 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:59.362+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:25:59.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:59 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:59.362+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:25:59.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:59 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:59.362+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:25:59.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:25:59 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:25:59.362+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:00.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:59 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:59.646+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:00.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:59 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:59.646+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:00.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:59 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:59.646+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:00.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:59 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:59.646+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:00.064 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:25:59 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:25:59.646+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:00.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:59 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:59.848+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:26:00.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:59 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:59.848+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:26:00.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:59 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:59.848+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:26:00.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:25:59 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:25:59.848+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.283089+0000 front 2024-05-09T00:25:32.283045+0000 (oldest deadline 2024-05-09T00:25:56.883326+0000) 2024-05-09T00:26:00.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:59 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:59.771+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:00.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:59 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:59.771+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:00.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:59 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:59.771+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:00.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:59 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:59.771+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:00.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:25:59 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:25:59.771+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:00.186 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:59 smithi142 podman[70397]: 2024-05-09 00:25:59.934564424 +0000 UTC m=+5.867469604 container cleanup 16b0dc942852f40f1e3be50698636aed0845abe516cbc4b4af41f0c327895ab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:26:00.186 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:25:59 smithi142 bash[70397]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7 2024-05-09T00:26:00.443 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:26:00 smithi142 podman[70415]: 2024-05-09 00:26:00.186198514 +0000 UTC m=+0.621799039 container remove 16b0dc942852f40f1e3be50698636aed0845abe516cbc4b4af41f0c327895ab0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T00:26:00.444 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:00 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:00.374+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:00.444 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:00 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:00.374+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:00.444 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:00 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:00.374+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:00.444 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:00 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:00.374+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:00.444 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:00 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:00.374+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:00.807 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:26:00 smithi142 podman[70505]: 2024-05-09 00:26:00.383598003 +0000 UTC m=+0.018210130 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:26:00.807 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:26:00 smithi142 podman[70505]: 2024-05-09 00:26:00.512174678 +0000 UTC m=+0.146786833 container create 690e84d8e05bec9ba30753ced4d9e2d8dccb62af224a9c35bd900033af3af06d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:26:00.807 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:26:00 smithi142 podman[70505]: 2024-05-09 00:26:00.715352753 +0000 UTC m=+0.349964862 container init 690e84d8e05bec9ba30753ced4d9e2d8dccb62af224a9c35bd900033af3af06d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-09T00:26:00.808 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:26:00 smithi142 podman[70505]: 2024-05-09 00:26:00.71833156 +0000 UTC m=+0.352943676 container start 690e84d8e05bec9ba30753ced4d9e2d8dccb62af224a9c35bd900033af3af06d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-09T00:26:00.808 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:26:00 smithi142 podman[70505]: 2024-05-09 00:26:00.807810163 +0000 UTC m=+0.442422282 container attach 690e84d8e05bec9ba30753ced4d9e2d8dccb62af224a9c35bd900033af3af06d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T00:26:01.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:00 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:00.689+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:01.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:00 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:00.689+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:01.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:00 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:00.689+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:01.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:00 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:00.690+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:01.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:00 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:00.690+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:01.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:00 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:00.808+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:26:01.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:00 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:00.808+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:26:01.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:00 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:00.808+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:26:01.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:00 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:00.808+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.283089+0000 front 2024-05-09T00:25:32.283045+0000 (oldest deadline 2024-05-09T00:25:56.883326+0000) 2024-05-09T00:26:01.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:00 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:00.747+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:01.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:00 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:00.747+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:01.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:00 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:00.747+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:01.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:00 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:00.747+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:01.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:00 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:00.747+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:01.193 INFO:journalctl@ceph.osd.7.smithi142.stdout:May 09 00:26:01 smithi142 podman[70505]: 2024-05-09 00:26:01.015134357 +0000 UTC m=+0.649746459 container died 690e84d8e05bec9ba30753ced4d9e2d8dccb62af224a9c35bd900033af3af06d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True) 2024-05-09T00:26:01.637 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:01 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:01.366+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:01.637 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:01 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:01.366+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:01.637 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:01 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:01.366+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:01.637 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:01 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:01.366+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:01.637 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:01 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:01.366+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:01.655 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.7.service' 2024-05-09T00:26:01.692 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:26:01.693 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-09T00:26:01.693 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-09T00:26:01.693 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.8 2024-05-09T00:26:01.999 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:01 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:01.712+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:01.999 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:01 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:01.713+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:02.000 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:01 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:01.713+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:02.000 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:01 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:01.713+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:02.000 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:01 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:01.713+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:02.000 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:01 smithi169 systemd[1]: Stopping Ceph osd.8 for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:26:02.000 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:01 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:01.827+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:26:02.000 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:01 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:01.827+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:26:02.000 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:01 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:01.827+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:26:02.000 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:01 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:01.827+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.283089+0000 front 2024-05-09T00:25:32.283045+0000 (oldest deadline 2024-05-09T00:25:56.883326+0000) 2024-05-09T00:26:02.000 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:02 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:01.999+0000 7f3ad8fdd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:26:02.000 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:02 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:01.999+0000 7f3ad8fdd700 -1 osd.8 83 *** Got signal Terminated *** 2024-05-09T00:26:02.000 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:02 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:01.999+0000 7f3ad8fdd700 -1 osd.8 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:26:02.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:01 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:01.642+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:02.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:01 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:01.642+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:02.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:01 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:01.642+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:02.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:01 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:01.642+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:02.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:01 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:01.642+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:02.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:02 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:02.364+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:02.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:02 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:02.364+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:02.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:02 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:02.364+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:02.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:02 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:02.364+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:02.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:02 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:02.364+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:03.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:02 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:02.647+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:03.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:02 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:02.647+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:03.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:02 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:02.647+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:03.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:02 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:02.647+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:03.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:02 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:02.647+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:03.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:02 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:02.752+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:03.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:02 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:02.752+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:03.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:02 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:02.752+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:03.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:02 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:02.752+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:03.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:02 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:02.752+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:03.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:02 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:02.782+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:26:03.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:02 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:02.782+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:26:03.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:02 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:02.782+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:26:03.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:02 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:02.782+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.283089+0000 front 2024-05-09T00:25:32.283045+0000 (oldest deadline 2024-05-09T00:25:56.883326+0000) 2024-05-09T00:26:03.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:03 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:03.368+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:03.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:03 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:03.368+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:03.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:03 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:03.368+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:03.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:03 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:03.368+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:03.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:03 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:03.368+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:04.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:03 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:03.605+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:04.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:03 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:03.605+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:04.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:03 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:03.605+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:04.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:03 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:03.605+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:04.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:03 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:03.605+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:04.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:03 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:03.759+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:26:04.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:03 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:03.760+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:26:04.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:03 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:03.760+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:26:04.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:03 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:03.760+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.283089+0000 front 2024-05-09T00:25:32.283045+0000 (oldest deadline 2024-05-09T00:25:56.883326+0000) 2024-05-09T00:26:04.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:03 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:03.706+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:04.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:03 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:03.706+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:04.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:03 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:03.706+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:04.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:03 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:03.706+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:04.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:03 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:03.706+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:04.595 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:04 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:04.324+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:04.596 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:04 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:04.324+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:04.596 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:04 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:04.324+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:04.596 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:04 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:04.324+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:04.596 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:04 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:04.324+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:04.596 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:04 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:04.324+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:05.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:04 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:04.585+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:05.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:04 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:04.585+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:05.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:04 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:04.585+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:05.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:04 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:04.585+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:05.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:04 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:04.585+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:05.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:04 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:04.742+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:26:05.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:04 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:04.742+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:26:05.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:04 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:04.742+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:26:05.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:04 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:04.742+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.283089+0000 front 2024-05-09T00:25:32.283045+0000 (oldest deadline 2024-05-09T00:25:56.883326+0000) 2024-05-09T00:26:05.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:04 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:04.717+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:05.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:04 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:04.717+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:05.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:04 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:04.717+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:05.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:04 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:04.717+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:05.108 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:04 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:04.717+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:05.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:05 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:05.302+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:05.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:05 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:05.302+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:05.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:05 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:05.302+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:05.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:05 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:05.302+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:05.695 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:05 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:05.302+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:05.695 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:05 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:05.302+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:06.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:05 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:05.582+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:06.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:05 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:05.582+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:06.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:05 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:05.582+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:06.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:05 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:05.582+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:06.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:05 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:05.582+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:06.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:05 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:05.791+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:26:06.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:05 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:05.791+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:26:06.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:05 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:05.791+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:26:06.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:05 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:05.791+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.283089+0000 front 2024-05-09T00:25:32.283045+0000 (oldest deadline 2024-05-09T00:25:56.883326+0000) 2024-05-09T00:26:06.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:05 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:05.757+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:06.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:05 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:05.757+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:06.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:05 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:05.757+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:06.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:05 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:05.757+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:06.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:05 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:05.757+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:06.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:06 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:06.335+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:06.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:06 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:06.335+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:06.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:06 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:06.335+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:06.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:06 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:06.335+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:06.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:06 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:06.335+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:06.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:06 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:06.335+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:07.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:06 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:06.629+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:07.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:06 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:06.629+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:07.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:06 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:06.629+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:07.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:06 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:06.629+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:07.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:06 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:06.629+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:07.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:06 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:06.629+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:40.807065+0000 front 2024-05-09T00:25:40.807217+0000 (oldest deadline 2024-05-09T00:26:06.107011+0000) 2024-05-09T00:26:07.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:06 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:06.796+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.377640+0000 front 2024-05-09T00:25:07.377702+0000 (oldest deadline 2024-05-09T00:25:32.677718+0000) 2024-05-09T00:26:07.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:06 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:06.796+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.679171+0000 front 2024-05-09T00:25:16.679241+0000 (oldest deadline 2024-05-09T00:25:40.179338+0000) 2024-05-09T00:26:07.106 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:06 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:06.796+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:22.981075+0000 front 2024-05-09T00:25:22.981006+0000 (oldest deadline 2024-05-09T00:25:47.080899+0000) 2024-05-09T00:26:07.107 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:06 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8[53483]: 2024-05-09T00:26:06.796+0000 7f3ad5593700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.283089+0000 front 2024-05-09T00:25:32.283045+0000 (oldest deadline 2024-05-09T00:25:56.883326+0000) 2024-05-09T00:26:07.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:06 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:06.771+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:07.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:06 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:06.771+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:07.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:06 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:06.771+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:07.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:06 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:06.771+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:07.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:06 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:06.771+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:07.641 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:07 smithi169 podman[71475]: 2024-05-09 00:26:07.371102495 +0000 UTC m=+5.480723975 container died 67e23aea7726ba76cbfebca827ab5000fa9411d5cd25bb6b370e7697a07b0fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T00:26:07.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:07 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:07.345+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:07.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:07 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:07.345+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:07.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:07 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:07.345+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:07.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:07 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:07.345+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:07.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:07 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:07.345+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:07.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:07 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:07.345+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:08.008 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:07 smithi169 podman[71475]: 2024-05-09 00:26:07.741098103 +0000 UTC m=+5.850719582 container cleanup 67e23aea7726ba76cbfebca827ab5000fa9411d5cd25bb6b370e7697a07b0fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:26:08.009 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:07 smithi169 bash[71475]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8 2024-05-09T00:26:08.009 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:07 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:07.806+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:08.009 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:07 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:07.806+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:08.009 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:07 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:07.806+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:08.009 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:07 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:07.806+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:08.009 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:07 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:07.806+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:08.009 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:07 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:07.806+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:08.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:07 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:07.651+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:08.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:07 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:07.651+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:08.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:07 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:07.651+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:08.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:07 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:07.651+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:08.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:07 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:07.651+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:08.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:07 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:07.651+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:40.807065+0000 front 2024-05-09T00:25:40.807217+0000 (oldest deadline 2024-05-09T00:26:06.107011+0000) 2024-05-09T00:26:08.356 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:08 smithi169 podman[71494]: 2024-05-09 00:26:08.008775039 +0000 UTC m=+0.627527749 container remove 67e23aea7726ba76cbfebca827ab5000fa9411d5cd25bb6b370e7697a07b0fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-09T00:26:08.612 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:08 smithi169 podman[71587]: 2024-05-09 00:26:08.359586989 +0000 UTC m=+0.109344933 container create 784e61d16da742778263b3f56819538bb239f031d2e3396387751035d56eb315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-09T00:26:08.612 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:08 smithi169 podman[71587]: 2024-05-09 00:26:08.268257905 +0000 UTC m=+0.018015817 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:26:08.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:08 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:08.333+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:08.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:08 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:08.333+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:08.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:08 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:08.333+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:08.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:08 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:08.333+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:08.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:08 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:08.333+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:08.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:08 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:08.333+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:08.913 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:08 smithi169 podman[71587]: 2024-05-09 00:26:08.612295733 +0000 UTC m=+0.362053647 container init 784e61d16da742778263b3f56819538bb239f031d2e3396387751035d56eb315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:26:08.913 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:08 smithi169 podman[71587]: 2024-05-09 00:26:08.615674913 +0000 UTC m=+0.365432823 container start 784e61d16da742778263b3f56819538bb239f031d2e3396387751035d56eb315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T00:26:08.913 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:08 smithi169 podman[71587]: 2024-05-09 00:26:08.695241465 +0000 UTC m=+0.444999380 container attach 784e61d16da742778263b3f56819538bb239f031d2e3396387751035d56eb315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:26:08.913 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:08 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:08.830+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:08.913 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:08 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:08.830+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:08.913 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:08 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:08.830+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:08.913 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:08 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:08.830+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:08.914 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:08 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:08.830+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:08.914 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:08 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:08.830+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:09.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:08 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:08.651+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:09.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:08 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:08.651+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:09.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:08 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:08.651+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:09.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:08 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:08.651+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:09.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:08 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:08.651+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:09.064 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:08 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:08.651+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:40.807065+0000 front 2024-05-09T00:25:40.807217+0000 (oldest deadline 2024-05-09T00:26:06.107011+0000) 2024-05-09T00:26:09.197 INFO:journalctl@ceph.osd.8.smithi169.stdout:May 09 00:26:08 smithi169 podman[71587]: 2024-05-09 00:26:08.913395047 +0000 UTC m=+0.663152956 container died 784e61d16da742778263b3f56819538bb239f031d2e3396387751035d56eb315 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:26:09.524 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.8.service' 2024-05-09T00:26:09.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:26:09.563 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-09T00:26:09.563 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-09T00:26:09.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.9 2024-05-09T00:26:09.695 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:09 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:09.328+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:09.695 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:09 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:09.328+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:09.695 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:09 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:09.328+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:09.695 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:09 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:09.328+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:09.695 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:09 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:09.328+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:09.695 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:09 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:09.328+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:09.888 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:09 smithi046 systemd[1]: Stopping Ceph osd.9 for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:26:09.888 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:09 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:09.635+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:09.889 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:09 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:09.635+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:09.889 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:09 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:09.635+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:09.889 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:09 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:09.635+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:09.889 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:09 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:09.635+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:09.889 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:09 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:09.635+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:40.807065+0000 front 2024-05-09T00:25:40.807217+0000 (oldest deadline 2024-05-09T00:26:06.107011+0000) 2024-05-09T00:26:09.889 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:09 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:09.888+0000 7f677aaa6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:26:09.889 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:09 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:09.888+0000 7f677aaa6700 -1 osd.9 83 *** Got signal Terminated *** 2024-05-09T00:26:09.889 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:09 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:09.888+0000 7f677aaa6700 -1 osd.9 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:26:10.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:09 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:09.859+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:10.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:09 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:09.859+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:10.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:09 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:09.859+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:10.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:09 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:09.859+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:10.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:09 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:09.859+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:10.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:09 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:09.859+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:10.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:10 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:10.332+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:10.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:10 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:10.332+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:10.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:10 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:10.332+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:10.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:10 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:10.332+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:10.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:10 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:10.332+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:10.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:10 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:10.332+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:11.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:10 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:10.608+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:11.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:10 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:10.608+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:11.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:10 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:10.608+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:11.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:10 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:10.608+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:11.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:10 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:10.608+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:11.064 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:10 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:10.608+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:40.807065+0000 front 2024-05-09T00:25:40.807217+0000 (oldest deadline 2024-05-09T00:26:06.107011+0000) 2024-05-09T00:26:11.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:10 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:10.903+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:11.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:10 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:10.903+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:11.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:10 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:10.903+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:11.106 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:10 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:10.903+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:11.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:10 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:10.903+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:11.107 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:10 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:10.903+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:11.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:11 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:11.314+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:11.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:11 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:11.314+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:11.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:11 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:11.314+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:11.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:11 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:11.314+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:11.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:11 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:11.314+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:11.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:11 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:11.314+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:12.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:11 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:11.641+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:12.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:11 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:11.641+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:12.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:11 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:11.641+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:12.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:11 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:11.641+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:12.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:11 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:11.641+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:12.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:11 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:11.641+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:40.807065+0000 front 2024-05-09T00:25:40.807217+0000 (oldest deadline 2024-05-09T00:26:06.107011+0000) 2024-05-09T00:26:12.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:11 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:11.878+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:12.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:11 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:11.878+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:12.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:11 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:11.878+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:12.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:11 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:11.878+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:12.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:11 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:11.878+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:12.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:11 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:11.878+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:12.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:12 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:12.358+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:12.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:12 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:12.358+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:12.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:12 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:12.358+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:12.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:12 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:12.358+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:12.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:12 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:12.358+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:12.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:12 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:12.358+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:12.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:12 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:12.358+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T00:25:50.568732+0000 front 2024-05-09T00:25:50.568694+0000 (oldest deadline 2024-05-09T00:26:12.268554+0000) 2024-05-09T00:26:13.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:12 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:12.682+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:13.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:12 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:12.682+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:13.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:12 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:12.682+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:13.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:12 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:12.682+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:13.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:12 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:12.682+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:13.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:12 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:12.682+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:40.807065+0000 front 2024-05-09T00:25:40.807217+0000 (oldest deadline 2024-05-09T00:26:06.107011+0000) 2024-05-09T00:26:13.258 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:12 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:12.912+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:13.258 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:12 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:12.912+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:13.258 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:12 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:12.912+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:13.258 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:12 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:12.912+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:13.258 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:12 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:12.912+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:13.258 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:12 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:12.912+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:13.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:13 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:13.377+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:13.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:13 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:13.377+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:13.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:13 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:13.377+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:13.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:13 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:13.377+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:13.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:13 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:13.377+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:13.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:13 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:13.377+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:13.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:13 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:13.377+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T00:25:50.568732+0000 front 2024-05-09T00:25:50.568694+0000 (oldest deadline 2024-05-09T00:26:12.268554+0000) 2024-05-09T00:26:14.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:13 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:13.654+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:14.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:13 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:13.654+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:14.064 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:13 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:13.654+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:14.064 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:13 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:13.654+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:14.064 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:13 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:13.654+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:14.064 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:13 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:13.654+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:40.807065+0000 front 2024-05-09T00:25:40.807217+0000 (oldest deadline 2024-05-09T00:26:06.107011+0000) 2024-05-09T00:26:14.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:13 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:13.934+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:14.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:13 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:13.934+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:14.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:13 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:13.934+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:14.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:13 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:13.934+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:14.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:13 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:13.934+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:14.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:13 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:13.934+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:14.679 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:14 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:14.420+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:14.679 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:14 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:14.420+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:14.679 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:14 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:14.420+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:14.679 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:14 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:14.420+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:14.679 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:14 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:14.420+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:14.680 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:14 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:14.420+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:14.680 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:14 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:14.420+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T00:25:50.568732+0000 front 2024-05-09T00:25:50.568694+0000 (oldest deadline 2024-05-09T00:26:12.268554+0000) 2024-05-09T00:26:15.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:14 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:14.665+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:25:01.898886+0000 front 2024-05-09T00:25:01.899132+0000 (oldest deadline 2024-05-09T00:25:27.799109+0000) 2024-05-09T00:26:15.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:14 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:14.665+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:07.799849+0000 front 2024-05-09T00:25:07.799826+0000 (oldest deadline 2024-05-09T00:25:33.699713+0000) 2024-05-09T00:26:15.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:14 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:14.665+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:18.301529+0000 front 2024-05-09T00:25:18.301411+0000 (oldest deadline 2024-05-09T00:25:40.001341+0000) 2024-05-09T00:26:15.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:14 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:14.665+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:25.302274+0000 front 2024-05-09T00:25:25.302330+0000 (oldest deadline 2024-05-09T00:25:52.303144+0000) 2024-05-09T00:26:15.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:14 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:14.665+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:34.505494+0000 front 2024-05-09T00:25:34.505451+0000 (oldest deadline 2024-05-09T00:25:57.405276+0000) 2024-05-09T00:26:15.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:14 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:14.665+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:40.807065+0000 front 2024-05-09T00:25:40.807217+0000 (oldest deadline 2024-05-09T00:26:06.107011+0000) 2024-05-09T00:26:15.063 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:14 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9[63390]: 2024-05-09T00:26:14.665+0000 7f6773054700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T00:25:48.908558+0000 front 2024-05-09T00:25:48.908479+0000 (oldest deadline 2024-05-09T00:26:14.208528+0000) 2024-05-09T00:26:15.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:14 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:14.949+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:15.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:14 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:14.949+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:15.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:14 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:14.949+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:15.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:14 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:14.949+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:15.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:14 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:14.949+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:15.358 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:14 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:14.949+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:15.505 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:15 smithi046 podman[117820]: 2024-05-09 00:26:15.231939983 +0000 UTC m=+5.455966656 container died b37f53f83511e037c1266964339464dd6d59cf93e2549522167600482a54d403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS) 2024-05-09T00:26:15.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:15 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:15.434+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:15.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:15 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:15.434+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:15.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:15 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:15.434+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:15.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:15 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:15.434+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:15.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:15 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:15.434+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:15.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:15 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:15.434+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:15.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:15 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:15.434+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T00:25:50.568732+0000 front 2024-05-09T00:25:50.568694+0000 (oldest deadline 2024-05-09T00:26:12.268554+0000) 2024-05-09T00:26:15.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:15 smithi046 podman[117820]: 2024-05-09 00:26:15.671068895 +0000 UTC m=+5.895095555 container cleanup b37f53f83511e037c1266964339464dd6d59cf93e2549522167600482a54d403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T00:26:15.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:15 smithi046 bash[117820]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9 2024-05-09T00:26:16.305 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:15 smithi046 podman[117836]: 2024-05-09 00:26:15.872468957 +0000 UTC m=+0.635410075 container remove b37f53f83511e037c1266964339464dd6d59cf93e2549522167600482a54d403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-05-09T00:26:16.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:15 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:15.992+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:16.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:15 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:15.992+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:16.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:15 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:15.992+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:16.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:15 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:15.992+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:16.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:15 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:15.992+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:16.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:15 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:15.992+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:16.562 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:16 smithi046 podman[117927]: 2024-05-09 00:26:16.208339712 +0000 UTC m=+0.018603962 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:26:16.562 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:16 smithi046 podman[117927]: 2024-05-09 00:26:16.323107734 +0000 UTC m=+0.133371965 container create a0bb9475fe936612a786296a4513c34afbf5eaa0f7d40196d2690fb29ba66319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-09T00:26:16.562 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:16 smithi046 podman[117927]: 2024-05-09 00:26:16.562652196 +0000 UTC m=+0.372916429 container init a0bb9475fe936612a786296a4513c34afbf5eaa0f7d40196d2690fb29ba66319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:26:16.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:16 smithi046 podman[117927]: 2024-05-09 00:26:16.566086092 +0000 UTC m=+0.376350331 container start a0bb9475fe936612a786296a4513c34afbf5eaa0f7d40196d2690fb29ba66319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:26:16.813 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:16 smithi046 podman[117927]: 2024-05-09 00:26:16.642088548 +0000 UTC m=+0.452352792 container attach a0bb9475fe936612a786296a4513c34afbf5eaa0f7d40196d2690fb29ba66319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:26:16.943 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:16 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:16.447+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:16.944 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:16 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:16.447+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:16.944 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:16 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:16.447+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:16.944 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:16 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:16.447+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:16.944 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:16 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:16.447+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:16.944 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:16 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:16.447+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:16.944 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:16 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:16.447+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T00:25:50.568732+0000 front 2024-05-09T00:25:50.568694+0000 (oldest deadline 2024-05-09T00:26:12.268554+0000) 2024-05-09T00:26:17.144 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:16 smithi046 podman[117927]: 2024-05-09 00:26:16.878970848 +0000 UTC m=+0.689235078 container died a0bb9475fe936612a786296a4513c34afbf5eaa0f7d40196d2690fb29ba66319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:26:17.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:17 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:17.007+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:17.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:17 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:17.007+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:17.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:17 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:17.007+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:17.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:17 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:17.007+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:17.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:17 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:17.007+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:17.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:17 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:17.007+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:17.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.9.service' 2024-05-09T00:26:17.411 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:17 smithi046 podman[117953]: 2024-05-09 00:26:17.363076568 +0000 UTC m=+0.477657201 container remove a0bb9475fe936612a786296a4513c34afbf5eaa0f7d40196d2690fb29ba66319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T00:26:17.411 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:17 smithi046 systemd[1]: ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.9.service: Deactivated successfully. 2024-05-09T00:26:17.411 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:17 smithi046 systemd[1]: Stopped Ceph osd.9 for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:26:17.411 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 00:26:17 smithi046 systemd[1]: ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.9.service: Consumed 10.355s CPU time. 2024-05-09T00:26:17.428 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:26:17.428 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-09T00:26:17.428 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-09T00:26:17.428 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.10 2024-05-09T00:26:17.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:17 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:17.431+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:17.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:17 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:17.431+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:17.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:17 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:17.431+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:17.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:17 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:17.431+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:17.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:17 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:17.431+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:17.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:17 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:17.431+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:17.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:17 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:17.431+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T00:25:50.568732+0000 front 2024-05-09T00:25:50.568694+0000 (oldest deadline 2024-05-09T00:26:12.268554+0000) 2024-05-09T00:26:17.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:17 smithi142 systemd[1]: Stopping Ceph osd.10 for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:26:18.193 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:17 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:17.739+0000 7f13037c3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:26:18.194 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:17 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:17.739+0000 7f13037c3700 -1 osd.10 83 *** Got signal Terminated *** 2024-05-09T00:26:18.194 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:17 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:17.739+0000 7f13037c3700 -1 osd.10 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:26:18.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:18 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:18.056+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:18.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:18 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:18.056+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:18.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:18 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:18.056+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:18.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:18 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:18.056+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:18.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:18 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:18.056+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:18.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:18 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:18.056+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:18.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:18 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:18.417+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:18.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:18 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:18.417+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:18.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:18 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:18.417+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:18.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:18 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:18.417+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:18.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:18 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:18.417+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:18.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:18 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:18.417+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:18.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:18 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:18.417+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T00:25:50.568732+0000 front 2024-05-09T00:25:50.568694+0000 (oldest deadline 2024-05-09T00:26:12.268554+0000) 2024-05-09T00:26:19.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:19 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:19.054+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:19.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:19 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:19.054+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:19.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:19 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:19.054+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:19.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:19 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:19.054+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:19.358 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:19 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:19.054+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:19.358 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:19 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:19.054+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:19.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:19 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:19.411+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:19.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:19 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:19.411+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:19.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:19 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:19.411+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:19.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:19 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:19.411+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:19.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:19 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:19.411+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:19.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:19 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:19.411+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:19.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:19 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:19.411+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T00:25:50.568732+0000 front 2024-05-09T00:25:50.568694+0000 (oldest deadline 2024-05-09T00:26:12.268554+0000) 2024-05-09T00:26:20.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:20 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:20.019+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:20.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:20 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:20.019+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:20.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:20 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:20.019+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:20.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:20 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:20.019+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:20.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:20 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:20.019+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:20.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:20 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:20.019+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:20.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:20 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:20.019+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-09T00:25:56.353455+0000 front 2024-05-09T00:25:56.353445+0000 (oldest deadline 2024-05-09T00:26:19.853368+0000) 2024-05-09T00:26:20.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:20 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:20.414+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:20.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:20 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:20.414+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:20.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:20 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:20.414+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:20.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:20 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:20.414+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:20.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:20 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:20.414+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:20.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:20 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:20.414+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:20.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:20 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:20.414+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T00:25:50.568732+0000 front 2024-05-09T00:25:50.568694+0000 (oldest deadline 2024-05-09T00:26:12.268554+0000) 2024-05-09T00:26:21.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:21 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:21.002+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:21.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:21 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:21.002+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:21.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:21 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:21.002+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:21.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:21 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:21.002+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:21.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:21 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:21.002+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:21.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:21 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:21.002+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:21.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:21 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:21.002+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-09T00:25:56.353455+0000 front 2024-05-09T00:25:56.353445+0000 (oldest deadline 2024-05-09T00:26:19.853368+0000) 2024-05-09T00:26:21.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:21 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:21.391+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:21.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:21 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:21.391+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:21.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:21 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:21.391+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:21.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:21 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:21.391+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:21.695 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:21 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:21.391+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:21.695 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:21 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:21.391+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:21.695 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:21 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:21.391+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T00:25:50.568732+0000 front 2024-05-09T00:25:50.568694+0000 (oldest deadline 2024-05-09T00:26:12.268554+0000) 2024-05-09T00:26:22.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.008+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:22.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.008+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:22.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.008+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:22.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.008+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:22.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.008+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:22.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.008+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:22.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.008+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-09T00:25:56.353455+0000 front 2024-05-09T00:25:56.353445+0000 (oldest deadline 2024-05-09T00:26:19.853368+0000) 2024-05-09T00:26:22.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:22 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:22.378+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:57.759355+0000 front 2024-05-09T00:24:57.759534+0000 (oldest deadline 2024-05-09T00:25:23.059386+0000) 2024-05-09T00:26:22.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:22 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:22.378+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.360525+0000 front 2024-05-09T00:25:05.360490+0000 (oldest deadline 2024-05-09T00:25:31.260358+0000) 2024-05-09T00:26:22.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:22 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:22.378+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:15.261723+0000 front 2024-05-09T00:25:15.261978+0000 (oldest deadline 2024-05-09T00:25:43.462425+0000) 2024-05-09T00:26:22.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:22 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:22.378+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:24.563861+0000 front 2024-05-09T00:25:24.563792+0000 (oldest deadline 2024-05-09T00:25:50.364110+0000) 2024-05-09T00:26:22.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:22 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:22.378+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.664902+0000 front 2024-05-09T00:25:32.664937+0000 (oldest deadline 2024-05-09T00:25:57.265467+0000) 2024-05-09T00:26:22.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:22 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:22.378+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:39.467382+0000 front 2024-05-09T00:25:39.467355+0000 (oldest deadline 2024-05-09T00:26:03.567227+0000) 2024-05-09T00:26:22.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:22 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10[59248]: 2024-05-09T00:26:22.378+0000 7f12fbd71700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T00:25:50.568732+0000 front 2024-05-09T00:25:50.568694+0000 (oldest deadline 2024-05-09T00:26:12.268554+0000) 2024-05-09T00:26:23.312 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.971+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:23.313 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.971+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:23.313 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.971+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:23.313 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.971+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:23.313 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.971+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:23.313 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.971+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:23.313 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:22 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:22.971+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-09T00:25:56.353455+0000 front 2024-05-09T00:25:56.353445+0000 (oldest deadline 2024-05-09T00:26:19.853368+0000) 2024-05-09T00:26:23.365 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:23 smithi142 podman[70621]: 2024-05-09 00:26:23.100696831 +0000 UTC m=+5.474743095 container died ede833d5601b0eb614cc08bf2104ffb9c0a5818e8d83f47ea09fedb2e980f0c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:26:23.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:23 smithi142 podman[70621]: 2024-05-09 00:26:23.45593427 +0000 UTC m=+5.829980533 container cleanup ede833d5601b0eb614cc08bf2104ffb9c0a5818e8d83f47ea09fedb2e980f0c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:26:23.693 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:23 smithi142 bash[70621]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10 2024-05-09T00:26:24.077 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:23 smithi142 podman[70638]: 2024-05-09 00:26:23.723719912 +0000 UTC m=+0.623298546 container remove ede833d5601b0eb614cc08bf2104ffb9c0a5818e8d83f47ea09fedb2e980f0c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T00:26:24.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:23 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:23.922+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:24.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:23 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:23.922+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:24.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:23 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:23.922+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:24.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:23 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:23.922+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:24.358 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:23 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:23.922+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:24.358 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:23 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:23.922+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:24.358 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:23 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:23.922+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-09T00:25:56.353455+0000 front 2024-05-09T00:25:56.353445+0000 (oldest deadline 2024-05-09T00:26:19.853368+0000) 2024-05-09T00:26:24.376 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:24 smithi142 podman[70730]: 2024-05-09 00:26:23.980418208 +0000 UTC m=+0.017905537 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:26:24.376 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:24 smithi142 podman[70730]: 2024-05-09 00:26:24.091041507 +0000 UTC m=+0.128528835 container create d68312b11e5e5004e411760da720306274f19b9b12dd7af2fcc545e43a941662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T00:26:24.377 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:24 smithi142 podman[70730]: 2024-05-09 00:26:24.301821427 +0000 UTC m=+0.339308755 container init d68312b11e5e5004e411760da720306274f19b9b12dd7af2fcc545e43a941662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T00:26:24.377 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:24 smithi142 podman[70730]: 2024-05-09 00:26:24.305340615 +0000 UTC m=+0.342827946 container start d68312b11e5e5004e411760da720306274f19b9b12dd7af2fcc545e43a941662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-05-09T00:26:24.377 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:24 smithi142 podman[70730]: 2024-05-09 00:26:24.376685355 +0000 UTC m=+0.414172692 container attach d68312b11e5e5004e411760da720306274f19b9b12dd7af2fcc545e43a941662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS) 2024-05-09T00:26:24.694 INFO:journalctl@ceph.osd.10.smithi142.stdout:May 09 00:26:24 smithi142 podman[70730]: 2024-05-09 00:26:24.603197603 +0000 UTC m=+0.640684931 container died d68312b11e5e5004e411760da720306274f19b9b12dd7af2fcc545e43a941662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T00:26:25.179 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.10.service' 2024-05-09T00:26:25.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:26:25.224 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-09T00:26:25.224 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-09T00:26:25.224 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.11 2024-05-09T00:26:25.251 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:24 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:24.918+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:25.251 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:24 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:24.918+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:25.251 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:24 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:24.918+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:25.251 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:24 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:24.918+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:25.251 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:24 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:24.918+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:25.251 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:24 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:24.918+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:25.251 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:24 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:24.918+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-09T00:25:56.353455+0000 front 2024-05-09T00:25:56.353445+0000 (oldest deadline 2024-05-09T00:26:19.853368+0000) 2024-05-09T00:26:25.542 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:25 smithi169 systemd[1]: Stopping Ceph osd.11 for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:26:25.856 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:25.542+0000 7f86aee5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:26:25.856 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:25.542+0000 7f86aee5f700 -1 osd.11 83 *** Got signal Terminated *** 2024-05-09T00:26:25.856 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:25.542+0000 7f86aee5f700 -1 osd.11 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T00:26:26.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:25.942+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:26.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:25.942+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:26.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:25.942+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:26.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:25.942+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:26.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:25.942+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:26.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:25.942+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:26.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:25 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:25.942+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-09T00:25:56.353455+0000 front 2024-05-09T00:25:56.353445+0000 (oldest deadline 2024-05-09T00:26:19.853368+0000) 2024-05-09T00:26:27.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:26 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:26.939+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:27.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:26 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:26.939+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:27.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:26 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:26.939+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:27.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:26 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:26.939+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:27.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:26 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:26.939+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:27.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:26 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:26.939+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:27.358 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:26 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:26.939+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-09T00:25:56.353455+0000 front 2024-05-09T00:25:56.353445+0000 (oldest deadline 2024-05-09T00:26:19.853368+0000) 2024-05-09T00:26:28.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:27 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:27.937+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:28.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:27 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:27.937+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:28.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:27 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:27.937+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:28.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:27 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:27.937+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:28.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:27 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:27.937+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:28.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:27 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:27.937+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:28.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:27 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:27.937+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-09T00:25:56.353455+0000 front 2024-05-09T00:25:56.353445+0000 (oldest deadline 2024-05-09T00:26:19.853368+0000) 2024-05-09T00:26:29.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:28 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:28.966+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:29.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:28 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:28.966+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:29.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:28 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:28.966+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:29.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:28 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:28.966+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:29.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:28 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:28.966+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:29.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:28 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:28.966+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:29.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:28 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:28.966+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-09T00:25:56.353455+0000 front 2024-05-09T00:25:56.353445+0000 (oldest deadline 2024-05-09T00:26:19.853368+0000) 2024-05-09T00:26:30.356 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:29 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:29.979+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T00:24:59.744210+0000 front 2024-05-09T00:24:59.744365+0000 (oldest deadline 2024-05-09T00:25:25.644365+0000) 2024-05-09T00:26:30.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:29 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:29.979+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-05-09T00:25:05.644886+0000 front 2024-05-09T00:25:05.644829+0000 (oldest deadline 2024-05-09T00:25:31.544916+0000) 2024-05-09T00:26:30.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:29 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:29.979+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-05-09T00:25:16.245949+0000 front 2024-05-09T00:25:16.245990+0000 (oldest deadline 2024-05-09T00:25:39.046825+0000) 2024-05-09T00:26:30.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:29 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:29.979+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T00:25:23.048893+0000 front 2024-05-09T00:25:23.048768+0000 (oldest deadline 2024-05-09T00:25:52.449069+0000) 2024-05-09T00:26:30.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:29 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:29.979+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-05-09T00:25:32.449861+0000 front 2024-05-09T00:25:32.449711+0000 (oldest deadline 2024-05-09T00:25:57.149614+0000) 2024-05-09T00:26:30.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:29 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:29.979+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-05-09T00:25:38.250530+0000 front 2024-05-09T00:25:38.250568+0000 (oldest deadline 2024-05-09T00:26:07.051198+0000) 2024-05-09T00:26:30.357 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:29 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:29.979+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-05-09T00:25:56.353455+0000 front 2024-05-09T00:25:56.353445+0000 (oldest deadline 2024-05-09T00:26:19.853368+0000) 2024-05-09T00:26:30.358 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:29 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11[60079]: 2024-05-09T00:26:29.979+0000 7f86ab415700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.169:6820 osd.8 since back 2024-05-09T00:26:03.954671+0000 front 2024-05-09T00:26:03.954614+0000 (oldest deadline 2024-05-09T00:26:29.254576+0000) 2024-05-09T00:26:31.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:30 smithi169 podman[71702]: 2024-05-09 00:26:30.898957743 +0000 UTC m=+5.474387711 container died 52249d1e3782003071b197a38c149e059427b9e41e6ed1bca258db05b1bc22a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:26:31.517 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:31 smithi169 podman[71702]: 2024-05-09 00:26:31.250411621 +0000 UTC m=+5.825841594 container cleanup 52249d1e3782003071b197a38c149e059427b9e41e6ed1bca258db05b1bc22a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-09T00:26:31.517 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:31 smithi169 bash[71702]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11 2024-05-09T00:26:31.856 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:31 smithi169 podman[71719]: 2024-05-09 00:26:31.517902277 +0000 UTC m=+0.618643103 container remove 52249d1e3782003071b197a38c149e059427b9e41e6ed1bca258db05b1bc22a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-09T00:26:32.146 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:31 smithi169 podman[71808]: 2024-05-09 00:26:31.868748438 +0000 UTC m=+0.109694966 container create b1d7870be5d6c5ab6e081d9e3780a0a429c6894af33cb5629311297da4022d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS) 2024-05-09T00:26:32.146 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:31 smithi169 podman[71808]: 2024-05-09 00:26:31.776577608 +0000 UTC m=+0.017524156 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T00:26:32.146 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:32 smithi169 podman[71808]: 2024-05-09 00:26:32.071351015 +0000 UTC m=+0.312297541 container init b1d7870be5d6c5ab6e081d9e3780a0a429c6894af33cb5629311297da4022d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T00:26:32.146 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:32 smithi169 podman[71808]: 2024-05-09 00:26:32.074334148 +0000 UTC m=+0.315280680 container start b1d7870be5d6c5ab6e081d9e3780a0a429c6894af33cb5629311297da4022d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:26:32.147 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:32 smithi169 podman[71808]: 2024-05-09 00:26:32.146105574 +0000 UTC m=+0.387052103 container attach b1d7870be5d6c5ab6e081d9e3780a0a429c6894af33cb5629311297da4022d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-09T00:26:32.606 INFO:journalctl@ceph.osd.11.smithi169.stdout:May 09 00:26:32 smithi169 podman[71808]: 2024-05-09 00:26:32.373923014 +0000 UTC m=+0.614869552 container died b1d7870be5d6c5ab6e081d9e3780a0a429c6894af33cb5629311297da4022d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-09T00:26:33.002 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@osd.11.service' 2024-05-09T00:26:33.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:26:33.080 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-09T00:26:33.080 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-09T00:26:33.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.a 2024-05-09T00:26:33.390 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:26:33 smithi046 systemd[1]: Stopping Ceph mds.a for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:26:33.813 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:26:33 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-a[70553]: 2024-05-09T00:26:33.390+0000 7f3abc150700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:26:33.813 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:26:33 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-a[70553]: 2024-05-09T00:26:33.390+0000 7f3abc150700 -1 mds.a *** got signal Terminated *** 2024-05-09T00:26:37.682 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:26:37 smithi046 podman[118044]: 2024-05-09 00:26:37.397570118 +0000 UTC m=+4.112488350 container died f835197cfca6fa27458448d2775723a88ac7b5b3012a4a918cb734f529827a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T00:26:38.023 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:26:37 smithi046 podman[118044]: 2024-05-09 00:26:37.755687738 +0000 UTC m=+4.470605979 container cleanup f835197cfca6fa27458448d2775723a88ac7b5b3012a4a918cb734f529827a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-a, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-05-09T00:26:38.023 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 00:26:37 smithi046 bash[118044]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-a 2024-05-09T00:26:38.191 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.a.service' 2024-05-09T00:26:38.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:26:38.227 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-09T00:26:38.227 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-09T00:26:38.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.d 2024-05-09T00:26:38.563 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 00:26:38 smithi046 systemd[1]: Stopping Ceph mds.d for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:26:39.063 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 00:26:38 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-d[69268]: 2024-05-09T00:26:38.701+0000 7f27932e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:26:39.063 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 00:26:38 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-d[69268]: 2024-05-09T00:26:38.701+0000 7f27932e0700 -1 mds.d *** got signal Terminated *** 2024-05-09T00:26:40.063 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 00:26:39 smithi046 podman[118163]: 2024-05-09 00:26:39.769269846 +0000 UTC m=+1.308700264 container died 516f9b745697662f50b75d63c578c20e5953c2b277a1a77609471124e04efda7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:26:40.519 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 00:26:40 smithi046 podman[118163]: 2024-05-09 00:26:40.092618377 +0000 UTC m=+1.632048797 container cleanup 516f9b745697662f50b75d63c578c20e5953c2b277a1a77609471124e04efda7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-d, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T00:26:40.519 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 00:26:40 smithi046 bash[118163]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-d 2024-05-09T00:26:40.519 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 00:26:40 smithi046 podman[118177]: 2024-05-09 00:26:40.285258832 +0000 UTC m=+0.514622309 container remove 516f9b745697662f50b75d63c578c20e5953c2b277a1a77609471124e04efda7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-d, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:26:40.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.d.service' 2024-05-09T00:26:40.572 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:26:40.573 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-09T00:26:40.573 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-09T00:26:40.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.g 2024-05-09T00:26:40.813 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:26:40 smithi046 systemd[1]: Stopping Ceph mds.g for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:26:41.313 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:26:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-g[70090]: 2024-05-09T00:26:40.913+0000 7fc573002700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:26:41.314 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:26:40 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-g[70090]: 2024-05-09T00:26:40.913+0000 7fc573002700 -1 mds.g *** got signal Terminated *** 2024-05-09T00:26:45.197 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:26:44 smithi046 podman[118284]: 2024-05-09 00:26:44.92749734 +0000 UTC m=+4.135197884 container died 552d4e2443f6f68ba0e9aa23a3dd69123367cdbc10fbcae1c6ca1a6fedbc8a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-09T00:26:45.555 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:26:45 smithi046 podman[118284]: 2024-05-09 00:26:45.371168902 +0000 UTC m=+4.578869435 container cleanup 552d4e2443f6f68ba0e9aa23a3dd69123367cdbc10fbcae1c6ca1a6fedbc8a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:26:45.555 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:26:45 smithi046 bash[118284]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-g 2024-05-09T00:26:45.812 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 00:26:45 smithi046 podman[118298]: 2024-05-09 00:26:45.555633253 +0000 UTC m=+0.626112875 container remove 552d4e2443f6f68ba0e9aa23a3dd69123367cdbc10fbcae1c6ca1a6fedbc8a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-05-09T00:26:45.816 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.g.service' 2024-05-09T00:26:45.849 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:26:45.850 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-09T00:26:45.850 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-09T00:26:45.850 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.j 2024-05-09T00:26:46.166 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:26:45 smithi046 systemd[1]: Stopping Ceph mds.j for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:26:46.563 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:26:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-j[68818]: 2024-05-09T00:26:46.165+0000 7f6b36f6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:26:46.563 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:26:46 smithi046 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-j[68818]: 2024-05-09T00:26:46.165+0000 7f6b36f6d700 -1 mds.j *** got signal Terminated *** 2024-05-09T00:26:47.800 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:26:47 smithi046 podman[118406]: 2024-05-09 00:26:47.538318098 +0000 UTC m=+1.469122699 container died 4b11d193045daa5b7e9e781152a25c7932e0b437afef9b3b3a4fe1382bf251b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-j, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-09T00:26:48.063 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:26:47 smithi046 podman[118406]: 2024-05-09 00:26:47.96645283 +0000 UTC m=+1.897257438 container cleanup 4b11d193045daa5b7e9e781152a25c7932e0b437afef9b3b3a4fe1382bf251b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T00:26:48.063 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:26:47 smithi046 bash[118406]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-j 2024-05-09T00:26:48.444 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 00:26:48 smithi046 podman[118420]: 2024-05-09 00:26:48.184382281 +0000 UTC m=+0.641743294 container remove 4b11d193045daa5b7e9e781152a25c7932e0b437afef9b3b3a4fe1382bf251b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-j, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:26:48.465 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.j.service' 2024-05-09T00:26:48.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:26:48.501 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-09T00:26:48.501 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-09T00:26:48.501 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.b 2024-05-09T00:26:48.789 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:26:48 smithi142 systemd[1]: Stopping Ceph mds.b for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:26:49.056 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:26:48 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-b[64936]: 2024-05-09T00:26:48.788+0000 7f207ea37700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:26:49.056 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:26:48 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-b[64936]: 2024-05-09T00:26:48.788+0000 7f207ea37700 -1 mds.b *** got signal Terminated *** 2024-05-09T00:26:50.305 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:26:50 smithi142 podman[70836]: 2024-05-09 00:26:50.038434296 +0000 UTC m=+1.341104102 container died 09a12a4d824b60e257bb0fb639ee8191c8f059f37bb37402734ac01dafb0dfcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-b, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T00:26:50.664 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:26:50 smithi142 podman[70836]: 2024-05-09 00:26:50.405232877 +0000 UTC m=+1.707902682 container cleanup 09a12a4d824b60e257bb0fb639ee8191c8f059f37bb37402734ac01dafb0dfcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:26:50.665 INFO:journalctl@ceph.mds.b.smithi142.stdout:May 09 00:26:50 smithi142 bash[70836]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-b 2024-05-09T00:26:50.831 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.b.service' 2024-05-09T00:26:50.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:26:50.866 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-09T00:26:50.866 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-09T00:26:50.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.e 2024-05-09T00:26:51.194 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:26:50 smithi142 systemd[1]: Stopping Ceph mds.e for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:26:51.194 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:26:51 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-e[64303]: 2024-05-09T00:26:51.166+0000 7f6b72bdc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:26:51.194 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:26:51 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-e[64303]: 2024-05-09T00:26:51.166+0000 7f6b72bdc700 -1 mds.e *** got signal Terminated *** 2024-05-09T00:26:54.318 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:26:54 smithi142 podman[70969]: 2024-05-09 00:26:54.047858623 +0000 UTC m=+2.971802455 container died 8c35d1b6fb6e103280bbbcfaa4c7dfab5b15e36db3c88f0eb359065e1bec628d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-e, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T00:26:54.678 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:26:54 smithi142 podman[70969]: 2024-05-09 00:26:54.417545981 +0000 UTC m=+3.341489812 container cleanup 8c35d1b6fb6e103280bbbcfaa4c7dfab5b15e36db3c88f0eb359065e1bec628d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-09T00:26:54.678 INFO:journalctl@ceph.mds.e.smithi142.stdout:May 09 00:26:54 smithi142 bash[70969]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-e 2024-05-09T00:26:54.845 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.e.service' 2024-05-09T00:26:54.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:26:54.877 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-09T00:26:54.877 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-09T00:26:54.877 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.h 2024-05-09T00:26:55.187 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:26:54 smithi142 systemd[1]: Stopping Ceph mds.h for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:26:55.443 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:26:55 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-h[64617]: 2024-05-09T00:26:55.188+0000 7fe4a3315700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:26:55.443 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:26:55 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-h[64617]: 2024-05-09T00:26:55.188+0000 7fe4a3315700 -1 mds.h *** got signal Terminated *** 2024-05-09T00:26:58.321 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:26:58 smithi142 podman[71090]: 2024-05-09 00:26:58.047138938 +0000 UTC m=+2.957141565 container died f07c30c5b6cc2c1b648145c06bc26f6599d9666762d3cf9b0ec2b2b8741ad286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:26:58.688 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:26:58 smithi142 podman[71090]: 2024-05-09 00:26:58.395781591 +0000 UTC m=+3.305784228 container cleanup f07c30c5b6cc2c1b648145c06bc26f6599d9666762d3cf9b0ec2b2b8741ad286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:26:58.688 INFO:journalctl@ceph.mds.h.smithi142.stdout:May 09 00:26:58 smithi142 bash[71090]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-h 2024-05-09T00:26:58.822 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.h.service' 2024-05-09T00:26:58.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:26:58.854 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-09T00:26:58.855 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-09T00:26:58.855 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.k 2024-05-09T00:26:58.943 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:26:58 smithi142 systemd[1]: Stopping Ceph mds.k for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:26:59.443 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:26:59 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-k[63981]: 2024-05-09T00:26:59.157+0000 7f0866aff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:26:59.444 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:26:59 smithi142 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-k[63981]: 2024-05-09T00:26:59.157+0000 7f0866aff700 -1 mds.k *** got signal Terminated *** 2024-05-09T00:27:03.444 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:27:03 smithi142 podman[71212]: 2024-05-09 00:27:03.163361053 +0000 UTC m=+4.089319801 container died 2b3a54540edf4897d3c8d56f6a276ee7137d2e90c84cad3076c00c422bfb3cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-k, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:27:03.785 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:27:03 smithi142 podman[71212]: 2024-05-09 00:27:03.592651888 +0000 UTC m=+4.518610631 container cleanup 2b3a54540edf4897d3c8d56f6a276ee7137d2e90c84cad3076c00c422bfb3cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-k, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T00:27:03.785 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:27:03 smithi142 bash[71212]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-k 2024-05-09T00:27:04.025 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.k.service' 2024-05-09T00:27:04.045 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:27:03 smithi142 podman[71226]: 2024-05-09 00:27:03.785058044 +0000 UTC m=+0.615448775 container remove 2b3a54540edf4897d3c8d56f6a276ee7137d2e90c84cad3076c00c422bfb3cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-k, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T00:27:04.045 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:27:04 smithi142 systemd[1]: ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.k.service: Deactivated successfully. 2024-05-09T00:27:04.045 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:27:04 smithi142 systemd[1]: Stopped Ceph mds.k for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:27:04.046 INFO:journalctl@ceph.mds.k.smithi142.stdout:May 09 00:27:04 smithi142 systemd[1]: ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.k.service: Consumed 1.177s CPU time. 2024-05-09T00:27:04.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:27:04.056 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-09T00:27:04.056 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-09T00:27:04.056 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.c 2024-05-09T00:27:04.356 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:27:04 smithi169 systemd[1]: Stopping Ceph mds.c for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:27:04.628 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:27:04 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c[64970]: 2024-05-09T00:27:04.397+0000 7f91ed11a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:27:04.629 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:27:04 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c[64970]: 2024-05-09T00:27:04.397+0000 7f91ed11a700 -1 mds.c *** got signal Terminated *** 2024-05-09T00:27:06.342 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:27:06 smithi169 podman[71914]: 2024-05-09 00:27:06.044518266 +0000 UTC m=+1.789080386 container died e161ca7048f4bc123c0ed39387229fa363c97aaa029a989a459255faf6bc13b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T00:27:06.606 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:27:06 smithi169 podman[71914]: 2024-05-09 00:27:06.431337178 +0000 UTC m=+2.175899299 container cleanup e161ca7048f4bc123c0ed39387229fa363c97aaa029a989a459255faf6bc13b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-09T00:27:06.607 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:27:06 smithi169 bash[71914]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c 2024-05-09T00:27:06.851 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.c.service' 2024-05-09T00:27:06.872 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:27:06 smithi169 podman[71940]: 2024-05-09 00:27:06.607498586 +0000 UTC m=+0.556789214 container remove e161ca7048f4bc123c0ed39387229fa363c97aaa029a989a459255faf6bc13b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T00:27:06.872 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:27:06 smithi169 systemd[1]: ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.c.service: Deactivated successfully. 2024-05-09T00:27:06.872 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:27:06 smithi169 systemd[1]: Stopped Ceph mds.c for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:27:06.873 INFO:journalctl@ceph.mds.c.smithi169.stdout:May 09 00:27:06 smithi169 systemd[1]: ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.c.service: Consumed 2.294s CPU time. 2024-05-09T00:27:06.885 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:27:06.886 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-09T00:27:06.886 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-09T00:27:06.886 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.f 2024-05-09T00:27:07.194 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:27:06 smithi169 systemd[1]: Stopping Ceph mds.f for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:27:07.606 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:27:07 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f[64658]: 2024-05-09T00:27:07.193+0000 7f7818d3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:27:07.606 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:27:07 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f[64658]: 2024-05-09T00:27:07.193+0000 7f7818d3f700 -1 mds.f *** got signal Terminated *** 2024-05-09T00:27:10.310 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:27:10 smithi169 podman[72048]: 2024-05-09 00:27:10.053289705 +0000 UTC m=+2.939644846 container died c8111ddbd5979f0b4ca84ec0d385bd531a9e80ab8ac3aeed343c300b99122aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T00:27:10.579 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:27:10 smithi169 podman[72048]: 2024-05-09 00:27:10.393391982 +0000 UTC m=+3.279747121 container cleanup c8111ddbd5979f0b4ca84ec0d385bd531a9e80ab8ac3aeed343c300b99122aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T00:27:10.579 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:27:10 smithi169 bash[72048]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f 2024-05-09T00:27:10.821 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.f.service' 2024-05-09T00:27:10.841 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:27:10 smithi169 podman[72063]: 2024-05-09 00:27:10.579625502 +0000 UTC m=+0.530367629 container remove c8111ddbd5979f0b4ca84ec0d385bd531a9e80ab8ac3aeed343c300b99122aad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T00:27:10.842 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:27:10 smithi169 systemd[1]: ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.f.service: Deactivated successfully. 2024-05-09T00:27:10.842 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:27:10 smithi169 systemd[1]: Stopped Ceph mds.f for eec67860-0d96-11ef-bc97-c7b262605968. 2024-05-09T00:27:10.842 INFO:journalctl@ceph.mds.f.smithi169.stdout:May 09 00:27:10 smithi169 systemd[1]: ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.f.service: Consumed 3.335s CPU time. 2024-05-09T00:27:10.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:27:10.855 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-09T00:27:10.855 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-09T00:27:10.855 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.i 2024-05-09T00:27:11.106 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:27:10 smithi169 systemd[1]: Stopping Ceph mds.i for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:27:11.606 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:27:11 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i[65278]: 2024-05-09T00:27:11.187+0000 7f87635cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:27:11.606 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:27:11 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i[65278]: 2024-05-09T00:27:11.187+0000 7f87635cf700 -1 mds.i *** got signal Terminated *** 2024-05-09T00:27:14.297 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:27:14 smithi169 podman[72173]: 2024-05-09 00:27:14.044591245 +0000 UTC m=+2.970975967 container died 83bac6934c60fb5fae0b83b8264586d348aca69738fab93dc1cc1725b0473481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T00:27:14.556 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:27:14 smithi169 podman[72173]: 2024-05-09 00:27:14.388308457 +0000 UTC m=+3.314693174 container cleanup 83bac6934c60fb5fae0b83b8264586d348aca69738fab93dc1cc1725b0473481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-09T00:27:14.556 INFO:journalctl@ceph.mds.i.smithi169.stdout:May 09 00:27:14 smithi169 bash[72173]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-i 2024-05-09T00:27:14.776 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.i.service' 2024-05-09T00:27:14.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:27:14.808 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-09T00:27:14.809 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-09T00:27:14.809 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.l 2024-05-09T00:27:15.106 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:27:14 smithi169 systemd[1]: Stopping Ceph mds.l for eec67860-0d96-11ef-bc97-c7b262605968... 2024-05-09T00:27:15.606 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:27:15 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-l[64348]: 2024-05-09T00:27:15.133+0000 7f8e302f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T00:27:15.607 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:27:15 smithi169 ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-l[64348]: 2024-05-09T00:27:15.133+0000 7f8e302f3700 -1 mds.l *** got signal Terminated *** 2024-05-09T00:27:16.517 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:27:16 smithi169 podman[72293]: 2024-05-09 00:27:16.246738797 +0000 UTC m=+1.226672700 container died b0c6cb191e9f9c1dd7d77e64e840f2f26f118b59ef84b29f9f9100cbb72496e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-l, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-05-09T00:27:16.856 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:27:16 smithi169 podman[72293]: 2024-05-09 00:27:16.600440354 +0000 UTC m=+1.580374257 container cleanup b0c6cb191e9f9c1dd7d77e64e840f2f26f118b59ef84b29f9f9100cbb72496e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-l, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T00:27:16.856 INFO:journalctl@ceph.mds.l.smithi169.stdout:May 09 00:27:16 smithi169 bash[72293]: ceph-eec67860-0d96-11ef-bc97-c7b262605968-mds-l 2024-05-09T00:27:17.017 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eec67860-0d96-11ef-bc97-c7b262605968@mds.l.service' 2024-05-09T00:27:17.048 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T00:27:17.048 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-09T00:27:17.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eec67860-0d96-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T00:27:17.250 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:27:19.845 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eec67860-0d96-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T00:27:20.035 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:27:22.401 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eec67860-0d96-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T00:27:22.591 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:27:24.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:27:24.950 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:27:24.982 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T00:27:25.008 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-09T00:27:25.010 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857/remote/smithi046/crash 2024-05-09T00:27:25.011 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/crash -- . 2024-05-09T00:27:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:tar: /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T00:27:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:tar: Error is not recoverable: exiting now 2024-05-09T00:27:25.055 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857/remote/smithi142/crash 2024-05-09T00:27:25.056 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/crash -- . 2024-05-09T00:27:25.096 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T00:27:25.096 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-05-09T00:27:25.098 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857/remote/smithi169/crash 2024-05-09T00:27:25.099 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/crash -- . 2024-05-09T00:27:25.132 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/eec67860-0d96-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T00:27:25.132 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now 2024-05-09T00:27:25.133 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-09T00:27:25.134 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-09T00:27:25.175 INFO:tasks.cephadm:Compressing logs... 2024-05-09T00:27:25.175 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T00:27:25.219 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T00:27:25.222 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T00:27:25.248 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T00:27:25.248 INFO:teuthology.orchestra.run.smithi169.stderr:: No such file or directory 2024-05-09T00:27:25.248 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.log 2024-05-09T00:27:25.248 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-09T00:27:25.248 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-09T00:27:25.248 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T00:27:25.248 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-09T00:27:25.248 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.audit.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T00:27:25.248 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-09T00:27:25.249 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-09T00:27:25.249 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.5.log: /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-09T00:27:25.249 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-09T00:27:25.249 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mgr.z.log: 88.1% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T00:27:25.259 INFO:teuthology.orchestra.run.smithi169.stderr: 90.5% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T00:27:25.259 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T00:27:25.259 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-09T00:27:25.259 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T00:27:25.259 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-09T00:27:25.259 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.11.log: 93.1% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-09T00:27:25.265 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T00:27:25.265 INFO:teuthology.orchestra.run.smithi142.stderr:: No such file or directory 2024-05-09T00:27:25.265 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.log 2024-05-09T00:27:25.265 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T00:27:25.265 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-09T00:27:25.265 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-09T00:27:25.265 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.7.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T00:27:25.265 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-09T00:27:25.266 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-09T00:27:25.266 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.b.log: /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-09T00:27:25.266 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-09T00:27:25.266 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-09T00:27:25.266 INFO:teuthology.orchestra.run.smithi142.stderr: 90.5% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T00:27:25.266 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.h.log: 88.2% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T00:27:25.266 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T00:27:25.266 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-09T00:27:25.266 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T00:27:25.266 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T00:27:25.267 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.e.log: 93.0% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-09T00:27:25.267 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-09T00:27:25.269 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T00:27:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T00:27:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T00:27:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.log 2024-05-09T00:27:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T00:27:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-09T00:27:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-09T00:27:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-09T00:27:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.g.log: /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-09T00:27:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-09T00:27:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-09T00:27:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mgr.x.log: 88.1% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr: 90.3% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.6.log: 94.7% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr: 94.6% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.a.log: /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.9% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-volume.log: 94.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr: 94.6% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-09T00:27:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.9.log: 91.0% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T00:27:25.276 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T00:27:25.276 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-09T00:27:25.277 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-09T00:27:25.300 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-volume.log: /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-09T00:27:25.304 INFO:teuthology.orchestra.run.smithi169.stderr: 91.0% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T00:27:25.304 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mon.b.log: /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-09T00:27:25.317 INFO:teuthology.orchestra.run.smithi142.stderr: 92.9% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-09T00:27:25.324 INFO:teuthology.orchestra.run.smithi142.stderr: 94.5% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-09T00:27:25.324 INFO:teuthology.orchestra.run.smithi142.stderr: 94.6% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-09T00:27:25.330 INFO:teuthology.orchestra.run.smithi169.stderr: 92.6% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-09T00:27:25.515 INFO:teuthology.orchestra.run.smithi169.stderr: 94.3% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-09T00:27:25.562 INFO:teuthology.orchestra.run.smithi046.stderr: 90.2% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-09T00:27:25.687 INFO:teuthology.orchestra.run.smithi169.stderr: 91.7% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-09T00:27:25.717 INFO:teuthology.orchestra.run.smithi142.stderr: 91.8% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-09T00:27:25.882 INFO:teuthology.orchestra.run.smithi169.stderr: 92.5% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-09T00:27:25.898 INFO:teuthology.orchestra.run.smithi169.stderr: 93.0% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-09T00:27:25.966 INFO:teuthology.orchestra.run.smithi046.stderr: 92.4% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-09T00:27:26.140 INFO:teuthology.orchestra.run.smithi142.stderr: 92.2% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-09T00:27:26.203 INFO:teuthology.orchestra.run.smithi142.stderr: 91.2% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-09T00:27:26.273 INFO:teuthology.orchestra.run.smithi046.stderr: 93.2% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-09T00:27:26.279 INFO:teuthology.orchestra.run.smithi046.stderr: 90.4% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-09T00:27:26.567 INFO:teuthology.orchestra.run.smithi046.stderr: 93.2% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-09T00:27:26.642 INFO:teuthology.orchestra.run.smithi046.stderr: 93.0% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-09T00:27:26.644 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-05-09T00:27:26.644 INFO:teuthology.orchestra.run.smithi046.stderr:real 0m1.410s 2024-05-09T00:27:26.644 INFO:teuthology.orchestra.run.smithi046.stderr:user 0m5.667s 2024-05-09T00:27:26.644 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m0.190s 2024-05-09T00:27:26.818 INFO:teuthology.orchestra.run.smithi142.stderr: 92.3% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-09T00:27:26.941 INFO:teuthology.orchestra.run.smithi169.stderr: 92.4% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-09T00:27:27.049 INFO:teuthology.orchestra.run.smithi169.stderr: 91.3% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-09T00:27:27.049 INFO:teuthology.orchestra.run.smithi169.stderr: 92.9% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-09T00:27:27.049 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-05-09T00:27:27.049 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m1.812s 2024-05-09T00:27:27.049 INFO:teuthology.orchestra.run.smithi169.stderr:user 0m7.103s 2024-05-09T00:27:27.049 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m0.271s 2024-05-09T00:27:27.173 INFO:teuthology.orchestra.run.smithi142.stderr: 92.1% -- replaced with /var/log/ceph/eec67860-0d96-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-09T00:27:27.175 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-05-09T00:27:27.175 INFO:teuthology.orchestra.run.smithi142.stderr:real 0m1.940s 2024-05-09T00:27:27.176 INFO:teuthology.orchestra.run.smithi142.stderr:user 0m5.843s 2024-05-09T00:27:27.176 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m0.206s 2024-05-09T00:27:27.176 INFO:tasks.cephadm:Archiving logs... 2024-05-09T00:27:27.176 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857/remote/smithi046/log 2024-05-09T00:27:27.177 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T00:27:28.061 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857/remote/smithi142/log 2024-05-09T00:27:28.062 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T00:27:29.107 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857/remote/smithi169/log 2024-05-09T00:27:29.108 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T00:27:30.206 INFO:tasks.cephadm:Removing cluster... 2024-05-09T00:27:30.207 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eec67860-0d96-11ef-bc97-c7b262605968 --force 2024-05-09T00:27:30.410 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:27:30.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eec67860-0d96-11ef-bc97-c7b262605968 --force 2024-05-09T00:27:30.878 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:27:31.150 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eec67860-0d96-11ef-bc97-c7b262605968 --force 2024-05-09T00:27:31.339 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: eec67860-0d96-11ef-bc97-c7b262605968 2024-05-09T00:27:31.607 INFO:tasks.cephadm:Removing cephadm ... 2024-05-09T00:27:31.607 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T00:27:31.627 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T00:27:31.643 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T00:27:31.658 INFO:tasks.cephadm:Teardown complete 2024-05-09T00:27:31.659 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T00:27:31.670 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T00:27:31.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T00:27:31.673 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T00:27:31.686 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T00:27:31.796 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T00:27:31.810 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T00:27:31.810 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T00:27:31.813 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T00:27:31.816 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T00:27:31.828 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-05-09T00:27:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-05-09T00:27:31.829 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-05-09T00:27:31.831 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T00:27:31.831 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-05-09T00:27:31.831 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 +827us[ +826us] +/- 48ms 2024-05-09T00:27:31.832 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 -931us[ -932us] +/- 54ms 2024-05-09T00:27:31.832 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -147us[ -149us] +/- 71ms 2024-05-09T00:27:31.832 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:27:31.833 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T00:27:31.833 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-05-09T00:27:31.833 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 +871us[ +845us] +/- 48ms 2024-05-09T00:27:31.833 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -906us[ -906us] +/- 54ms 2024-05-09T00:27:31.833 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -127us[ -154us] +/- 71ms 2024-05-09T00:27:31.833 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:27:31.835 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T00:27:31.835 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-05-09T00:27:31.835 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 +826us[ +822us] +/- 48ms 2024-05-09T00:27:31.835 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 -927us[ -931us] +/- 54ms 2024-05-09T00:27:31.835 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -141us[ -141us] +/- 71ms 2024-05-09T00:27:31.835 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T00:27:31.836 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T00:27:31.886 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T00:27:31.887 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T00:27:31.918 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T00:27:32.012 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T00:27:32.076 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T00:27:32.141 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T00:27:32.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:27:32.173 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T00:27:32.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:27:32.200 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T00:27:32.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:27:32.226 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T00:27:32.237 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T00:27:32.276 INFO:teuthology.task.internal:Duration was 2096.382010 seconds 2024-05-09T00:27:32.276 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T00:27:32.309 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T00:27:32.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T00:27:32.312 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T00:27:32.314 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T00:27:32.347 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T00:27:32.350 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T00:27:32.353 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T00:27:32.858 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T00:27:32.858 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T00:27:32.859 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T00:27:32.925 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-05-09T00:27:32.926 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T00:27:32.946 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-05-09T00:27:32.947 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T00:27:32.968 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T00:27:32.968 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T00:27:32.971 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T00:27:32.989 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T00:27:33.035 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T00:27:33.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T00:27:33.094 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T00:27:33.098 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T00:27:34.341 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T00:27:34.354 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T00:27:34.354 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T00:27:34.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T00:27:34.409 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T00:27:34.439 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T00:27:34.455 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T00:27:34.459 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T00:27:34.462 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T00:27:34.481 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-05-09T00:27:34.484 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-05-09T00:27:34.504 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-05-09T00:27:34.557 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T00:27:34.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:27:34.572 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T00:27:34.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:27:34.588 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T00:27:34.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T00:27:34.613 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T00:27:34.626 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T00:27:34.627 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857/remote/smithi046 2024-05-09T00:27:34.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T00:27:34.696 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857/remote/smithi142 2024-05-09T00:27:34.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T00:27:34.735 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7698857/remote/smithi169 2024-05-09T00:27:34.735 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T00:27:34.774 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T00:27:34.774 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T00:27:34.777 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T00:27:34.780 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T00:27:34.826 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T00:27:34.842 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T00:27:34.842 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T00:27:34.866 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T00:27:34.866 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T00:27:34.869 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T00:27:34.871 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T00:27:34.882 INFO:teuthology.orchestra.run.smithi046.stdout: 265304 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 00:27 /home/ubuntu/cephtest 2024-05-09T00:27:34.886 INFO:teuthology.orchestra.run.smithi142.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 00:27 /home/ubuntu/cephtest 2024-05-09T00:27:34.887 INFO:teuthology.orchestra.run.smithi169.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 00:27 /home/ubuntu/cephtest 2024-05-09T00:27:34.889 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T00:27:34.901 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T00:27:34.965 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/fs/test_o_trunc}} duration: 2096.3820104599 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-09T00:27:34.965 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T00:27:35.049 INFO:teuthology.run:pass